December 2011 - tor-commits - lists.torproject.org

r25245: {website} grammar fix. (website/trunk/about/en)
by Andrew Lewman 06 Dec '11

06 Dec '11

Author: phobos Date: 2011-12-06 16:12:36 +0000 (Tue, 06 Dec 2011) New Revision: 25245 Modified: website/trunk/about/en/contact.wml Log: grammar fix. Modified: website/trunk/about/en/contact.wml =================================================================== --- website/trunk/about/en/contact.wml 2011-12-06 16:10:28 UTC (rev 25244) +++ website/trunk/about/en/contact.wml 2011-12-06 16:12:36 UTC (rev 25245) @@ -23,7 +23,7 @@ <ul> <li><i>help at rt.torproject.org</i> is address to use for Tor support requests. This is staffed by volunteers helping people - with questions about all things Tor. There is best effort service + with questions about all things Tor. This is best effort service with no guarantees.</li> <li><i>tor-translation at lists.torproject.org</i> can put new <a href="<page getinvolved/translation>">website translations</a> into place,

1 0

r25244: {website} fix the android package path. (website/trunk/download/en)
by Andrew Lewman 06 Dec '11

06 Dec '11

Author: phobos Date: 2011-12-06 16:10:28 +0000 (Tue, 06 Dec 2011) New Revision: 25244 Modified: website/trunk/download/en/download.wml Log: fix the android package path. Modified: website/trunk/download/en/download.wml =================================================================== --- website/trunk/download/en/download.wml 2011-12-05 00:59:37 UTC (rev 25243) +++ website/trunk/download/en/download.wml 2011-12-06 16:10:28 UTC (rev 25244) @@ -223,7 +223,7 @@  <div class="package" style="padding-top: 13px; border-top: 0px;"> <div class="downloads"> - <a class="button" style="margin-bottom: 5px;" href="<package-androidbundle-alpha>"><span class="strong">Download</span><span class="normal">Android Bundle</span></a> + <a class="button" style="margin-bottom: 5px;" href="../<package-androidbundle-alpha>"><span class="strong">Download</span><span class="normal">Android Bundle</span></a> <a class="additional" href="<page docs/android>">Installation Instructions</a> </div>

1 0

[translation/tsum] Update translations for tsum
by translation＠torproject.org 06 Dec '11

06 Dec '11

commit b89cb04d022584688843d5c46e1edec4bf7f3d1d Author: Translation commit bot <translation(a)torproject.org> Date: Tue Dec 6 15:15:08 2011 +0000 Update translations for tsum --- fi/tsum.po | 604 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 604 insertions(+), 0 deletions(-) diff --git a/fi/tsum.po b/fi/tsum.po new file mode 100644 index 0000000..d00e07c --- /dev/null +++ b/fi/tsum.po @@ -0,0 +1,604 @@ +# SOME DESCRIPTIVE TITLE +# Copyright (C) YEAR The Tor Project, Inc +# This file is distributed under the same license as the PACKAGE package. +# +# Translators: +msgid "" +msgstr "" +"Project-Id-Version: The Tor Project\n" +"Report-Msgid-Bugs-To: https://trac.torproject.org/projects/tor\n" +"POT-Creation-Date: 2011-11-05 17:16+0000\n" +"PO-Revision-Date: 2011-10-12 13:07+0000\n" +"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n" +"Language-Team: LANGUAGE <LL(a)li.org>\n" +"MIME-Version: 1.0\n" +"Content-Type: text/plain; charset=UTF-8\n" +"Content-Transfer-Encoding: 8bit\n" +"Language: fi\n" +"Plural-Forms: nplurals=2; plural=(n != 1)\n" + +#. type: Plain text +#: en/tsum.text:2 +msgid "# The Short User Manual\n" +msgstr "" + +#. type: Plain text +#: en/tsum.text:7 +msgid "" +"This user manual contains information about how to download Tor, how to use " +"it, and what to do if Tor is unable to connect to the network. If you can't " +"find the answer to your question in this document, email " +"help(a)rt.torproject.org." +msgstr "" + +#. type: Plain text +#: en/tsum.text:11 +msgid "" +"Please note that we are providing support on a voluntary basis, and we get a" +" lot of emails every single day. There is no need to worry if we don't get " +"back to you straight away." +msgstr "" + +#. type: Plain text +#: en/tsum.text:13 +#, no-wrap +msgid "" +"How Tor works\n" +"-------------\n" +msgstr "" + +#. type: Plain text +#: en/tsum.text:19 +msgid "" +"Tor is a network of virtual tunnels that allows you to improve your privacy " +"and security on the Internet. Tor works by sending your traffic through " +"three random servers (also known as *relays*) in the Tor network, before the" +" traffic is sent out onto the public Internet." +msgstr "" + +#. type: Plain text +#: en/tsum.text:21 +msgid "![How Tor Works](images/how-tor-works.png)" +msgstr "" + +#. type: Plain text +#: en/tsum.text:26 +msgid "" +"The image above illustrates a user browsing to different websites over Tor. " +"The green monitors represent relays in the Tor network, while the three keys" +" represent the layers of encryption between the user and each relay." +msgstr "" + +#. type: Plain text +#: en/tsum.text:31 +msgid "" +"Tor will anonymize the origin of your traffic, and it will encrypt " +"everything between you and the Tor network. Tor will also encrypt your " +"traffic inside the Tor network, but it cannot encrypt your traffic between " +"the Tor network and its final destination." +msgstr "" + +#. type: Plain text +#: en/tsum.text:36 +msgid "" +"If you are communicating sensitive information, for example when logging on " +"to a website with a username and password, make sure that you are using " +"HTTPS (e.g. **https**://torproject.org/, not **http**://torproject.org/)." +msgstr "" + +#. type: Plain text +#: en/tsum.text:38 +#, no-wrap +msgid "" +"How to download Tor\n" +"-------------------\n" +msgstr "" + +#. type: Plain text +#: en/tsum.text:44 +msgid "" +"The bundle we recommend to most users is the [Tor Browser " +"Bundle](https://www.torproject.org/projects/torbrowser.html). This bundle " +"contains a browser preconfigured to safely browse the Internet through Tor, " +"and requires no installation. You download the bundle, extract the archive, " +"and start Tor." +msgstr "" + +#. type: Plain text +#: en/tsum.text:48 +msgid "" +"There are two different ways to get hold of the Tor software. You can either" +" browse to the [Tor Project website](https://www.torproject.org/) and " +"download it there, or you can use GetTor, the email autoresponder." +msgstr "" + +#. type: Plain text +#: en/tsum.text:53 +msgid "" +"### How to get Tor via email\n" +"To receive the English Tor Browser Bundle for Windows, send an email to gettor(a)torproject.org with **windows** in the body of the message. You can leave the subject blank." +msgstr "" + +#. type: Plain text +#: en/tsum.text:57 +msgid "" +"You can also request the Tor Browser Bundle for Mac OS X (write " +"**macos-i386**), and Linux (write **linux-i386** for 32-bit systems or " +"**linux-x86_64** for 64-bit systems)." +msgstr "" + +#. type: Plain text +#: en/tsum.text:61 +msgid "" +"If you want a translated version of Tor, write **help** instead. You will " +"then receive an email with instructions and a list of available languages." +msgstr "" + +#. type: Plain text +#: en/tsum.text:67 +#, no-wrap +msgid "" +"**Note**: The Tor Browser Bundles for Linux and Mac OS X are rather\n" +"large, and you will not be able to receive any of these bundles with a\n" +"Gmail, Hotmail or Yahoo account. If you cannot receive the bundle you\n" +"want, send an email to help(a)rt.torproject.org and we\n" +"will give you a list of website mirrors to use.\n" +msgstr "" + +#. type: Plain text +#: en/tsum.text:73 +msgid "" +"#### How to get Tor as several small sized packages\n" +"It is possible to request the Tor Browser Bundle for Windows as several small sized packages, instead of one big package. This can be useful if you don't have a lot of bandwidth available, or if your email provider does not allow you to receive large attachments." +msgstr "" + +#. type: Plain text +#: en/tsum.text:76 +msgid "" +"Send an email to gettor(a)torproject.org with the following words in the body " +"of the email:" +msgstr "" + +#. type: Plain text +#: en/tsum.text:79 +#, no-wrap +msgid "" +"windows \n" +"split\n" +msgstr "" + +#. type: Plain text +#: en/tsum.text:83 +msgid "" +"It is important that you include the keyword *split* on its own line. See " +"*What to do with split packages* for information on how to unpack and re-" +"assemble the small sized packages." +msgstr "" + +#. type: Plain text +#: en/tsum.text:89 +msgid "" +"### Tor for smartphones\n" +"You can get Tor on your Android device by installing the package named *Orbot*. For information about how to download and install Orbot, please see the [Tor Project website](https://www.torproject.org/docs/android.html.en)." +msgstr "" + +#. type: Plain text +#: en/tsum.text:93 +msgid "" +"We also have experimental packages for [Nokia " +"Maemo/N900](https://www.torproject.org/docs/N900.html.en) and [Apple " +"iOS](http://sid77.slackware.it/iphone/)." +msgstr "" + +#. type: Plain text +#: en/tsum.text:97 +msgid "" +"### How to verify that you have the right version\n" +"Before running the Tor Browser Bundle, you should make sure that you have the right version." +msgstr "" + +#. type: Plain text +#: en/tsum.text:102 +msgid "" +"The software you receive is accompanied by a file with the same name as the " +"bundle and the extension **.asc**. This .asc file is a GPG signature, and " +"will allow you to verify the file you've downloaded is exactly the one that " +"we intended you to get." +msgstr "" + +#. type: Plain text +#: en/tsum.text:105 +#, no-wrap +msgid "" +"Before you can verify the signature, you will have to download and\n" +"install GnuPG: \n" +msgstr "" + +#. type: Plain text +#: en/tsum.text:109 +#, no-wrap +msgid "" +"**Windows**: [http://gpg4win.org/download.html](http://gpg4win.org/download.html) \n" +"**Mac OS X**: [http://macgpg.sourceforge.net/](http://macgpg.sourceforge.net/) \n" +"**Linux**: Most Linux distributions come with GnuPG preinstalled.\n" +msgstr "" + +#. type: Plain text +#: en/tsum.text:112 +msgid "" +"Please note that you may need to edit the paths and the commands used below " +"to get it to work on your system." +msgstr "" + +#. type: Plain text +#: en/tsum.text:115 +msgid "" +"Erinn Clark signs the Tor Browser Bundles with key 0x63FEE659. To import " +"Erinn's key, run:" +msgstr "" + +#. type: Plain text +#: en/tsum.text:117 +#, no-wrap +msgid "\tgpg --keyserver hkp://keys.gnupg.net --recv-keys 0x63FEE659\n" +msgstr "" + +#. type: Plain text +#: en/tsum.text:119 +msgid "After importing the key, verify that the fingerprint is correct:" +msgstr "" + +#. type: Plain text +#: en/tsum.text:121 +#, no-wrap +msgid "\tgpg --fingerprint 0x63FEE659\n" +msgstr "" + +#. type: Plain text +#: en/tsum.text:123 +msgid "You should see:" +msgstr "" + +#. type: Plain text +#: en/tsum.text:130 +#, no-wrap +msgid "" +"\tpub 2048R/63FEE659 2003-10-16\n" +"\t Key fingerprint = 8738 A680 B84B 3031 A630 F2DB 416F 0610 63FE E659\n" +"\tuid Erinn Clark <erinn(a)torproject.org>\n" +"\tuid Erinn Clark <erinn(a)debian.org>\n" +"\tuid Erinn Clark <erinn(a)double-helix.org>\n" +"\tsub 2048R/EB399FD7 2003-10-16\n" +msgstr "" + +#. type: Plain text +#: en/tsum.text:132 +msgid "" +"To verify the signature of the package you downloaded, run the following " +"command:" +msgstr "" + +#. type: Plain text +#: en/tsum.text:134 +#, no-wrap +msgid "" +"\tgpg --verify tor-browser-2.2.33-2_en-US.exe.asc tor-browser-2.2.33-2_en-" +"US.exe\n" +msgstr "" + +#. type: Plain text +#: en/tsum.text:140 +msgid "" +"The output should say *\"Good signature\"*. A bad signature means that the " +"file may have been tampered with. If you see a bad signature, send details " +"about where you downloaded the package from, how you verified the signature," +" and the output from GnuPG in an email to help(a)rt.torproject.org." +msgstr "" + +#. type: Plain text +#: en/tsum.text:147 +msgid "" +"Once you have verified the signature and seen the *\"Good signature\"* " +"output, go ahead and extract the package archive. You should then see a " +"directory similar to **tor-browser_en-US**. Inside that directory is another" +" directory called **Docs**, which contains a file called **changelog**. You " +"want to make sure that the version number on the top line of the changelog " +"file matches the version number in the filename." +msgstr "" + +#. type: Plain text +#: en/tsum.text:153 +msgid "" +"### How to use the Tor Browser Bundle\n" +"After downloading the Tor Browser Bundle and extracting the package, you should have a directory with a few files in it. One of the files is an executable called \"Start Tor Browser\" (or \"start-tor-browser\", depending on your operating system)." +msgstr "" + +#. type: Plain text +#: en/tsum.text:160 +msgid "" +"When you start the Tor Browser Bundle, you will first see Vidalia start up " +"and connect you to the Tor network. After that, you will see a browser " +"confirming that you are now using Tor. This is done by displaying " +"[https://check.torproject.org/](https://check.torproject.org/). You can now " +"browse the Internet through Tor." +msgstr "" + +#. type: Plain text +#: en/tsum.text:163 +#, no-wrap +msgid "" +"*Please note that it is important that you use the browser that comes\n" +"with the bundle, and not your own browser.*\n" +msgstr "" + +#. type: Plain text +#: en/tsum.text:169 +msgid "" +"### What to do when Tor does not connect\n" +"Some users will notice that Vidalia gets stuck when trying to connect to the Tor network. If this happens, make sure that you are connected to the Internet. If you need to connect to a proxy server, see *How to use an open proxy* below." +msgstr "" + +#. type: Plain text +#: en/tsum.text:174 +msgid "" +"If your normal Internet connection is working, but Tor still can't connect " +"to the network, try the following; open the Vidalia control panel, click on " +"*Message Log* and select the *Advanced* tab. It may be that Tor won't " +"connect because:" +msgstr "" + +#. type: Plain text +#: en/tsum.text:178 +#, no-wrap +msgid "" +"**Your system clock is off**: Make sure that the date and time on your\n" +"system is correct, and restart Tor. You may need to synchronize your\n" +"system clock with an Internet time server. \n" +msgstr "" + +#. type: Plain text +#: en/tsum.text:183 +#, no-wrap +msgid "" +"**You are behind a restrictive firewall**: To tell Tor to only try port\n" +"80 and port 443, open the Vidalia control panel, click on *Settings* and\n" +"*Network*, and tick the box that says *My firewall only lets me connect\n" +"to certain ports*. \n" +msgstr "" + +#. type: Plain text +#: en/tsum.text:187 +#, no-wrap +msgid "" +"**Your anti-virus program is blocking Tor**: Make sure that your\n" +"anti-virus program is not preventing Tor from making network\n" +"connections.\n" +msgstr "" + +#. type: Plain text +#: en/tsum.text:191 +msgid "" +"If Tor still doesn't work, it's likely that your Internet Service Provider " +"(ISP) is blocking Tor. Very often this can be worked around with **Tor " +"bridges**, hidden relays that aren't as easy to block." +msgstr "" + +#. type: Plain text +#: en/tsum.text:195 +msgid "" +"If you need help with figuring out why Tor can't connect, send an email to " +"help(a)rt.torproject.org and include the relevant parts from the log file." +msgstr "" + +#. type: Plain text +#: en/tsum.text:203 +msgid "" +"### How to find a bridge\n" +"To use a bridge, you will first have to locate one; you can either browse to [bridges.torproject.org](https://bridges.torproject.org/), or you can send an email to bridges(a)torproject.org. If you do send an email, please make sure that you write **get bridges** in the body of the email. Without this, you will not get a reply. Note that you need to send this email from either a gmail.com or a yahoo.com address." +msgstr "" + +#. type: Plain text +#: en/tsum.text:208 +msgid "" +"Configuring more than one bridge address will make your Tor connection more " +"stable, in case some of the bridges become unreachable. There is no " +"guarantee that the bridge you are using now will work tomorrow, so you " +"should make a habit of updating your list of bridges every so often." +msgstr "" + +#. type: Plain text +#: en/tsum.text:214 +msgid "" +"### How to use a bridge\n" +"Once you have a set of bridges to use, open the Vidalia control panel, click on *Settings*, *Network* and tick the box that says *My ISP blocks connections to the Tor network*. Enter the bridges in the box below, hit *OK* and start Tor again." +msgstr "" + +#. type: Plain text +#: en/tsum.text:220 +msgid "" +"### How to use an open proxy\n" +"If using a bridge does not work, try configuring Tor to use any HTTPS or SOCKS proxy to get access to the Tor network. This means even if Tor is blocked by your local network, open proxies can be safely used to connect to the Tor Network and on to the uncensored Internet." +msgstr "" + +#. type: Plain text +#: en/tsum.text:223 +msgid "" +"The steps below assume you have a functional Tor/Vidalia configuration, and " +"you have found a list of HTTPS, SOCKS4, or SOCKS5 proxies." +msgstr "" + +#. type: Bullet: '1. ' +#: en/tsum.text:235 +msgid "Open the Vidalia control panel, click on *Settings*." +msgstr "" + +#. type: Bullet: '2. ' +#: en/tsum.text:235 +msgid "Click *Network*. Select *I use a proxy to access the Internet*." +msgstr "" + +#. type: Plain text +#: en/tsum.text:235 +#, no-wrap +msgid "" +"3. On the *Address* line, enter the open proxy address. This can be a\n" +"hostname or an IP Address.\n" +"4. Enter the port for the proxy.\n" +"5. Generally, you do not need a username and password. If you do,\n" +"enter the information in the proper fields.\n" +"6. Choose the *Type* of proxy you are using, whether HTTP/HTTPS, SOCKS4,\n" +"or SOCKS5.\n" +"7. Push the *OK* button. Vidalia and Tor are now configured to use a\n" +"proxy to access the rest of the Tor network.\n" +msgstr "" + +#. type: Plain text +#: en/tsum.text:240 +msgid "" +"## Frequently Asked Questions\n" +"This section will answer some of the most common questions. If your question is not mentioned here, please send an email to help(a)rt.torproject.org." +msgstr "" + +#. type: Plain text +#: en/tsum.text:244 +msgid "" +"### Unable to extract the archive\n" +"If you are using Windows and find that you cannot extract the archive, download and install [7-Zip](http://www.7-zip.org/)." +msgstr "" + +#. type: Plain text +#: en/tsum.text:248 +msgid "" +"If you are unable to download 7-Zip, try to rename the file from .z to .zip " +"and use winzip to extract the archive. Before renaming the file, tell " +"Windows to show file extensions:" +msgstr "" + +#. type: Plain text +#: en/tsum.text:254 +msgid "" +"#### Windows XP\n" +"1. Open *My Computer*\n" +"2. Click on *Tools* and choose *Folder Options...* in the menu\n" +"3. Click on the *View* tab\n" +"4. Uncheck *Hide extensions for known file types* and click *OK*" +msgstr "" + +#. type: Plain text +#: en/tsum.text:260 +msgid "" +"#### Windows Vista\n" +"1. Open *Computer*\n" +"2. Click on *Organize* and choose *Folder and search options* in the menu\n" +"3. Click on the *View* tab\n" +"4. Uncheck *Hide extensions for known file types* and click *OK*" +msgstr "" + +#. type: Plain text +#: en/tsum.text:266 +msgid "" +"#### Windows 7\n" +"1. Open *Computer*\n" +"2. Click on *Organize* and choose *Folder and search options* in the menu\n" +"3. Click on the *View* tab\n" +" 4. Uncheck *Hide extensions for known file types* and click *OK*" +msgstr "" + +#. type: Plain text +#: en/tsum.text:273 +msgid "" +"### What to do with split packages\n" +"When you request a split package, the parts may arrive out of order. You need to make sure you have received all the parts before you attempt to unpack them. Save all the parts into one directory on your computer, unzip them, and double-click the file that ends in \"..split.part01.exe\" to start the unpacking process." +msgstr "" + +#. type: Plain text +#: en/tsum.text:277 +msgid "" +"Once the unpacking process has completed, you should see a newly created " +"\".exe\" file in the same directory. Run this file and wait for the Tor " +"Browser Bundle to start." +msgstr "" + +#. type: Plain text +#: en/tsum.text:281 +msgid "" +"### Vidalia asks for a password\n" +"You should not have to enter a password when starting Vidalia. If you are prompted for one, you are likely affected by one of these problems:" +msgstr "" + +#. type: Plain text +#: en/tsum.text:287 +#, no-wrap +msgid "" +"**You are already running Vidalia and Tor**:\n" +"For example, this situation can happen if you installed the Vidalia\n" +"bundle and now you're trying to run the Tor Browser Bundle. In that\n" +"case, you will need to close the old Vidalia and Tor before you can run\n" +"this one.\n" +msgstr "" + +#. type: Plain text +#: en/tsum.text:294 +#, no-wrap +msgid "" +"**Vidalia crashed, but left Tor running**:\n" +"If the dialog that prompts you for a control password has a Reset\n" +"button, you can click the button and Vidalia will restart Tor with a new\n" +"random control password. If you do not see a Reset button, or if Vidalia\n" +"is unable to restart Tor for you; go into your process or task manager,\n" +"and terminate the Tor process. Then use Vidalia to restart Tor.\n" +msgstr "" + +#. type: Plain text +#: en/tsum.text:298 +msgid "" +"For more information, see the " +"[FAQ](https://torproject.org/docs/faq.html#VidaliaPassword) on the Tor " +"Project website." +msgstr "" + +#. type: Plain text +#: en/tsum.text:303 +msgid "" +"### Flash does not work\n" +"For security reasons, Flash, Java, and other plugins are currently disabled for Tor. Plugins operate independently from Firefox and can perform activity on your computer that ruins your anonymity." +msgstr "" + +#. type: Plain text +#: en/tsum.text:308 +msgid "" +"Most YouTube videos work with HTML5, and it is possible to view these videos" +" over Tor. You need to join the [HTML5 trial](https://www.youtube.com/html5)" +" on the YouTube website before you can use the HTML5 player." +msgstr "" + +#. type: Plain text +#: en/tsum.text:312 +msgid "" +"Note that the browser will not remember that you joined the trial once you " +"close it, so you will need to re-join the trial the next time you run the " +"Tor Browser Bundle." +msgstr "" + +#. type: Plain text +#: en/tsum.text:315 +msgid "" +"Please see the [Torbutton FAQ](https://www.torproject.org/torbutton" +"/torbutton-faq.html#noflash) for more information." +msgstr "" + +#. type: Plain text +#: en/tsum.text:321 +msgid "" +"### I want to use another browser\n" +"For security reasons, we recommend that you only browse the web through Tor using the Tor Browser Bundle. It is technically possible to use Tor with other browsers, but by doing so you open yourself up to potential attacks." +msgstr "" + +#. type: Plain text +#: en/tsum.text:325 +msgid "" +"### Why Tor is slow\n" +"Tor can sometimes be a bit slower than your normal Internet connection. After all, your traffic is sent through many different countries, sometimes across oceans around the world!" +msgstr "" + +

1 0

[doctor/master] Remove an error message that is too loud.
by karsten＠torproject.org 06 Dec '11

06 Dec '11

commit b9eb17d6635d8289345ef60e714587eb587fa7fd Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Tue Dec 6 13:45:24 2011 +0100 Remove an error message that is too loud. It's quite possible that we fail to download a consensus from a directory authority. We should expect that. We'll also include a warning in the later reports. But this is not an error case of the program itself. --- src/org/torproject/doctor/Downloader.java | 3 --- 1 files changed, 0 insertions(+), 3 deletions(-) diff --git a/src/org/torproject/doctor/Downloader.java b/src/org/torproject/doctor/Downloader.java index 1e3b05e..3125062 100755 --- a/src/org/torproject/doctor/Downloader.java +++ b/src/org/torproject/doctor/Downloader.java @@ -56,9 +56,6 @@ public class Downloader { if (responses.containsKey(url)) { String response = responses.get(url); this.downloadedConsensuses.put(nickname, response); - } else { - System.err.println("Could not download consensus from directory " - + "authority " + nickname + ". Ignoring."); } } if (responses.isEmpty()) {

1 0

[doctor/master] Consider all bwauth* params valid on website output.
by karsten＠torproject.org 06 Dec '11

06 Dec '11

commit 078a795113b9b5525505a193b4c68c6e9404b316 Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Tue Dec 6 07:29:45 2011 +0100 Consider all bwauth* params valid on website output. --- .../torproject/doctor/MetricsWebsiteReport.java | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) diff --git a/src/org/torproject/doctor/MetricsWebsiteReport.java b/src/org/torproject/doctor/MetricsWebsiteReport.java index b7f9888..b03f652 100755 --- a/src/org/torproject/doctor/MetricsWebsiteReport.java +++ b/src/org/torproject/doctor/MetricsWebsiteReport.java @@ -403,7 +403,7 @@ public class MetricsWebsiteReport implements Report { ("circwindow,CircuitPriorityHalflifeMsec,refuseunknownexits," + "cbtdisabled,cbtnummodes,cbtrecentcount,cbtmaxtimeouts," + "cbtmincircs,cbtquantile,cbtclosequantile,cbttestfreq," - + "cbtmintimeout,cbtinitialtimeout,bwauthpid").split(","))); + + "cbtmintimeout,cbtinitialtimeout").split(","))); Map<String, String> consensusConsensusParams = downloadedConsensus.getConsensusParams(); for (Status vote : this.downloadedVotes) { @@ -416,7 +416,8 @@ public class MetricsWebsiteReport implements Report { if (!consensusConsensusParams.containsKey(e.getKey()) || !consensusConsensusParams.get(e.getKey()).equals( e.getValue()) || - !validParameters.contains(e.getKey())) { + (!validParameters.contains(e.getKey()) && + !e.getKey().startsWith("bwauth"))) { conflictOrInvalid = true; break; }

1 0

[metrics-web/master] Remove consensus-health checker.
by karsten＠torproject.org 06 Dec '11

06 Dec '11

commit 8af25f86effec8bb9b7a577310cc569adb2879cc Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Mon Dec 5 20:23:44 2011 +0100 Remove consensus-health checker. --- build.xml | 11 - src/org/torproject/chc/Checker.java | 285 ------- src/org/torproject/chc/Downloader.java | 251 ------- src/org/torproject/chc/Main.java | 46 -- src/org/torproject/chc/MetricsWebsiteReport.java | 875 ---------------------- src/org/torproject/chc/Parser.java | 167 ---- src/org/torproject/chc/Report.java | 23 - src/org/torproject/chc/Status.java | 176 ----- src/org/torproject/chc/StatusEntry.java | 48 -- src/org/torproject/chc/StatusFileReport.java | 197 ----- src/org/torproject/chc/Warning.java | 46 -- 11 files changed, 0 insertions(+), 2125 deletions(-) diff --git a/build.xml b/build.xml index 570c973..e148a13 100644 --- a/build.xml +++ b/build.xml @@ -21,7 +21,6 @@ <copy file="${contextxmltemplate}" tofile="${contextxml}"/> <copy file="config.template" tofile="config"/> <mkdir dir="${classes}"/> - <mkdir dir="website"/> </target>  @@ -60,16 +59,6 @@ </java> </target> -  - <target name="chc" depends="compile"> - <java fork="true" - maxmemory="512m" - classname="org.torproject.chc.Main" - error="chc-stderr.txt"> - <classpath refid="classpath"/> - </java> - </target> -  <target name="run" depends="compile"> <java fork="true" diff --git a/src/org/torproject/chc/Checker.java b/src/org/torproject/chc/Checker.java deleted file mode 100644 index 02c70b7..0000000 --- a/src/org/torproject/chc/Checker.java +++ /dev/null @@ -1,285 +0,0 @@ -/* Copyright 2011 The Tor Project - * See LICENSE for licensing information */ -package org.torproject.chc; - -import java.io.*; -import java.text.*; -import java.util.*; - -/* Check a given consensus and votes for irregularities and write results - * to a warnings map consisting of warning type and details. */ -public class Checker { - - /* Warning messages consisting of type and details. */ - private SortedMap<Warning, String> warnings = - new TreeMap<Warning, String>(); - - public SortedMap<Warning, String> getWarnings() { - return this.warnings; - } - - /* Date-time format to format timestamps. */ - private static SimpleDateFormat dateTimeFormat; - static { - dateTimeFormat = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss"); - dateTimeFormat.setTimeZone(TimeZone.getTimeZone("UTC")); - } - - /* Downloaded consensus and corresponding votes for processing. */ - private SortedMap<String, Status> downloadedConsensuses; - private Status downloadedConsensus; - private SortedSet<Status> downloadedVotes; - public void processDownloadedConsensuses( - SortedMap<String, Status> downloadedConsensuses) { - this.downloadedConsensuses = downloadedConsensuses; - this.findMostRecentConsensus(); - this.checkMissingConsensuses(); - this.checkAllConsensusesFresh(); - if (this.downloadedConsensus != null) { - if (this.isConsensusFresh(this.downloadedConsensus)) { - this.checkConsensusMethods(); - this.checkRecommendedVersions(); - this.checkConsensusParameters(); - this.checkAuthorityKeys(); - this.checkMissingVotes(); - this.checkBandwidthScanners(); - } - } else { - this.warnings.put(Warning.NoConsensusKnown, ""); - } - } - - /* Find most recent consensus and corresponding votes. */ - private void findMostRecentConsensus() { - long mostRecentValidAfterMillis = -1L; - for (Status downloadedConsensus : downloadedConsensuses.values()) { - if (downloadedConsensus.getValidAfterMillis() > - mostRecentValidAfterMillis) { - this.downloadedConsensus = downloadedConsensus; - mostRecentValidAfterMillis = - downloadedConsensus.getValidAfterMillis(); - } - } - if (this.downloadedConsensus != null) { - this.downloadedVotes = this.downloadedConsensus.getVotes(); - } - } - - /* Check if any directory authority didn't tell us a consensus. */ - private void checkMissingConsensuses() { - SortedSet<String> missingConsensuses = new TreeSet<String>( - Arrays.asList(("gabelmoo,tor26,ides,maatuska,dannenberg,urras," - + "moria1,dizum").split(","))); - missingConsensuses.removeAll(this.downloadedConsensuses.keySet()); - if (!missingConsensuses.isEmpty()) { - StringBuilder sb = new StringBuilder(); - for (String nickname : missingConsensuses) { - sb.append(", " + nickname); - } - this.warnings.put(Warning.ConsensusDownloadTimeout, - sb.toString().substring(2)); - } - } - - /* Check if all consensuses are fresh. */ - private void checkAllConsensusesFresh() { - long fresh = System.currentTimeMillis() - 60L * 60L * 1000L; - SortedSet<String> nonFresh = new TreeSet<String>(); - for (Map.Entry<String, Status> e : downloadedConsensuses.entrySet()) { - String nickname = e.getKey(); - Status downloadedConsensus = e.getValue(); - if (downloadedConsensus.getValidAfterMillis() < fresh) { - nonFresh.add(nickname); - } - } - if (!nonFresh.isEmpty()) { - StringBuilder sb = new StringBuilder(); - for (String nickname : nonFresh) { - sb.append(", " + nickname); - } - this.warnings.put(Warning.ConsensusNotFresh, - sb.toString().substring(2)); - } - } - - /* Check if the most recent consensus is older than 1 hour. */ - private boolean isConsensusFresh(Status consensus) { - return (consensus.getValidAfterMillis() >= - System.currentTimeMillis() - 60L * 60L * 1000L); - } - - /* Check supported consensus methods of all votes. */ - private void checkConsensusMethods() { - SortedSet<String> dirs = new TreeSet<String>(); - for (Status vote : this.downloadedVotes) { - if (!vote.getConsensusMethods().contains( - this.downloadedConsensus.getConsensusMethods().last())) { - dirs.add(vote.getNickname()); - } - } - if (!dirs.isEmpty()) { - StringBuilder sb = new StringBuilder(); - for (String dir : dirs) { - sb.append(", " + dir); - } - this.warnings.put(Warning.ConsensusMethodNotSupported, - sb.toString().substring(2)); - } - } - - /* Check if the recommended versions in a vote are different from the - * recommended versions in the consensus. */ - private void checkRecommendedVersions() { - SortedSet<String> unrecommendedClientVersions = new TreeSet<String>(), - unrecommendedServerVersions = new TreeSet<String>(); - for (Status vote : this.downloadedVotes) { - if (vote.getRecommendedClientVersions() != null && - !downloadedConsensus.getRecommendedClientVersions().equals( - vote.getRecommendedClientVersions())) { - StringBuilder message = new StringBuilder(); - message.append(vote.getNickname()); - for (String version : vote.getRecommendedClientVersions()) { - message.append(" " + version); - } - unrecommendedClientVersions.add(message.toString()); - } - if (vote.getRecommendedServerVersions() != null && - !downloadedConsensus.getRecommendedServerVersions().equals( - vote.getRecommendedServerVersions())) { - StringBuilder message = new StringBuilder(); - message.append(vote.getNickname()); - for (String version : vote.getRecommendedServerVersions()) { - message.append(" " + version); - } - unrecommendedServerVersions.add(message.toString()); - } - } - if (!unrecommendedServerVersions.isEmpty()) { - StringBuilder sb = new StringBuilder(); - for (String dir : unrecommendedServerVersions) { - sb.append(", " + dir); - } - this.warnings.put(Warning.DifferentRecommendedServerVersions, - sb.toString().substring(2)); - } - if (!unrecommendedClientVersions.isEmpty()) { - StringBuilder sb = new StringBuilder(); - for (String dir : unrecommendedClientVersions) { - sb.append(", " + dir); - } - this.warnings.put(Warning.DifferentRecommendedClientVersions, - sb.toString().substring(2)); - } - } - - /* Check if a vote contains conflicting or invalid consensus - * parameters. */ - private void checkConsensusParameters() { - Set<String> validParameters = new HashSet<String>(Arrays.asList( - ("circwindow,CircuitPriorityHalflifeMsec,refuseunknownexits," - + "cbtdisabled,cbtnummodes,cbtrecentcount,cbtmaxtimeouts," - + "cbtmincircs,cbtquantile,cbtclosequantile,cbttestfreq," - + "cbtmintimeout,cbtinitialtimeout,perconnbwburst,perconnbwrate"). - split(","))); - SortedSet<String> conflicts = new TreeSet<String>(); - for (Status vote : this.downloadedVotes) { - Map<String, String> voteConsensusParams = - vote.getConsensusParams(); - boolean conflictOrInvalid = false; - if (voteConsensusParams != null) { - for (Map.Entry<String, String> e : - voteConsensusParams.entrySet()) { - if (!downloadedConsensus.getConsensusParams().containsKey( - e.getKey()) || - !downloadedConsensus.getConsensusParams().get(e.getKey()). - equals(e.getValue()) || - (!validParameters.contains(e.getKey()) && - !e.getKey().startsWith("bwauth"))) { - StringBuilder message = new StringBuilder(); - message.append(vote.getNickname()); - for (Map.Entry<String, String> p : - voteConsensusParams.entrySet()) { - message.append(" " + p.getKey() + "=" + p.getValue()); - } - conflicts.add(message.toString()); - break; - } - } - } - } - if (!conflicts.isEmpty()) { - StringBuilder sb = new StringBuilder(); - for (String dir : conflicts) { - sb.append(", " + dir); - } - this.warnings.put(Warning.ConflictingOrInvalidConsensusParams, - sb.toString().substring(2)); - } - } - - /* Check whether any of the authority keys expire in the next 14 - * days. */ - private void checkAuthorityKeys() { - SortedMap<String, String> expiringCertificates = - new TreeMap<String, String>(); - long now = System.currentTimeMillis(); - for (Status vote : this.downloadedVotes) { - long voteDirKeyExpiresMillis = vote.getDirKeyExpiresMillis(); - if (voteDirKeyExpiresMillis - 14L * 24L * 60L * 60L * 1000L < now) { - expiringCertificates.put(vote.getNickname(), - dateTimeFormat.format(voteDirKeyExpiresMillis)); - } - } - if (!expiringCertificates.isEmpty()) { - StringBuilder sb = new StringBuilder(); - for (Map.Entry<String, String> e : - expiringCertificates.entrySet()) { - String dir = e.getKey(); - String timestamp = e.getValue(); - sb.append(", " + dir + " " + timestamp); - } - this.warnings.put(Warning.CertificateExpiresSoon, - sb.toString().substring(2)); - } - } - - /* Check if any votes are missing. */ - private void checkMissingVotes() { - SortedSet<String> knownAuthorities = new TreeSet<String>( - Arrays.asList(("dannenberg,dizum,gabelmoo,ides,maatuska,moria1," - + "tor26,urras").split(","))); - SortedSet<String> missingVotes = - new TreeSet<String>(knownAuthorities); - for (Status vote : this.downloadedVotes) { - missingVotes.remove(vote.getNickname()); - } - if (!missingVotes.isEmpty()) { - StringBuilder sb = new StringBuilder(); - for (String missingDir : missingVotes) { - sb.append(", " + missingDir); - } - this.warnings.put(Warning.VotesMissing, - sb.toString().substring(2)); - } - } - - /* Check if any bandwidth scanner results are missing. */ - private void checkBandwidthScanners() { - SortedSet<String> missingBandwidthScanners = new TreeSet<String>( - Arrays.asList("ides,urras,moria1,gabelmoo,maatuska".split(","))); - for (Status vote : this.downloadedVotes) { - if (vote.getBandwidthWeights() > 0) { - missingBandwidthScanners.remove(vote.getNickname()); - } - } - if (!missingBandwidthScanners.isEmpty()) { - StringBuilder sb = new StringBuilder(); - for (String dir : missingBandwidthScanners) { - sb.append(", " + dir); - } - this.warnings.put(Warning.BandwidthScannerResultsMissing, - sb.toString().substring(2)); - } - } -} - diff --git a/src/org/torproject/chc/Downloader.java b/src/org/torproject/chc/Downloader.java deleted file mode 100644 index e56d20d..0000000 --- a/src/org/torproject/chc/Downloader.java +++ /dev/null @@ -1,251 +0,0 @@ -/* Copyright 2011 The Tor Project - * See LICENSE for licensing information */ -package org.torproject.chc; - -import java.io.*; -import java.net.*; -import java.text.*; -import java.util.*; -import java.util.zip.*; - -/* Download the latest network status consensus and corresponding - * votes. */ -public class Downloader { - - /* List of directory authorities to download consensuses and votes - * from. */ - private SortedMap<String, String> authorities = - new TreeMap<String, String>(); - public Downloader() { - this.authorities.put("gabelmoo", "212.112.245.170"); - this.authorities.put("tor26", "86.59.21.38"); - this.authorities.put("ides", "216.224.124.114:9030"); - this.authorities.put("maatuska", "213.115.239.118:443"); - this.authorities.put("dannenberg", "193.23.244.244"); - this.authorities.put("urras", "208.83.223.34:443"); - this.authorities.put("moria1", "128.31.0.34:9131"); - this.authorities.put("dizum", "194.109.206.212"); - } - - /* Download a new consensus and corresponding votes. */ - public void downloadFromAuthorities() { - this.downloadConsensus(); - if (!this.downloadedConsensuses.isEmpty()) { - this.parseConsensusToFindReferencedVotes(); - this.downloadReferencedVotes(); - } - } - - /* Download the most recent consensus from all authorities. */ - private SortedMap<String, String> downloadedConsensuses = - new TreeMap<String, String>(); - private void downloadConsensus() { - Map<String, String> urls = new HashMap<String, String>(); - for (Map.Entry<String, String> e : this.authorities.entrySet()) { - String nickname = e.getKey(); - String address = e.getValue(); - String resource = "/tor/status-vote/current/consensus.z"; - String fullUrl = "http://" + address + resource; - urls.put(nickname, fullUrl); - } - Map<String, String> responses = - this.downloadFromAuthority(new HashSet<String>(urls.values())); - for (Map.Entry<String, String> e : urls.entrySet()) { - String nickname = e.getKey(); - String url = e.getValue(); - if (responses.containsKey(url)) { - String response = responses.get(url); - this.downloadedConsensuses.put(nickname, response); - } else { - System.err.println("Could not download consensus from directory " - + "authority " + nickname + ". Ignoring."); - } - } - if (responses.isEmpty()) { - System.err.println("Could not download consensus from any of the " - + "directory authorities. Ignoring."); - } - } - - /* Downloads a consensus or vote in a separate thread that can be - * interrupted after a timeout. */ - private static class DownloadRunnable implements Runnable { - Thread mainThread; - String url; - String response; - boolean finished = false; - public DownloadRunnable(String url) { - this.mainThread = Thread.currentThread(); - this.url = url; - } - public void run() { - try { - URL u = new URL(this.url); - HttpURLConnection huc = (HttpURLConnection) u.openConnection(); - huc.setRequestMethod("GET"); - huc.connect(); - int responseCode = huc.getResponseCode(); - if (responseCode == 200) { - BufferedInputStream in = new BufferedInputStream( - new InflaterInputStream(huc.getInputStream())); - ByteArrayOutputStream baos = new ByteArrayOutputStream(); - int len; - byte[] data = new byte[1024]; - while (!this.finished && - (len = in.read(data, 0, 1024)) >= 0) { - baos.write(data, 0, len); - } - if (this.finished) { - return; - } - in.close(); - byte[] allData = baos.toByteArray(); - this.response = new String(allData); - this.finished = true; - this.mainThread.interrupt(); - } - } catch (IOException e) { - /* Can't do much except leaving this.response at null. */ - } - this.finished = true; - } - } - - /* Download one or more consensuses or votes from one or more directory - * authorities using a timeout of 60 seconds. */ - private Map<String, String> downloadFromAuthority(Set<String> urls) { - Set<DownloadRunnable> downloadRunnables = - new HashSet<DownloadRunnable>(); - for (String url : urls) { - DownloadRunnable downloadRunnable = new DownloadRunnable(url); - downloadRunnables.add(downloadRunnable); - new Thread(downloadRunnable).start(); - } - long started = System.currentTimeMillis(), sleep; - while ((sleep = started + 60L * 1000L - System.currentTimeMillis()) - > 0L) { - try { - Thread.sleep(sleep); - } catch (InterruptedException e) { - /* Do nothing. */ - } - boolean unfinished = false; - for (DownloadRunnable downloadRunnable : downloadRunnables) { - if (!downloadRunnable.finished) { - unfinished = true; - break; - } - } - if (!unfinished) { - break; - } - } - Map<String, String> responses = new HashMap<String, String>(); - for (DownloadRunnable downloadRunnable : downloadRunnables) { - String url = downloadRunnable.url; - String response = downloadRunnable.response; - if (response != null) { - responses.put(url, response); - } - downloadRunnable.finished = true; - } - return responses; - } - - /* Date-time formats to parse and format timestamps. */ - private static SimpleDateFormat dateTimeFormat; - static { - dateTimeFormat = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss"); - dateTimeFormat.setTimeZone(TimeZone.getTimeZone("UTC")); - } - - /* Parse the downloaded consensus to find fingerprints of directory - * authorities publishing the corresponding votes. */ - private SortedSet<String> fingerprints = new TreeSet<String>(); - private void parseConsensusToFindReferencedVotes() { - for (String downloadedConsensus : - this.downloadedConsensuses.values()) { - try { - BufferedReader br = new BufferedReader(new StringReader( - downloadedConsensus)); - String line; - while ((line = br.readLine()) != null) { - if (line.startsWith("valid-after ")) { - try { - long validAfterMillis = dateTimeFormat.parse(line.substring( - "valid-after ".length())).getTime(); - if (validAfterMillis + 60L * 60L * 1000L < - System.currentTimeMillis()) { - /* Consensus is more than 1 hour old. We won't be able to - * download the corresponding votes anymore. */ - break; - } - } catch (ParseException e) { - System.err.println("Could not parse valid-after timestamp " - + "in line '" + line + "' of a downloaded consensus. " - + "Not downloading votes."); - break; - } - } else if (line.startsWith("dir-source ")) { - String[] parts = line.split(" "); - if (parts.length < 3) { - System.err.println("Bad dir-source line '" + line - + "' in downloaded consensus. Skipping."); - continue; - } - String nickname = parts[1]; - if (nickname.endsWith("-legacy")) { - continue; - } - String fingerprint = parts[2]; - this.fingerprints.add(fingerprint); - } - } - br.close(); - } catch (IOException e) { - System.err.println("Could not parse consensus to find referenced " - + "votes in it. Skipping."); - } - } - } - - /* Download the votes published by directory authorities listed in the - * consensus. */ - private List<String> downloadedVotes = new ArrayList<String>(); - private void downloadReferencedVotes() { - for (String fingerprint : this.fingerprints) { - String downloadedVote = null; - List<String> authorities = new ArrayList<String>( - this.authorities.values()); - Collections.shuffle(authorities); - for (String authority : authorities) { - if (downloadedVote != null) { - break; - } - String resource = "/tor/status-vote/current/" + fingerprint - + ".z"; - String fullUrl = "http://" + authority + resource; - Set<String> urls = new HashSet<String>(); - urls.add(fullUrl); - Map<String, String> downloadedVotes = - this.downloadFromAuthority(urls); - if (downloadedVotes.containsKey(fullUrl)) { - downloadedVote = downloadedVotes.get(fullUrl); - this.downloadedVotes.add(downloadedVote); - } - } - } - } - - /* Return the previously downloaded (unparsed) consensus string by - * authority nickname. */ - public SortedMap<String, String> getConsensusStrings() { - return this.downloadedConsensuses; - } - - /* Return the previously downloaded (unparsed) vote strings. */ - public List<String> getVoteStrings() { - return this.downloadedVotes; - } -} - diff --git a/src/org/torproject/chc/Main.java b/src/org/torproject/chc/Main.java deleted file mode 100644 index a0a3c19..0000000 --- a/src/org/torproject/chc/Main.java +++ /dev/null @@ -1,46 +0,0 @@ -/* Copyright 2011 The Tor Project - * See LICENSE for licensing information */ -package org.torproject.chc; - -import java.util.*; - -/* Coordinate the process of downloading consensus and votes to check - * Tor's consensus health. */ -public class Main { - public static void main(String[] args) { - - /* Initialize reports. */ - List<Report> reports = new ArrayList<Report>(); - reports.add(new MetricsWebsiteReport( - "website/consensus-health.html")); - reports.add(new StatusFileReport()); - - /* Download consensus and corresponding votes from the directory - * authorities. */ - Downloader downloader = new Downloader(); - downloader.downloadFromAuthorities(); - - /* Parse consensus and votes. */ - Parser parser = new Parser(); - SortedMap<String, Status> parsedDownloadedConsensuses = parser.parse( - downloader.getConsensusStrings(), downloader.getVoteStrings()); - - /* Check consensus and votes for possible problems. */ - Checker checker = new Checker(); - checker.processDownloadedConsensuses(parsedDownloadedConsensuses); - SortedMap<Warning, String> warnings = checker.getWarnings(); - - /* Pass warnings, consensuses, and votes to the reports, and finish - * writing them. */ - for (Report report : reports) { - report.processWarnings(warnings); - report.processDownloadedConsensuses(parsedDownloadedConsensuses); - report.writeReport(); - } - - /* Terminate the program including any download threads that may still - * be running. */ - System.exit(0); - } -} - diff --git a/src/org/torproject/chc/MetricsWebsiteReport.java b/src/org/torproject/chc/MetricsWebsiteReport.java deleted file mode 100644 index 8aa52b5..0000000 --- a/src/org/torproject/chc/MetricsWebsiteReport.java +++ /dev/null @@ -1,875 +0,0 @@ -/* Copyright 2011 The Tor Project - * See LICENSE for licensing information */ -package org.torproject.chc; - -import java.io.*; -import java.text.*; -import java.util.*; - -/* Transform the most recent consensus and corresponding votes into an - * HTML page showing possible irregularities. */ -public class MetricsWebsiteReport implements Report { - - /* Date-time format to format timestamps. */ - private static SimpleDateFormat dateTimeFormat; - static { - dateTimeFormat = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss"); - dateTimeFormat.setTimeZone(TimeZone.getTimeZone("UTC")); - } - - /* Output file to write report to. */ - private File htmlOutputFile; - - /* Initialize this report. */ - public MetricsWebsiteReport(String htmlOutputFilename) { - this.htmlOutputFile = new File(htmlOutputFilename); - } - - /* Process warnings. */ - public void processWarnings(SortedMap<Warning, String> warnings) { - /* We could use these warnings instead of running all checks - * ourselves. But we're not doing that yet. */ - } - - /* Store the downloaded consensus and corresponding votes for later - * processing. */ - private Status downloadedConsensus; - private SortedSet<Status> downloadedVotes; - public void processDownloadedConsensuses( - SortedMap<String, Status> downloadedConsensuses) { - long mostRecentValidAfterMillis = -1L; - for (Status downloadedConsensus : downloadedConsensuses.values()) { - if (downloadedConsensus.getValidAfterMillis() > - mostRecentValidAfterMillis) { - this.downloadedConsensus = downloadedConsensus; - mostRecentValidAfterMillis = - downloadedConsensus.getValidAfterMillis(); - } - } - if (this.downloadedConsensus != null) { - this.downloadedVotes = this.downloadedConsensus.getVotes(); - } - } - - /* Writer to write all HTML output to. */ - private BufferedWriter bw; - - /* Write HTML output file for the metrics website. */ - public void writeReport() { - - if (this.downloadedConsensus != null) { - try { - this.htmlOutputFile.getParentFile().mkdirs(); - this.bw = new BufferedWriter(new FileWriter(this.htmlOutputFile)); - writePageHeader(); - writeValidAfterTime(); - writeKnownFlags(); - writeNumberOfRelaysVotedAbout(); - writeConsensusMethods(); - writeRecommendedVersions(); - writeConsensusParameters(); - writeAuthorityKeys(); - writeBandwidthScannerStatus(); - writeAuthorityVersions(); - writeRelayFlagsTable(); - writeRelayFlagsSummary(); - writePageFooter(); - this.bw.close(); - } catch (IOException e) { - System.err.println("Could not write HTML output file '" - + this.htmlOutputFile.getAbsolutePath() + "'. Ignoring."); - } - } - } - - /* Write the HTML page header including the metrics website - * navigation. */ - private void writePageHeader() throws IOException { - this.bw.write("<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.0 " - + "Transitional//EN\">\n" - + "<html>\n" - + " <head>\n" - + " <title>Tor Metrics Portal: Consensus health</title>\n" - + " <meta http-equiv=\"content-type\" content=\"text/html; " - + "charset=ISO-8859-1\">\n" - + " <link href=\"/css/stylesheet-ltr.css\" type=\"text/css\" " - + "rel=\"stylesheet\">\n" - + " <link href=\"/images/favicon.ico\" " - + "type=\"image/x-icon\" rel=\"shortcut icon\">\n" - + " </head>\n" - + " <body>\n" - + " <div class=\"center\">\n" - + " <table class=\"banner\" border=\"0\" cellpadding=\"0\" " - + "cellspacing=\"0\" summary=\"\">\n" - + " <tr>\n" - + " <td class=\"banner-left\"><a " - + "href=\"/index.html\"><img src=\"/images/top-left.png\" " - + "alt=\"Click to go to home page\" width=\"193\" " - + "height=\"79\"></a></td>\n" - + " <td class=\"banner-middle\">\n" - + " <a href=\"/\">Home</a>\n" - + " <a href=\"graphs.html\">Graphs</a>\n" - + " <a href=\"research.html\">Research</a>\n" - + " <a href=\"status.html\">Status</a>\n" - + " <br>\n" - + " <font size=\"2\">\n" - + " <a href=\"networkstatus.html\">Network " - + "Status</a>\n" - + " <a href=\"exonerator.html\">ExoneraTor</a>\n" - + " <a href=\"relay-search.html\">Relay Search</a>\n" - + " <a class=\"current\">Consensus Health</a>\n" - + " </font>\n" - + " </td>\n" - + " <td class=\"banner-right\"></td>\n" - + " </tr>\n" - + " </table>\n" - + " <div class=\"main-column\">\n" - + " <h2>Tor Metrics Portal: Consensus Health</h2>\n" - + " <br>\n" - + " <p>This page shows statistics about the current " - + "consensus and votes to facilitate debugging of the " - + "directory consensus process.</p>\n"); - } - - /* Write the valid-after time of the downloaded consensus. */ - private void writeValidAfterTime() throws IOException { - this.bw.write(" <br>\n" - + " <a name=\"validafter\">\n" - + " <h3><a href=\"#validafter\" class=\"anchor\">" - + "Valid-after time</a></h3>\n" - + " <br>\n" - + " <p>Consensus was published "); - if (this.downloadedConsensus.getValidAfterMillis() < - System.currentTimeMillis() - 3L * 60L * 60L * 1000L) { - this.bw.write("<font color=\"red\">" - + dateTimeFormat.format( - this.downloadedConsensus.getValidAfterMillis()) + "</font>"); - } else { - this.bw.write(dateTimeFormat.format( - this.downloadedConsensus.getValidAfterMillis())); - } - this.bw.write(". <i>Note that it takes up to 15 to learn about new " - + "consensus and votes and process them.</i></p>\n"); - } - - /* Write the lists of known flags. */ - private void writeKnownFlags() throws IOException { - this.bw.write(" <br>\n" - + " <a name=\"knownflags\">\n" - + " <h3><a href=\"#knownflags\" class=\"anchor\">Known " - + "flags</a></h3>\n" - + " <br>\n" - + " <table border=\"0\" cellpadding=\"4\" " - + "cellspacing=\"0\" summary=\"\">\n" - + " <colgroup>\n" - + " <col width=\"160\">\n" - + " <col width=\"640\">\n" - + " </colgroup>\n"); - if (this.downloadedVotes.size() < 1) { - this.bw.write(" <tr><td>(No votes.)</td><td></td></tr>\n"); - } else { - for (Status vote : this.downloadedVotes) { - this.bw.write(" <tr>\n" - + " <td>" + vote.getNickname() + "</td>\n" - + " <td>known-flags"); - for (String knownFlag : vote.getKnownFlags()) { - this.bw.write(" " + knownFlag); - } - this.bw.write("</td>\n" - + " </tr>\n"); - } - } - this.bw.write(" <tr>\n" - + " <td><font color=\"blue\">consensus</font>" - + "</td>\n" - + " <td><font color=\"blue\">known-flags"); - for (String knownFlag : this.downloadedConsensus.getKnownFlags()) { - this.bw.write(" " + knownFlag); - } - this.bw.write("</font></td>\n" - + " </tr>\n" - + " </table>\n"); - } - - /* Write the number of relays voted about. */ - private void writeNumberOfRelaysVotedAbout() throws IOException { - this.bw.write(" <br>\n" - + " <a name=\"numberofrelays\">\n" - + " <h3><a href=\"#numberofrelays\" class=\"anchor\">" - + "Number of relays voted about</a></h3>\n" - + " <br>\n" - + " <table border=\"0\" cellpadding=\"4\" " - + "cellspacing=\"0\" summary=\"\">\n" - + " <colgroup>\n" - + " <col width=\"160\">\n" - + " <col width=\"320\">\n" - + " <col width=\"320\">\n" - + " </colgroup>\n"); - if (this.downloadedVotes.size() < 1) { - this.bw.write(" <tr><td>(No votes.)</td><td></td><td></td>" - + "</tr>\n"); - } else { - for (Status vote : this.downloadedVotes) { - this.bw.write(" <tr>\n" - + " <td>" + vote.getNickname() + "</td>\n" - + " <td>" + vote.getStatusEntries().size() - + " total</td>\n" - + " <td>" + vote.getRunningRelays() - + " Running</td>\n" - + " </tr>\n"); - } - } - this.bw.write(" <tr>\n" - + " <td><font color=\"blue\">consensus</font>" - + "</td>\n" - + " <td><font color=\"blue\">" - + this.downloadedConsensus.getStatusEntries().size() - + " total</font></td>\n" - + " <td><font color=\"blue\">" - + this.downloadedConsensus.getRunningRelays() - + " Running</font></td>\n" - + " </tr>\n" - + " </table>\n"); - } - - /* Write the supported consensus methods of directory authorities and - * the resulting consensus method. */ - private void writeConsensusMethods() throws IOException { - this.bw.write(" <br>\n" - + " <a name=\"consensusmethods\">\n" - + " <h3><a href=\"#consensusmethods\" class=\"anchor\">" - + "Consensus methods</a></h3>\n" - + " <br>\n" - + " <table border=\"0\" cellpadding=\"4\" " - + "cellspacing=\"0\" summary=\"\">\n" - + " <colgroup>\n" - + " <col width=\"160\">\n" - + " <col width=\"640\">\n" - + " </colgroup>\n"); - if (this.downloadedVotes.size() < 1) { - this.bw.write(" <tr><td>(No votes.)</td><td></td></tr>\n"); - } else { - for (Status vote : this.downloadedVotes) { - SortedSet<Integer> consensusMethods = - vote.getConsensusMethods(); - if (consensusMethods.contains( - this.downloadedConsensus.getConsensusMethods().last())) { - this.bw.write(" <tr>\n" - + " <td>" + vote.getNickname() + "</td>\n" - + " <td>consensus-methods"); - for (int consensusMethod : consensusMethods) { - this.bw.write(" " + String.valueOf(consensusMethod)); - } - this.bw.write("</td>\n" - + " </tr>\n"); - } else { - this.bw.write(" <tr>\n" - + " <td><font color=\"red\">" - + vote.getNickname() + "</font></td>\n" - + " <td><font color=\"red\">" - + "consensus-methods"); - for (int consensusMethod : consensusMethods) { - this.bw.write(" " + String.valueOf(consensusMethod)); - } - this.bw.write("</font></td>\n" - + " </tr>\n"); - } - } - } - this.bw.write(" <tr>\n" - + " <td><font color=\"blue\">consensus</font>" - + "</td>\n" - + " <td><font color=\"blue\">consensus-method " - + this.downloadedConsensus.getConsensusMethods().last() - + "</font></td>\n" - + " </tr>\n" - + " </table>\n"); - } - - /* Write recommended versions. */ - private void writeRecommendedVersions() throws IOException { - this.bw.write(" <br>\n" - + " <a name=\"recommendedversions\">\n" - + " <h3><a href=\"#recommendedversions\" class=\"anchor\">" - + "Recommended versions</a></h3>\n" - + " <br>\n" - + " <table border=\"0\" cellpadding=\"4\" " - + "cellspacing=\"0\" summary=\"\">\n" - + " <colgroup>\n" - + " <col width=\"160\">\n" - + " <col width=\"640\">\n" - + " </colgroup>\n"); - if (this.downloadedVotes.size() < 1) { - this.bw.write(" <tr><td>(No votes.)</td><td></td></tr>\n"); - } else { - for (Status vote : this.downloadedVotes) { - SortedSet<String> voteRecommendedClientVersions = - vote.getRecommendedClientVersions(); - if (voteRecommendedClientVersions != null) { - if (downloadedConsensus.getRecommendedClientVersions().equals( - voteRecommendedClientVersions)) { - this.bw.write(" <tr>\n" - + " <td>" + vote.getNickname() + "</td>\n" - + " <td>client-versions "); - int i = 0; - for (String version : voteRecommendedClientVersions) { - this.bw.write((i++ > 0 ? "," : "") + version); - } - this.bw.write("</td>\n" - + " </tr>\n"); - } else { - this.bw.write(" <tr>\n" - + " <td><font color=\"red\">" - + vote.getNickname() - + "</font></td>\n" - + " <td><font color=\"red\">client-versions "); - int i = 0; - for (String version : voteRecommendedClientVersions) { - this.bw.write((i++ > 0 ? "," : "") + version); - } - this.bw.write("</font></td>\n" - + " </tr>\n"); - } - } - SortedSet<String> voteRecommendedServerVersions = - vote.getRecommendedServerVersions(); - if (voteRecommendedServerVersions != null) { - if (downloadedConsensus.getRecommendedServerVersions().equals( - voteRecommendedServerVersions)) { - this.bw.write(" <tr>\n" - + " <td></td>\n" - + " <td>server-versions "); - int i = 0; - for (String version : voteRecommendedServerVersions) { - this.bw.write((i++ > 0 ? "," : "") + version); - } - this.bw.write("</td>\n" - + " </tr>\n"); - } else { - this.bw.write(" <tr>\n" - + " <td></td>\n" - + " <td><font color=\"red\">server-versions "); - int i = 0; - for (String version : voteRecommendedServerVersions) { - this.bw.write((i++ > 0 ? "," : "") + version); - } - this.bw.write("</font></td>\n" - + " </tr>\n"); - } - } - } - } - this.bw.write(" <tr>\n" - + " <td><font color=\"blue\">consensus</font>" - + "</td>\n" - + " <td><font color=\"blue\">client-versions "); - int i = 0; - for (String version : - downloadedConsensus.getRecommendedClientVersions()) { - this.bw.write((i++ > 0 ? "," : "") + version); - } - this.bw.write("</font></td>\n" - + " </tr>\n" - + " <tr>\n" - + " <td></td>\n" - + " <td><font color=\"blue\">server-versions "); - i = 0; - for (String version : - downloadedConsensus.getRecommendedServerVersions()) { - this.bw.write((i++ > 0 ? "," : "") + version); - } - this.bw.write("</font></td>\n" - + " </tr>\n" - + " </table>\n"); - } - - /* Write consensus parameters. */ - private void writeConsensusParameters() throws IOException { - this.bw.write(" <br>\n" - + " <a name=\"consensusparams\">\n" - + " <h3><a href=\"#consensusparams\" class=\"anchor\">" - + "Consensus parameters</a></h3>\n" - + " <br>\n" - + " <table border=\"0\" cellpadding=\"4\" " - + "cellspacing=\"0\" summary=\"\">\n" - + " <colgroup>\n" - + " <col width=\"160\">\n" - + " <col width=\"640\">\n" - + " </colgroup>\n"); - if (this.downloadedVotes.size() < 1) { - this.bw.write(" <tr><td>(No votes.)</td><td></td></tr>\n"); - } else { - Set<String> validParameters = new HashSet<String>(Arrays.asList( - ("circwindow,CircuitPriorityHalflifeMsec,refuseunknownexits," - + "cbtdisabled,cbtnummodes,cbtrecentcount,cbtmaxtimeouts," - + "cbtmincircs,cbtquantile,cbtclosequantile,cbttestfreq," - + "cbtmintimeout,cbtinitialtimeout,bwauthpid").split(","))); - Map<String, String> consensusConsensusParams = - downloadedConsensus.getConsensusParams(); - for (Status vote : this.downloadedVotes) { - Map<String, String> voteConsensusParams = - vote.getConsensusParams(); - boolean conflictOrInvalid = false; - if (voteConsensusParams != null) { - for (Map.Entry<String, String> e : - voteConsensusParams.entrySet()) { - if (!consensusConsensusParams.containsKey(e.getKey()) || - !consensusConsensusParams.get(e.getKey()).equals( - e.getValue()) || - !validParameters.contains(e.getKey())) { - conflictOrInvalid = true; - break; - } - } - } - if (conflictOrInvalid) { - this.bw.write(" <tr>\n" - + " <td><font color=\"red\">" - + vote.getNickname() + "</font></td>\n" - + " <td><font color=\"red\">params"); - for (Map.Entry<String, String> e : - voteConsensusParams.entrySet()) { - this.bw.write(" " + e.getKey() + "=" + e.getValue()); - } - this.bw.write("</font></td>\n" - + " </tr>\n"); - } else { - this.bw.write(" <tr>\n" - + " <td>" + vote.getNickname() + "</td>\n" - + " <td>params"); - for (Map.Entry<String, String> e : - voteConsensusParams.entrySet()) { - this.bw.write(" " + e.getKey() + "=" + e.getValue()); - } - this.bw.write("</td>\n" - + " </tr>\n"); - } - } - } - this.bw.write(" <tr>\n" - + " <td><font color=\"blue\">consensus</font>" - + "</td>\n" - + " <td><font color=\"blue\">params"); - for (Map.Entry<String, String> e : - this.downloadedConsensus.getConsensusParams().entrySet()) { - this.bw.write(" " + e.getKey() + "=" + e.getValue()); - } - this.bw.write("</font></td>\n" - + " </tr>\n" - + " </table>\n"); - } - - /* Write authority keys and their expiration dates. */ - private void writeAuthorityKeys() throws IOException { - this.bw.write(" <br>\n" - + " <a name=\"authoritykeys\">\n" - + " <h3><a href=\"#authoritykeys\" class=\"anchor\">" - + "Authority keys</a></h3>\n" - + " <br>\n" - + " <table border=\"0\" cellpadding=\"4\" " - + "cellspacing=\"0\" summary=\"\">\n" - + " <colgroup>\n" - + " <col width=\"160\">\n" - + " <col width=\"640\">\n" - + " </colgroup>\n"); - if (this.downloadedVotes.size() < 1) { - this.bw.write(" <tr><td>(No votes.)</td><td></td></tr>\n"); - } else { - for (Status vote : this.downloadedVotes) { - long voteDirKeyExpiresMillis = vote.getDirKeyExpiresMillis(); - if (voteDirKeyExpiresMillis - 14L * 24L * 60L * 60L * 1000L < - System.currentTimeMillis()) { - this.bw.write(" <tr>\n" - + " <td><font color=\"red\">" - + vote.getNickname() + "</font></td>\n" - + " <td><font color=\"red\">dir-key-expires " - + dateTimeFormat.format(voteDirKeyExpiresMillis) - + "</font></td>\n" - + " </tr>\n"); - } else { - this.bw.write(" <tr>\n" - + " <td>" + vote.getNickname() + "</td>\n" - + " <td>dir-key-expires " - + dateTimeFormat.format(voteDirKeyExpiresMillis) - + "</td>\n" - + " </tr>\n"); - } - } - } - this.bw.write(" </table>\n" - + " <br>\n" - + " <p><i>Note that expiration dates of legacy keys are " - + "not included in votes and therefore not listed here!</i>" - + "</p>\n"); - } - - /* Write the status of bandwidth scanners and results being contained - * in votes. */ - private void writeBandwidthScannerStatus() throws IOException { - this.bw.write(" <br>\n" - + " <a name=\"bwauthstatus\">\n" - + " <h3><a href=\"#bwauthstatus\" class=\"anchor\">" - + "Bandwidth scanner status</a></h3>\n" - + " <br>\n" - + " <table border=\"0\" cellpadding=\"4\" " - + "cellspacing=\"0\" summary=\"\">\n" - + " <colgroup>\n" - + " <col width=\"160\">\n" - + " <col width=\"640\">\n" - + " </colgroup>\n"); - if (this.downloadedVotes.size() < 1) { - this.bw.write(" <tr><td>(No votes.)</td><td></td></tr>\n"); - } else { - for (Status vote : this.downloadedVotes) { - if (vote.getBandwidthWeights() > 0) { - this.bw.write(" <tr>\n" - + " <td>" + vote.getNickname() + "</td>\n" - + " <td>" + vote.getBandwidthWeights() - + " Measured values in w lines</td>\n" - + " </tr>\n"); - } - } - } - this.bw.write(" </table>\n"); - } - - /* Write directory authority versions. */ - private void writeAuthorityVersions() throws IOException { - this.bw.write(" <br>\n" - + " <a name=\"authorityversions\">\n" - + " <h3><a href=\"#authorityversions\" class=\"anchor\">" - + "Authority versions</a></h3>\n" - + " <br>\n"); - Map<String, String> authorityVersions = - this.downloadedConsensus.getAuthorityVersions(); - if (authorityVersions.size() < 1) { - this.bw.write(" <p>(No relays with Authority flag found.)" - + "</p>\n"); - } else { - this.bw.write(" <table border=\"0\" cellpadding=\"4\" " - + "cellspacing=\"0\" summary=\"\">\n" - + " <colgroup>\n" - + " <col width=\"160\">\n" - + " <col width=\"640\">\n" - + " </colgroup>\n"); - for (Map.Entry<String, String> e : authorityVersions.entrySet()) { - String nickname = e.getKey(); - String versionString = e.getValue(); - this.bw.write(" <tr>\n" - + " <td>" + nickname + "</td>\n" - + " <td>" + versionString + "</td>\n" - + " </tr>\n"); - } - this.bw.write(" </table>\n" - + " <br>\n" - + " <p><i>Note that this list of relays with the " - + "Authority flag may be different from the list of v3 " - + "directory authorities!</i></p>\n"); - } - } - - /* Write the (huge) table containing relay flags contained in votes and - * the consensus for each relay. */ - private void writeRelayFlagsTable() throws IOException { - this.bw.write(" <br>\n" - + " <a name=\"relayflags\">\n" - + " <h3><a href=\"#relayflags\" class=\"anchor\">Relay " - + "flags</a></h3>\n" - + " <br>\n" - + " <p>The semantics of flags written in the table is " - + "as follows:</p>\n" - + " <ul>\n" - + " <li><b>In vote and consensus:</b> Flag in vote " - + "matches flag in consensus, or relay is not listed in " - + "consensus (because it doesn't have the Running " - + "flag)</li>\n" - + " <li><b><font color=\"red\">Only in " - + "vote:</font></b> Flag in vote, but missing in the " - + "consensus, because there was no majority for the flag or " - + "the flag was invalidated (e.g., Named gets invalidated by " - + "Unnamed)</li>\n" - + " <li><b><font color=\"gray\"><s>Only in " - + "consensus:</s></font></b> Flag in consensus, but missing " - + "in a vote of a directory authority voting on this " - + "flag</li>\n" - + " <li><b><font color=\"blue\">In " - + "consensus:</font></b> Flag in consensus</li>\n" - + " </ul>\n" - + " <br>\n" - + " <p>See also the summary below the table.</p>\n" - + " <table border=\"0\" cellpadding=\"4\" " - + "cellspacing=\"0\" summary=\"\">\n" - + " <colgroup>\n" - + " <col width=\"120\">\n" - + " <col width=\"80\">\n"); - for (int i = 0; i < this.downloadedVotes.size(); i++) { - this.bw.write(" <col width=\"" - + (640 / this.downloadedVotes.size()) + "\">\n"); - } - this.bw.write(" </colgroup>\n"); - SortedMap<String, String> allRelays = new TreeMap<String, String>(); - for (Status vote : this.downloadedVotes) { - for (StatusEntry statusEntry : vote.getStatusEntries().values()) { - allRelays.put(statusEntry.getFingerprint(), - statusEntry.getNickname()); - } - } - for (StatusEntry statusEntry : - this.downloadedConsensus.getStatusEntries().values()) { - allRelays.put(statusEntry.getFingerprint(), - statusEntry.getNickname()); - } - int linesWritten = 0; - for (Map.Entry<String, String> e : allRelays.entrySet()) { - if (linesWritten++ % 10 == 0) { - this.writeRelayFlagsTableHeader(); - } - String fingerprint = e.getKey(); - String nickname = e.getValue(); - this.writeRelayFlagsTableRow(fingerprint, nickname); - } - this.bw.write(" </table>\n"); - } - - /* Write the table header that is repeated every ten relays and that - * contains the directory authority names. */ - private void writeRelayFlagsTableHeader() throws IOException { - this.bw.write(" <tr><td><br><b>Fingerprint</b></td>" - + "<td><br><b>Nickname</b></td>\n"); - for (Status vote : this.downloadedVotes) { - String shortDirName = vote.getNickname().length() > 6 ? - vote.getNickname().substring(0, 5) + "." : - vote.getNickname(); - this.bw.write("<td><br><b>" + shortDirName + "</b></td>"); - } - this.bw.write("<td><br><b>consensus</b></td></tr>\n"); - } - - /* Write a single row in the table of relay flags. */ - private void writeRelayFlagsTableRow(String fingerprint, - String nickname) throws IOException { - this.bw.write(" <tr>\n"); - if (this.downloadedConsensus.containsStatusEntry(fingerprint) && - this.downloadedConsensus.getStatusEntry(fingerprint).getFlags(). - contains("Named") && - !Character.isDigit(nickname.charAt(0))) { - this.bw.write(" <td id=\"" + nickname - + "\"><a href=\"relay.html?fingerprint=" - + fingerprint + "\" target=\"_blank\">" - + fingerprint.substring(0, 8) + "</a></td>\n"); - } else { - this.bw.write(" <td><a href=\"relay.html?fingerprint=" - + fingerprint + "\" target=\"_blank\">" - + fingerprint.substring(0, 8) + "</a></td>\n"); - } - this.bw.write(" <td>" + nickname + "</td>\n"); - SortedSet<String> relevantFlags = new TreeSet<String>(); - for (Status vote : this.downloadedVotes) { - if (vote.containsStatusEntry(fingerprint)) { - relevantFlags.addAll(vote.getStatusEntry(fingerprint).getFlags()); - } - } - SortedSet<String> consensusFlags = null; - if (this.downloadedConsensus.containsStatusEntry(fingerprint)) { - consensusFlags = this.downloadedConsensus. - getStatusEntry(fingerprint).getFlags(); - relevantFlags.addAll(consensusFlags); - } - for (Status vote : this.downloadedVotes) { - if (vote.containsStatusEntry(fingerprint)) { - SortedSet<String> flags = vote.getStatusEntry(fingerprint). - getFlags(); - this.bw.write(" <td>"); - int flagsWritten = 0; - for (String flag : relevantFlags) { - this.bw.write(flagsWritten++ > 0 ? "<br>" : ""); - if (flags.contains(flag)) { - if (consensusFlags == null || - consensusFlags.contains(flag)) { - this.bw.write(flag); - } else { - this.bw.write("<font color=\"red\">" + flag + "</font>"); - } - } else if (consensusFlags != null && - vote.getKnownFlags().contains(flag) && - consensusFlags.contains(flag)) { - this.bw.write("<font color=\"gray\"><s>" + flag - + "</s></font>"); - } - } - this.bw.write("</td>\n"); - } else { - this.bw.write(" <td></td>\n"); - } - } - if (consensusFlags != null) { - this.bw.write(" <td>"); - int flagsWritten = 0; - for (String flag : relevantFlags) { - this.bw.write(flagsWritten++ > 0 ? "<br>" : ""); - if (consensusFlags.contains(flag)) { - this.bw.write("<font color=\"blue\">" + flag + "</font>"); - } - } - this.bw.write("</td>\n"); - } else { - this.bw.write(" <td></td>\n"); - } - this.bw.write(" </tr>\n"); - } - - /* Write the relay flag summary. */ - private void writeRelayFlagsSummary() throws IOException { - this.bw.write(" <br>\n" - + " <a name=\"overlap\">\n" - + " <h3><a href=\"#overlap\" class=\"anchor\">Overlap " - + "between votes and consensus</a></h3>\n" - + " <br>\n" - + " <p>The semantics of columns is similar to the " - + "table above:</p>\n" - + " <ul>\n" - + " <li><b>In vote and consensus:</b> Flag in vote " - + "matches flag in consensus, or relay is not listed in " - + "consensus (because it doesn't have the Running " - + "flag)</li>\n" - + " <li><b><font color=\"red\">Only in " - + "vote:</font></b> Flag in vote, but missing in the " - + "consensus, because there was no majority for the flag or " - + "the flag was invalidated (e.g., Named gets invalidated by " - + "Unnamed)</li>\n" - + " <li><b><font color=\"gray\"><s>Only in " - + "consensus:</s></font></b> Flag in consensus, but missing " - + "in a vote of a directory authority voting on this " - + "flag</li>\n" - + " </ul>\n" - + " <br>\n" - + " <table border=\"0\" cellpadding=\"4\" " - + "cellspacing=\"0\" summary=\"\">\n" - + " <colgroup>\n" - + " <col width=\"160\">\n" - + " <col width=\"210\">\n" - + " <col width=\"210\">\n" - + " <col width=\"210\">\n" - + " </colgroup>\n" - + " <tr><td></td><td><b>Only in vote</b></td>" - + "<td><b>In vote and consensus</b></td>" - + "<td><b>Only in consensus</b></td>\n"); - Set<String> allFingerprints = new HashSet<String>(); - for (Status vote : this.downloadedVotes) { - allFingerprints.addAll(vote.getStatusEntries().keySet()); - } - allFingerprints.addAll(this.downloadedConsensus.getStatusEntries(). - keySet()); - SortedMap<String, SortedMap<String, Integer>> flagsAgree = - new TreeMap<String, SortedMap<String, Integer>>(); - SortedMap<String, SortedMap<String, Integer>> flagsLost = - new TreeMap<String, SortedMap<String, Integer>>(); - SortedMap<String, SortedMap<String, Integer>> flagsMissing = - new TreeMap<String, SortedMap<String, Integer>>(); - for (String fingerprint : allFingerprints) { - SortedSet<String> consensusFlags = - this.downloadedConsensus.containsStatusEntry(fingerprint) ? - this.downloadedConsensus.getStatusEntry(fingerprint).getFlags() : - null; - for (Status vote : this.downloadedVotes) { - String dir = vote.getNickname(); - if (vote.containsStatusEntry(fingerprint)) { - SortedSet<String> flags = vote.getStatusEntry(fingerprint). - getFlags(); - for (String flag : this.downloadedConsensus.getKnownFlags()) { - SortedMap<String, SortedMap<String, Integer>> sums = null; - if (flags.contains(flag)) { - if (consensusFlags == null || - consensusFlags.contains(flag)) { - sums = flagsAgree; - } else { - sums = flagsLost; - } - } else if (consensusFlags != null && - vote.getKnownFlags().contains(flag) && - consensusFlags.contains(flag)) { - sums = flagsMissing; - } - if (sums != null) { - SortedMap<String, Integer> sum = null; - if (sums.containsKey(dir)) { - sum = sums.get(dir); - } else { - sum = new TreeMap<String, Integer>(); - sums.put(dir, sum); - } - sum.put(flag, sum.containsKey(flag) ? - sum.get(flag) + 1 : 1); - } - } - } - } - } - for (Status vote : this.downloadedVotes) { - String dir = vote.getNickname(); - int i = 0; - for (String flag : vote.getKnownFlags()) { - this.bw.write(" <tr>\n" - + " <td>" + (i++ == 0 ? dir : "") - + "</td>\n"); - if (flagsLost.containsKey(dir) && - flagsLost.get(dir).containsKey(flag)) { - this.bw.write(" <td><font color=\"red\"> " - + flagsLost.get(dir).get(flag) + " " + flag - + "</font></td>\n"); - } else { - this.bw.write(" <td></td>\n"); - } - if (flagsAgree.containsKey(dir) && - flagsAgree.get(dir).containsKey(flag)) { - this.bw.write(" <td>" + flagsAgree.get(dir).get(flag) - + " " + flag + "</td>\n"); - } else { - this.bw.write(" <td></td>\n"); - } - if (flagsMissing.containsKey(dir) && - flagsMissing.get(dir).containsKey(flag)) { - this.bw.write(" <td><font color=\"gray\"><s>" - + flagsMissing.get(dir).get(flag) + " " + flag - + "</s></font></td>\n"); - } else { - this.bw.write(" <td></td>\n"); - } - this.bw.write(" </tr>\n"); - } - } - this.bw.write(" </table>\n"); - } - - /* Write the footer of the HTML page containing the blurb that is on - * every page of the metrics website. */ - private void writePageFooter() throws IOException { - this.bw.write(" </div>\n" - + " </div>\n" - + " <div class=\"bottom\" id=\"bottom\">\n" - + " <p>This material is supported in part by the " - + "National Science Foundation under Grant No. " - + "CNS-0959138. Any opinions, finding, and conclusions " - + "or recommendations expressed in this material are " - + "those of the author(s) and do not necessarily reflect " - + "the views of the National Science Foundation.</p>\n" - + " <p>\"Tor\" and the \"Onion Logo\" are <a " - + "href=\"https://www.torproject.org/docs/trademark-faq.html" - + ".en\">" - + "registered trademarks</a> of The Tor Project, " - + "Inc.</p>\n" - + " <p>Data on this site is freely available under a " - + "<a href=\"http://creativecommons.org/publicdomain/" - + "zero/1.0/\">CC0 no copyright declaration</a>: To the " - + "extent possible under law, the Tor Project has waived " - + "all copyright and related or neighboring rights in " - + "the data. Graphs are licensed under a <a " - + "href=\"http://creativecommons.org/licenses/by/3.0/" - + "us/\">Creative Commons Attribution 3.0 United States " - + "License</a>.</p>\n" - + " </div>\n" - + " </body>\n" - + "</html>"); - } -} - diff --git a/src/org/torproject/chc/Parser.java b/src/org/torproject/chc/Parser.java deleted file mode 100644 index 053c7b4..0000000 --- a/src/org/torproject/chc/Parser.java +++ /dev/null @@ -1,167 +0,0 @@ -/* Copyright 2011 The Tor Project - * See LICENSE for licensing information */ -package org.torproject.chc; - -import java.io.*; -import java.text.*; -import java.util.*; -import org.apache.commons.codec.binary.*; - -/* Parse a network status consensus or vote. */ -public class Parser { - - /* Parse and return a consensus and corresponding votes, or null if - * something goes wrong. */ - public SortedMap<String, Status> parse( - SortedMap<String, String> consensusStrings, - List<String> voteStrings) { - SortedSet<Status> parsedVotes = new TreeSet<Status>(); - for (String voteString : voteStrings) { - Status parsedVote = this.parseConsensusOrVote(voteString, false); - if (parsedVote != null) { - parsedVotes.add(parsedVote); - } - } - SortedMap<String, Status> parsedConsensuses = - new TreeMap<String, Status>(); - for (Map.Entry<String, String> e : consensusStrings.entrySet()) { - String nickname = e.getKey(); - String consensusString = e.getValue(); - Status parsedConsensus = this.parseConsensusOrVote(consensusString, - true); - if (parsedConsensus != null) { - for (Status parsedVote : parsedVotes) { - if (parsedConsensus.getValidAfterMillis() == - parsedVote.getValidAfterMillis()) { - parsedConsensus.addVote(parsedVote); - } - } - parsedConsensuses.put(nickname, parsedConsensus); - } - } - return parsedConsensuses; - } - - /* Date-time formats to parse and format timestamps. */ - private static SimpleDateFormat dateTimeFormat; - static { - dateTimeFormat = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss"); - dateTimeFormat.setTimeZone(TimeZone.getTimeZone("UTC")); - } - - /* Parse a consensus or vote string into a Status instance. */ - private Status parseConsensusOrVote(String statusString, - boolean isConsensus) { - if (statusString == null) { - return null; - } - Status status = new Status(); - status.setUnparsedString(statusString); - try { - BufferedReader br = new BufferedReader(new StringReader( - statusString)); - String line, rLine = null, sLine = null; - int totalRelays = 0, runningRelays = 0, bandwidthWeights = 0; - while ((line = br.readLine()) != null) { - if (line.startsWith("consensus-method ") || - line.startsWith("consensus-methods ")) { - SortedSet<Integer> consensusMethods = new TreeSet<Integer>(); - String[] parts = line.split(" "); - for (int i = 1; i < parts.length; i++) { - consensusMethods.add(Integer.parseInt(parts[i])); - } - status.setConsensusMethods(consensusMethods); - } else if (line.startsWith("valid-after ")) { - try { - status.setValidAfterMillis(dateTimeFormat.parse( - line.substring("valid-after ".length())).getTime()); - } catch (ParseException e) { - System.err.println("Could not parse valid-after timestamp in " - + "line '" + line + "' of a " - + (isConsensus ? "consensus" : "vote") + ". Skipping."); - return null; - } - } else if (line.startsWith("client-versions ")) { - status.setRecommendedClientVersions( - new TreeSet<String>(Arrays.asList( - line.split(" ")[1].split(",")))); - } else if (line.startsWith("server-versions ")) { - status.setRecommendedServerVersions( - new TreeSet<String>(Arrays.asList( - line.split(" ")[1].split(",")))); - } else if (line.startsWith("known-flags ")) { - for (String flag : line.substring("known-flags ".length()). - split(" ")) { - status.addKnownFlag(flag); - } - } else if (line.startsWith("params ")) { - if (line.length() > "params ".length()) { - for (String param : - line.substring("params ".length()).split(" ")) { - String paramName = param.split("=")[0]; - String paramValue = param.split("=")[1]; - status.addConsensusParam(paramName, paramValue); - } - } - } else if (line.startsWith("dir-source ") && !isConsensus) { - status.setNickname(line.split(" ")[1]); - status.setFingerprint(line.split(" ")[2]); - } else if (line.startsWith("dir-key-expires ")) { - try { - status.setDirKeyExpiresMillis(dateTimeFormat.parse( - line.substring("dir-key-expires ".length())).getTime()); - } catch (ParseException e) { - System.err.println("Could not parse dir-key-expires " - + "timestamp in line '" + line + "' of a " - + (isConsensus ? "consensus" : "vote") + ". Skipping."); - return null; - } - } else if (line.startsWith("r ") || - line.equals("directory-footer")) { - if (rLine != null) { - StatusEntry statusEntry = new StatusEntry(); - statusEntry.setNickname(rLine.split(" ")[1]); - statusEntry.setFingerprint(Hex.encodeHexString( - Base64.decodeBase64(rLine.split(" ")[2] + "=")). - toUpperCase()); - SortedSet<String> flags = new TreeSet<String>(); - if (sLine.length() > 2) { - for (String flag : sLine.substring(2).split(" ")) { - flags.add(flag); - } - } - statusEntry.setFlags(flags); - status.addStatusEntry(statusEntry); - } - if (line.startsWith("r ")) { - rLine = line; - } else { - break; - } - } else if (line.startsWith("s ") || line.equals("s")) { - sLine = line; - if (line.contains(" Running")) { - runningRelays++; - } - } else if (line.startsWith("v ") && - sLine.contains(" Authority")) { - String nickname = rLine.split(" ")[1]; - String versionString = line.substring(2); - status.addAuthorityVersion(nickname, versionString); - } else if (line.startsWith("w ") && !isConsensus && - line.contains(" Measured")) { - bandwidthWeights++; - } - } - br.close(); - status.setRunningRelays(runningRelays); - status.setBandwidthWeights(bandwidthWeights); - } catch (IOException e) { - System.err.println("Caught an IOException while parsing a " - + (isConsensus ? "consensus" : "vote") + " string. Skipping."); - return null; - } - return status; - } -} - diff --git a/src/org/torproject/chc/Report.java b/src/org/torproject/chc/Report.java deleted file mode 100644 index 48015eb..0000000 --- a/src/org/torproject/chc/Report.java +++ /dev/null @@ -1,23 +0,0 @@ -/* Copyright 2011 The Tor Project - * See LICENSE for licensing information */ -package org.torproject.chc; - -import java.util.*; - -/* Transform findings from parsing consensuses and votes into a report of - * some form. */ -public interface Report { - - /* Process the downloaded current consensus and corresponding votes to - * find irregularities between them. */ - public abstract void processDownloadedConsensuses( - SortedMap<String, Status> downloadedConsensuses); - - /* Process warnings consisting of warning type and details. */ - public abstract void processWarnings( - SortedMap<Warning, String> warnings); - - /* Finish writing report. */ - public abstract void writeReport(); -} - diff --git a/src/org/torproject/chc/Status.java b/src/org/torproject/chc/Status.java deleted file mode 100644 index cbf7395..0000000 --- a/src/org/torproject/chc/Status.java +++ /dev/null @@ -1,176 +0,0 @@ -/* Copyright 2011 The Tor Project - * See LICENSE for licensing information */ -package org.torproject.chc; - -import java.util.*; - -/* Contains the unparsed string and parsed fields from a network status - * consensus or vote. */ -public class Status implements Comparable<Status> { - - /* Helper methods to implement the Comparable interface; Status - * instances are compared by nickname of the publishing directory - * authorities. */ - public int compareTo(Status o) { - return this.nickname.compareTo(o.nickname); - } - public boolean equals(Object o) { - return (o instanceof Status && - this.nickname.equals(((Status) o).nickname)); - } - - /* Unparsed string that was downloaded or read from disk and that can - * be written to disk. */ - private String unparsedString; - public void setUnparsedString(String unparsedString) { - this.unparsedString = unparsedString; - } - public String getUnparsedString() { - return this.unparsedString; - } - - /* Votes published at the same time as this consensus; votes don't - * reference any statuses. */ - private SortedSet<Status> votes = new TreeSet<Status>(); - public void addVote(Status vote) { - this.votes.add(vote); - } - public SortedSet<Status> getVotes() { - return this.votes; - } - - /* Fingerprint of the directory authority publishing this vote; left - * empty for consensuses. */ - private String fingerprint; - public void setFingerprint(String fingerprint) { - this.fingerprint = fingerprint; - } - public String getFingerprint() { - return this.fingerprint; - } - - /* Nickname of the directory authority publishing this vote; left empty - * for consensuses. */ - private String nickname; - public void setNickname(String nickname) { - this.nickname= nickname; - } - public String getNickname() { - return this.nickname; - } - - /* Valid-after time in milliseconds. */ - private long validAfterMillis; - public void setValidAfterMillis(long validAfterMillis) { - this.validAfterMillis = validAfterMillis; - } - public long getValidAfterMillis() { - return this.validAfterMillis; - } - - /* Consensus parameters. */ - private SortedMap<String, String> consensusParams = - new TreeMap<String, String>(); - public void addConsensusParam(String paramName, String paramValue) { - this.consensusParams.put(paramName, paramValue); - } - public SortedMap<String, String> getConsensusParams() { - return this.consensusParams; - } - - /* Consensus methods supported by the directory authority sending a vote - * or of the produced consensus. */ - private SortedSet<Integer> consensusMethods; - public void setConsensusMethods(SortedSet<Integer> consensusMethods) { - this.consensusMethods = consensusMethods; - } - public SortedSet<Integer> getConsensusMethods() { - return this.consensusMethods; - } - - /* Recommended server versions. */ - private SortedSet<String> recommendedServerVersions; - public void setRecommendedServerVersions( - SortedSet<String> recommendedServerVersions) { - this.recommendedServerVersions = recommendedServerVersions; - } - public SortedSet<String> getRecommendedServerVersions() { - return this.recommendedServerVersions; - } - - /* Recommended client versions. */ - private SortedSet<String> recommendedClientVersions; - public void setRecommendedClientVersions( - SortedSet<String> recommendedClientVersions) { - this.recommendedClientVersions = recommendedClientVersions; - } - public SortedSet<String> getRecommendedClientVersions() { - return this.recommendedClientVersions; - } - - /* Expiration times of directory signing keys. */ - private long dirKeyExpiresMillis; - public void setDirKeyExpiresMillis(long dirKeyExpiresMillis) { - this.dirKeyExpiresMillis = dirKeyExpiresMillis; - } - public long getDirKeyExpiresMillis() { - return this.dirKeyExpiresMillis; - } - - /* Known flags by the directory authority sending a vote or of the - * produced consensus. */ - private SortedSet<String> knownFlags = new TreeSet<String>(); - public void addKnownFlag(String knownFlag) { - this.knownFlags.add(knownFlag); - } - public SortedSet<String> getKnownFlags() { - return this.knownFlags; - } - - /* Number of status entries with the Running flag. */ - private int runningRelays; - public void setRunningRelays(int runningRelays) { - this.runningRelays = runningRelays; - } - public int getRunningRelays() { - return this.runningRelays; - } - - /* Number of status entries containing bandwidth weights (only relevant - * in votes). */ - private int bandwidthWeights; - public void setBandwidthWeights(int bandwidthWeights) { - this.bandwidthWeights = bandwidthWeights; - } - public int getBandwidthWeights() { - return this.bandwidthWeights; - } - - /* Status entries contained in this status. */ - private SortedMap<String, StatusEntry> statusEntries = - new TreeMap<String, StatusEntry>(); - public void addStatusEntry(StatusEntry statusEntry) { - this.statusEntries.put(statusEntry.getFingerprint(), statusEntry); - } - public SortedMap<String, StatusEntry> getStatusEntries() { - return this.statusEntries; - } - public boolean containsStatusEntry(String fingerprint) { - return this.statusEntries.containsKey(fingerprint); - } - public StatusEntry getStatusEntry(String fingerprint) { - return this.statusEntries.get(fingerprint); - } - - /* Versions of directory authorities (only set in a consensus). */ - private SortedMap<String, String> authorityVersions = - new TreeMap<String, String>(); - public void addAuthorityVersion(String fingerprint, - String versionString) { - this.authorityVersions.put(fingerprint, versionString); - } - public SortedMap<String, String> getAuthorityVersions() { - return this.authorityVersions; - } -} - diff --git a/src/org/torproject/chc/StatusEntry.java b/src/org/torproject/chc/StatusEntry.java deleted file mode 100644 index 0377937..0000000 --- a/src/org/torproject/chc/StatusEntry.java +++ /dev/null @@ -1,48 +0,0 @@ -/* Copyright 2011 The Tor Project - * See LICENSE for licensing information */ -package org.torproject.chc; - -import java.util.*; - -/* Contains the parsed data from a network status entry contained in a - * network status consensus or vote. */ -public class StatusEntry implements Comparable<StatusEntry> { - - /* Helper methods to implement the Comparable interface; StatusEntry - * instances are compared by fingerprint. */ - public int compareTo(StatusEntry o) { - return this.fingerprint.compareTo(o.fingerprint); - } - public boolean equals(Object o) { - return (o instanceof StatusEntry && - this.fingerprint.equals(((StatusEntry) o).fingerprint)); - } - - /* Relay fingerprint. */ - private String fingerprint; - public void setFingerprint(String fingerprint) { - this.fingerprint = fingerprint; - } - public String getFingerprint() { - return this.fingerprint; - } - - /* Relay nickname. */ - private String nickname; - public void setNickname(String nickname) { - this.nickname = nickname; - } - public String getNickname() { - return this.nickname; - } - - /* Relay flags. */ - private SortedSet<String> flags; - public void setFlags(SortedSet<String> flags) { - this.flags = flags; - } - public SortedSet<String> getFlags() { - return this.flags; - } -} - diff --git a/src/org/torproject/chc/StatusFileReport.java b/src/org/torproject/chc/StatusFileReport.java deleted file mode 100644 index 4aef195..0000000 --- a/src/org/torproject/chc/StatusFileReport.java +++ /dev/null @@ -1,197 +0,0 @@ -/* Copyright 2011 The Tor Project - * See LICENSE for licensing information */ -package org.torproject.chc; - -import java.io.*; -import java.text.*; -import java.util.*; - -/* Check a given consensus and votes for irregularities and write results - * to stdout while rate-limiting warnings based on severity. */ -public class StatusFileReport implements Report { - - /* Date-time format to format timestamps. */ - private static SimpleDateFormat dateTimeFormat; - static { - dateTimeFormat = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss"); - dateTimeFormat.setTimeZone(TimeZone.getTimeZone("UTC")); - } - - /* Downloaded consensus and corresponding votes for later - * processing. */ - private SortedMap<String, Status> downloadedConsensuses; - private Status downloadedConsensus; - private SortedSet<Status> downloadedVotes; - public void processDownloadedConsensuses( - SortedMap<String, Status> downloadedConsensuses) { - this.downloadedConsensuses = downloadedConsensuses; - } - - /* Warnings obtained from checking the current consensus and votes. */ - private SortedMap<Warning, String> warnings; - public void processWarnings(SortedMap<Warning, String> warnings) { - this.warnings = warnings; - } - - /* Check consensuses and votes for irregularities and write output to - * stdout. */ - public void writeReport() { - this.readLastWarned(); - this.prepareReports(); - this.writeStatusFiles(); - this.writeLastWarned(); - } - - /* Warning messages of the last 24 hours that is used to implement - * rate limiting. */ - private Map<String, Long> lastWarned = new HashMap<String, Long>(); - - /* Read when we last emitted a warning to rate-limit some of them. */ - private void readLastWarned() { - long now = System.currentTimeMillis(); - File lastWarnedFile = new File("stats/chc-last-warned"); - try { - if (lastWarnedFile.exists()) { - BufferedReader br = new BufferedReader(new FileReader( - lastWarnedFile)); - String line; - while ((line = br.readLine()) != null) { - if (!line.contains(": ")) { - System.err.println("Bad line in stats/chc-last-warned: '" + line - + "'. Ignoring this line."); - continue; - } - long warnedMillis = Long.parseLong(line.substring(0, - line.indexOf(": "))); - if (warnedMillis < now - 24L * 60L * 60L * 1000L) { - /* Remove warnings that are older than 24 hours. */ - continue; - } - String message = line.substring(line.indexOf(": ") + 2); - lastWarned.put(message, warnedMillis); - } - } - } catch (IOException e) { - System.err.println("Could not read file '" - + lastWarnedFile.getAbsolutePath() + "' to learn which " - + "warnings have been sent out before. Ignoring."); - } - } - - /* Prepare a report to be written to stdout. */ - private String allWarnings = null, newWarnings = null; - private void prepareReports() { - SortedMap<String, Long> warningStrings = new TreeMap<String, Long>(); - for (Map.Entry<Warning, String> e : this.warnings.entrySet()) { - Warning type = e.getKey(); - String details = e.getValue(); - switch (type) { - case NoConsensusKnown: - break; - case ConsensusDownloadTimeout: - warningStrings.put("The following directory authorities did " - + "not return a consensus within a timeout of 60 seconds: " - + details, 150L * 60L * 1000L); - break; - case ConsensusNotFresh: - warningStrings.put("The consensuses published by the following " - + "directory authorities are more than 1 hour old and " - + "therefore not fresh anymore: " + details, - 150L * 60L * 1000L); - break; - case ConsensusMethodNotSupported: - warningStrings.put("The following directory authorities do not " - + "support the consensus method that the consensus uses: " - + details, 24L * 60L * 60L * 1000L); - break; - case DifferentRecommendedClientVersions: - warningStrings.put("The following directory authorities " - + "recommend other client versions than the consensus: " - + details, 150L * 60L * 1000L); - break; - case DifferentRecommendedServerVersions: - warningStrings.put("The following directory authorities " - + "recommend other server versions than the consensus: " - + details, 150L * 60L * 1000L); - break; - case ConflictingOrInvalidConsensusParams: - warningStrings.put("The following directory authorities set " - + "conflicting or invalid consensus parameters: " + details, - 150L * 60L * 1000L); - break; - case CertificateExpiresSoon: - warningStrings.put("The certificates of the following " - + "directory authorities expire within the next 14 days: " - + details, 24L * 60L * 60L * 1000L); - break; - case VotesMissing: - warningStrings.put("We're missing votes from the following " - + "directory authorities: " + details, 150L * 60L * 1000L); - break; - case BandwidthScannerResultsMissing: - warningStrings.put("The following directory authorities are " - + "not reporting bandwidth scanner results: " + details, - 150L * 60L * 1000L); - break; - } - } - long now = System.currentTimeMillis(); - StringBuilder allSb = new StringBuilder(), - newSb = new StringBuilder(); - for (Map.Entry<String, Long> e : warningStrings.entrySet()) { - String message = e.getKey(); - allSb.append(message + "\n"); - long warnInterval = e.getValue(); - if (!lastWarned.containsKey(message) || - lastWarned.get(message) + warnInterval < now) { - newSb.append(message + "\n"); - } - } - if (newSb.length() > 0) { - this.allWarnings = allSb.toString(); - this.newWarnings = newSb.toString(); - for (String message : warningStrings.keySet()) { - this.lastWarned.put(message, now); - } - } - } - - /* Write report to stdout. */ - private void writeStatusFiles() { - try { - BufferedWriter allBw = new BufferedWriter(new FileWriter( - "all-warnings")), newBw = new BufferedWriter(new FileWriter( - "new-warnings")); - if (this.allWarnings != null) { - allBw.write(this.allWarnings); - } - if (this.newWarnings != null) { - newBw.write(this.newWarnings); - } - allBw.close(); - newBw.close(); - } catch (IOException e) { - System.err.println("Could not write status files 'all-warnings' " - + "and/or 'new-warnings'. Ignoring."); - } - } - - /* Write timestamps when warnings were last sent to disk. */ - private void writeLastWarned() { - File lastWarnedFile = new File("stats/chc-last-warned"); - try { - lastWarnedFile.getParentFile().mkdirs(); - BufferedWriter bw = new BufferedWriter(new FileWriter( - lastWarnedFile)); - for (Map.Entry<String, Long> e : lastWarned.entrySet()) { - bw.write(String.valueOf(e.getValue()) + ": " + e.getKey() + "\n"); - } - bw.close(); - } catch (IOException e) { - System.err.println("Could not write file '" - + lastWarnedFile.getAbsolutePath() + "' to remember which " - + "warnings have been sent out before. Ignoring."); - } - } -} - diff --git a/src/org/torproject/chc/Warning.java b/src/org/torproject/chc/Warning.java deleted file mode 100644 index 3425c6c..0000000 --- a/src/org/torproject/chc/Warning.java +++ /dev/null @@ -1,46 +0,0 @@ -/* Copyright 2011 The Tor Project - * See LICENSE for licensing information */ -package org.torproject.chc; - -/* Warning about irregularities in parsed consensuses and votes. */ -public enum Warning { - - /* No consensus is known that can be checked. */ - NoConsensusKnown, - - /* One or more directory authorities did not return a consensus within a - * timeout of 60 seconds. */ - ConsensusDownloadTimeout, - - /* One or more directory authorities published a consensus that is more - * than 1 hour old and therefore not fresh anymore. */ - ConsensusNotFresh, - - /* One or more directory authorities does not support the consensus - * method that the consensus uses. */ - ConsensusMethodNotSupported, - - /* One or more directory authorities recommends different client - * versions than the ones in the consensus. */ - DifferentRecommendedClientVersions, - - /* One or more directory authorities recommends different server - * versions than the ones in the consensus. */ - DifferentRecommendedServerVersions, - - /* One or more directory authorities set conflicting or invalid - * consensus parameters. */ - ConflictingOrInvalidConsensusParams, - - /* The certificate(s) of one or more directory authorities expire within - * the next 14 days. */ - CertificateExpiresSoon, - - /* The vote(s) of one or more directory authorities are missing. */ - VotesMissing, - - /* One or more directory authorities are not reporting bandwidth scanner - * results. */ - BandwidthScannerResultsMissing -} -

1 0

[torflow/master] And forgot to reset a var...
by mikeperry＠torproject.org 06 Dec '11

06 Dec '11

commit f967c8b45f63c38459011d780b248009a4f67c8a Author: Mike Perry <mikeperry-git(a)fscked.org> Date: Mon Dec 5 19:39:41 2011 -0800 And forgot to reset a var... --- NetworkScanners/BwAuthority/aggregate.py | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/NetworkScanners/BwAuthority/aggregate.py b/NetworkScanners/BwAuthority/aggregate.py index f8573a4..93cad8a 100755 --- a/NetworkScanners/BwAuthority/aggregate.py +++ b/NetworkScanners/BwAuthority/aggregate.py @@ -513,6 +513,7 @@ def main(argv): # FIXME: This may be expensive pid_avg = filt_avg prev_pid_avg = 2*pid_avg + f_nodes = nodes.values() while prev_pid_avg > pid_avg: f_nodes = filter(lambda n: n.desc_bw >= pid_avg, f_nodes)

1 0

[torflow/master] Fix c_node exception typo.
by mikeperry＠torproject.org 06 Dec '11

06 Dec '11

commit 02562661714b840417c8d6ef8ac2c47f7a8ee47c Author: Mike Perry <mikeperry-git(a)fscked.org> Date: Mon Dec 5 19:29:42 2011 -0800 Fix c_node exception typo. --- NetworkScanners/BwAuthority/aggregate.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/NetworkScanners/BwAuthority/aggregate.py b/NetworkScanners/BwAuthority/aggregate.py index 1207b07..f8573a4 100755 --- a/NetworkScanners/BwAuthority/aggregate.py +++ b/NetworkScanners/BwAuthority/aggregate.py @@ -515,7 +515,7 @@ def main(argv): prev_pid_avg = 2*pid_avg while prev_pid_avg > pid_avg: - f_nodes = filter(lambda n: n.desc_bw >= pid_avg, c_nodes) + f_nodes = filter(lambda n: n.desc_bw >= pid_avg, f_nodes) prev_pid_avg = pid_avg pid_avg = sum(map(lambda n: n.filt_bw, f_nodes))/float(len(f_nodes))

1 0

[torflow/master] Improve and demote some noisy loglines.
by mikeperry＠torproject.org 06 Dec '11

06 Dec '11

commit a0de14874f23873728320ba378c364676b9740dd Author: Mike Perry <mikeperry-git(a)fscked.org> Date: Sat Dec 3 21:41:50 2011 -0800 Improve and demote some noisy loglines. --- NetworkScanners/BwAuthority/aggregate.py | 34 ++++++++++++++++------------- 1 files changed, 19 insertions(+), 15 deletions(-) diff --git a/NetworkScanners/BwAuthority/aggregate.py b/NetworkScanners/BwAuthority/aggregate.py index 7af68aa..1fd6e29 100755 --- a/NetworkScanners/BwAuthority/aggregate.py +++ b/NetworkScanners/BwAuthority/aggregate.py @@ -594,33 +594,37 @@ def main(argv): if circ_error < 0: n.pid_error = min(circ_error,n.pid_error) - # Don't punish gimpy nodes too hard + # Don't accumulate too much amplification for fast nodes + if cs_junk.use_desc_bw: + if n.pid_error_sum > cs_junk.pid_max and n.pid_error > 0: + plog("INFO", "Capping feedback for node %s=%s desc=%d ns=%d pid_error_sum=%f" % + (n.nick, n.idhex, n.desc_bw, n.ns_bw, n.pid_error_sum)) + n.pid_error_sum = cs_junk.pid_max + else: + if float(n.ns_bw)/n.desc_bw > cs_junk.pid_max and n.pid_error > 0: + plog("INFO", "Capping feedback for node %s=%s desc=%d ns=%d pid_error=%f" % + (n.nick, n.idhex, n.desc_bw, n.ns_bw, n.pid_error)) + n.pid_error = 0 + n.pid_error_sum = 0 + + # Don't punish gimpy nodes too hard. if cs_junk.use_mercy: if not cs_junk.use_desc_bw: # If node was demoted in the past and we plan to demote it again, # let's just not and say we did. if n.desc_bw > n.ns_bw and n.pid_error < 0: - plog("INFO", "Showing mercy for node "+n.nick+"="+n.idhex) + plog("DEBUG", "Showing mercy for node %s=%s desc=%d ns=%d pid_error=%f" % + (n.nick, n.idhex, n.desc_bw, n.ns_bw, n.pid_error)) n.use_bw = n.desc_bw if n.pid_error_sum < 0 and n.pid_error < 0: - plog("INFO", "Showing mercy for node "+n.nick+"="+n.idhex+", pid_error_sum="+str(n.pid_error_sum)) - n.pid_error_sum = 0 - - # Similarly, don't accumulate too much amplification for - # fast nodes - if cs_junk.use_desc_bw: - if n.pid_error_sum > cs_junk.pid_max and n.pid_error > 0: - plog("INFO", "Capping feedback for node "+n.nick+"="+n.idhex+", pid_error_sum="+str(n.pid_error_sum)) - n.pid_error_sum = cs_junk.pid_max - else: - if float(n.ns_bw)/n.desc_bw > cs_junk.pid_max and n.pid_error > 0: - plog("INFO", "Capping feedback for node "+n.nick+"="+n.idhex) - n.pid_error = 0 + plog("DEBUG", "Showing mercy for node %s=%s desc=%d ns=%d pid_error_sum=%f" % + (n.nick, n.idhex, n.desc_bw, n.ns_bw, n.pid_error_sum)) n.pid_error_sum = 0 if n.idhex in prev_votes.vote_map: # If there is a new sample, let's use it for all but guards if n.measured_at > prev_votes.vote_map[n.idhex].measured_at: + # Nodes with the Guard flag will respond slowly to feedback, # so they should be sampled less often, and in proportion to # the appropriate Wgx weight.

1 0

[torflow/master] The old code should use circ failures..
by mikeperry＠torproject.org 06 Dec '11

06 Dec '11

commit 2416763b6351d5ad2e7514677412fc501e9dda4e Author: Mike Perry <mikeperry-git(a)fscked.org> Date: Mon Dec 5 16:18:06 2011 -0800 The old code should use circ failures.. I think this was added because circ_fail_rate used to get set to 0 by a consensus param that we later changed. --- NetworkScanners/BwAuthority/aggregate.py | 6 ++---- 1 files changed, 2 insertions(+), 4 deletions(-) diff --git a/NetworkScanners/BwAuthority/aggregate.py b/NetworkScanners/BwAuthority/aggregate.py index 4c5fc28..1207b07 100755 --- a/NetworkScanners/BwAuthority/aggregate.py +++ b/NetworkScanners/BwAuthority/aggregate.py @@ -529,10 +529,8 @@ def main(argv): else: plog("INFO", "PID control disabled") - filt_avg = sum(map(lambda n: n.filt_bw*(1.0-n.circ_fail_rate), - nodes.itervalues()))/float(len(nodes)) - strm_avg = sum(map(lambda n: n.strm_bw*(1.0-n.circ_fail_rate), - nodes.itervalues()))/float(len(nodes)) + filt_avg = sum(map(lambda n: n.filt_bw, nodes.itervalues()))/float(len(nodes)) + strm_avg = sum(map(lambda n: n.strm_bw, nodes.itervalues()))/float(len(nodes)) for cl in ["Guard+Exit", "Guard", "Exit", "Middle"]: true_filt_avg[cl] = filt_avg true_strm_avg[cl] = strm_avg

1 0