- tor-commits - lists.torproject.org

r25155: {website} Update TBB design doc based on comments from pde. (website/trunk/projects/torbrowser/design)
by Mike Perry 07 Oct '11

07 Oct '11

Author: mikeperry Date: 2011-10-07 00:35:15 +0000 (Fri, 07 Oct 2011) New Revision: 25155 Modified: website/trunk/projects/torbrowser/design/index.html.en Log: Update TBB design doc based on comments from pde. Modified: website/trunk/projects/torbrowser/design/index.html.en =================================================================== --- website/trunk/projects/torbrowser/design/index.html.en 2011-10-06 16:18:04 UTC (rev 25154) +++ website/trunk/projects/torbrowser/design/index.html.en 2011-10-07 00:35:15 UTC (rev 25155) @@ -1,6 +1,6 @@ <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> -<html xmlns="http://www.w3.org/1999/xhtml"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /><title>The Design and Implementation of the Tor Browser [DRAFT]</title><meta name="generator" content="DocBook XSL Stylesheets V1.75.2" /></head><body><div class="article" title="The Design and Implementation of the Tor Browser [DRAFT]"><div class="titlepage"><div><div><h2 class="title"><a id="design"></a>The Design and Implementation of the Tor Browser [DRAFT]</h2></div><div><div class="author"><h3 class="author"><span class="firstname">Mike</span> <span class="surname">Perry</span></h3><div class="affiliation"><div class="address"><p><code class="email"><<a class="email" href="mailto:mikeperry#torproject org">mikeperry#torproject org</a>></code></p></div></div></div></div><div><div class="author"><h3 class="author"><span class="firstname">Erinn</span> <span class="surname">Clark</span></h3><div class="affiliation"><div class="address"><p><code class= "email"><<a class="email" href="mailto:erinn#torproject org">erinn#torproject org</a>></code></p></div></div></div></div><div><div class="author"><h3 class="author"><span class="firstname">Steven</span> <span class="surname">Murdoch</span></h3><div class="affiliation"><div class="address"><p><code class="email"><<a class="email" href="mailto:sjmurdoch#torproject org">sjmurdoch#torproject org</a>></code></p></div></div></div></div><div><p class="pubdate">Oct 4 2011</p></div></div><hr /></div><div class="toc"><p><b>Table of Contents</b></p><dl><dt><span class="sect1"><a href="#id2857732">1. Introduction</a></span></dt><dd><dl><dt><span class="sect2"><a href="#adversary">1.1. Adversary Model</a></span></dt></dl></dd><dt><span class="sect1"><a href="#DesignRequirements">2. Design Requirements and Philosophy</a></span></dt><dd><dl><dt><span class="sect2"><a href="#security">2.1. Security Requirements</a></span></dt><dt><span class="sect2"><a href="#privacy">2.2. Pri vacy Requirements</a></span></dt><dt><span class="sect2"><a href="#philosophy">2.3. Philosophy</a></span></dt></dl></dd><dt><span class="sect1"><a href="#Implementation">3. Implementation</a></span></dt><dd><dl><dt><span class="sect2"><a href="#proxy-obedience">3.1. Proxy Obedience</a></span></dt><dt><span class="sect2"><a href="#state-separation">3.2. State Separation</a></span></dt><dt><span class="sect2"><a href="#disk-avoidance">3.3. Disk Avoidance</a></span></dt><dt><span class="sect2"><a href="#app-data-isolation">3.4. Application Data Isolation</a></span></dt><dt><span class="sect2"><a href="#identifier-linkability">3.5. Cross-Origin Identifier Unlinkability</a></span></dt><dt><span class="sect2"><a href="#fingerprinting-linkability">3.6. Cross-Origin Fingerprinting Unlinkability</a></span></dt><dt><span class="sect2"><a href="#new-identity">3.7. Long-Term Unlinkability via "New Identity" button</a></span></dt><dt><span class="sect2"><a href="#click-to-play">3.8. Clic k-to-play for plugins and invasive content</a></span></dt><dt><span class="sect2"><a href="#firefox-patches">3.9. Description of Firefox Patches</a></span></dt></dl></dd><dt><span class="sect1"><a href="#Packaging">4. Packaging</a></span></dt><dd><dl><dt><span class="sect2"><a href="#build-security">4.1. Build Process Security</a></span></dt><dt><span class="sect2"><a href="#addons">4.2. External Addons</a></span></dt><dt><span class="sect2"><a href="#prefs">4.3. Pref Changes</a></span></dt><dt><span class="sect2"><a href="#update-mechanism">4.4. Update Security</a></span></dt></dl></dd><dt><span class="sect1"><a href="#Testing">5. Testing</a></span></dt><dd><dl><dt><span class="sect2"><a href="#SingleStateTesting">5.1. Single state testing</a></span></dt></dl></dd></dl></div><div class="sect1" title="1. Introduction"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a id="id2857732"></a>1. Introduction</h2></div></div></div><p> +<html xmlns="http://www.w3.org/1999/xhtml"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /><title>The Design and Implementation of the Tor Browser [DRAFT]</title><meta name="generator" content="DocBook XSL Stylesheets V1.75.2" /></head><body><div class="article" title="The Design and Implementation of the Tor Browser [DRAFT]"><div class="titlepage"><div><div><h2 class="title"><a id="design"></a>The Design and Implementation of the Tor Browser [DRAFT]</h2></div><div><div class="author"><h3 class="author"><span class="firstname">Mike</span> <span class="surname">Perry</span></h3><div class="affiliation"><div class="address"><p><code class="email"><<a class="email" href="mailto:mikeperry#torproject org">mikeperry#torproject org</a>></code></p></div></div></div></div><div><div class="author"><h3 class="author"><span class="firstname">Erinn</span> <span class="surname">Clark</span></h3><div class="affiliation"><div class="address"><p><code class= "email"><<a class="email" href="mailto:erinn#torproject org">erinn#torproject org</a>></code></p></div></div></div></div><div><div class="author"><h3 class="author"><span class="firstname">Steven</span> <span class="surname">Murdoch</span></h3><div class="affiliation"><div class="address"><p><code class="email"><<a class="email" href="mailto:sjmurdoch#torproject org">sjmurdoch#torproject org</a>></code></p></div></div></div></div><div><p class="pubdate">Oct 6 2011</p></div></div><hr /></div><div class="toc"><p><b>Table of Contents</b></p><dl><dt><span class="sect1"><a href="#id2597772">1. Introduction</a></span></dt><dd><dl><dt><span class="sect2"><a href="#adversary">1.1. Adversary Model</a></span></dt></dl></dd><dt><span class="sect1"><a href="#DesignRequirements">2. Design Requirements and Philosophy</a></span></dt><dd><dl><dt><span class="sect2"><a href="#security">2.1. Security Requirements</a></span></dt><dt><span class="sect2"><a href="#privacy">2.2. Pri vacy Requirements</a></span></dt><dt><span class="sect2"><a href="#philosophy">2.3. Philosophy</a></span></dt></dl></dd><dt><span class="sect1"><a href="#Implementation">3. Implementation</a></span></dt><dd><dl><dt><span class="sect2"><a href="#proxy-obedience">3.1. Proxy Obedience</a></span></dt><dt><span class="sect2"><a href="#state-separation">3.2. State Separation</a></span></dt><dt><span class="sect2"><a href="#disk-avoidance">3.3. Disk Avoidance</a></span></dt><dt><span class="sect2"><a href="#app-data-isolation">3.4. Application Data Isolation</a></span></dt><dt><span class="sect2"><a href="#identifier-linkability">3.5. Cross-Origin Identifier Unlinkability</a></span></dt><dt><span class="sect2"><a href="#fingerprinting-linkability">3.6. Cross-Origin Fingerprinting Unlinkability</a></span></dt><dt><span class="sect2"><a href="#new-identity">3.7. Long-Term Unlinkability via "New Identity" button</a></span></dt><dt><span class="sect2"><a href="#click-to-play">3.8. Clic k-to-play for plugins and invasive content</a></span></dt><dt><span class="sect2"><a href="#firefox-patches">3.9. Description of Firefox Patches</a></span></dt></dl></dd><dt><span class="sect1"><a href="#Packaging">4. Packaging</a></span></dt><dd><dl><dt><span class="sect2"><a href="#build-security">4.1. Build Process Security</a></span></dt><dt><span class="sect2"><a href="#addons">4.2. External Addons</a></span></dt><dt><span class="sect2"><a href="#prefs">4.3. Pref Changes</a></span></dt><dt><span class="sect2"><a href="#update-mechanism">4.4. Update Security</a></span></dt></dl></dd><dt><span class="sect1"><a href="#Testing">5. Testing</a></span></dt><dd><dl><dt><span class="sect2"><a href="#SingleStateTesting">5.1. Single state testing</a></span></dt></dl></dd></dl></div><div class="sect1" title="1. Introduction"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a id="id2597772"></a>1. Introduction</h2></div></div></div><p> This document describes the <a class="link" href="#adversary" title="1.1. Adversary Model">adversary model</a>, <a class="link" href="#DesignRequirements" title="2. Design Requirements and Philosophy">design requirements</a>, @@ -58,11 +58,11 @@ The adversary can run exit nodes, or alternatively, they may control routers upstream of exit nodes. Both of these scenarios have been observed in the wild. - </p></li><li class="listitem"><span class="command"><strong>Adservers and/or Malicious Websites</strong></span><p> + </p></li><li class="listitem"><span class="command"><strong>Ad servers and/or Malicious Websites</strong></span><p> The adversary can also run websites, or more likely, they can contract out -ad space from a number of different adservers and inject content that way. For -some users, the adversary may be the adservers themselves. It is not -inconceivable that adservers may try to subvert or reduce a user's anonymity +ad space from a number of different ad servers and inject content that way. For +some users, the adversary may be the ad servers themselves. It is not +inconceivable that ad servers may try to subvert or reduce a user's anonymity through Tor for marketing purposes. </p></li><li class="listitem"><span class="command"><strong>Local Network/ISP/Upstream Router</strong></span><p> The adversary can also inject malicious content at the user's upstream router @@ -80,7 +80,7 @@ positions to accomplish various aspects of their goals. It should be noted that many of these attacks (especially those involving IP address leakage) are often performed by accident by websites that simply have Javascript, dynamic -CSS elements, and plugins. Others are performed by adservers seeking to +CSS elements, and plugins. Others are performed by ad servers seeking to correlate users' activity across different IP addresses, and still others are performed by malicious agents on the Tor network and at national firewalls. @@ -187,7 +187,8 @@ </p></li></ol></div></div></div></div><div class="sect1" title="2. Design Requirements and Philosophy"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a id="DesignRequirements"></a>2. Design Requirements and Philosophy</h2></div></div></div><p> The Tor Browser Design Requirements are meant to describe the properties of a -Private Browsing Mode that defends against both network and forensic adversaries. +Private Browsing Mode that defends against both network and local forensic +adversaries. </p><p> @@ -237,7 +238,9 @@ of private browsing to disk outside of the application's control. The user must be able to ensure that secure removal of the software is sufficient to remove evidence of the use of the software. All exceptions and shortcomings -due to operating system behavior MUST BE wiped by an uninstaller. +due to operating system behavior MUST BE wiped by an uninstaller. However, due +to permissions issues with access to swap, implementations MAY choose to leave +it out of scope, and/or leave it to the user to implement encrypted swap. </p></li><li class="listitem"><span class="command"><strong>Update Safety</strong></span><p>The browser SHOULD NOT perform unsafe updates or upgrades.</p></li></ol></div></div><div class="sect2" title="2.2. Privacy Requirements"><div class="titlepage"><div><div><h3 class="title"><a id="privacy"></a>2.2. Privacy Requirements</h3></div></div></div><p> @@ -259,10 +262,12 @@ </p><div class="orderedlist"><ol class="orderedlist" type="1"><li class="listitem"><span class="command"><strong>Cross-Origin Identifier Unlinkability</strong></span><p> User activity on one url bar origin MUST NOT be linkable to their activity in -any other url bar origin by any third party. This property specifically applies to -linkability from stored browser identifiers, authentication tokens, and shared -state. This functionality SHOULD NOT interfere with federated login in a -substantial way. +any other url bar origin by any third party automatically or without user +interaction or approval. This requirement specifically applies to linkability +from stored browser identifiers, authentication tokens, and shared state. The +requirement does not apply to linkable information the user manually submits +to sites, or due information submitted during manual link traversal. This +functionality SHOULD NOT interfere with federated login in a substantial way. </p></li><li class="listitem"><span class="command"><strong>Cross-Origin Fingerprinting Unlinkability</strong></span><p> @@ -417,13 +422,13 @@ Tor Browser State is separated from existing browser state through use of a custom Firefox profile. Furthermore, plugins are disabled, which prevents Flash cookies from leaking from a pre-existing Flash directory. - </p></div><div class="sect2" title="3.3. Disk Avoidance"><div class="titlepage"><div><div><h3 class="title"><a id="disk-avoidance"></a>3.3. Disk Avoidance</h3></div></div></div><div class="sect3" title="Design Goal:"><div class="titlepage"><div><div><h4 class="title"><a id="id2886678"></a>Design Goal:</h4></div></div></div><div class="blockquote"><blockquote class="blockquote"> + </p></div><div class="sect2" title="3.3. Disk Avoidance"><div class="titlepage"><div><div><h3 class="title"><a id="disk-avoidance"></a>3.3. Disk Avoidance</h3></div></div></div><div class="sect3" title="Design Goal:"><div class="titlepage"><div><div><h4 class="title"><a id="id2616664"></a>Design Goal:</h4></div></div></div><div class="blockquote"><blockquote class="blockquote"> Tor Browser MUST (at user option) prevent all disk records of browser activity. The user should be able to optionally enable URL history and other history features if they so desire. Once we <a class="ulink" href="https://trac.torproject.org/projects/tor/ticket/3100" target="_top">simplify the preferences interface</a>, we will likely just enable Private Browsing mode by default to handle this goal. - </blockquote></div></div><div class="sect3" title="Implementation Status:"><div class="titlepage"><div><div><h4 class="title"><a id="id2874561"></a>Implementation Status:</h4></div></div></div><div class="blockquote"><blockquote class="blockquote"> + </blockquote></div></div><div class="sect3" title="Implementation Status:"><div class="titlepage"><div><div><h4 class="title"><a id="id2606128"></a>Implementation Status:</h4></div></div></div><div class="blockquote"><blockquote class="blockquote"> For now, Tor Browser blocks write access to the disk through Torbutton using several Firefox preferences. @@ -488,7 +493,7 @@ context-menu option to drill down into specific types of state or permissions. An example of this simplification can be seen in Figure 1. - </p><div class="figure"><a id="id2867838"></a><p class="title"><b>Figure 1. Improving the Privacy UI</b></p><div class="figure-contents"><div class="mediaobject" align="center"><img src="CookieManagers.png" align="middle" alt="Improving the Privacy UI" /></div><div class="caption"><p></p> + </p><div class="figure"><a id="id2612402"></a><p class="title"><b>Figure 1. Improving the Privacy UI</b></p><div class="figure-contents"><div class="mediaobject" align="center"><img src="CookieManagers.png" align="middle" alt="Improving the Privacy UI" /></div><div class="caption"><p></p> On the left is the standard Firefox cookie manager. On the right is a mock-up of how isolating identifiers to the URL bar origin might simplify the privacy @@ -578,6 +583,7 @@ make this behavior unlinkable, we wish to include a settings file for all platforms that disables flash cookies using the <a class="ulink" href="http://www.macromedia.com/support/documentation/en/flashplayer/help/setting…" target="_top">Flash settings manager</a>. + </p><p><span class="command"><strong>Implementation Status:</strong></span> We are currently <a class="ulink" href="https://trac.torproject.org/projects/tor/ticket/3974" target="_top">having @@ -608,11 +614,17 @@ MUST prompt users before following redirects that would cause the user to automatically navigate between two different url bar origins. - </p><p><span class="command"><strong>Implementation status:</strong></span> +</p><p> -There are numerous ways for the user to be redirected, and the Firefox API -support to detect each of them is poor. We have a <a class="ulink" href="https://trac.torproject.org/projects/tor/ticket/3600" target="_top">trac bug -open</a> to implement what we can. +However, to +reduce the occurrence of warning fatigue, these warning messages MAY be limited +to automated redirect cycles only. For example, the automated redirect +sequence <span class="command"><strong>User Click -> t.co -> bit.ly -> cnn.com</strong></span> can be +assumed to be benign, but the redirect sequence <span class="command"><strong>User Click -> t.co -> +bit.ly -> cnn.com -> 2o7.net -> scorecardresearch.net -> cnn.com</strong></span> is +clearly due to tracking. Non-automated redirect cycles that require +user input at some step (such as federated login systems) need not be +interrupted by the UI. </p><p> @@ -622,6 +634,12 @@ especially with frequent use of the <a class="link" href="#new-identity" title="3.7. Long-Term Unlinkability via "New Identity" button">New Identity</a> button. + </p><p><span class="command"><strong>Implementation status:</strong></span> + +There are numerous ways for the user to be redirected, and the Firefox API +support to detect each of them is poor. We have a <a class="ulink" href="https://trac.torproject.org/projects/tor/ticket/3600" target="_top">trac bug +open</a> to implement what we can. + </p></li><li class="listitem">window.name <p> @@ -639,7 +657,36 @@ for the duration of a link-driven navigation session, but as soon as the user enters a new URL or navigates between https/http schemes, the property is cleared. - </p></li><li class="listitem">Exit node usage + </p></li><li class="listitem">Auto form-fill + <p> + +We disable the password saving functionality in the browser as part of our +<a class="link" href="#disk-avoidance" title="3.3. Disk Avoidance">Disk Avoidance</a> requirement. However, +since users may decide to re-enable disk history records and password saving, +we also set the <a class="ulink" href="http://kb.mozillazine.org/Signon.autofillForms" target="_top">signon.autofillForms</a> +preference to false to prevent saved values from immediately populating +fields upon page load. Since Javascript can read these values as soon as they +appear, setting this preference prevents automatic linkability from stored passwords. + + </p></li><li class="listitem">HSTS supercookies + <p> +An extreme (but not impossible) attack to mount is the creation of <a class="ulink" href="https://secure.wikimedia.org/wikipedia/en/wiki/HTTP_Strict_Transport_Securi…" target="_top">HSTS</a> +supercookies. Since HSTS effectively stores one bit of information per domain +name, an adversary in possession of numerous domains can use them to construct +cookies based on stored HSTS state. + + </p><p><span class="command"><strong>Design Goal:</strong></span> + +There appears to be three options for us: 1. Disable HSTS entirely, and rely +instead on HTTPS-Everywhere. 2. Restrict the number of HSTS-enabled third +parties allowed per url bar origin. 3. Prevent third parties from storing HSTS +rules. We have not yet decided upon the best approach. + + </p><p><span class="command"><strong>Implementation Status:</strong></span> Currently, HSTS state is +cleared by <a class="link" href="#new-identity" title="3.7. Long-Term Unlinkability via "New Identity" button">New Identity</a>, but we don't +defend against the creation of these cookies between <span class="command"><strong>New +Identity</strong></span> invocations. + </p></li><li class="listitem">Exit node usage <p><span class="command"><strong>Design Goal:</strong></span> Every distinct navigation session (as defined by a non-blank referer header) @@ -715,11 +762,22 @@ pre-built list to query, a large amount of fingerprintable information may still be available. + </p><p> + +The sure-fire way to address font linkability is to ship the browser with a +font for every language, typeface, and style in use in the world, and to only +use those fonts at the exclusion of system fonts. However, this set may be +impractically large. It is possible that a smaller <a class="ulink" href="https://secure.wikimedia.org/wikipedia/en/wiki/Unicode_typeface#List_of_Uni…" target="_top">common +subset</a> may be found that provides total coverage. However, we believe +that with strong url bar origin identifier isolation, a simpler approach can reduce the +number of bits available to the adversary while avoiding the rendering and +language issues of supporting a global font set. + </p><p><span class="command"><strong>Design Goal:</strong></span> -To address the Javascript issue, we intend to <a class="ulink" href="https://trac.torproject.org/projects/tor/ticket/2872" target="_top">limit the number of -fonts</a> an origin can load, gracefully degrading to built-in and/or -remote fonts once the limit is reached. +We intend to <a class="ulink" href="https://trac.torproject.org/projects/tor/ticket/2872" target="_top">limit the number of +fonts</a> a url bar origin can load, gracefully degrading to built-in +and/or remote fonts once the limit is reached. </p><p><span class="command"><strong>Implementation Status:</strong></span> @@ -805,7 +863,7 @@ even with the default precision in most browsers, they required up to 120 seconds of amortization and repeated trials to get stable results from their feature set. We intend to work with the research community to establish the -optimum tradeoff between quantization+jitter and amortization time. +optimum trade-off between quantization+jitter and amortization time. </p><p><span class="command"><strong>Implementation Status:</strong></span> @@ -852,22 +910,24 @@ </p></li></ol></div></div><div class="sect2" title="3.7. Long-Term Unlinkability via "New Identity" button"><div class="titlepage"><div><div><h3 class="title"><a id="new-identity"></a>3.7. Long-Term Unlinkability via "New Identity" button</h3></div></div></div><p> In order to avoid long-term linkability, we provide a "New Identity" context menu option in Torbutton. - </p><div class="sect3" title="Design Goal:"><div class="titlepage"><div><div><h4 class="title"><a id="id2853903"></a>Design Goal:</h4></div></div></div><div class="blockquote"><blockquote class="blockquote"> + </p><div class="sect3" title="Design Goal:"><div class="titlepage"><div><div><h4 class="title"><a id="id2626323"></a>Design Goal:</h4></div></div></div><div class="blockquote"><blockquote class="blockquote"> All linkable identifiers and browser state MUST be cleared by this feature. - </blockquote></div></div><div class="sect3" title="Implementation Status:"><div class="titlepage"><div><div><h4 class="title"><a id="id2874701"></a>Implementation Status:</h4></div></div></div><div class="blockquote"><blockquote class="blockquote"> - First, Torbutton disables -all open tabs and windows via nsIContentPolicy blocking, and then closes each -tab and window. The extra step for blocking tabs is done as a precaution to -ensure that any asynchronous Javascript is in fact properly disabled. After -closing all of the windows, we then clear the following state: OCSP (by -toggling security.OCSP.enabled), cache, site-specific zoom and content -preferences, Cookies, DOM storage, safe browsing key, the Google wifi -geolocation token (if exists), HTTP auth, SSL Session IDs, and the last opened URL -field (via the pref general.open_location.last_url). After clearing the -browser state, we then send the NEWNYM signal to the Tor control port to cause -a new circuit to be created. + </blockquote></div></div><div class="sect3" title="Implementation Status:"><div class="titlepage"><div><div><h4 class="title"><a id="id2612376"></a>Implementation Status:</h4></div></div></div><div class="blockquote"><blockquote class="blockquote"> + + First, Torbutton disables all open tabs and windows via nsIContentPolicy +blocking, and then closes each tab and window. The extra step for blocking +tabs is done as a precaution to ensure that any asynchronous Javascript is in +fact properly disabled. After closing all of the windows, we then clear the +following state: OCSP (by toggling security.OCSP.enabled), cache, +site-specific zoom and content preferences, Cookies, DOM storage, safe +browsing key, the Google wifi geolocation token (if exists), HTTP auth, SSL +Session IDs, HSTS state, and the last opened URL field (via the pref +general.open_location.last_url). After clearing the browser state, we then +send the NEWNYM signal to the Tor control port to cause a new circuit to be +created. + </blockquote></div></div></div><div class="sect2" title="3.8. Click-to-play for plugins and invasive content"><div class="titlepage"><div><div><h3 class="title"><a id="click-to-play"></a>3.8. Click-to-play for plugins and invasive content</h3></div></div></div><p> Some content types are too invasive and/or too opaque for us to properly eliminate their linkability properties. For these content types, we use @@ -895,7 +955,8 @@ This patch exposes a pref 'permissions.memory_only' that properly isolates the permissions manager to memory, which is responsible for all user specified -site permissions, as well as stored HTTPS STS policy from visited sites. +site permissions, as well as stored <a class="ulink" href="https://secure.wikimedia.org/wikipedia/en/wiki/HTTP_Strict_Transport_Securi…" target="_top">HSTS</a> +policy from visited sites. The pref does successfully clear the permissions manager memory if toggled. It does not need to be set in prefs.js, and can be handled by Torbutton. @@ -952,7 +1013,7 @@ This patch prevents random URLs from being inserted into content-prefs.sqllite in the profile directory as content prefs change (includes site-zoom and perhaps other site prefs?). - </p></li></ol></div></div></div><div class="sect1" title="4. Packaging"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a id="Packaging"></a>4. Packaging</h2></div></div></div><p> </p><div class="sect2" title="4.1. Build Process Security"><div class="titlepage"><div><div><h3 class="title"><a id="build-security"></a>4.1. Build Process Security</h3></div></div></div><p> </p></div><div class="sect2" title="4.2. External Addons"><div class="titlepage"><div><div><h3 class="title"><a id="addons"></a>4.2. External Addons</h3></div></div></div><p> </p><div class="sect3" title="Included Addons"><div class="titlepage"><div><div><h4 class="title"><a id="id2886800"></a>Included Addons</h4></div></div></div></div><div class="sect3" title="Excluded Addons"><div class="titlepage"><div><div><h4 class="title"><a id="id2882777"></a>Excluded Addons</h4></div></div></div></div><div class="sect3" title="Dangerous Addons"><div class="titlepage"><div><div><h4 cla ss="title"><a id="id2864076"></a>Dangerous Addons</h4></div></div></div></div></div><div class="sect2" title="4.3. Pref Changes"><div class="titlepage"><div><div><h3 class="title"><a id="prefs"></a>4.3. Pref Changes</h3></div></div></div><p> </p></div><div class="sect2" title="4.4. Update Security"><div class="titlepage"><div><div><h3 class="title"><a id="update-mechanism"></a>4.4. Update Security</h3></div></div></div><p> </p></div></div><div class="sect1" title="5. Testing"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a id="Testing"></a>5. Testing</h2></div></div></div><p> + </p></li></ol></div></div></div><div class="sect1" title="4. Packaging"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a id="Packaging"></a>4. Packaging</h2></div></div></div><p> </p><div class="sect2" title="4.1. Build Process Security"><div class="titlepage"><div><div><h3 class="title"><a id="build-security"></a>4.1. Build Process Security</h3></div></div></div><p> </p></div><div class="sect2" title="4.2. External Addons"><div class="titlepage"><div><div><h3 class="title"><a id="addons"></a>4.2. External Addons</h3></div></div></div><p> </p><div class="sect3" title="Included Addons"><div class="titlepage"><div><div><h4 class="title"><a id="id2621568"></a>Included Addons</h4></div></div></div></div><div class="sect3" title="Excluded Addons"><div class="titlepage"><div><div><h4 class="title"><a id="id2614080"></a>Excluded Addons</h4></div></div></div></div><div class="sect3" title="Dangerous Addons"><div class="titlepage"><div><div><h4 cla ss="title"><a id="id2613296"></a>Dangerous Addons</h4></div></div></div></div></div><div class="sect2" title="4.3. Pref Changes"><div class="titlepage"><div><div><h3 class="title"><a id="prefs"></a>4.3. Pref Changes</h3></div></div></div><p> </p></div><div class="sect2" title="4.4. Update Security"><div class="titlepage"><div><div><h3 class="title"><a id="update-mechanism"></a>4.4. Update Security</h3></div></div></div><p> </p></div></div><div class="sect1" title="5. Testing"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a id="Testing"></a>5. Testing</h2></div></div></div><p> The purpose of this section is to cover all the known ways that Tor browser security can be subverted from a penetration testing perspective. The hope

1 0

[torbutton/master] Bug 4197: Torbutton formfill blocking can't be undone
by mikeperry＠torproject.org 06 Oct '11

06 Oct '11

commit 9e2cbd2993c595abf1e67bdf887578bd086c73b5 Author: Mike Perry <mikeperry-git(a)fscked.org> Date: Thu Oct 6 14:10:08 2011 -0700 Bug 4197: Torbutton formfill blocking can't be undone We still require a toggle, which is very difficult for TBB users. Filed #4198 for that, as several prefs are involved, and we probably want to wait for bug #3100 anyways. Also, we should set signon.autoformFill to false during Tor usage regardless of this pref, for unlinkability. So we now do so. --- src/chrome/content/torbutton.js | 14 ++++++++++++++ 1 files changed, 14 insertions(+), 0 deletions(-) diff --git a/src/chrome/content/torbutton.js b/src/chrome/content/torbutton.js index 8b0837e..1c76f03 100644 --- a/src/chrome/content/torbutton.js +++ b/src/chrome/content/torbutton.js @@ -300,6 +300,8 @@ var torbutton_unique_pref_observer = torbutton_update_status( m_tb_prefs.getBoolPref("extensions.torbutton.tor_enabled"), true); + // XXX: We should try to get rid of these warnings now that toggle + // is not supported in TBB. case "extensions.torbutton.disable_domstorage": case "extensions.torbutton.no_updates": case "extensions.torbutton.no_search": @@ -1838,8 +1840,16 @@ function torbutton_update_status(mode, force_update) { false, mode, changed); torbutton_setBoolPref("signon.rememberSignons", "remember_signons", false, mode, changed); + } else { + torbutton_setBoolPref("browser.formfill.enable", "formfill", + true, mode, changed); + torbutton_setBoolPref("signon.rememberSignons", "remember_signons", + true, mode, changed); } + torbutton_setBoolPref("signon.autofillForms", "autofillForms", + false, mode, changed); + if (m_tb_ff4) { if(m_tb_prefs.getBoolPref('extensions.torbutton.block_thwrite')) { m_tb_prefs.setBoolPref("places.history.enabled", false); @@ -1866,6 +1876,10 @@ function torbutton_update_status(mode, force_update) { torbutton_setBoolPref("signon.rememberSignons", "remember_signons", false, mode, changed); } + + torbutton_setBoolPref("signon.autofillForms", "autofillForms", + true, mode, changed); + if (m_tb_ff4) { if(m_tb_prefs.getBoolPref('extensions.torbutton.block_nthwrite')) { m_tb_prefs.setBoolPref("places.history.enabled", false);

1 0

[translation/vidalia_help] Update translations for vidalia_help
by translation＠torproject.org 06 Oct '11

06 Oct '11

commit 443f020f9de919363a71becc841e445dab9e808c Author: Translation commit bot <translation(a)torproject.org> Date: Thu Oct 6 19:04:39 2011 +0000 Update translations for vidalia_help --- de/running.po | 63 ++++++++++++++++++++++++++------------------------------ 1 files changed, 29 insertions(+), 34 deletions(-) diff --git a/de/running.po b/de/running.po index b73d5d3..688c035 100644 --- a/de/running.po +++ b/de/running.po @@ -1,27 +1,19 @@ # -# $Id$ -# -# This file is part of Vidalia, and is subject to the license terms in -# the LICENSE file, found in the top level directory of this -# distribution. If you did not receive the LICENSE file with this -# file, you may obtain it from the Vidalia source package distributed -# by the Vidalia Project at http://www.torproject.org/projects/vidalia.html. -# No part of Vidalia, including this file, may be copied, modified, -# propagated, or distributed except according to the terms described -# in the LICENSE file. -# +# Translators: +# runasand <runa.sandvik(a)gmail.com>, 2011. msgid "" msgstr "" -"Project-Id-Version: PACKAGE VERSION\n" -"Report-Msgid-Bugs-To: \n" +"Project-Id-Version: The Tor Project\n" +"Report-Msgid-Bugs-To: https://trac.torproject.org/projects/tor\n" "POT-Creation-Date: 2010-06-26 17:00+0200\n" -"PO-Revision-Date: 2010-06-26 15:18-0600\n" -"Last-Translator: Runa Sandvik <runa.sandvik(a)gmail.com>\n" +"PO-Revision-Date: 2010-11-30 05:03+0000\n" +"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n" "Language-Team: LANGUAGE <LL(a)li.org>\n" "MIME-Version: 1.0\n" "Content-Type: text/plain; charset=UTF-8\n" "Content-Transfer-Encoding: 8bit\n" -"X-Generator: Pootle 1.1.0\n" +"Language: de\n" +"Plural-Forms: nplurals=2; plural=(n != 1)\n" #. type: Content of: <html><body><h1> #: en/running.html:16 @@ -80,8 +72,8 @@ msgstr "" msgid "" "If Vidalia is unable to start Tor, Vidalia will display an error message " "telling you what went wrong. You can also look at your <a " -"href=\"log.html\">message log</a> to see if Tor printed any more information " -"about what went wrong." +"href=\"log.html\">message log</a> to see if Tor printed any more information" +" about what went wrong." msgstr "" "Falls Vidalia Tor nicht starten konnte, wird Ihnen eine Fehlermeldung mit " "entsprechenden Hinweisen angezeigt. Sie können außerdem im <a " @@ -97,8 +89,8 @@ msgstr "Um Tor zu <i>stoppen</i>," #: en/running.html:45 msgid "Select <i>Stop</i> from Vidalia's tray menu or press <i>Ctrl+T</i>" msgstr "" -"wählen Sie <i>Stop</i> aus Vidalias Traymenü oder drücken Sie " -"<i>Strg+T</i>" +"wählen Sie <i>Stop</i> aus Vidalias Traymenü oder drücken Sie" +" <i>Strg+T</i>" #. type: Content of: <html><body><p><ol><li> #: en/running.html:47 @@ -114,8 +106,8 @@ msgstr "" msgid "" "If Vidalia was unable to stop Tor, Vidalia will display an error message " "telling you what went wrong. You can also look at your <a " -"href=\"log.html\">message log</a> to see if Tor printed any more information " -"about what went wrong." +"href=\"log.html\">message log</a> to see if Tor printed any more information" +" about what went wrong." msgstr "" "Falls Vidalia Tor nicht beenden konnte, wird Ihnen eine Fehlermeldung mit " "entsprechenden Hinweisen angezeigt. Sie können außerdem im <a " @@ -145,7 +137,8 @@ msgstr "" #. type: Content of: <html><body><p><table><tr><td> #: en/running.html:69 -msgid "Tor is stopped. Select <i>Start</i> from the Vidalia menu to start Tor." +msgid "" +"Tor is stopped. Select <i>Start</i> from the Vidalia menu to start Tor." msgstr "" "Tor wurde gestoppt. Wählen Sie <i>Start</i> aus dem Vidalia-Menü " "um Tor zu starten." @@ -163,15 +156,15 @@ msgstr "" #. type: Content of: <html><body><p><table><tr><td> #: en/running.html:84 msgid "" -"Tor is running. If you want to stop Tor, select <i>Stop</i> from the Vidalia " -"menu. Tor will print informational messages to the <a " +"Tor is running. If you want to stop Tor, select <i>Stop</i> from the Vidalia" +" menu. Tor will print informational messages to the <a " "href=\"log.html\">message log</a> while it is running, if you want to see " "what Tor is doing." msgstr "" "Tor wird ausgeführt. Falls Sie Tor stoppen wollen, wählen Sie " "<i>Stop</i> aus dem Vidalia-Menü. Tor speichert während des " -"Betriebs Informationen im <a href=\\\"log.html\\\">Nachrichtenprotokoll</a>. Um " -"weitere Informationen zu erfahren, können Sie dort nachschauen." +"Betriebs Informationen im <a href=\\\"log.html\\\">Nachrichtenprotokoll</a>." +" Um weitere Informationen zu erfahren, können Sie dort nachschauen." #. type: Content of: <html><body><p><table><tr><td> #: en/running.html:92 @@ -182,12 +175,14 @@ msgstr "Tor wird gerade gestoppt." #: en/running.html:98 msgid "" "If Tor exits unexpectedly, Vidalia will change its icon to the dark onion " -"with a red X and display an error message letting you know what went " -"wrong. You can also check the <a href=\"log.html\">message log</a> for " -"details about any problems Tor encountered before it exited." +"with a red X and display an error message letting you know what went wrong. " +"You can also check the <a href=\"log.html\">message log</a> for details " +"about any problems Tor encountered before it exited." msgstr "" -"Falls Tor unerwartet beendet wird, verändert sich das Vidalia-Symbol zu " -"einer dunklen Zwiebel mit einem roten X und eine entsprechende " +"Falls Tor unerwartet beendet wird, verändert sich das Vidalia-Symbol zu" +" einer dunklen Zwiebel mit einem roten X und eine entsprechende " "Fehlermeldung wird angezeigt. Sie können außerdem das <a " -"href=\\\"log.html\\\">Nachrichtenprotokoll</a> auf weitere Details zu Problemen " -"untersuchen, die zur Beendigung von Tor geführt haben." +"href=\\\"log.html\\\">Nachrichtenprotokoll</a> auf weitere Details zu " +"Problemen untersuchen, die zur Beendigung von Tor geführt haben." + +

1 0

[translation/vidalia_help] Update translations for vidalia_help
by translation＠torproject.org 06 Oct '11

06 Oct '11

commit 9f0a041fad2d4883b48d3aec28be80b87865a2ca Author: Translation commit bot <translation(a)torproject.org> Date: Thu Oct 6 18:34:39 2011 +0000 Update translations for vidalia_help --- zh_CN/troubleshooting.po | 96 +++++++++++++++++++++------------------------ 1 files changed, 45 insertions(+), 51 deletions(-) diff --git a/zh_CN/troubleshooting.po b/zh_CN/troubleshooting.po index 92dddad..d07ac05 100644 --- a/zh_CN/troubleshooting.po +++ b/zh_CN/troubleshooting.po @@ -1,34 +1,24 @@ # -# $Id$ -# -# This file is part of Vidalia, and is subject to the license terms in -# the LICENSE file, found in the top level directory of this -# distribution. If you did not receive the LICENSE file with this -# file, you may obtain it from the Vidalia source package distributed -# by the Vidalia Project at http://www.torproject.org/projects/vidalia.html. -# No part of Vidalia, including this file, may be copied, modified, -# propagated, or distributed except according to the terms described -# in the LICENSE file. -# +# Translators: +# jhk <j_hk001(a)126.com>, 2011. msgid "" msgstr "" -"Project-Id-Version: PACKAGE VERSION\n" -"Report-Msgid-Bugs-To: \n" +"Project-Id-Version: The Tor Project\n" +"Report-Msgid-Bugs-To: https://trac.torproject.org/projects/tor\n" "POT-Creation-Date: 2010-08-03 15:49-0300\n" -"PO-Revision-Date: 2010-11-18 16:08+0200\n" -"Last-Translator: jonchil <abcd666(a)gmail.com>\n" +"PO-Revision-Date: 2011-03-22 16:45+0000\n" +"Last-Translator: jhk <j_hk001(a)126.com>\n" "Language-Team: LANGUAGE <LL(a)li.org>\n" -"Language: zh_CN\n" "MIME-Version: 1.0\n" "Content-Type: text/plain; charset=UTF-8\n" "Content-Transfer-Encoding: 8bit\n" -"Plural-Forms: nplurals=1; plural=0;\n" -"X-Generator: Pootle 2.0.5\n" +"Language: zh_CN\n" +"Plural-Forms: nplurals=1; plural=0\n" # type: Content of: <html><body><h1> #: en/troubleshooting.html:16 msgid "Troubleshooting" -msgstr "故障处理" +msgstr "疑难解答" # type: Content of: <html><body> #: en/troubleshooting.html:19 @@ -38,8 +28,8 @@ msgid "" "having, check out our website at <i>www.vidalia-project.net</i> for more " "support and information. <a name=\"start\"/>" msgstr "" -"以下列表中是一些运行 Tor 时常见的问题。如果以下没有您所遇到的问题。请访问我们的网站来获取更多的支持和信息<i>www.vidalia-" -"project.net</i><a name=\"start\"/>" +"下面列出常见的问题或疑问。如果你不能找到任何有关于你的具体问题，请访问<i>www.vidalia-" +"project.net</i>以获得更多的信息和支持。" # type: Content of: <html><body><h3> #: en/troubleshooting.html:25 @@ -51,25 +41,27 @@ msgstr "我无法启动 Tor" msgid "" "The most likely reason that Vidalia could not start Tor is because Vidalia " "is looking for your Tor installation in the wrong directory. You can tell " -"Vidalia where Tor is located by updating the <i>Tor Executable</i> option in " -"the <a href=\"config.html#general\">general configuration settings</a>." +"Vidalia where Tor is located by updating the <i>Tor Executable</i> option in" +" the <a href=\"config.html#general\">general configuration settings</a>." msgstr "" +"Vidalia 无法启动Tor最可能的原因是因为Vidalia在错误的目录寻找你的Tor安装。你可以通过查阅<a " +"href=\"config.html#general\">一般设置</a>并修改<i>Tor执行文件</i>选项以告诉Vidalia Tor的正确路径。" # type: Content of: <html><body><p> #: en/troubleshooting.html:33 msgid "" "Another possible reason that Tor cannot start is because there is already " -"another Tor process running. Check your list of running process and stop the " -"previous Tor process, if you find one. Then, try running Tor again." -msgstr "" +"another Tor process running. Check your list of running process and stop the" +" previous Tor process, if you find one. Then, try running Tor again." +msgstr "Tor不能启动的另一个可能原因是因为已经有另一个Tor进程运行。检查您的运行进程列表，并停止以前的Tor进程。然后，尝试再次运行Tor。" # type: Content of: <html><body><p> #: en/troubleshooting.html:38 msgid "" -"If that did not help, check your <a href=\"log.html\">message log</a> to see " -"if Tor printed any information about errors it encountered while trying to " +"If that did not help, check your <a href=\"log.html\">message log</a> to see" +" if Tor printed any information about errors it encountered while trying to " "start." -msgstr "" +msgstr "如果这样没有帮助，检查您的<a href=\"log.html\">消息日志</a> ，看看Tor是否记录有尝试启动时遇到错误的任何信息。" # type: Content of: <html><body> #: en/troubleshooting.html:43 @@ -83,7 +75,8 @@ msgstr "" # type: Content of: <html><body><p> #: en/troubleshooting.html:46 -msgid "Vidalia manages Tor by communicating with it via Tor's <i>control port</i>." +msgid "" +"Vidalia manages Tor by communicating with it via Tor's <i>control port</i>." msgstr "" # type: Content of: <html><body><p> @@ -101,9 +94,9 @@ msgid "" "If Tor is listening on a different port than Vidalia expects, Vidalia will " "be unable to connect to Tor. You rarely need to change this setting, but if " "there is another service running on your machine that conflicts with Tor's " -"control port, you will need to specify a different port. You can change this " -"setting in Vidalia's <a href=\"config.html#advanced\">advanced configuration " -"settings</a>." +"control port, you will need to specify a different port. You can change this" +" setting in Vidalia's <a href=\"config.html#advanced\">advanced " +"configuration settings</a>." msgstr "" # type: Content of: <html><body> @@ -147,9 +140,9 @@ msgstr "" # type: Content of: <html><body><ul><li><p> #: en/troubleshooting.html:89 msgid "" -"Vidalia crashed, but left Tor running with the last known random " -"password. After you restart Vidalia, it generates a new random password, but " -"Vidalia can't talk to Tor, because the random passwords are different." +"Vidalia crashed, but left Tor running with the last known random password. " +"After you restart Vidalia, it generates a new random password, but Vidalia " +"can't talk to Tor, because the random passwords are different." msgstr "" # type: Content of: <html><body><ul><li><p> @@ -174,11 +167,11 @@ msgstr "" msgid "" "You had previously set Tor to run as a service. When Tor is set to run as a " "service, it starts up when the system boots. If you configured Tor to start " -"as a service through Vidalia, a random password was set and saved in " -"Tor. When you reboot, Tor starts up and uses the random password it saved. " -"You login and start up Vidalia. Vidalia attempts to talk to the already " -"running Tor. Vidalia generates a random password, but it is different than " -"the saved password in the Tor service." +"as a service through Vidalia, a random password was set and saved in Tor. " +"When you reboot, Tor starts up and uses the random password it saved. You " +"login and start up Vidalia. Vidalia attempts to talk to the already running " +"Tor. Vidalia generates a random password, but it is different than the saved" +" password in the Tor service." msgstr "" # type: Content of: <html><body><ul><li><p> @@ -186,8 +179,8 @@ msgstr "" msgid "" "You need to reconfigure Tor to not be a service. See the Tor wiki page on " "running <a " -"href=\"https://wiki.torproject.org/noreply/TheOnionRouter/TorFAQ#WinNTService\"> " -"Tor as a service</a> for more information on how to remove the Tor service." +"href=\"https://wiki.torproject.org/noreply/TheOnionRouter/TorFAQ#WinNTService\">" +" Tor as a service</a> for more information on how to remove the Tor service." msgstr "" # type: Content of: <html><body> @@ -203,18 +196,17 @@ msgstr "" # type: Content of: <html><body><p> #: en/troubleshooting.html:125 msgid "" -"If Tor exits immediately after trying to start, you most likely have another " -"Tor process already running. Check the <a href=\"log.html\">message log</a> " -"to see if any of the last few messages in the list are highlighted in yellow " -"and contain a message similar to the following:" +"If Tor exits immediately after trying to start, you most likely have another" +" Tor process already running. Check the <a href=\"log.html\">message log</a>" +" to see if any of the last few messages in the list are highlighted in " +"yellow and contain a message similar to the following:" msgstr "" # type: Content of: <html><body><pre> #: en/troubleshooting.html:131 #, no-wrap msgid "" -"connection_create_listener(): Could not bind to 127.0.0.1:9050: Address " -"already in use. \n" +"connection_create_listener(): Could not bind to 127.0.0.1:9050: Address already in use. \n" "Is Tor already running?\n" msgstr "" @@ -232,8 +224,8 @@ msgstr "" #: en/troubleshooting.html:141 msgid "" "If Tor had been running successfully for awhile (that is, longer than a few " -"seconds), then you should check the <a href=\"log.html\">message log</a> for " -"information about any errors Tor experienced before it exited. Such errors " +"seconds), then you should check the <a href=\"log.html\">message log</a> for" +" information about any errors Tor experienced before it exited. Such errors " "will be highlighted in either red or yellow." msgstr "" @@ -254,3 +246,5 @@ msgid "" "href=\"log.html\">message log</a> to see if Tor reported any errors while " "trying to exit." msgstr "" + +

1 0

[stem/master] Removing checked in binaries
by atagar＠torproject.org 06 Oct '11

06 Oct '11

commit e1c31c060c0246a62e911643a383372889214300 Author: Damian Johnson <atagar(a)torproject.org> Date: Thu Oct 6 09:57:24 2011 -0700 Removing checked in binaries Forgot to establish a gitignore before the prior commit. An extra copy of run_tests.py also snuck in (haven't a clue how). --- stem/__init__.pyc | Bin 228 -> 0 bytes stem/run_tests.py | 13 ------------- stem/types.pyc | Bin 3406 -> 0 bytes test/__init__.pyc | Bin 237 -> 0 bytes test/version.pyc | Bin 3982 -> 0 bytes 5 files changed, 0 insertions(+), 13 deletions(-) diff --git a/stem/__init__.pyc b/stem/__init__.pyc deleted file mode 100644 index 43c6f7d..0000000 Binary files a/stem/__init__.pyc and /dev/null differ diff --git a/stem/run_tests.py b/stem/run_tests.py deleted file mode 100644 index ad52f9c..0000000 --- a/stem/run_tests.py +++ /dev/null @@ -1,13 +0,0 @@ -#!/usr/bin/env python - -""" -Runs unit and integration tests. -""" - -import unittest -import test.types - -if __name__ == '__main__': - suite = unittest.TestLoader().loadTestsFromTestCase(test.types.TestVerionFunctions) - unittest.TextTestRunner(verbosity=2).run(suite) - diff --git a/stem/types.pyc b/stem/types.pyc deleted file mode 100644 index 27a5bb1..0000000 Binary files a/stem/types.pyc and /dev/null differ diff --git a/test/__init__.pyc b/test/__init__.pyc deleted file mode 100644 index 6b41a10..0000000 Binary files a/test/__init__.pyc and /dev/null differ diff --git a/test/version.pyc b/test/version.pyc deleted file mode 100644 index c459fa7..0000000 Binary files a/test/version.pyc and /dev/null differ

1 0

[stem/master] Class and function for Tor Versions
by atagar＠torproject.org 06 Oct '11

06 Oct '11

commit 69083da95c5ab536adcd23f19a424ed9395015bb Author: Damian Johnson <atagar(a)torproject.org> Date: Thu Oct 6 09:52:04 2011 -0700 Class and function for Tor Versions Starting with a simple (but non-trivial) class that will be needed for handling PROTOCOLINFO responses. This is partly to establish conventions for documentation and unit tests. --- run_tests.py | 13 ++++++ stem/__init__.py | 6 +++ stem/__init__.pyc | Bin 0 -> 228 bytes stem/run_tests.py | 13 ++++++ stem/types.py | 99 +++++++++++++++++++++++++++++++++++++++++++ stem/types.pyc | Bin 0 -> 3406 bytes test/__init__.py | 6 +++ test/__init__.pyc | Bin 0 -> 237 bytes test/version.py | 121 +++++++++++++++++++++++++++++++++++++++++++++++++++++ test/version.pyc | Bin 0 -> 3982 bytes 10 files changed, 258 insertions(+), 0 deletions(-) diff --git a/run_tests.py b/run_tests.py old mode 100644 new mode 100755 index e69de29..0fe6365 --- a/run_tests.py +++ b/run_tests.py @@ -0,0 +1,13 @@ +#!/usr/bin/env python + +""" +Runs unit and integration tests. +""" + +import unittest +import test.version + +if __name__ == '__main__': + suite = unittest.TestLoader().loadTestsFromTestCase(test.version.TestVerionFunctions) + unittest.TextTestRunner(verbosity=2).run(suite) + diff --git a/stem/__init__.py b/stem/__init__.py new file mode 100644 index 0000000..01c350c --- /dev/null +++ b/stem/__init__.py @@ -0,0 +1,6 @@ +""" +Library for working with the tor process. +""" + +__all__ = ["types"] + diff --git a/stem/__init__.pyc b/stem/__init__.pyc new file mode 100644 index 0000000..43c6f7d Binary files /dev/null and b/stem/__init__.pyc differ diff --git a/stem/run_tests.py b/stem/run_tests.py new file mode 100644 index 0000000..ad52f9c --- /dev/null +++ b/stem/run_tests.py @@ -0,0 +1,13 @@ +#!/usr/bin/env python + +""" +Runs unit and integration tests. +""" + +import unittest +import test.types + +if __name__ == '__main__': + suite = unittest.TestLoader().loadTestsFromTestCase(test.types.TestVerionFunctions) + unittest.TextTestRunner(verbosity=2).run(suite) + diff --git a/stem/types.py b/stem/types.py new file mode 100644 index 0000000..0f7030b --- /dev/null +++ b/stem/types.py @@ -0,0 +1,99 @@ +""" +Classes for miscellaneous tor object. This includes... + +types.Version - Tor versioning information. + * get_version(versionStr) + Converts a version string to a types.Version instance. +""" + +import re + +class Version: + """ + Comparable tor version, as per the 'new version' of the version-spec... + https://gitweb.torproject.org/torspec.git/blob/HEAD:/version-spec.txt + + Attributes: + major (int) - major version + minor (int) - minor version + micro (int) - micro version + patch (int) - optional patch level (None if undefined) + status (str) - optional status tag without the preceding dash such as + 'alpha', 'beta-dev', etc (None if undefined) + """ + + def __init__(self, major, minor, micro, patch = None, status = None): + self.major = major + self.minor = minor + self.micro = micro + self.patch = patch + self.status = status + + def __str__(self): + """ + Provides the normal representation for the version, for instance: + "0.2.2.23-alpha" + """ + + suffix = "" + + if self.patch: + suffix += ".%i" % self.patch + + if self.status: + suffix += "-%s" % self.status + + return "%i.%i.%i%s" % (self.major, self.minor, self.micro, suffix) + + def __cmp__(self, other): + """ + Simple comparision of versions. An undefined patch level is treated as zero + and status tags are compared lexically (as per the version spec). + """ + + if not isinstance(other, Version): + raise ValueError("types.Version can only be compared with other Version instances") + + for attr in ("major", "minor", "micro", "patch"): + myVersion = max(0, self.__dict__[attr]) + otherVersion = max(0, other.__dict__[attr]) + + if myVersion > otherVersion: return 1 + elif myVersion < otherVersion: return -1 + + myStatus = self.status if self.status else "" + otherStatus = other.status if other.status else "" + + return cmp(myStatus, otherStatus) + +def get_version(versionStr): + """ + Parses a version string, providing back a types.Version instance. + + Arguments: + versionStr (str) - string representing a tor version (ex. "0.2.2.23-alpha") + + Returns: + types.Version instance + + Throws: + ValueError if input isn't a valid tor version + """ + + if not isinstance(versionStr, str): + raise ValueError("argument is not a string") + + m = re.match(r'^([0-9]+).([0-9]+).([0-9]+)(.[0-9]+)?(-\S*)?$', versionStr) + + if m: + major, minor, micro, patch, status = m.groups() + + # The patch and status matches are optional (may be None) and have an extra + # proceeding period or dash if they exist. Stripping those off. + + if patch: patch = int(patch[1:]) + if status: status = status[1:] + + return Version(int(major), int(minor), int(micro), patch, status) + else: raise ValueError("'%s' isn't a properly formatted tor version" % versionStr) + diff --git a/stem/types.pyc b/stem/types.pyc new file mode 100644 index 0000000..27a5bb1 Binary files /dev/null and b/stem/types.pyc differ diff --git a/test/__init__.py b/test/__init__.py new file mode 100644 index 0000000..6d651bb --- /dev/null +++ b/test/__init__.py @@ -0,0 +1,6 @@ +""" +Unit and integration tests for the stem library. +""" + +__all__ = ["version"] + diff --git a/test/__init__.pyc b/test/__init__.pyc new file mode 100644 index 0000000..6b41a10 Binary files /dev/null and b/test/__init__.pyc differ diff --git a/test/version.py b/test/version.py new file mode 100644 index 0000000..fd32886 --- /dev/null +++ b/test/version.py @@ -0,0 +1,121 @@ +""" +Unit tests for types functions and classes. +""" + +import unittest +import stem.types + +class TestVerionFunctions(unittest.TestCase): + """ + Tests methods and functions related to 'types.Version'. + """ + + def test_parsing(self): + """ + Tests parsing by the 'get_version' function. + """ + + # valid versions with various number of compontents to the version + version = stem.types.get_version("0.1.2.3-tag") + self.assert_versions_match(version, 0, 1, 2, 3, "tag") + + version = stem.types.get_version("0.1.2.3") + self.assert_versions_match(version, 0, 1, 2, 3, None) + + version = stem.types.get_version("0.1.2-tag") + self.assert_versions_match(version, 0, 1, 2, None, "tag") + + version = stem.types.get_version("0.1.2") + self.assert_versions_match(version, 0, 1, 2, None, None) + + # checks an empty tag + version = stem.types.get_version("0.1.2.3-") + self.assert_versions_match(version, 0, 1, 2, 3, "") + + version = stem.types.get_version("0.1.2-") + self.assert_versions_match(version, 0, 1, 2, None, "") + + # checks invalid version strings + self.assertRaises(ValueError, stem.types.get_version, "") + self.assertRaises(ValueError, stem.types.get_version, "1.2.3.4nodash") + self.assertRaises(ValueError, stem.types.get_version, "1.2.3.a") + self.assertRaises(ValueError, stem.types.get_version, "1.2.a.4") + self.assertRaises(ValueError, stem.types.get_version, "12.3") + self.assertRaises(ValueError, stem.types.get_version, "1.-2.3") + + def test_comparison(self): + """ + Tests comparision between Version instances. + """ + + # check for basic incrementing in each portion + self.assert_version_is_greater("1.1.2.3-tag", "0.1.2.3-tag") + self.assert_version_is_greater("0.2.2.3-tag", "0.1.2.3-tag") + self.assert_version_is_greater("0.1.3.3-tag", "0.1.2.3-tag") + self.assert_version_is_greater("0.1.2.4-tag", "0.1.2.3-tag") + self.assert_version_is_greater("0.1.2.3-ugg", "0.1.2.3-tag") + self.assert_version_is_equal("0.1.2.3-tag", "0.1.2.3-tag") + + # checks that a missing patch level equals zero + self.assert_version_is_equal("0.1.2", "0.1.2.0") + self.assert_version_is_equal("0.1.2-tag", "0.1.2.0-tag") + + # checks for missing patch or status + self.assert_version_is_greater("0.1.2.3-tag", "0.1.2.3") + self.assert_version_is_greater("0.1.2.3-tag", "0.1.2-tag") + self.assert_version_is_greater("0.1.2.3-tag", "0.1.2") + + self.assert_version_is_equal("0.1.2.3", "0.1.2.3") + self.assert_version_is_equal("0.1.2", "0.1.2") + + def test_string(self): + """ + Tests the Version -> string conversion. + """ + + # checks conversion with various numbers of arguments + + self.assert_string_matches("0.1.2.3-tag") + self.assert_string_matches("0.1.2.3") + self.assert_string_matches("0.1.2") + + def assert_versions_match(self, version, major, minor, micro, patch, status): + """ + Asserts that the values for a types.Version instance match the given + values. + """ + + self.assertEqual(version.major, major) + self.assertEqual(version.minor, minor) + self.assertEqual(version.micro, micro) + self.assertEqual(version.patch, patch) + self.assertEqual(version.status, status) + + def assert_version_is_greater(self, firstVersion, secondVersion): + """ + Asserts that the parsed version of the first version is greate than the + second (also checking the inverse). + """ + + version1 = stem.types.get_version(firstVersion) + version2 = stem.types.get_version(secondVersion) + self.assertEqual(version1 > version2, True) + self.assertEqual(version1 < version2, False) + + def assert_version_is_equal(self, firstVersion, secondVersion): + """ + Asserts that the parsed version of the first version equals the second. + """ + + version1 = stem.types.get_version(firstVersion) + version2 = stem.types.get_version(secondVersion) + self.assertEqual(version1, version2) + + def assert_string_matches(self, version): + """ + Parses the given version string then checks that its string representation + matches the input. + """ + + self.assertEqual(version, str(stem.types.get_version(version))) + diff --git a/test/version.pyc b/test/version.pyc new file mode 100644 index 0000000..c459fa7 Binary files /dev/null and b/test/version.pyc differ

1 0

r25154: {website} add Tunisian mirror, remove xpdm for being non-responsive. (in website/trunk: . include)
by Andrew Lewman 06 Oct '11

06 Oct '11

Author: phobos Date: 2011-10-06 16:18:04 +0000 (Thu, 06 Oct 2011) New Revision: 25154 Modified: website/trunk/include/mirrors-table.wmi website/trunk/update-mirrors.pl Log: add Tunisian mirror, remove xpdm for being non-responsive. Modified: website/trunk/include/mirrors-table.wmi =================================================================== --- website/trunk/include/mirrors-table.wmi 2011-10-06 01:38:19 UTC (rev 25153) +++ website/trunk/include/mirrors-table.wmi 2011-10-06 16:18:04 UTC (rev 25154) @@ -1,49 +1,49 @@ <tr> - <td>NL</td> + <td>TN</td> <td></td> <td>Up to date</td> <td> - </td> - <td><a href="http://www.torproject.nl/dist/">http</a></td> - <td><a href="http://www.torproject.nl/">http</a></td> - <td><a href="https://www.torproject.nl/dist/">https</a></td> - <td><a href="https://www.torproject.nl/">https</a></td> + <td><a href="http://tor.mirror.tn/dist/">http</a></td> + <td><a href="http://tor.mirror.tn/">http</a></td> <td> - </td> <td> - </td> + <td> - </td> + <td> - </td> </tr> <tr> <td>NL</td> - <td>CCC</td> + <td></td> <td>Up to date</td> <td> - </td> - <td><a href="http://tor.ccc.de/dist/">http</a></td> - <td><a href="http://tor.ccc.de/">http</a></td> + <td><a href="http://www.torproject.nl/dist/">http</a></td> + <td><a href="http://www.torproject.nl/">http</a></td> + <td><a href="https://www.torproject.nl/dist/">https</a></td> + <td><a href="https://www.torproject.nl/">https</a></td> <td> - </td> <td> - </td> - <td> - </td> - <td> - </td> </tr> <tr> - <td>DK</td> + <td>NL</td> - <td>Zentrum der Gesundheit</td> + <td>CCC</td> <td>Up to date</td> <td> - </td> - <td><a href="http://tor.idnr.ws/dist/">http</a></td> - <td><a href="http://tor.idnr.ws/">http</a></td> + <td><a href="http://tor.ccc.de/dist/">http</a></td> + <td><a href="http://tor.ccc.de/">http</a></td> <td> - </td> <td> - </td> <td> - </td> @@ -188,6 +188,23 @@ <tr> + <td>AT</td> + + <td>cyberarmy</td> + + <td>Up to date</td> + + <td> - </td> + <td><a href="http://tor.cyberarmy.at/dist">http</a></td> + <td><a href="http://tor.cyberarmy.at">http</a></td> + <td> - </td> + <td> - </td> + <td> - </td> + <td> - </td> +</tr> + +<tr> + <td>US</td> <td>searchprivate</td> @@ -290,32 +307,32 @@ <tr> - <td>US</td> + <td>IL</td> - <td>Xpdm</td> + <td>Host4site</td> <td>Up to date</td> <td> - </td> - <td><a href="http://torproj.xpdm.us/dist/">http</a></td> - <td><a href="http://torproj.xpdm.us/">http</a></td> - <td><a href="https://torproj.xpdm.us/dist/">https</a></td> - <td><a href="https://torproj.xpdm.us/">https</a></td> + <td><a href="http://mirror.host4site.co.il/torproject.org/dist">http</a></td> + <td><a href="http://mirror.host4site.co.il/torproject.org/">http</a></td> <td> - </td> <td> - </td> + <td> - </td> + <td> - </td> </tr> <tr> - <td>IL</td> + <td>US</td> - <td>Host4site</td> + <td>AskApache</td> <td>Up to date</td> <td> - </td> - <td><a href="http://mirror.host4site.co.il/torproject.org/dist">http</a></td> - <td><a href="http://mirror.host4site.co.il/torproject.org/">http</a></td> + <td><a href="http://tor.askapache.com/dist/">http</a></td> + <td><a href="http://tor.askapache.com/">http</a></td> <td> - </td> <td> - </td> <td> - </td> @@ -392,15 +409,15 @@ <tr> - <td>US</td> + <td>DK</td> - <td>AskApache</td> + <td>Zentrum der Gesundheit</td> <td>Up to date</td> <td> - </td> - <td><a href="http://tor.askapache.com/dist/">http</a></td> - <td><a href="http://tor.askapache.com/">http</a></td> + <td><a href="http://tor.idnr.ws/dist/">http</a></td> + <td><a href="http://tor.idnr.ws/">http</a></td> <td> - </td> <td> - </td> <td> - </td> @@ -511,36 +528,36 @@ <tr> - <td>AT</td> + <td>NL</td> - <td>cyberarmy</td> + <td></td> - <td>Unknown</td> + <td>Up to date</td> <td> - </td> - <td><a href="http://tor.cyberarmy.at/dist">http</a></td> - <td><a href="http://tor.cyberarmy.at">http</a></td> <td> - </td> <td> - </td> + <td><a href="https://www.coevoet.nl/tor/dist">https</a></td> <td> - </td> <td> - </td> + <td> - </td> </tr> <tr> - <td>NL</td> + <td>US</td> - <td></td> + <td>Xpdm</td> - <td>Up to date</td> + <td>Unknown</td> <td> - </td> + <td><a href="http://torproj.xpdm.us/dist/">http</a></td> + <td><a href="http://torproj.xpdm.us/">http</a></td> + <td><a href="https://torproj.xpdm.us/dist/">https</a></td> + <td><a href="https://torproj.xpdm.us/">https</a></td> <td> - </td> <td> - </td> - <td><a href="https://www.coevoet.nl/tor/dist">https</a></td> - <td> - </td> - <td> - </td> - <td> - </td> </tr> <tr> Modified: website/trunk/update-mirrors.pl =================================================================== --- website/trunk/update-mirrors.pl 2011-10-06 01:38:19 UTC (rev 25153) +++ website/trunk/update-mirrors.pl 2011-10-06 16:18:04 UTC (rev 25154) @@ -247,27 +247,6 @@ rsyncDistMirror => "", updateDate => "Unknown", }, - - mirror014 => { - adminContact => "tormaster AT xpdm DOT us", - orgName => "Xpdm", - isoCC => "US", - subRegion => "", - region => "North America", - ipv4 => "True", - ipv6 => "False", - loadBalanced => "Unknown", - httpWebsiteMirror => "http://torproj.xpdm.us/", - httpsWebsiteMirror => "https://torproj.xpdm.us/", - rsyncWebsiteMirror => "", - ftpWebsiteMirror => "", - httpDistMirror => "http://torproj.xpdm.us/dist/", - httpsDistMirror => "https://torproj.xpdm.us/dist/", - rsyncDistMirror => "", - hiddenServiceMirror => "http://h3prhz46uktgm4tt.onion/", - updateDate => "Unknown", - }, - mirror016 => { adminContact => "security AT hostoffice DOT hu", orgName => "Unknown", @@ -714,6 +693,24 @@ httpsDistMirror => "https://www.torproject.nl/dist/", rsyncDistMirror => "", hiddenServiceMirror => "", + }, + mirror045 => { + adminContact => "", + orgName => "", + isoCC => "TN", + subRegion => "", + region => "TN", + ipv4 => "True", + ipv6 => "False", + loadBalanced => "No", + httpWebsiteMirror => "http://tor.mirror.tn/", + httpsWebsiteMirror => "", + rsyncWebsiteMirror => "", + ftpWebsiteMirror => "", + httpDistMirror => "http://tor.mirror.tn/dist/", + httpsDistMirror => "", + rsyncDistMirror => "", + hiddenServiceMirror => "", } );

1 0

[metrics-tasks/master] Add George's v2 detector script (#2718).
by karsten＠torproject.org 06 Oct '11

06 Oct '11

commit 13a1a7911d8a84923cd071160775eccc24d8b47e Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Wed Oct 5 08:37:06 2011 +0200 Add George's v2 detector script (#2718). --- task-2718/.gitignore | 2 + task-2718/detectorv2.py | 531 +++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 533 insertions(+), 0 deletions(-) diff --git a/task-2718/.gitignore b/task-2718/.gitignore index 917bb1b..0c965b5 100644 --- a/task-2718/.gitignore +++ b/task-2718/.gitignore @@ -1,3 +1,5 @@ *.csv *.pdf +*.aux +*.log diff --git a/task-2718/detectorv2.py b/task-2718/detectorv2.py new file mode 100755 index 0000000..5dce711 --- /dev/null +++ b/task-2718/detectorv2.py @@ -0,0 +1,531 @@ +## Copyright (c) 2011 George Danezis <gdane(a)microsoft.com> +## +## All rights reserved. +## +## Redistribution and use in source and binary forms, with or without +## modification, are permitted (subject to the limitations in the +## disclaimer below) provided that the following conditions are met: +## +## * Redistributions of source code must retain the above copyright +## notice, this list of conditions and the following disclaimer. +## +## * Redistributions in binary form must reproduce the above copyright +## notice, this list of conditions and the following disclaimer in the +## documentation and/or other materials provided with the +## distribution. +## +## * Neither the name of <Owner Organization> nor the names of its +## contributors may be used to endorse or promote products derived +## from this software without specific prior written permission. +## +## NO EXPRESS OR IMPLIED LICENSES TO ANY PARTY'S PATENT RIGHTS ARE +## GRANTED BY THIS LICENSE. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT +## HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED +## WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +## MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +## DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE +## LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR +## CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +## SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR +## BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, +## WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE +## OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN +## IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +## +## (Clear BSD license: http://labs.metacarta.com/license-explanation.html#license) + +## This script reads a .csv file of the number of Tor users and finds +## anomalies that might be indicative of censorship. + +# Dep: matplotlib +from pylab import * +import matplotlib + +# Dep: numpy +import numpy + +# Dep: scipy +import scipy.stats +from scipy.stats.distributions import norm +from scipy.stats.distributions import poisson +from scipy.stats.distributions import gamma + +# Std lib +from datetime import date +from datetime import timedelta +import os.path + +import random + +days = ["Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun"] + +# read the .csv file +class torstatstore: + def __init__(self, file_name, DAYS): + self.DAYS = DAYS + f = file(file_name) + country_codes = f.readline() + country_codes = country_codes.strip().split(",") + + store = {} + MAX_INDEX = 0 + for i, line in enumerate(f): + MAX_INDEX += 1 + line_parsed = line.strip().split(",") + for j, (ccode, val) in enumerate(zip(country_codes,line_parsed)): + processed_val = None + if ccode == "date": + try: + year, month, day = int(val[:4]), int(val[5:7]), int(val[8:10]) + processed_val = date(year, month, day) + except Exception, e: + print "Parsing error (ignoring line %s):" % j + print "%s" % val,e + break + + elif val != "NA": + processed_val = int(val) + store[(ccode, i)] = processed_val + + # min and max + date_min = store[("date", 0)] + date_max = store[("date", i)] + + all_dates = [] + d = date_min + dt = timedelta(days=1) + while d <= date_max: + all_dates += [d] + d = d + dt + + # Save for later + self.store = store + self.all_dates = all_dates + self.country_codes = country_codes + self.MAX_INDEX = MAX_INDEX + self.date_min = date_min + self.date_max = date_max + + def get_country_series(self, ccode): + assert ccode in self.country_codes + series = {} + for d in self.all_dates: + series[d] = None + for i in range(self.MAX_INDEX): + series[self.store[("date", i)]] = self.store[(ccode, i)] + sx = [] + for d in self.all_dates: + sx += [series[d]] + return sx[-self.DAYS:] + + def get_dates(self): + return self.all_dates[-self.DAYS:] + + def get_largest(self, number): + exclude = set(["all", "??", "date"]) + l = [(self.store[(c, self.MAX_INDEX-1)], c) for c in self.country_codes if c not in exclude] + l.sort() + l.reverse() + return [c for _, c in l][:number] + + def get_largest_locations(self, number): + l = self.get_largest(number) + res = {} + for ccode in l[:number]: + res[ccode] = self.get_country_series(ccode) + return res + + def get_codes(self): + return self.country_codes + [] + +## Run a particle filter based inference algorithm +## given a data series and a model of traffic over time +def particle_filter_detector(ser1, taps, models): + # particle : (id, rate, censor, last_censor prev_particle) + + # Model paramaters + normal_std_factor = 4 + censorship_std_factor = 7 + censorship_prior_model = 0.01 + change_tap_prior_model = 0.1 + + # Sampling parameters + change_tap_sample = 0.2 + censorship_prior_sample = 0.3 + particle_number = 1000 + mult_particles = 1 + + # Check consistancy once + for t in models: + assert len(ser1) == len(models[t]) + + # Clean up a bit the data + series2 = [] + last = None + first = None + # Process series + for s in ser1: + if s == None: + series2 += [last] + else: + if first == None: + first = s + series2 += [s] + last = s + + series2 = [s if s != None else first for s in series2] + series = series2 + + # Data structures to keep logs + particles = {} + outputlog = [(series[0],series[0])] + + # Initial particles: + particles[0] = [] + G = gamma(max(1,series[0]), 1) + for pi, r in enumerate(G.rvs(particle_number)): + particles[0] += [(pi, r, False, None, 0, random.choice(taps), False)] + + # Now run the sampler for all times + for pi in range(1, len(series)): + assert models != None + assert taps != None + + # Normal distributions from taps and the model standard deviation for normality and censorship + round_models = {} + for ti in taps: + NoCensor = norm(models[ti][pi][0], (models[ti][pi][1] * normal_std_factor)**2) + Censor = norm(models[ti][pi][0], (models[ti][pi][1] * censorship_std_factor)**2) + round_models[ti] = (NoCensor, Censor) + + # Store for expanded pool of particles + temporary_particles = [] + + # Expand the distribution + for p in particles[pi-1]: + p_old, C_old, j = tracebackp(particles, p, pi-1, p[5] - 1) # taps[0] - 1) + + # Serial number of old particle + p_old_num = None + if p_old != None: + p_old_num = p_old[0] + + # Create a number of candidate particles from each previous particle + for _ in range(mult_particles): + + # Sample a new tap for the candidate particle + new_tap = p[5] + if random.random() < change_tap_sample: + new_tap = random.choice(taps) + + # Update this censorship flag + C = False + if random.random() < censorship_prior_sample: + C = True + + # Determine new rate + new_p = None + if p_old == None: + new_p = p[1] # continue as before + if C | C_old: + while new_p == None or new_p < 0: + new_p = p_old[1] * (1 + round_models[new_tap][1].rvs(1)[0]) ## censor models + else: + while new_p == None or new_p < 0: + new_p = p_old[1] * (1 + round_models[new_tap][0].rvs(1)[0]) ## no censor models + + # Build and register new particle + newpi = (None, new_p, C, p[0], pi, new_tap, C | C_old) + temporary_particles += [newpi] + + + # Assign a weight to each sampled candidtae particle + weights = [] + for px in temporary_particles: + wx = 1.0 + + # Adjust weight to observation + if not series[pi] == None: + poisson_prob = poisson.pmf(series[pi], px[1]) + #print poisson_prob, px + wx *= poisson_prob + + # Adjust the probability of censorship + if px[2]: + wx *= censorship_prior_model / censorship_prior_sample + else: + wx *= (1 - censorship_prior_model) / (1 - censorship_prior_sample) + + # Adjust the probability of changing the tap + if px[5] == particles[pi-1][px[3]][5]: + wx *= (1 - change_tap_prior_model) / (((1-change_tap_sample) + change_tap_sample*(1.0 / len(taps)))) + else: + wx *= (change_tap_prior_model) / (1 - (((1-change_tap_sample) + change_tap_sample*(1.0 / len(taps))))) + + weights += [wx] + + weights_sum = sum(weights) + + ## Resample according to weight + particles[pi] = [] + for pid in range(particle_number): + px = samplep(weights, weights_sum, temporary_particles) + px = (pid, px[1], px[2], px[3], px[4], px[5], px[6]) + particles[pi] += [px] + + ## Collect some statistics + + ## stats + Ci = 0 + mean = 0 + for px in particles[pi]: + if px[2]: + Ci += 1 + mean += px[1] + mean = mean / len(particles[pi]) + + # Diversity + Div = len(set([pv[3] for pv in particles[pi]])) + + # Range of values + range_normal = sorted([pn[1] for pn in temporary_particles if not pn[2]]) + Base = range_normal[len(range_normal)/2] + Mn = range_normal[len(range_normal)*1/100] + Mx = range_normal[len(range_normal)*99/100] + outputlog += [(Mn, Mx)] + + # How many are using the censorship model at any time? + censor_model_stat = len([1 for pn in particles[pi] if pn[6]])* 100 / len(particles[pi]) + + # Build histogram of taps + tap_hist = {} + for px in particles[pi]: + tap_hist[px[5]] = tap_hist.get(px[5], 0) + 1 + + print "%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s" % (pi, Ci, mean, series[pi], tap_hist, Base, Mn, Mx, Div, censor_model_stat) + # print " [%s - %s]" % (key_series_point*(1+NoCensor.ppf(0.00001)), key_series_point*(1+NoCensor.ppf(0.99999))) + + return particles, outputlog + +## Get number of censorship particles, particles that use previous censorship models, +## and total number of particles over time. +def get_events(particles): + events = [] + for ps in sorted(particles): + censor_model_stat = len([1 for pn in particles[ps] if pn[6]]) + events += [(len([1 for p in particles[ps] if p[2]]), censor_model_stat, len(particles[ps]))] + return events + +## Make pretty graphs of the data and censorship events +def plotparticles(series, particles, outputlog, labels, xtitle, events): + assert len(xtitle) == 3 + fname, stitle, slegend = xtitle + + font = {'family' : 'Bitstream Vera Sans', + 'weight' : 'normal', + 'size' : 8} + matplotlib.rc('font', **font) + + mmx = max(series) + if mmx == None: + return # There is no data here! + diff = abs(-mmx*0.1 - mmx*1.1) + + ylim( (-mmx*0.1, 1+mmx*1.1) ) + plot(labels, series, linewidth=1.0, label="Users") + + F = gcf() + + wherefill = [] + minc, maxc = [], [] + for mm,mx in outputlog: + wherefill += [not (mm == None and mx == None)] + assert mm <= mx or (mm == None and mx == None) + minc += [mm] + maxc += [mx] + + fill_between(labels, minc, maxc, where=wherefill, color="gray", label="Prediction") + + vdown = [] + vup = [] + active_region_20 = [] + active_region_50 = [] + for i,v in enumerate(series): + if minc[i] == None or maxc[i] == None: + continue + + mean = (minc[i] + maxc[i]) / 2 + + v2 = v + if v2 == None: + v2 = 0 + + if events[i][0] * 100 / events[i][2] > 10: + if v2 <= mean: + vdown += [(labels[i], v2, events[i][0], events[i][2])] + # print vdown[-1] + else: + vup += [(labels[i], v2, events[i][0], events[i][2])] + + active_region_20 += [events[i][1] * 100 / events[i][2] > 20] + active_region_50 += [events[i][1] * 100 / events[i][2] > 50] + + fill_between(labels, -mmx*0.1*ones(len(labels)), (1+mmx*1.1)*ones(len(labels)), where=active_region_20, color="r", alpha=0.15) + fill_between(labels, -mmx*0.1*ones(len(labels)), (1+mmx*1.1)*ones(len(labels)), where=active_region_50, color="r", alpha=0.15) + + x = [p[0] for p in vdown] + y = [p[1] for p in vdown] + s = [20 + p[2]*100 / p[3] for p in vdown] + if len(x) > 0: + scatter(x,y,s=s, marker='v', c='r') + for xi,yi, score, total in vdown: + if 100 * score / total > 10: + text(xi, yi - diff*5 / 100, "%2d%%" % (100 * float(score) / total), color="r") + + x = [p[0] for p in vup] + y = [p[1] for p in vup] + s = [20+ p[2]*100 / p[3] for p in vup] + if len(x) > 0: + scatter(x,y,s=s, marker='^', c='g') + for xi,yi, score, total in vup: + if 100 * score / total > 10: + text(xi, yi+diff*5 / 100, "%2d%%" % (100 * float(score) / total), color="g") + + + legend(loc=2) + + xlabel('Time (days)') + ylabel('Users') + title(stitle) + grid(True) + + + F.set_size_inches(10,5) + F.savefig(fname, format="png", dpi = (150)) + close() + +## Get a particle from a trace at time current_round - delay +def tracebackp(particles, start_particle, current_round, delay): + if current_round - delay < 0: + return None, False, 0 + + j = current_round + this_particle = start_particle + C = False + r = None + # print "-----" + while not j < current_round - delay: + # print this_particle + C |= this_particle[2] # set the censorship flag + j = j-1 + if not (not j < current_round - delay): + break + this_particle = particles[j][this_particle[3]] + assert j+1 == this_particle[4] == current_round - delay + return (this_particle, C, j+1) + +# Sample a number of items according to their weights +def samplep(weights, total, samples): + rx = random.random() * total + stotal = 0.0 + for i,w in enumerate(weights): + stotal += w + if stotal >= rx: + return samples[i] + + assert False + +# Makes an estimate of the rate from sample observations +def infer_sample_rate(series): + seriesNone = series + [] + series = series + [] # we need a fresh copy! + for i,s in enumerate(series): + if seriesNone[i] == None: + series[i] = 0 + if series[i] == 0: + series[i] = 0.001 + rates = list(gamma.rvs(series, 1)) + for i,r in enumerate(rates): + if seriesNone[i] == None or seriesNone[i] == 0: + rates[i] = None + return rates + +# Get (mean, std) for the top-50 series and different day delays (e.g. taps=[1,7]) +def make_daytoday_normal_models(tss, taps): + codes = tss.get_largest_locations(50) + series = {} + L = len(codes.values()[0]) + for c in codes: + series[(c, 0)] = infer_sample_rate(tss.get_country_series(c)) + assert len(series[(c, 0)]) == L + for d in taps: + series[(c, d)] = historic_frac(series[(c, 0)], d) + assert len(series[(c, d)]) == L + + models = {} + for d in taps: + models[d] = [] + for i in range(L): + v = [] + for c in codes: + vi = series[(c, d)][i] + if not vi == None: + v += [vi] + if len(v) > 1: + v.sort() + v = v[len(v)*5/100:len(v)*95/100:] + if (numpy.mean(v) > 10) or (numpy.mean(v) < -10): + # models[d] += [(0.0, 0.02)] + print i, [(numpy.mean(v), numpy.std(v))] + models[d] += [(numpy.mean(v), numpy.std(v))] + else: + models[d] += [(numpy.mean(v), numpy.std(v))] + else: + models[d] += [(0.0, 0.02)] + # print d, i, models[d][-1] # , v[:5] + + return models + +# Get historic fractions of traffic to train models +def historic_frac(rates, delay): + assert delay > 0 + diff= [] + for i,r in enumerate(rates): + if i - delay < 0 or rates[i-delay] == None or rates[i] == None or rates[i-delay] == 0: + diff += [None] + else: + diff += [(rates[i] - rates[i-delay]) / rates[i-delay]] + return diff + +def plot_country(tss, models, taps, country_code, GRAPH_DIR): + series = tss.get_country_series(country_code) + particles, outputlog = particle_filter_detector(series, taps, models) ## Run the inference algorithm + events = get_events(particles) ## Extract events from particles -- % censorship over time + labels = tss.get_dates() + xtitle = (os.path.join(GRAPH_DIR, "%s-censor.png" % country_code), "Tor report for %s" % country_code,"") + plotparticles(series, particles, outputlog, labels, xtitle, events) + + +#def main(): +if True: + # Change these to customize script + ## (Model parameters are still in particle filter function) + CSV_FILE = "direct-users.csv" + GRAPH_DIR = "img2" + DAYS= 4 * 31 + + tss = torstatstore(CSV_FILE, DAYS) + gr = tss.get_country_series('gr') + rates_gr = infer_sample_rate(gr) + # print historic_frac(rates_gr, 1) + + models = make_daytoday_normal_models(tss, [1, 7]) + + # for country_code in tss.get_largest(250): + #country_code = "kr" + for country_code in ["cn", "eg", "ly", "kr", "de", "mm"]: + print country_code + plot_country(tss, models, [1, 7], country_code, GRAPH_DIR) + +#if __name__ == "__main__": +# main()

1 0

r25153: {website} add the torbutton signature link. (website/trunk/torbutton/en)
by Andrew Lewman 06 Oct '11

06 Oct '11

Author: phobos Date: 2011-10-06 01:38:19 +0000 (Thu, 06 Oct 2011) New Revision: 25153 Modified: website/trunk/torbutton/en/index.wml Log: add the torbutton signature link. Modified: website/trunk/torbutton/en/index.wml =================================================================== --- website/trunk/torbutton/en/index.wml 2011-10-05 11:24:10 UTC (rev 25152) +++ website/trunk/torbutton/en/index.wml 2011-10-06 01:38:19 UTC (rev 25153) @@ -94,7 +94,7 @@ <strong>Install Stable:</strong> Click to <a href="https://www.torproject.org/dist/torbutton/torbutton-current.xpi" hash="<version-hash-torbutton>" onclick="return - install(event);">install from this website</a>.<br/> + install(event);">install from this website</a>. Verify the <a href="https://www.torproject.org/dist/torbutton/torbutton-current.xpi.asc">signature</a>.<br/> <!-- <strong>Install Alpha:</strong> Click to <a href="https://www.torproject.org/dist/torbutton/torbutton-current-alpha.xpi"

1 0

[translation/gettor] Update translations for gettor
by translation＠torproject.org 05 Oct '11

05 Oct '11

commit 9867ba074ed4773e5ace7cc491a9f0241ae3faf9 Author: Translation commit bot <translation(a)torproject.org> Date: Wed Oct 5 19:15:11 2011 +0000 Update translations for gettor --- ar/gettor.po | 13 ++++++++++++- 1 files changed, 12 insertions(+), 1 deletions(-) diff --git a/ar/gettor.po b/ar/gettor.po index ebb6996..d2458c4 100644 --- a/ar/gettor.po +++ b/ar/gettor.po @@ -11,7 +11,7 @@ msgstr "" "Project-Id-Version: The Tor Project\n" "Report-Msgid-Bugs-To: https://trac.torproject.org/projects/tor\n" "POT-Creation-Date: 2011-09-10 01:52-0400\n" -"PO-Revision-Date: 2011-10-05 18:44+0000\n" +"PO-Revision-Date: 2011-10-05 18:51+0000\n" "Last-Translator: redeemer <fadi.redeemer.mansour(a)gmail.com>\n" "Language-Team: LANGUAGE <LL(a)li.org>\n" "MIME-Version: 1.0\n" @@ -183,6 +183,10 @@ msgid "" " windows\n" " split" msgstr "" +"ببساطة أضف الكلمة \"split\" لوحدها في سطر مستقل (هذا الجزء مهم)، مثال:\n" +"\n" +" windows\n" +" split" #: lib/gettor/i18n.py:102 msgid "" @@ -210,6 +214,9 @@ msgid "" "a fresh folder before, simply unzip all files in that folder. If you don't\n" "know how to unzip the .z files, please see the UNPACKING THE FILES section." msgstr "" +"2- قم بفك ضغط كل الملفات ذات اللاحقة \".z\". إن قمت بحفظ الملفات في مجلد " +"جديد، قم ببساطة بفك ضغط كل الملفات في ذاك المجلد. إن كنت لا تعرف كيف تفك ضغط" +" الملفات .z، قم بمراجعة الفقرة \"فك ضغط الملفات\"." #: lib/gettor/i18n.py:114 msgid "" @@ -225,6 +232,8 @@ msgid "" "clicking the file ending in \"..split.part01.exe\". This should start the \n" "process automatically." msgstr "" +"4- الآن قم بفك ضغط الأرشيف متعدد الملفات بالضغط المزدوج على الملف ذو اللاحقة" +" \".split.part01.exe\". سيبدأ هذا العملية بشكل آلي." #: lib/gettor/i18n.py:121 msgid "" @@ -245,6 +254,8 @@ msgid "" "SUPPORT\n" "=======" msgstr "" +"الدعم الفني\n" +"======" #: lib/gettor/i18n.py:133 msgid ""

1 0