- tor-commits - lists.torproject.org

[torbutton/master] Reorganize options into their tab groups.
by mikeperry＠torproject.org 04 Apr '11

04 Apr '11

commit ee8c7d2e0366343df2a062d08a73de067dffde58 Author: Mike Perry <mikeperry-git(a)fscked.org> Date: Sun Apr 3 19:09:06 2011 -0700 Reorganize options into their tab groups. --- website/design/design.xml | 277 ++++++++++++++++++++++----------------------- 1 files changed, 137 insertions(+), 140 deletions(-) diff --git a/website/design/design.xml b/website/design/design.xml index e97c07c..403bb3b 100644 --- a/website/design/design.xml +++ b/website/design/design.xml @@ -1110,12 +1110,15 @@ requirements. <sect1> <title>Description of Options</title> - <para>This section provides a detailed description of Torbutton's options. Each option is presented as the string from the preferences window, a summary, the preferences it touches, and the effect this has on the components, chrome, and browser properties.</para> + <sect2> + <title>Proxy Settings</title> + <sect3> <title>Test Settings</title> <para> This button under the Proxy Settings tab provides a way to verify that the @@ -1138,8 +1141,11 @@ callback <function>torbutton_prefs_test_settings()</function> in <ulink url="https://gitweb.torproject.org/torbutton.git/blob_plain/HEAD:/src/chrome/con…">preferences.js</ulink>. </para> + </sect3> </sect2> - <sect2 id="plugins"> + <sect2> + <title>Dynamic Content Settings</title> + <sect3 id="plugins"> <title>Disable plugins on Tor Usage (crucial)</title> <para>Option: <command>extensions.torbutton.no_tor_plugins</command></para> @@ -1223,8 +1229,8 @@ performed by this setting are crucial to satisfying the <link linkend="proxy">Proxy Obedience</link> requirement. </para> -</sect2> -<sect2> +</sect3> +<sect3> <title>Isolate Dynamic Content to Tor State (crucial)</title> <para>Option: <command>extensions.torbutton.isolate_content</command></para> @@ -1272,8 +1278,8 @@ This setting is responsible for satisfying the <link linkend="isolation">Network Isolation</link> requirement. </para> -</sect2> -<sect2 id="jshooks"> +</sect3> +<sect3 id="jshooks"> <title>Hook Dangerous Javascript</title> @@ -1312,8 +1318,8 @@ We are still looking for a workaround as of Torbutton 1.3.2.  </para> -</sect2> -<sect2> +</sect3> +<sect3> <title>Resize windows to multiples of 50px during Tor usage (recommended)</title> <para>Option: <command>extensions.torbutton.resize_windows</command></para> @@ -1357,8 +1363,31 @@ infer toolbar size/presence by the distance to the nearest 50 pixel roundoff). This setting helps to meet the <link linkend="setpreservation">Anonymity Set Preservation</link> requirements. </para> -</sect2> -<sect2> +</sect3> +<sect3> + +<title>Disable Search Suggestions during Tor (recommended)</title> + + <para>Option: <command>extensions.torbutton.no_search</command></para> + +<para> +This setting causes Torbutton to disable <ulink +url="http://kb.mozillazine.org/Browser.search.suggest.enabled"><command>browser.search.suggest.enabled</command></ulink> +during Tor usage. +This governs if you get Google search suggestions during Tor +usage. Your Google cookie is transmitted with google search suggestions, hence +this is recommended to be disabled. + +</para> +<para> +While this setting doesn't satisfy any Torbutton requirements, the fact that +cookies are transmitted for partially typed queries does not seem desirable +for Tor usage. +</para> +</sect3> + + +<sect3> <title>Disable Updates During Tor</title> <para>Option: <command>extensions.torbutton.no_updates</command></para> @@ -1377,8 +1406,8 @@ update settings</ulink> during Tor This setting satisfies the <link linkend="updates">Update Safety</link> requirement. </para> -</sect2> -<sect2> +</sect3> +<sect3> <title>Redirect Torbutton Updates Via Tor (recommended)</title> <para>Option: <command>extensions.torbutton.update_torbutton_via_tor</command></para> @@ -1395,30 +1424,9 @@ help censored users meet the <link linkend="undiscoverability">Tor Undiscoverability</link> requirement. </para> -</sect2> - -<sect2> - -<title>Disable Search Suggestions during Tor (recommended)</title> - - <para>Option: <command>extensions.torbutton.no_search</command></para> - -<para> -This setting causes Torbutton to disable <ulink -url="http://kb.mozillazine.org/Browser.search.suggest.enabled"><command>browser.search.suggest.enabled</command></ulink> -during Tor usage. -This governs if you get Google search suggestions during Tor -usage. Your Google cookie is transmitted with google search suggestions, hence -this is recommended to be disabled. +</sect3> -</para> -<para> -While this setting doesn't satisfy any Torbutton requirements, the fact that -cookies are transmitted for partially typed queries does not seem desirable -for Tor usage. -</para> -</sect2> -<sect2> +<sect3> <title>Disable livemarks updates during Tor usage (recommended)</title> <para>Option: <simplelist> @@ -1446,8 +1454,8 @@ Isolation</link> and <link linkend="setpreservation">Anonymity Set Preservation</link> requirements. </para> -</sect2> -<sect2> +</sect3> +<sect3> <title>Block Tor/Non-Tor access to network from file:// urls (recommended)</title> <para>Options: <simplelist> @@ -1478,13 +1486,13 @@ operations in opposite Tor states. Also, allowing pages to submit arbitrary files to arbitrary sites just generally seems like a bad idea. </para> -</sect2> +</sect3>   -<sect2> +<sect3> <title>Close all Tor/Non-Tor tabs and windows on toggle (optional)</title> @@ -1520,9 +1528,11 @@ While this setting doesn't satisfy any Torbutton requirements, the fact that cookies are transmitted for partially typed queries does not seem desirable for Tor usage. </para> -</sect2> - -<sect2> +</sect3> + </sect2> + <sect2> + <title>History and Forms Settings</title> +<sect3> <title>Isolate Access to History navigation to Tor state (crucial)</title> <para>Option: <command>extensions.torbutton.block_js_history</command></para> <para> @@ -1560,10 +1570,10 @@ Separation</link> and (until Bug 409737 is fixed) <link linkend="isolation">Netw requirements. </para> -</sect2> +</sect3> -<sect2> +<sect3> <title>History Access Settings</title> <para>Options: @@ -1601,8 +1611,8 @@ linkend="state">State Separation</link> and <link linkend="disk">Disk Avoidance</link> requirements. </para> -</sect2> -<sect2> +</sect3> +<sect3> <title>Clear History During Tor Toggle (optional)</title> @@ -1619,9 +1629,8 @@ This setting is an optional way to help satisfy the <link linkend="state">State Separation</link> requirement. </para> -</sect2> -<sect2> - +</sect3> +<sect3> <title>Block Password+Form saving during Tor/Non-Tor</title> <para>Options: @@ -1644,8 +1653,11 @@ linkend="state">State Separation</link> and <link linkend="disk">Disk Avoidance</link> requirements. </para> -</sect2> -<sect2> +</sect3> + </sect2> + <sect2> + <title>Cache Settings</title> +<sect3> <title>Block Tor disk cache and clear all cache on Tor Toggle</title> <para>Option: <command>extensions.torbutton.clear_cache</command> @@ -1663,8 +1675,8 @@ linkend="state">State Separation</link> and <link linkend="disk">Disk Avoidance</link> requirements. </para> -</sect2> -<sect2> +</sect3> +<sect3> <title>Block disk and memory cache during Tor</title> <para>Option: <command>extensions.torbutton.block_cache</command></para> @@ -1683,8 +1695,11 @@ linkend="state">State Separation</link> and <link linkend="disk">Disk Avoidance</link> requirements. </para> -</sect2> -<sect2> +</sect3> + </sect2> + <sect2> + <title>Cookie and Auth Settings</title> +<sect3> <title>Clear Cookies on Tor Toggle</title> <para>Option: <command>extensions.torbutton.clear_cookies</command> @@ -1706,8 +1721,8 @@ linkend="state">State Separation</link> and <link linkend="disk">Disk Avoidance</link> requirements. </para> -</sect2> -<sect2> +</sect3> +<sect3> <title>Store Non-Tor cookies in a protected jar</title> @@ -1736,8 +1751,8 @@ linkend="disk">Disk Avoidance</link> requirements. </para> -</sect2> -<sect2> +</sect3> +<sect3> <title>Store both Non-Tor and Tor cookies in a protected jar (dangerous)</title> @@ -1757,22 +1772,22 @@ linkend="state">State Separation</link> requirement. </para> -</sect2> +</sect3>  -<sect2> +<sect3> <title>Manage My Own Cookies (dangerous)</title> <para>Options: None</para> <para>This setting disables all Torbutton cookie handling by setting the above cookie prefs all to false.</para> -</sect2> -<sect2> +</sect3> +<sect3> -<sect2> +<sect3> <title>Do not write Tor/Non-Tor cookies to disk</title> <para>Options: <simplelist> @@ -1796,7 +1811,7 @@ This allows Torbutton to provide an option to preserve a user's cookies while still satisfying the <link linkend="disk">Disk Avoidance</link> requirement. </para> -</sect2> +</sect3> <title>Disable DOM Storage during Tor usage (crucial)</title> @@ -1816,9 +1831,9 @@ This setting helps to satisfy the <link linkend="state">State Separation</link> requirement. </para> -</sect2> +</sect3> -<sect2> +<sect3> <title>Clear HTTP Auth on Tor Toggle (recommended)</title> <para>Option: <command>extensions.torbutton.clear_http_auth</command> </para> @@ -1833,63 +1848,14 @@ every time Tor is toggled. This setting helps to satisfy the <link linkend="state">State Separation</link> requirement. </para> -</sect2> - - -<sect2> - - <title>Clear cookies on Tor/Non-Tor shutdown</title> - -<para>Option: <command>extensions.torbutton.shutdown_method</command> - </para> - -<para> This option variable can actually take 3 values: 0, 1, and 2. 0 means no -cookie clearing, 1 means clear only during Tor-enabled shutdown, and 2 means -clear for both Tor and Non-Tor shutdown. When set to 1 or 2, Torbutton listens -for the <ulink -url="http://developer.mozilla.org/en/docs/Observer_Notifications#Application_shu…">quit-application-granted</ulink> event in -<function>https://gitweb.torproject.org/torbutton.git/blob_plain/HEAD:/src/components…</function> and use <ulink -url="https://gitweb.torproject.org/torbutton.git/blob_plain/HEAD:/src/components…">@torproject.org/cookie-jar-selector;2</ulink> -to clear out all cookies and all cookie jars upon shutdown. </para> -<para> -This setting helps to satisfy the <link -linkend="state">State Separation</link> requirement. -</para> - - -</sect2> -<sect2> - - <title>Reload cookie jar/clear cookies on Firefox crash</title> - <para>Options: - <simplelist> - <member><command>extensions.torbutton.reload_crashed_jar</command></member> - <member><command>extensions.torbutton.crashed</command></member> - </simplelist> - </para> - - <para>This is no longer a user visible option, and is enabled by default. In -the event of a crash, the Torbutton <ulink -url="https://gitweb.torproject.org/torbutton.git/blob_plain/HEAD:/src/components…">components/crash-observer.js</ulink> - component will notify the Chrome (via the - <command>extensions.torbutton.crashed</command> pref and a <ulink -url="https://developer.mozilla.org/en/NsIPrefBranch2#addObserver.28.29">pref -observer</ulink> in -the chrome that listens for this update), and Torbutton will load the - correct jar for the current Tor state via the <ulink -url="https://gitweb.torproject.org/torbutton.git/blob_plain/HEAD:/src/components…">@torproject.org/cookie-jar-selector;2</ulink> - component.</para> - -<para> -This setting helps to satisfy the <link -linkend="state">State Separation</link> requirement in the event of Firefox -crashes. -</para> - -</sect2> - +</sect3> + </sect2> + <sect2> + <title>Startup Settings</title> + +  -<sect2> +<sect3> <title>On crash recovery or session restored startup, restore via: Tor, Non-Tor</title> <para>Options: <simplelist> @@ -1916,9 +1882,10 @@ requirement in the event of Firefox crashes by ensuring all cookies, settings and saved sessions are reloaded from a fixed Tor state. </para> -</sect2> +</sect3> -<sect2> + +<sect3> <title>On normal startup, set state to: Tor, Non-Tor, Shutdown State</title> <para>Options: @@ -1940,9 +1907,9 @@ Firefox exit and checks this value as well during startup. </para> -</sect2> +</sect3> -<sect2> +<sect3> <title>Prevent session store from saving Non-Tor/Tor-loaded tabs</title> <para>Options: @@ -1965,9 +1932,36 @@ crashes. </para> -</sect2> +</sect3> + </sect2> + <sect2> + <title>Shutdown Settings</title> +<sect3> + + <title>Clear cookies on Tor/Non-Tor shutdown</title> + +<para>Option: <command>extensions.torbutton.shutdown_method</command> + </para> + +<para> This option variable can actually take 3 values: 0, 1, and 2. 0 means no +cookie clearing, 1 means clear only during Tor-enabled shutdown, and 2 means +clear for both Tor and Non-Tor shutdown. When set to 1 or 2, Torbutton listens +for the <ulink +url="http://developer.mozilla.org/en/docs/Observer_Notifications#Application_shu…">quit-application-granted</ulink> event in +<function>https://gitweb.torproject.org/torbutton.git/blob_plain/HEAD:/src/components…</function> and use <ulink +url="https://gitweb.torproject.org/torbutton.git/blob_plain/HEAD:/src/components…">@torproject.org/cookie-jar-selector;2</ulink> +to clear out all cookies and all cookie jars upon shutdown. </para> +<para> +This setting helps to satisfy the <link +linkend="state">State Separation</link> requirement. +</para> -<sect2> + +</sect3> + </sect2> + <sect2> + <title>Header Settings</title> +<sect3> <title>Set user agent during Tor usage (crucial)</title> <para>Options: @@ -2016,8 +2010,8 @@ linkend="setpreservation">Anonymity Set Preservation</link> requirement. </para> -</sect2> -<sect2> +</sect3> +<sect3> <title>Spoof US English Browser</title> <para>Options: @@ -2043,15 +2037,15 @@ linkend="setpreservation">Anonymity Set Preservation</link> and <link linkend="location">Location Neutrality</link> requirements. </para> -</sect2> -<sect2> +</sect3> +<sect3> <title>Don't send referrer during Tor Usage</title> <para>Option: <command>extensions.torbutton.disable_referer</command> </para> -<para> +<para> This option causes Torbutton to set <ulink url="http://kb.mozillazine.org/Network.http.sendSecureXSiteReferrer">network.http.sendSecureXSiteReferrer</ulink> and <ulink @@ -2061,11 +2055,11 @@ url="http://kb.mozillazine.org/Network.http.sendRefererHeader">network.http.send This setting also does not directly satisfy any Torbutton requirement, but some may desire to mask their referrer for general privacy concerns. </para> -</sect2> +</sect3>  -<sect2> +<sect3> <title>Strip platform and language off of Google Search Box queries</title> <para>Option: <command>extensions.torbutton.fix_google_srch</command> @@ -2084,9 +2078,9 @@ platform information. This setting strips off that info while Tor is enabled. This setting helps Torbutton to fulfill its <link linkend="setpreservation">Anonymity Set Preservation</link> requirement. </para> -</sect2> +</sect3> -<sect2> +<sect3> <title>Automatically use an alternate search engine when presented with a Google Captcha</title> @@ -2125,9 +2119,9 @@ encoded in the preferences <command>extensions.torbutton.redir_url.[1-5]</command>. </para> -</sect2> +</sect3> -<sect2> +<sect3> <title>Store SSL/CA Certs in separate jars for Tor/Non-Tor (recommended)</title> @@ -2170,6 +2164,9 @@ is currently not exposed via the preferences UI. </para> +</sect3> + + </sect2> </sect1>

1 0

[torbutton/master] Reorganize, document observers, and fix gitweb urls.
by mikeperry＠torproject.org 04 Apr '11

04 Apr '11

commit 6adaae4b281932e73a923d9d56c72de235dcf8a8 Author: Mike Perry <mikeperry-git(a)fscked.org> Date: Sun Apr 3 18:26:07 2011 -0700 Reorganize, document observers, and fix gitweb urls. I probably should have broken these into separate commits. Too late now. --- website/design/design.xml | 595 +++++++++++++++++++++++++++------------------ 1 files changed, 356 insertions(+), 239 deletions(-) diff --git a/website/design/design.xml b/website/design/design.xml index b137caf..e97c07c 100644 --- a/website/design/design.xml +++ b/website/design/design.xml @@ -11,7 +11,7 @@ <address><email>mikeperry.fscked/org</email></address> </affiliation> </author> - <pubdate>Mar 25 2011</pubdate> + <pubdate>Apr 3 2011</pubdate> </articleinfo> <sect1> @@ -19,7 +19,7 @@ <para> This document describes the goals, operation, and testing procedures of the -Torbutton Firefox extension. It is current as of Torbutton 1.2.5. +Torbutton Firefox extension. It is current as of Torbutton 1.3.2. </para> <sect2 id="adversary"> @@ -219,6 +219,7 @@ adserver-class adversaries. </para> </listitem> + <listitem id="fingerprinting"><command>Fingerprint users based on browser attributes</command> <para> @@ -234,7 +235,11 @@ resolution information available in the <ulink url="http://developer.mozilla.org/en/docs/DOM:window">window</ulink> and <ulink url="http://developer.mozilla.org/en/docs/DOM:window.screen">window.screen</ulink> -objects. Browser window resolution information provides something like +objects. + + + +Browser window resolution information provides something like (1280-640)*(1024-480)=348160 different anonymity sets. Desktop resolution information contributes about another factor of 5 (for about 5 resolutions in typical use). In addition, the dimensions and position of the desktop taskbar @@ -252,14 +257,25 @@ information alone. </para> <para> -Of course, this space is non-uniform and prone to incremental changes. -However, if a bit vector space consisting of the above extracted attributes -were used instead of the hash approach from <ulink -url="http://mandark.fr/0x000000/articles/Total_Recall_On_Firefox..html">The Hacker -Webzine article above</ulink>, minor changes in browser window resolution will -no longer generate totally new identifiers. +Of course, this space is non-uniform in user density and prone to incremental +changes. The <ulink +url="https://wiki.mozilla.org/Fingerprinting#Data">Panopticlick study +done</ulink> by the EFF attempts to measure the actual entropy - the number of +identifying bits of information encoded in browser properties. Their result +data is definitely useful, and the metric is probably the appropriate one for +determining how identifying a particular browser property is. However, some +quirks of their study means that they do not extract as much information as +they could from display information: they only use desktop resolution (which +Torbutton reports as the window resolution) and do not attempt to infer the +size of toolbars. </para> + </listitem> <listitem><command>Remotely or locally exploit browser and/or OS</command> @@ -377,7 +393,7 @@ is called <ulink url="http://developer.mozilla.org/en/docs/XUL_Reference">XUL</ulink>.</para> </sect2> </sect1> -<sect1> +<sect1 id="components"> <title>Components</title> <para> @@ -387,38 +403,13 @@ services to other pieces of the extension. </para> - <sect2> + <sect2 id="hookedxpcom"> <title>Hooked Components</title> <para>Torbutton makes extensive use of Contract ID hooking, and implements some of its own standalone components as well. Let's discuss the hooked components first.</para> -<sect3 id="sessionstore"> - <title><ulink -url="http://developer.mozilla.org/en/docs/nsISessionStore">@mozilla.org/browser/sessionstore;1</ulink> - -<ulink -url="https://git.torproject.org/checkout/torbutton/master/src/components/nsSessi…">components/nsSessionStore36.js</ulink></title> - -<para>These components address the <link linkend="disk">Disk Avoidance</link> -requirements of Torbutton. As stated in the requirements, Torbutton needs to -prevent Tor tabs from being written to disk by the Firefox session store for a -number of reasons, primary among them is the fact that Firefox can crash at -any time, and a restart can cause you to fetch tabs in the incorrect Tor -state.</para> - -<para>These components illustrate a complication with Firefox hooking: you can -only hook member functions of a class if they are published in an -interface that the class implements. Unfortunately, the sessionstore has no -published interface that is amenable to disabling the writing out of Tor tabs -in specific. As such, Torbutton had to include the <emphasis>entire</emphasis> -nsSessionStore from both Firefox 2.0, 3.0, 3.5 and 3.6 -with a couple of modifications to prevent tabs that were loaded with Tor -enabled from being written to disk, and some version detection code to -determine which component to load. The <ulink -url="https://git.torproject.org/checkout/torbutton/master/src/components/nsSessi…">diff against the original session -store</ulink> is included in the git repository.</para> -</sect3> <sect3 id="appblocker"> <title><ulink url="http://www.oxymoronical.com/experiments/xpcomref/applications/Firefox/3.5/c…">@mozilla.org/uriloader/external-protocol-service;1 @@ -426,7 +417,7 @@ url="http://www.oxymoronical.com/experiments/xpcomref/applications/Firefox/3.5/c url="http://www.oxymoronical.com/experiments/xpcomref/applications/Firefox/3.5/c…">@mozilla.org/uriloader/external-helper-app-service;1</ulink>, and <ulink url="http://www.oxymoronical.com/experiments/xpcomref/applications/Firefox/3.5/c…">@mozilla.org/mime;1</ulink> - <ulink - url="https://git.torproject.org/checkout/torbutton/master/src/components/externa…">components/external-app-blocker.js</ulink></title> + url="https://gitweb.torproject.org/torbutton.git/blob_plain/HEAD:/src/components…">components/external-app-blocker.js</ulink></title> <para> Due to <link linkend="FirefoxBugs">Firefox Bug</link> <ulink url="https://bugzilla.mozilla.org/show_bug.cgi?id=440892">440892</ulink> allowing Firefox 3.x to automatically launch some @@ -440,37 +431,9 @@ Obedience</link> Requirement. </para> </sect3> <sect3> -<title><ulink -url="http://lxr.mozilla.org/seamonkey/source/browser/components/sessionstore/src…">@mozilla.org/browser/sessionstartup;1</ulink> - - <ulink -url="https://git.torproject.org/checkout/torbutton/master/src/components/crash-o…">components/crash-observer.js</ulink></title> - -<para>This component wraps the Firefox Session Startup component that is in -charge of <ulink -url="http://developer.mozilla.org/en/docs/Session_store_API">restoring saved -sessions</ulink>. The wrapper's only job is to intercept the -<function>doRestore()</function> function, which is called by Firefox if it is determined that the -browser crashed and the session needs to be restored. The wrapper notifies the -Torbutton chrome that the browser crashed by setting the pref -<command>extensions.torbutton.crashed</command>, or that it is a normal -startup via the pref <command>extensions.torbutton.noncrashed</command>. The Torbutton Chrome <ulink -url="https://developer.mozilla.org/en/NsIPrefBranch2#addObserver.28.29">listens for a -preference change</ulink> for this value and then does the appropriate cleanup. This -includes setting the Tor state to the one the user selected for crash recovery -in the preferences window (<command>extensions.torbutton.restore_tor</command>), and -restoring cookies for the corresponding cookie jar, if it exists.</para> - -<para>By performing this notification, this component assists in the -<link linkend="proxy">Proxy Obedience</link>, and <link -linkend="isolation">Network Isolation</link> requirements. -</para> - - -</sect3> -<sect3> <title><ulink url="http://www.oxymoronical.com/experiments/xpcomref/applications/Firefox/3.5/c…">@mozilla.org/browser/global-history;2</ulink> - <ulink - url="https://git.torproject.org/checkout/torbutton/master/src/components/ignore-…">components/ignore-history.js</ulink></title> + url="https://gitweb.torproject.org/torbutton.git/blob_plain/HEAD:/src/components…">components/ignore-history.js</ulink></title> <para>This component was contributed by <ulink url="http://www.collinjackson.com/">Collin Jackson</ulink> as a method for defeating @@ -486,14 +449,18 @@ preferences. </para> <para> This component helps satisfy the <link linkend="state">State Separation</link> -and <link linkend="disk">Disk Avoidance</link> requirements of Torbutton. +and <link linkend="disk">Disk Avoidance</link> requirements of Torbutton. It +is only needed for Firefox 3.x. On Firefox 4, we omit this component in favor +of the <ulink +url="https://developer.mozilla.org/en/CSS/Privacy_and_the_%3avisited_selector">built-in +history protections</ulink>. </para> </sect3> <sect3 id="livemarks"> <title><ulink url="http://www.oxymoronical.com/experiments/xpcomref/applications/Firefox/3.5/c…">@mozilla.org/browser/livemark-service;2</ulink> - <ulink - url="https://git.torproject.org/checkout/torbutton/master/src/components/block-l…">components/block-livemarks.js</ulink></title> + url="https://gitweb.torproject.org/torbutton.git/blob_plain/HEAD:/src/components…">components/block-livemarks.js</ulink></title> <para> The <ulink @@ -518,18 +485,19 @@ Preservation</link> requirements. extension. These components do not hook any interfaces, nor are they used anywhere besides Torbutton itself.</para> -<sect3> +<sect3 id="cookiejar"> <title><ulink -url="https://git.torproject.org/checkout/torbutton/master/src/components/cookie-…">@torproject.org/cookie-jar-selector;2 +url="https://gitweb.torproject.org/torbutton.git/blob_plain/HEAD:/src/components…">@torproject.org/cookie-jar-selector;2 - components/cookie-jar-selector.js</ulink></title> <para>The cookie jar selector (also based on code from <ulink url="http://www.collinjackson.com/">Collin Jackson</ulink>) is used by the Torbutton chrome to switch between -Tor and Non-Tor cookies. Its operations are simple: sync cookies to disk, then -move the current cookies.txt file to the appropriate backup location -(cookies-tor.txt or cookies-nontor.txt), and then moving the other cookie jar -into place.</para> +Tor and Non-Tor cookies. It stores an XML representation of the current +cookie state in memory and/or on disk. When Tor is toggled, it syncs the +current cookies to this XML store, and then loads the cookies for the other +state from the XML store. +</para> <para> This component helps to address the <link linkend="state">State @@ -539,7 +507,7 @@ Isolation</link> requirement of Torbutton. </sect3> <sect3> <title><ulink -url="https://git.torproject.org/checkout/torbutton/master/src/components/torbutt…">@torproject.org/torbutton-logger;1 +url="https://gitweb.torproject.org/torbutton.git/blob_plain/HEAD:/src/components…">@torproject.org/torbutton-logger;1 - components/torbutton-logger.js</ulink></title> <para>The torbutton logger component allows on-the-fly redirection of torbutton @@ -554,7 +522,7 @@ change the loglevel on the fly by changing <sect3 id="windowmapper"> <title><ulink -url="https://git.torproject.org/checkout/torbutton/master/src/components/window-…">@torproject.org/content-window-mapper;1 +url="https://gitweb.torproject.org/torbutton.git/blob_plain/HEAD:/src/components…">@torproject.org/content-window-mapper;1 - components/window-mapper.js</ulink></title> <para>Torbutton tags Firefox <ulink @@ -573,9 +541,43 @@ and page loading in general can generate hundreds of these lookups, this result is cached inside the component. </para> </sect3> +<sect3> + <title><ulink +url="https://gitweb.torproject.org/torbutton.git/blob_plain/HEAD:/src/components…">@torproject.org/crash-observer;1</ulink></title> + <para> + +This component detects when Firefox crashes by altering Firefox prefs during +runtime and checking for the same values at startup. It <ulink +url="https://developer.mozilla.org/en/XPCOM_Interface_Reference/nsIPrefService#s…">synchronizes +the preference service</ulink> to ensure the altered prefs are written to disk +immediately. + + </para> +</sect3> +<sect3> + <title><ulink +url="https://gitweb.torproject.org/torbutton.git/blob_plain/HEAD:/src/components…">@torproject.org/torbutton-ss-blocker;1</ulink></title> + <para> + +This component subscribes to the Firefox +<ulink +url="https://developer.mozilla.org/en/Observer_Notifications#Session_Store">sessionstore-state-write</ulink> +observer event to filter out URLs from tabs loaded during Tor, to prevent them +from being written to disk. This is a rather expensive operation that involves +potentially very large JSON evaluations and object tree traversals, but it +preferable to replacing the Firefox session store with our own implementation, +which is what was done in years past. + + </para> +</sect3> + + + <sect3 id="contentpolicy"> <title><ulink -url="https://git.torproject.org/checkout/torbutton/master/src/components/cssbloc…">@torproject.org/cssblocker;1 +url="https://gitweb.torproject.org/torbutton.git/blob_plain/HEAD:/src/components…">@torproject.org/cssblocker;1 - components/cssblocker.js</ulink></title> <para>This is a key component to Torbutton's security measures. When Tor is @@ -598,7 +600,8 @@ linkend="isolation">Network Isolation</link> requirements of Torbutton. <para>In addition, the content policy also blocks website javascript from -<ulink url="http://pseudo-flaw.net/content/tor/torbutton/">querying for +<ulink +url="http://webdevwonders.com/detecting-firefox-add-ons/">querying for versions and existence of extension chrome</ulink> while Tor is enabled, and also masks the presence of Torbutton to website javascript while Tor is disabled. </para> @@ -608,7 +611,7 @@ disabled. </para> Finally, some of the work that logically belongs to the content policy is instead handled by the <command>torbutton_http_observer</command> and <command>torbutton_weblistener</command> in <ulink -url="https://git.torproject.org/checkout/torbutton/master/src/chrome/content/tor…">torbutton.js</ulink>. These two objects handle blocking of +url="https://gitweb.torproject.org/torbutton.git/blob_plain/HEAD:/src/chrome/con…">torbutton.js</ulink>. These two objects handle blocking of Firefox 3 favicon loads, popups, and full page plugins, which for whatever reason are not passed to the Firefox content policy itself (see Firefox Bugs <ulink @@ -674,21 +677,113 @@ Torbutton.</para> <title>Chrome</title> <para>The chrome is where all the torbutton graphical elements and windows are -located. Each window is described as an <ulink +located. </para> +<sect2> + <title>XUL Windows and Overlays</title> +<para> +Each window is described as an <ulink url="http://developer.mozilla.org/en/docs/XUL_Reference">XML file</ulink>, with zero or more Javascript files attached. The scope of these Javascript files is their containing -window.</para> +window. XUL files that add new elements and script to existing Firefox windows +are called overlays.</para> -<sect2 id="browseroverlay"> +<sect3 id="browseroverlay"> <title>Browser Overlay - <ulink -url="https://git.torproject.org/checkout/torbutton/master/src/chrome/content/tor…">torbutton.xul</ulink></title> +url="https://gitweb.torproject.org/torbutton.git/blob_plain/HEAD:/src/chrome/con…">torbutton.xul</ulink></title> <para>The browser overlay, torbutton.xul, defines the toolbar button, the status bar, and events for toggling the button. The overlay code is in <ulink -url="https://git.torproject.org/checkout/torbutton/master/src/chrome/content/tor…">chrome/content/torbutton.js</ulink>. +url="https://gitweb.torproject.org/torbutton.git/blob_plain/HEAD:/src/chrome/con…">chrome/content/torbutton.js</ulink>. It contains event handlers for preference update, shutdown, upgrade, and location change events.</para> +</sect3> +<sect3> + <title>Preferences Window - <ulink +url="https://gitweb.torproject.org/torbutton.git/blob_plain/HEAD:/src/chrome/con…">preferences.xul</ulink></title> + +<para>The preferences window of course lays out the Torbutton preferences, with +handlers located in <ulink +url="https://gitweb.torproject.org/torbutton.git/blob_plain/HEAD:/src/chrome/con…">chrome/content/preferences.js</ulink>.</para> +</sect3> +<sect3> + <title>Other Windows</title> + +<para>There are additional windows that describe popups for right clicking on +the status bar, the toolbutton, and the about page.</para> + +</sect3> +</sect2> +<sect2> + <title>Major Chrome Observers</title> + <para> +In addition to the <link linkend="components">components described +above</link>, Torbutton also instantiates several observers in the browser +overlay window. These mostly grew due to scoping convenience, and many should +probably be relocated into their own components. + </para> + <orderedlist> + <listitem><command>torbutton_window_pref_observer</command> + <para> +This is an observer that listens for Torbutton state changes, for the purposes +of updating the Torbutton button graphic as the Tor state changes. + </para> + </listitem> + + <listitem><command>torbutton_unique_pref_observer</command> + <para> + +This is an observer that only runs in one window, called the main window. It +listens for changes to all of the Torbutton preferences, as well as Torbutton +controlled Firefox preferences. It is what carries out the toggle path when +the proxy settings change. When the main window is closed, the +torbutton_close_window event handler runs to dub a new window the "main +window". + + </para> + </listitem> + + <listitem><command>tbHistoryListener</command> + <para> +The tbHistoryListener exists to prevent client window Javascript from +interacting with window.history to forcibly navigate a user to a tab session +history entry from a different Tor state. It also expunges the window.history +entries during toggle. This listener helps Torbutton +satisfy the <link linkend="isolation">Network Isolation</link> requirement as +well as the <link linkend="state">State Separation</link> requirement. + + </para> + </listitem> + + <listitem><command>torbutton_http_observer</command> + <para> + +The torbutton_http_observer performs some of the work that logically belongs +to the content policy. This handles blocking of +Firefox 3 favicon loads, which for whatever +reason are not passed to the Firefox content policy itself (see Firefox Bugs +<ulink +url="https://bugzilla.mozilla.org/show_bug.cgi?id=437014">437014</ulink> and +<ulink +url="https://bugzilla.mozilla.org/show_bug.cgi?id=401296">401296</ulink>). + + </para> + <para> +The observer is also responsible for redirecting users to alternate +search engines when Google presents them with a Captcha, as well as copying +Google Captcha-related cookies between international Google domains. + </para> + </listitem> + + <listitem><command>torbutton_proxyservice</command> + <para> +The Torbutton proxy service handles redirecting Torbutton-related update +checks on addons.mozilla.org through Tor. This is done to help satisfy the +<link linkend="undiscoverability">Tor Undiscoverability</link> requirement. + </para> + </listitem> + + <listitem><command>torbutton_weblistener</command> <para>The <ulink url="https://developer.mozilla.org/en/nsIWebProgressListener#onLocationChange">location change</ulink> <ulink @@ -706,11 +801,103 @@ url="https://developer.mozilla.org/en/DOM/window.screen">window.screen</ulink> object to obfuscate browser and desktop resolution information. </para> + </listitem> + + </orderedlist> + </sect2> +</sect1> + +<sect1> + <title>Toggle Code Path</title> + <para> + +The act of toggling is connected to <function>torbutton_toggle()</function> +via the <ulink +url="https://gitweb.torproject.org/torbutton.git/blob_plain/HEAD:/src/chrome/con…">torbutton.xul</ulink> +and <ulink +url="https://gitweb.torproject.org/torbutton.git/blob_plain/HEAD:/src/chrome/con…">popup.xul</ulink> +overlay files. Most of the work in the toggling process is present in <ulink +url="https://gitweb.torproject.org/torbutton.git/blob_plain/HEAD:/src/chrome/con…">torbutton.js</ulink> +</para> <para> -The browser overlay helps to satisfy a number of Torbutton requirements. These -are better enumerated in each of the Torbutton preferences below. However, -there are also a number of Firefox preferences set in + +Toggling is a 3 stage process: Button Click, Proxy Update, and +Settings Update. These stages are reflected in the prefs +<command>extensions.torbutton.tor_enabled</command>, +<command>extensions.torbutton.proxies_applied</command>, and +<command>extensions.torbutton.settings_applied</command>. The reason for the +three stage preference update is to ensure immediate enforcement of <link +linkend="isolation">Network Isolation</link> via the <link +linkend="contentpolicy">content policy</link>. Since the content window +javascript runs on a different thread than the chrome javascript, it is +important to properly convey the stages to the content policy to avoid race +conditions and leakage, especially with <ulink +url="https://bugzilla.mozilla.org/show_bug.cgi?id=409737">Firefox Bug +409737</ulink> unfixed. The content policy does not allow any network activity +whatsoever during this three stage transition. + + </para> + <sect2> + <title>Button Click</title> + <para> + +This is the first step in the toggling process. When the user clicks the +toggle button or the toolbar, <function>torbutton_toggle()</function> is +called. This function checks the current Tor status by comparing the current +proxy settings to the selected Tor settings, and then sets the proxy settings +to the opposite state, and sets the pref +<command>extensions.torbutton.tor_enabled</command> to reflect the new state. +It is this proxy pref update that gives notification via the <ulink +url="https://developer.mozilla.org/en/NsIPrefBranch2#addObserver.28.29">pref +observer</ulink> +<command>torbutton_unique_pref_observer</command> to perform the rest of the +toggle. + + </para> + </sect2> + <sect2> + <title>Proxy Update</title> + <para> + +When Torbutton receives any proxy change notifications via its +<command>torbutton_unique_pref_observer</command>, it calls +<function>torbutton_set_status()</function> which checks against the Tor +settings to see if the Tor proxy settings match the current settings. If so, +it calls <function>torbutton_update_status()</function>, which determines if +the Tor state has actually changed, and sets +<command>extensions.torbutton.proxies_applied</command> to the appropriate Tor +state value, and ensures that +<command>extensions.torbutton.tor_enabled</command> is also set to the correct +value. This is decoupled from the button click functionalty via the pref +observer so that other addons (such as SwitchProxy) can switch the proxy +settings between multiple proxies. + + </para> + </sect2> + + <sect2> + <title>Settings Update</title> + <para> + +The next stage is also handled by +<function>torbutton_update_status()</function>. This function sets scores of +Firefox preferences, saving the original values to prefs under +<command>extensions.torbutton.saved.*</command>, and performs the <link +linkend="cookiejar">cookie jaring</link>, state clearing (such as window.name +and DOM storage), and <link linkend="preferences">preference +toggling</link>. At the +end of its work, it sets +<command>extensions.torbutton.settings_applied</command>, which signifies the +completion of the toggle operation to the <link +linkend="contentpolicy">content policy</link>. + + </para> + </sect2> +<sect2 id="preferences"> +<title>Firefox preferences touched during Toggle</title> +<para> +There are also a number of Firefox preferences set in <function>torbutton_update_status()</function> that aren't governed by any Torbutton setting. These are: </para> @@ -810,7 +997,8 @@ restoring it to the previous user value upon toggle. </para> </listitem> - <listitem><command>security.enable_ssl2</command> + <listitem><command>security.enable_ssl2</command> or <ulink +url="http://www.oxymoronical.com/experiments/xpcomref/applications/Firefox/3.5/i…">nsIDOMCrypto::logout()</ulink> <para> TLS Session IDs can persist for an indefinite duration, providing an identifier that is sent to TLS sites that can be used to link activity. This @@ -819,19 +1007,36 @@ in Firefox 3: The OCSP server can use this Session ID to build a history of TLS sites someone visits, and also correlate their activity as users move from network to network (such as home to work to coffee shop, etc), inside and outside of Tor. To handle this and to help satisfy our <link -linkend="state">State Separation Requirement</link>, we currently -toggle +linkend="state">State Separation Requirement</link>, we call the logout() +function of nsIDOMCrypto. Since this may be absent, or may fail, we fall back +to toggling <command>security.enable_ssl2</command>, which clears the SSL Session ID cache via the pref observer at <ulink -url="http://mxr.mozilla.org/security/source/security/manager/ssl/src/nsNSSCompon…">nsNSSComponent.cpp -line 2134</ulink>. This is an arcane and potentially fragile fix. It would be -better if there were a more standard interface for accomplishing the same -thing. <link linkend="FirefoxBugs">Firefox Bug</link> <ulink -url="https://bugzilla.mozilla.org/show_bug.cgi?id=448747">448747</ulink> has -been filed for this. - +url="http://mxr.mozilla.org/security/source/security/manager/ssl/src/nsNSSCompon…">nsNSSComponent.cpp</ulink>. + </para> + </listitem> + <listitem><command>security.OCSP.enabled</command> + <para> +Similarly, we toggle <command>security.OCSP.enabled</command>, which clears the OCSP certificate +validation cache via the pref observer at <ulink +url="http://mxr.mozilla.org/security/source/security/manager/ssl/src/nsNSSCompon…">nsNSSComponent.cpp</ulink>. +In this way, exit nodes will not be able to fingerprint you +based the fact that non-Tor OCSP lookups were obviously previously cached. +To handle this and to help satisfy our <link +linkend="state">State Separation Requirement</link>, </para> </listitem> + <listitem><command><ulink +url="http://kb.mozillazine.org/Updating_extensions#Disabling_update_checks_for_i…">extensions.e0204bd5-9d31-402b-a99d-a6aa8ffebdca.getAddons.cache.enabled</ulink></command> + <para> +We permanently disable addon usage statistic reporting to the +addons.mozilla.org statistics engine. These statistics send version +information about Torbutton users via non-Tor, allowing their Tor use to be +uncovered. Disabling this reporting helps Torbutton to satisfy its <link +linkend="undiscoverability">Tor Undiscoverability</link> requirement. + + </para> + </listitem> <listitem><command><ulink url="http://www.mozilla.com/en-US/firefox/geolocation/">geo.enabled</ulink></command> <para> @@ -900,112 +1105,12 @@ requirements. </orderedlist> </sect2> -<sect2> - <title>Preferences Window - <ulink -url="https://git.torproject.org/checkout/torbutton/master/src/chrome/content/pre…">preferences.xul</ulink></title> - -<para>The preferences window of course lays out the Torbutton preferences, with -handlers located in <ulink -url="https://git.torproject.org/checkout/torbutton/master/src/chrome/content/pre…">chrome/content/preferences.js</ulink>.</para> -</sect2> -<sect2> - <title>Other Windows</title> -<para>There are additional windows that describe popups for right clicking on -the status bar, the toolbutton, and the about page.</para> - -</sect2> -</sect1> - -<sect1> - <title>Toggle Code Path</title> - <para> - -The act of toggling is connected to <function>torbutton_toggle()</function> -via the <ulink -url="https://git.torproject.org/checkout/torbutton/master/src/chrome/content/tor…">torbutton.xul</ulink> -and <ulink -url="https://git.torproject.org/checkout/torbutton/master/src/chrome/content/pop…">popup.xul</ulink> -overlay files. Most of the work in the toggling process is present in <ulink -url="https://git.torproject.org/checkout/torbutton/master/src/chrome/content/tor…">torbutton.js</ulink> - -</para> -<para> - -Toggling is a 3 stage process: Button Click, Proxy Update, and -Settings Update. These stages are reflected in the prefs -<command>extensions.torbutton.tor_enabled</command>, -<command>extensions.torbutton.proxies_applied</command>, and -<command>extensions.torbutton.settings_applied</command>. The reason for the -three stage preference update is to ensure immediate enforcement of <link -linkend="isolation">Network Isolation</link> via the <link -linkend="contentpolicy">content policy</link>. Since the content window -javascript runs on a different thread than the chrome javascript, it is -important to properly convey the stages to the content policy to avoid race -conditions and leakage, especially with <ulink -url="https://bugzilla.mozilla.org/show_bug.cgi?id=409737">Firefox Bug -409737</ulink> unfixed. The content policy does not allow any network activity -whatsoever during this three stage transition. - - </para> - <sect2> - <title>Button Click</title> - <para> - -This is the first step in the toggling process. When the user clicks the -toggle button or the toolbar, <function>torbutton_toggle()</function> is -called. This function checks the current Tor status by comparing the current -proxy settings to the selected Tor settings, and then sets the proxy settings -to the opposite state, and sets the pref -<command>extensions.torbutton.tor_enabled</command> to reflect the new state. -It is this proxy pref update that gives notification via the <ulink -url="https://developer.mozilla.org/en/NsIPrefBranch2#addObserver.28.29">pref -observer</ulink> -<command>torbutton_unique_pref_observer</command> to perform the rest of the -toggle. - - </para> - </sect2> - <sect2> - <title>Proxy Update</title> - <para> - -When Torbutton receives any proxy change notifications via its -<command>torbutton_unique_pref_observer</command>, it calls -<function>torbutton_set_status()</function> which checks against the Tor -settings to see if the Tor proxy settings match the current settings. If so, -it calls <function>torbutton_update_status()</function>, which determines if -the Tor state has actually changed, and sets -<command>extensions.torbutton.proxies_applied</command> to the appropriate Tor -state value, and ensures that -<command>extensions.torbutton.tor_enabled</command> is also set to the correct -value. This is decoupled from the button click functionalty via the pref -observer so that other addons (such as SwitchProxy) can switch the proxy -settings between multiple proxies. - - </para> - </sect2> - <sect2> - <title>Settings Update</title> - <para> - -The next stage is also handled by -<function>torbutton_update_status()</function>. This function sets scores of -Firefox preferences, saving the original values to prefs under -<command>extensions.torbutton.saved.*</command>, and performs the history -clearing, cookie jaring, and ssl certificate jaring work of Torbutton. At the -end of its work, it sets -<command>extensions.torbutton.settings_applied</command>, which signifies the -completion of the toggle operation to the <link -linkend="contentpolicy">content policy</link>. - - </para> - </sect2> </sect1> <sect1> <title>Description of Options</title> - + <para>This section provides a detailed description of Torbutton's options. Each option is presented as the string from the preferences window, a summary, the preferences it touches, and the effect this has on the components, chrome, and @@ -1025,12 +1130,12 @@ Torbutton can easily inspect for a hidden link with an id of or <command>failure</command> to indicate if the user hit the page from a Tor IP, a non-Tor IP. This check is handled in <function>torbutton_test_settings()</function> in <ulink -url="https://git.torproject.org/checkout/torbutton/master/src/chrome/content/tor…">torbutton.js</ulink>. +url="https://gitweb.torproject.org/torbutton.git/blob_plain/HEAD:/src/chrome/con…">torbutton.js</ulink>. Presenting the results to the user is handled by the <ulink -url="https://git.torproject.org/checkout/torbutton/master/src/chrome/content/pre…">preferences +url="https://gitweb.torproject.org/torbutton.git/blob_plain/HEAD:/src/chrome/con…">preferences window</ulink> callback <function>torbutton_prefs_test_settings()</function> in <ulink -url="https://git.torproject.org/checkout/torbutton/master/src/chrome/content/pre…">preferences.js</ulink>. +url="https://gitweb.torproject.org/torbutton.git/blob_plain/HEAD:/src/chrome/con…">preferences.js</ulink>. </para> </sect2> @@ -1063,7 +1168,7 @@ event occurs (<function>torbutton_update_tags()</function>), and every time the tor state is changed (<function>torbutton_update_status()</function>). As a backup measure, plugins are also prevented from loading by the content policy in <ulink -url="https://git.torproject.org/checkout/torbutton/master/src/components/cssbloc…">@torproject.org/cssblocker;1</ulink> if Tor is +url="https://gitweb.torproject.org/torbutton.git/blob_plain/HEAD:/src/components…">@torproject.org/cssblocker;1</ulink> if Tor is enabled and this option is set. </para> @@ -1125,7 +1230,7 @@ linkend="proxy">Proxy Obedience</link> requirement. <para>Option: <command>extensions.torbutton.isolate_content</command></para> <para>Enabling this preference is what enables the <ulink -url="https://git.torproject.org/checkout/torbutton/master/src/components/cssbloc…">@torproject.org/cssblocker;1</ulink> content policy +url="https://gitweb.torproject.org/torbutton.git/blob_plain/HEAD:/src/components…">@torproject.org/cssblocker;1</ulink> content policy mentioned above, and causes it to block content load attempts in pages an opposite Tor state from the current state. Freshly loaded <ulink url="https://developer.mozilla.org/en/XUL/tabbrowser">browser @@ -1175,7 +1280,7 @@ linkend="isolation">Network Isolation</link> requirement. <para>Option: <command>extensions.torbutton.kill_bad_js</command></para> <para>This setting enables injection of the <ulink -url="https://git.torproject.org/checkout/torbutton/master/src/chrome/content/jsh…">Javascript +url="https://gitweb.torproject.org/torbutton.git/blob_plain/HEAD:/src/chrome/con…">Javascript hooking code</ulink>. This is done in the chrome in <function>torbutton_hookdoc()</function>, which is called ultimately by both the <ulink @@ -1202,7 +1307,7 @@ to retrieve the original screen values by using <ulink url="http://pseudo-flaw.net/tor/torbutton/unmask-sandbox-xpcnativewrapper.html">XPCNativeWrapper</ulink> or <ulink url="http://pseudo-flaw.net/tor/torbutton/unmask-components-lookupmethod.html">Components.lookupMethod</ulink>. -We are still looking for a workaround as of Torbutton 1.2.5. +We are still looking for a workaround as of Torbutton 1.3.2.  @@ -1322,6 +1427,8 @@ for Tor usage. </para> <para> + + This option causes Torbutton to prevent Firefox from loading <ulink url="http://www.mozilla.com/firefox/livebookmarks.html">Livemarks</ulink> during Tor usage. Because people often have very personalized Livemarks (such as RSS @@ -1372,6 +1479,11 @@ files to arbitrary sites just generally seems like a bad idea. </para> </sect2> + + + + + <sect2> <title>Close all Tor/Non-Tor tabs and windows on toggle (optional)</title> @@ -1455,6 +1567,7 @@ requirements. <title>History Access Settings</title> <para>Options: + <simplelist> <member><command>extensions.torbutton.block_thread</command></member> <member><command>extensions.torbutton.block_nthread</command></member> @@ -1464,7 +1577,7 @@ requirements. </para> <para>These four settings govern the behavior of the <ulink -url="https://git.torproject.org/checkout/torbutton/master/src/components/ignore-…">components/ignore-history.js</ulink> +url="https://gitweb.torproject.org/torbutton.git/blob_plain/HEAD:/src/components…">components/ignore-history.js</ulink> history blocker component mentioned above. By hooking the browser's view of the history itself via the <ulink url="http://www.oxymoronical.com/experiments/xpcomref/applications/Firefox/3.5/c…">@mozilla.org/browser/global-history;2</ulink> @@ -1604,7 +1717,7 @@ linkend="disk">Disk Avoidance</link> requirements. <para> This setting causes Torbutton to use <ulink -url="https://git.torproject.org/checkout/torbutton/master/src/components/cookie-…">@torproject.org/cookie-jar-selector;2</ulink> to store +url="https://gitweb.torproject.org/torbutton.git/blob_plain/HEAD:/src/components…">@torproject.org/cookie-jar-selector;2</ulink> to store non-tor cookies in a cookie jar during Tor usage, and clear the Tor cookies before restoring the jar. </para> @@ -1634,7 +1747,7 @@ linkend="disk">Disk Avoidance</link> requirements. <para> This setting causes Torbutton to use <ulink -url="https://git.torproject.org/checkout/torbutton/master/src/components/cookie-…">@torproject.org/cookie-jar-selector;2</ulink> to store +url="https://gitweb.torproject.org/torbutton.git/blob_plain/HEAD:/src/components…">@torproject.org/cookie-jar-selector;2</ulink> to store both Tor and Non-Tor cookies into protected jars. </para> @@ -1646,6 +1759,8 @@ linkend="state">State Separation</link> requirement. </sect2> + <sect2> @@ -1720,6 +1835,7 @@ linkend="state">State Separation</link> requirement. </para> </sect2> + <sect2> <title>Clear cookies on Tor/Non-Tor shutdown</title> @@ -1732,8 +1848,8 @@ cookie clearing, 1 means clear only during Tor-enabled shutdown, and 2 means clear for both Tor and Non-Tor shutdown. When set to 1 or 2, Torbutton listens for the <ulink url="http://developer.mozilla.org/en/docs/Observer_Notifications#Application_shu…">quit-application-granted</ulink> event in -<function>https://git.torproject.org/checkout/torbutton/master/src/components/crash-o…</function> and use <ulink -url="https://git.torproject.org/checkout/torbutton/master/src/components/cookie-…">@torproject.org/cookie-jar-selector;2</ulink> +<function>https://gitweb.torproject.org/torbutton.git/blob_plain/HEAD:/src/components…</function> and use <ulink +url="https://gitweb.torproject.org/torbutton.git/blob_plain/HEAD:/src/components…">@torproject.org/cookie-jar-selector;2</ulink> to clear out all cookies and all cookie jars upon shutdown. </para> <para> This setting helps to satisfy the <link @@ -1754,14 +1870,14 @@ linkend="state">State Separation</link> requirement. <para>This is no longer a user visible option, and is enabled by default. In the event of a crash, the Torbutton <ulink -url="https://git.torproject.org/checkout/torbutton/master/src/components/crash-o…">components/crash-observer.js</ulink> +url="https://gitweb.torproject.org/torbutton.git/blob_plain/HEAD:/src/components…">components/crash-observer.js</ulink> component will notify the Chrome (via the <command>extensions.torbutton.crashed</command> pref and a <ulink url="https://developer.mozilla.org/en/NsIPrefBranch2#addObserver.28.29">pref observer</ulink> in the chrome that listens for this update), and Torbutton will load the correct jar for the current Tor state via the <ulink -url="https://git.torproject.org/checkout/torbutton/master/src/components/cookie-…">@torproject.org/cookie-jar-selector;2</ulink> +url="https://gitweb.torproject.org/torbutton.git/blob_plain/HEAD:/src/components…">@torproject.org/cookie-jar-selector;2</ulink> component.</para> <para> @@ -1772,7 +1888,7 @@ crashes. </sect2> - + <sect2> <title>On crash recovery or session restored startup, restore via: Tor, Non-Tor</title> <para>Options: @@ -1784,7 +1900,7 @@ crashes. </para> <para>This option works with the Torbutton <ulink -url="https://git.torproject.org/checkout/torbutton/master/src/components/crash-o…">crash-observer.js</ulink> +url="https://gitweb.torproject.org/torbutton.git/blob_plain/HEAD:/src/components…">crash-observer.js</ulink> to set the Tor state after a crash is detected (via the <command>extensions.torbutton.crashed</command> pref). To confirm for false positives (such as session restore failures, upgrade, normal @@ -1814,7 +1930,7 @@ settings and saved sessions are reloaded from a fixed Tor state. </para> <para>This option also works with the Torbutton <ulink -url="https://git.torproject.org/checkout/torbutton/master/src/components/crash-o…">crash-observer.js</ulink> +url="https://gitweb.torproject.org/torbutton.git/blob_plain/HEAD:/src/components…">crash-observer.js</ulink> to set the Tor state after a normal startup is detected (via the <command>extensions.torbutton.noncrashed</command> pref). To confirm for false positives @@ -1837,7 +1953,7 @@ Firefox exit and checks this value as well during startup. </para> <para>If these options are enabled, the <ulink -url="https://git.torproject.org/checkout/torbutton/master/src/components/nsSessi…">replacement nsSessionStore.js</ulink> +url="https://gitweb.torproject.org/torbutton.git/blob_plain/HEAD:/src/components…">replacement nsSessionStore.js</ulink> component checks the <command>__tb_tor_fetched</command> tag of tabs before writing them out. If the tag is from a blocked Tor state, the tab is not written to disk. </para> @@ -1946,6 +2062,9 @@ This setting also does not directly satisfy any Torbutton requirement, but some may desire to mask their referrer for general privacy concerns. </para> </sect2> + + + <sect2> <title>Strip platform and language off of Google Search Box queries</title> @@ -2023,7 +2142,7 @@ encoded in the preferences These settings govern if Torbutton attempts to isolate the user's SSL certificates into separate jars for each Tor state. This isolation is implemented in <function>torbutton_jar_certs()</function> in <ulink -url="https://git.torproject.org/checkout/torbutton/master/src/chrome/content/tor…">chrome/content/torbutton.js</ulink>, +url="https://gitweb.torproject.org/torbutton.git/blob_plain/HEAD:/src/chrome/con…">chrome/content/torbutton.js</ulink>, which calls <function>torbutton_jar_cert_type()</function> and <function>torbutton_unjar_cert_type()</function> for each certificate type in the <ulink @@ -2252,6 +2371,22 @@ to resize maximized and minimized windows when it should not. </para> </listitem> <listitem><ulink +url="https://bugzilla.mozilla.org/show_bug.cgi?id=629820">nsIContentPolicy::shouldLoad not +called for web request in Firefox Mobile</ulink> + <para> + +The new <ulink +url="https://wiki.mozilla.org/Mobile/Fennec/Extensions/Electrolysis">Electrolysis</ulink> +multiprocess system appears to have some pretty rough edge cases with respect +to registering XPCOM category managers such as the nsIContentPolicy, which +make it difficult to do a straight-forward port of Torbutton or +HTTPS-Everywhere to Firefox Mobile. It probably also has similar issues with +wrapping existing <link linkend="hookedxpcom">Firefox XPCOM components</link>, +which will also cause more problems for porting TOrbutton. + + </para> + </listitem> + <listitem><ulink url="https://bugzilla.mozilla.org/show_bug.cgi?id=290456">Bug 290456 - Block/clear Flash MX "cookies" as well</ulink> <para> @@ -2279,6 +2414,8 @@ FoxyProxy) difficult to impossible to implement securely. </para> </listitem> +  - <listitem><ulink url="https://bugzilla.mozilla.org/show_bug.cgi?id=419598">Bug 419598 - 'var Date' is deletable</ulink>

1 0

[torbutton/master] Update option documentation.
by mikeperry＠torproject.org 04 Apr '11

04 Apr '11

commit 7a0b3316abfe0bfd718a777fe15405fa1b2e3e0f Author: Mike Perry <mikeperry-git(a)fscked.org> Date: Sun Apr 3 21:32:50 2011 -0700 Update option documentation. --- website/design/design.xml | 75 +++++++++++++++------------------------------ 1 files changed, 25 insertions(+), 50 deletions(-) diff --git a/website/design/design.xml b/website/design/design.xml index 403bb3b..81c27ff 100644 --- a/website/design/design.xml +++ b/website/design/design.xml @@ -571,7 +571,9 @@ which is what was done in years past. </para> </sect3> - + + @@ -1435,7 +1437,6 @@ Undiscoverability</link> requirement. </para> <para> - This option causes Torbutton to prevent Firefox from loading <ulink url="http://www.mozilla.com/firefox/livebookmarks.html">Livemarks</ulink> during @@ -1488,10 +1489,6 @@ files to arbitrary sites just generally seems like a bad idea. </para> </sect3> - - - - <sect3> <title>Close all Tor/Non-Tor tabs and windows on toggle (optional)</title> @@ -1577,7 +1574,6 @@ requirements. <title>History Access Settings</title> <para>Options: - <simplelist> <member><command>extensions.torbutton.block_thread</command></member> <member><command>extensions.torbutton.block_nthread</command></member> @@ -1586,7 +1582,7 @@ requirements. </simplelist> </para> -<para>These four settings govern the behavior of the <ulink +<para>On Firefox 3.x, these four settings govern the behavior of the <ulink url="https://gitweb.torproject.org/torbutton.git/blob_plain/HEAD:/src/components…">components/ignore-history.js</ulink> history blocker component mentioned above. By hooking the browser's view of the history itself via the <ulink @@ -1606,6 +1602,15 @@ Database</ulink> and the older Firefox 2 mechanisms. </para> <para> +On Firefox 4, Mozilla finally <ulink +url="https://developer.mozilla.org/en/CSS/Privacy_and_the_%3avisited_selector">addressed +these issues</ulink>, so we can effectively ignore the "read" pair of the +above prefs. We then only need to link the write prefs to +<command>places.history.enabled</command>, which disabled writing to the +history store while set. +</para> + +<para> This setting helps to satisfy the <link linkend="state">State Separation</link> and <link linkend="disk">Disk Avoidance</link> requirements. @@ -1852,18 +1857,12 @@ linkend="state">State Separation</link> requirement. </sect2> <sect2> <title>Startup Settings</title> - - - <sect3> - <title>On crash recovery or session restored startup, restore via: Tor, Non-Tor</title> + <title>On Browser Startup, set Tor state to: Tor, Non-Tor</title> <para>Options: - <simplelist> - <member><command>extensions.torbutton.restore_tor</command></member> - <member><command>extensions.torbutton.crashed</command></member> - <member><command>extensions.torbutton.normal_exit</command></member> - </simplelist> + <command>extensions.torbutton.restore_tor</command> </para> + <para>This option works with the Torbutton <ulink url="https://gitweb.torproject.org/torbutton.git/blob_plain/HEAD:/src/components…">crash-observer.js</ulink> @@ -1884,32 +1883,9 @@ settings and saved sessions are reloaded from a fixed Tor state. </para> </sect3> - -<sect3> - <title>On normal startup, set state to: Tor, Non-Tor, Shutdown State</title> - - <para>Options: - <simplelist> - <member><command>extensions.torbutton.startup_state</command></member> - <member><command>extensions.torbutton.noncrashed</command></member> - <member><command>extensions.torbutton.normal_exit</command></member> - </simplelist> - </para> - - <para>This option also works with the Torbutton <ulink -url="https://gitweb.torproject.org/torbutton.git/blob_plain/HEAD:/src/components…">crash-observer.js</ulink> - to set the Tor state after a normal startup is detected (via the - <command>extensions.torbutton.noncrashed</command> pref). To confirm for -false positives -(such as session restore failures, etc), Torbutton also sets the pref -extensions.torbutton.normal_exit in torbutton_uninstall_observer() during -Firefox exit and checks this value as well during startup. - -</para> - -</sect3> <sect3> + <title>Prevent session store from saving Non-Tor/Tor-loaded tabs</title> <para>Options: @@ -2038,27 +2014,26 @@ linkend="location">Location Neutrality</link> requirements. </para> </sect3> -<sect3> - <title>Don't send referrer during Tor Usage</title> +<sect3> + <title>Referer Spoofing Options</title> -<para>Option: <command>extensions.torbutton.disable_referer</command> +<para>Option: <command>extensions.torbutton.refererspoof</command> </para> + <para> -This option causes Torbutton to set <ulink -url="http://kb.mozillazine.org/Network.http.sendSecureXSiteReferrer">network.http.sendSecureXSiteReferrer</ulink> and -<ulink -url="http://kb.mozillazine.org/Network.http.sendRefererHeader">network.http.sendRefererHeader</ulink> during Tor usage.</para> +This option variable has three values. If it is 0, "smart" referer spoofing is +enabled. If it is 1, the referer behaves as normal. If it is 2, no referer is +sent. +</para> <para> This setting also does not directly satisfy any Torbutton requirement, but some may desire to mask their referrer for general privacy concerns. </para> </sect3> - - <sect3> <title>Strip platform and language off of Google Search Box queries</title>

1 0

[tor/master] Small fixes for the 2702 implementation
by nickm＠torproject.org 04 Apr '11

04 Apr '11

commit e97f78d7899372238c04bc6de019996ed9968f5d Author: Sebastian Hahn <sebastian(a)torproject.org> Date: Sun Mar 27 05:29:00 2011 +0200 Small fixes for the 2702 implementation Improve the INSTALL documentation for static builds, remove a few unnecessary lines from configure.in and tweak the changelog message slightly. --- INSTALL | 38 +++++++++++++------------------------- changes/bug2702 | 8 ++++---- configure.in | 18 +----------------- 3 files changed, 18 insertions(+), 46 deletions(-) diff --git a/INSTALL b/INSTALL index 1007b43..ddb790b 100644 --- a/INSTALL +++ b/INSTALL @@ -23,12 +23,20 @@ If it doesn't build for you: Lastly, check out https://www.torproject.org/docs/faq#DoesntWork -An example of how to build a mostly static Tor: +How to do static builds of tor: -Libevent should be built with: -% ./configure --disable-shared --enable-static --with-pic +Tor supports linking each of the libraries it needs statically. Use the +--enable-static-X ./configure option in conjunction with the --with-X-dir +option for libevent, zlib, and openssl. For this to work sanely, libevent +should be built with --disable-shared --enable-static --with-pic, and +OpenSSL should be built with no-shared no-dso. -An example of how to build a mostly static Tor: +If you need to build tor so that system libraries are also statically linked, +use the --enable-static-tor ./configure option. This won't work on OS X +unless you build the required crt0.o yourself. It is also incompatible with +the --enable-gcc-hardening option. + +An example of how to build a mostly static tor: ./configure --enable-static-libevent \ --enable-static-openssl \ --enable-static-zlib \ @@ -36,29 +44,9 @@ An example of how to build a mostly static Tor: --with-openssl-dir=/tmp/static-tor/openssl-0.9.8r/ \ --with-zlib-dir=/tmp/static-tor/zlib-1.2.5 -An example of how to build an entirely static Tor (no Mac OS X support, sorry): +An example of how to build an entirely static tor: ./configure --enable-static-tor \ - --enable-static-libevent \ - --enable-static-openssl \ - --enable-static-zlib \ --with-libevent-dir=/tmp/static-tor/libevent-1.4.14b-stable \ --with-openssl-dir=/tmp/static-tor/openssl-0.9.8r/ \ --with-zlib-dir=/tmp/static-tor/zlib-1.2.5 -This currently does not work with --enable-gcc-hardening because of libevent issues: - -configure:6176: gcc -o conftest -D_FORTIFY_SOURCE=2 -fstack-protector-all -fwrapv -fPIE -Wstack-protector ---param ssp-buffer-size=1 -I/tmp/static-tor/libevent-1.4.14b-stable -I${top_srcdir}/src/common - -L/tmp/static-tor/libevent-1.4.14b-stable -pie conftest.c -lpthread -ldl -levent -lrt >&5 -/usr/bin/ld: /tmp/static-tor/libevent-1.4.14b-stable/libevent.a(event.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC -/tmp/static-tor/libevent-1.4.14b-stable/libevent.a: could not read symbols: Bad value - collect2: ld returned 1 exit status - configure:6176: $? = 1 - -This produces the following Tor binaries on Gnu/Linux x86-64: - -% file src/or/tor -src/or/tor: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), statically linked, for GNU/Linux 2.6.15, not stripped - -% ldd src/or/tor - not a dynamic executable diff --git a/changes/bug2702 b/changes/bug2702 index cd004f5..b119f69 100644 --- a/changes/bug2702 +++ b/changes/bug2702 @@ -1,5 +1,5 @@ - o Minor enhancement - - Implements --enable-static-tor for configure time. Closes bug 2702. - Idea, general hackery and thoughts from Alexei Czeskis, John Gilmore, - Jacob Appelbaum. + o Minor features: + - Implements --enable-static-tor for configure time. Implements ticket + 2702. Idea, general hackery and thoughts from Alexei Czeskis, John + Gilmore, Jacob Appelbaum. diff --git a/configure.in b/configure.in index f7875df..b7e4180 100644 --- a/configure.in +++ b/configure.in @@ -33,14 +33,12 @@ AC_ARG_ENABLE(static-libevent, AC_ARG_ENABLE(static-zlib, AS_HELP_STRING(--enable-static-zlib, Link against a static zlib library. Requires --with-zlib-dir)) AC_ARG_ENABLE(static-tor, - AS_HELP_STRING(--enable-static-tor, Create an entirely static Tor binary. Requires --static-openssl - --static-libevent and --static-zlib)) + AS_HELP_STRING(--enable-static-tor, Create an entirely static Tor binary. Requires --with-openssl-dir and --with-libevent-dir and --with-zlib-dir)) if test "$enable_static_tor" = "yes"; then enable_static_libevent="yes"; enable_static_openssl="yes"; enable_static_zlib="yes"; - AC_MSG_NOTICE("We're attempting to build a static Tor.") CFLAGS="$CFLAGS -static" fi @@ -527,20 +525,6 @@ else fi AC_SUBST(TOR_ZLIB_LIBS) -if test "$enable_static_tor" = "yes"; then - if test "$enable_static_libevent" = "no"; then - AC_MSG_ERROR("You must configure with --enable-static-libevent") - fi - if test "$enable_static_openssl" = "no"; then - AC_MSG_ERROR("You must configure with --enable-static-openssl") - fi - if test "$enable_static_zlib" = "no"; then - AC_MSG_ERROR("You must configure with --enable-static-zlib") - fi - AC_MSG_NOTICE("We're attempting to build a static Tor.") - CFLAGS="$CFLAGS -static" -fi - dnl Make sure to enable support for large off_t if available.

1 0

[tor/master] add --enable-static-tor to our configure script
by nickm＠torproject.org 04 Apr '11

04 Apr '11

commit fe051a43c1fc5d8528ed46deaa818031b16a79eb Author: Jacob Appelbaum <jacob(a)appelbaum.net> Date: Thu Mar 10 01:22:32 2011 -0800 add --enable-static-tor to our configure script This implements the feature request in bug #2702 --- INSTALL | 39 +++++++++++++++++++++++++++++++++++++++ changes/bug2702 | 5 +++++ configure.in | 25 +++++++++++++++++++++++++ 3 files changed, 69 insertions(+), 0 deletions(-) diff --git a/INSTALL b/INSTALL index 5da50a6..1007b43 100644 --- a/INSTALL +++ b/INSTALL @@ -23,3 +23,42 @@ If it doesn't build for you: Lastly, check out https://www.torproject.org/docs/faq#DoesntWork +An example of how to build a mostly static Tor: + +Libevent should be built with: +% ./configure --disable-shared --enable-static --with-pic + +An example of how to build a mostly static Tor: +./configure --enable-static-libevent \ + --enable-static-openssl \ + --enable-static-zlib \ + --with-libevent-dir=/tmp/static-tor/libevent-1.4.14b-stable \ + --with-openssl-dir=/tmp/static-tor/openssl-0.9.8r/ \ + --with-zlib-dir=/tmp/static-tor/zlib-1.2.5 + +An example of how to build an entirely static Tor (no Mac OS X support, sorry): +./configure --enable-static-tor \ + --enable-static-libevent \ + --enable-static-openssl \ + --enable-static-zlib \ + --with-libevent-dir=/tmp/static-tor/libevent-1.4.14b-stable \ + --with-openssl-dir=/tmp/static-tor/openssl-0.9.8r/ \ + --with-zlib-dir=/tmp/static-tor/zlib-1.2.5 + +This currently does not work with --enable-gcc-hardening because of libevent issues: + +configure:6176: gcc -o conftest -D_FORTIFY_SOURCE=2 -fstack-protector-all -fwrapv -fPIE -Wstack-protector +--param ssp-buffer-size=1 -I/tmp/static-tor/libevent-1.4.14b-stable -I${top_srcdir}/src/common - +L/tmp/static-tor/libevent-1.4.14b-stable -pie conftest.c -lpthread -ldl -levent -lrt >&5 +/usr/bin/ld: /tmp/static-tor/libevent-1.4.14b-stable/libevent.a(event.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC +/tmp/static-tor/libevent-1.4.14b-stable/libevent.a: could not read symbols: Bad value + collect2: ld returned 1 exit status + configure:6176: $? = 1 + +This produces the following Tor binaries on Gnu/Linux x86-64: + +% file src/or/tor +src/or/tor: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), statically linked, for GNU/Linux 2.6.15, not stripped + +% ldd src/or/tor + not a dynamic executable diff --git a/changes/bug2702 b/changes/bug2702 new file mode 100644 index 0000000..cd004f5 --- /dev/null +++ b/changes/bug2702 @@ -0,0 +1,5 @@ + o Minor enhancement + - Implements --enable-static-tor for configure time. Closes bug 2702. + Idea, general hackery and thoughts from Alexei Czeskis, John Gilmore, + Jacob Appelbaum. + diff --git a/configure.in b/configure.in index 83fd044..f7875df 100644 --- a/configure.in +++ b/configure.in @@ -32,6 +32,17 @@ AC_ARG_ENABLE(static-libevent, AS_HELP_STRING(--enable-static-libevent, Link against a static libevent library. Requires --with-libevent-dir)) AC_ARG_ENABLE(static-zlib, AS_HELP_STRING(--enable-static-zlib, Link against a static zlib library. Requires --with-zlib-dir)) +AC_ARG_ENABLE(static-tor, + AS_HELP_STRING(--enable-static-tor, Create an entirely static Tor binary. Requires --static-openssl + --static-libevent and --static-zlib)) + +if test "$enable_static_tor" = "yes"; then + enable_static_libevent="yes"; + enable_static_openssl="yes"; + enable_static_zlib="yes"; + AC_MSG_NOTICE("We're attempting to build a static Tor.") + CFLAGS="$CFLAGS -static" +fi if test x$enable_buf_freelists != xno; then AC_DEFINE(ENABLE_BUF_FREELISTS, 1, @@ -516,6 +527,20 @@ else fi AC_SUBST(TOR_ZLIB_LIBS) +if test "$enable_static_tor" = "yes"; then + if test "$enable_static_libevent" = "no"; then + AC_MSG_ERROR("You must configure with --enable-static-libevent") + fi + if test "$enable_static_openssl" = "no"; then + AC_MSG_ERROR("You must configure with --enable-static-openssl") + fi + if test "$enable_static_zlib" = "no"; then + AC_MSG_ERROR("You must configure with --enable-static-zlib") + fi + AC_MSG_NOTICE("We're attempting to build a static Tor.") + CFLAGS="$CFLAGS -static" +fi + dnl Make sure to enable support for large off_t if available.

1 0

[tor/master] Changes file for static-work branch
by nickm＠torproject.org 04 Apr '11

04 Apr '11

commit ad1ae44154b023be76ce19278c3a18a3f1961055 Author: Nick Mathewson <nickm(a)torproject.org> Date: Mon Apr 4 15:35:16 2011 -0400 Changes file for static-work branch --- changes/static-flag | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/changes/static-flag b/changes/static-flag new file mode 100644 index 0000000..2cf9e10 --- /dev/null +++ b/changes/static-flag @@ -0,0 +1,4 @@ + o Minor features + - New "--enable-static-tor" configuration flag to try to build Tor + to link statically against as much as possible. Doesn't work on + all platforms.

1 0

[tor/master] Merge branch 'static-work'
by nickm＠torproject.org 04 Apr '11

04 Apr '11

commit 3cf23be712cad5537a77029e63895154c1c900ea Merge: 1587e0d ad1ae44 Author: Nick Mathewson <nickm(a)torproject.org> Date: Mon Apr 4 15:35:23 2011 -0400 Merge branch 'static-work' INSTALL | 27 +++++++++++++++++++++++++++ changes/bug2702 | 5 +++++ changes/static-flag | 4 ++++ configure.in | 9 +++++++++ 4 files changed, 45 insertions(+), 0 deletions(-)

1 0

r24556: {arm} Other version bumps for arm release. (in arm: resources resources/build/debian trunk trunk/src)
by Damian Johnson 04 Apr '11

04 Apr '11

Author: atagar Date: 2011-04-04 15:28:03 +0000 (Mon, 04 Apr 2011) New Revision: 24556 Modified: arm/resources/build/debian/changelog arm/resources/deb-prep.sh arm/trunk/ChangeLog arm/trunk/src/version.py Log: Other version bumps for arm release. Modified: arm/resources/build/debian/changelog =================================================================== --- arm/resources/build/debian/changelog 2011-04-04 15:22:31 UTC (rev 24555) +++ arm/resources/build/debian/changelog 2011-04-04 15:28:03 UTC (rev 24556) @@ -1,3 +1,9 @@ +tor-arm (1.4.2.0-1) unstable; urgency=low + + * Upstream release 1.4.2.0. + + -- Damian Johnson <atagar(a)torproject.org> Sat, 4 Apr 2011 1:30:43 -0700 + tor-arm (1.4.1.2-1) unstable; urgency=low * Upstream release 1.4.1.2. Modified: arm/resources/deb-prep.sh =================================================================== --- arm/resources/deb-prep.sh 2011-04-04 15:22:31 UTC (rev 24555) +++ arm/resources/deb-prep.sh 2011-04-04 15:28:03 UTC (rev 24556) @@ -19,6 +19,6 @@ # /usr/share/doc/arm/armrc.sample -> /usr/share/doc/tor-arm/armrc.sample.gz sed -i 's/\/usr\/share\/doc\/arm\/armrc.sample/\/usr\/share\/doc\/tor-arm\/armrc.sample.gz/g' release_deb/arm.1 -tar czf tor-arm_1.4.1.2.orig.tar.gz release_deb +tar czf tor-arm_1.4.2.0.orig.tar.gz release_deb svn export resources/build/debian release_deb/debian Modified: arm/trunk/ChangeLog =================================================================== --- arm/trunk/ChangeLog 2011-04-04 15:22:31 UTC (rev 24555) +++ arm/trunk/ChangeLog 2011-04-04 15:28:03 UTC (rev 24556) @@ -1,6 +1,6 @@ CHANGE LOG -4/4/11 - version 1.4.2 +4/4/11 - version 1.4.2 (r24555) This release chiefly consists of a fully reimplemented connection panel. Besides being a sane, maintainable implementation this includes numerous new features and improvements like full circuit paths, applications involved for local connections, and better type identification. * added: full rewrite of the connection panel, providing: Modified: arm/trunk/src/version.py =================================================================== --- arm/trunk/src/version.py 2011-04-04 15:22:31 UTC (rev 24555) +++ arm/trunk/src/version.py 2011-04-04 15:28:03 UTC (rev 24556) @@ -2,6 +2,6 @@ Provides arm's version and release date. """ -VERSION = '1.4.2_dev' -LAST_MODIFIED = "January 7, 2011" +VERSION = '1.4.3_dev' +LAST_MODIFIED = "April 4, 2011"

1 0

r24555: {arm} Arm version 1.4.2 release. (in arm/release: . src src/interface src/interface/connections src/interface/graphing src/util)
by Damian Johnson 04 Apr '11

04 Apr '11

Author: atagar Date: 2011-04-04 15:22:31 +0000 (Mon, 04 Apr 2011) New Revision: 24555 Added: arm/release/src/interface/connections/ arm/release/src/interface/connections/__init__.py arm/release/src/interface/connections/circEntry.py arm/release/src/interface/connections/connEntry.py arm/release/src/interface/connections/connPanel.py arm/release/src/interface/connections/entries.py arm/release/src/util/enum.py Removed: arm/release/TODO arm/release/src/interface/connections/__init__.py arm/release/src/interface/connections/circEntry.py arm/release/src/interface/connections/connEntry.py arm/release/src/interface/connections/connPanel.py arm/release/src/interface/connections/entries.py Modified: arm/release/ arm/release/ChangeLog arm/release/README arm/release/armrc.sample arm/release/src/interface/configPanel.py arm/release/src/interface/connPanel.py arm/release/src/interface/controller.py arm/release/src/interface/descriptorPopup.py arm/release/src/interface/graphing/__init__.py arm/release/src/interface/graphing/bandwidthStats.py arm/release/src/interface/graphing/connStats.py arm/release/src/interface/graphing/graphPanel.py arm/release/src/interface/headerPanel.py arm/release/src/interface/logPanel.py arm/release/src/interface/torrcPanel.py arm/release/src/settings.cfg arm/release/src/starter.py arm/release/src/test.py arm/release/src/util/__init__.py arm/release/src/util/conf.py arm/release/src/util/connections.py arm/release/src/util/log.py arm/release/src/util/panel.py arm/release/src/util/procTools.py arm/release/src/util/sysTools.py arm/release/src/util/torConfig.py arm/release/src/util/torTools.py arm/release/src/util/uiTools.py arm/release/src/version.py Log: Arm version 1.4.2 release. Property changes on: arm/release ___________________________________________________________________ Modified: svn:mergeinfo - /arm/trunk:22227-24074 + /arm/trunk:22227-24554 Modified: arm/release/ChangeLog =================================================================== --- arm/release/ChangeLog 2011-04-04 15:15:20 UTC (rev 24554) +++ arm/release/ChangeLog 2011-04-04 15:22:31 UTC (rev 24555) @@ -1,5 +1,53 @@ CHANGE LOG +4/4/11 - version 1.4.2 +This release chiefly consists of a fully reimplemented connection panel. Besides being a sane, maintainable implementation this includes numerous new features and improvements like full circuit paths, applications involved for local connections, and better type identification. + + * added: full rewrite of the connection panel, providing: + o listing the full paths involved in active circuits + o identification of socks, hidden service, and controller applications (arm, vidalia, polipo, etc) + o identification of exit connections with the common usage for the port they're using + o display of the local -> internal -> external address when room is available (original patch by Fabian Keil) + o better accuracy and performance in identifying client and directory connections + o marking the uptimes for initial connections (arm only tracks connection uptimes since starting, so these entries are just minimum durations) + o lazily loading the initial IP -> fingerprint mappings to improve the startup time + o using the circuit-status to disambiguating multiple relays on the same IP address + o smarter space utilization, filling in smaller columns if there isn't room for higher priority but larger entries + o connection details popup changes: + + using the consensus exit policies rather than the longer descriptor versions when available + + displaying connection details no longer freezes the rest of the display + + detail panel uses the full screen width and is dynamically resizable + + more resilient to missing descriptors + * change: hiding most tor config values by default (idea by arma) + * change: dropping warning suggesting that users set the FetchUselessDescriptors option (suggestion by Sebastian and others) + * change: always starting the bandwidth field from zero rather than using the state file total, which only contains the last day's worth of data (thanks to guilhem) + * change: suggesting authentication and giving steps for it in the readme (suggestion by Sebastian) + * change: caching config display lines, which reduces the CPU usage when scrolling by around 40% + * change: added summaries for the remaining tor configuration options + * change: using a dedicated enum class rather than tuple sets + * fix: torrc validation requires 'GETINFO config-text' which was introduced in Tor verison 0.2.2.7 (caught by Sjon, talion, and torland, https://trac.torproject.org/projects/tor/ticket/2501) + * fix: off-by-one issue with the displayed line numbers for torrc errors (caught by Sjon) + * fix: bin function wasn't available before python 2.6 (caught by Paul Menzel) + * fix: mis-parsing family entries when there's no entry after the comma (caught by StrangeCharm, https://trac.torproject.org/projects/tor/ticket/2414) + * fix: preventing SOCKS and CONTROL connections from being expanded (patch by Fabian Keil) + * fix: disabling name resolution for application queries to avoid leaking to resolvers (patch by Fabian Keil) + * fix: reversing src and dst addresses of SOCKS and CONTROL connections (caught by Fabian Keil) + * fix: changing the 'APPLICATION' type to 'SOCKS' since the previous label was too long (caught by Fabian Keil) + * fix: crashing issue from unknown relay nicknames (caught by krkhan) + * fix: concurrency bug occasionally causing "syshook" stacktraces when shutting down + * fix: header panel displayed the wrong IP address if it changed since we first started (https://trac.torproject.org/projects/tor/ticket/2776) + * fix: unchecked OSError could cause us to crash when making directories (for instance if there was a permissions issue) + * fix: the availability check for bsd resolvers was broken, probably causing resolution to fail for a few seconds on that platform + * fix: dropping the pointless 'Log notice stdout' entry provided by config-text queries (https://trac.torproject.org/projects/tor/ticket/2362) + * fix: taking DirServer and AlternateDirAuthority into account when determining the directory authorities + * fix: consuming a little extra space in the connection panel when scrollbars aren't visible + * fix: dropping the deprecated 'features.config.descriptions.persistPath' config option + * fix: failed connection attempts to the control port were generating zombie connections (https://trac.torproject.org/projects/tor/ticket/2812) + * fix: concurrency bug in joining on the TorCtl thread when tor shut down + * fix: the 'startup.dataDirectory' config option was being ignored + * fix: recognizing the proper private ip ranges of the 172.* block + * fix: missing 'is default' option from config sort ordering + 1/7/11 - version 1.4.1 (r24054) Platform specific enhancements including BSD compatibility and vastly improved performance on Linux. Modified: arm/release/README =================================================================== --- arm/release/README 2011-04-04 15:15:20 UTC (rev 24554) +++ arm/release/README 2011-04-04 15:22:31 UTC (rev 24555) @@ -25,6 +25,22 @@ ... starting Tor with '--controlport <PORT>' ... or including 'ControlPort <PORT>' in your torrc +It's also highly suggested for the control port to require authentication. +This can be done either with a cookie or password: + * Cookie Authentication - Controllers authenticate to Tor by providing the + contents of the control_auth_cookie file. To set this up... + - add "CookieAuthentication 1" to your torrc + - either restart Tor or run "pkill -sighup tor" + - this method of authentication is automatically handled by arm, so you + can still start arm as you normally would + + * Password Authentication - Attaching to the control port requires a + password. To set this up... + - run "tor --hash-password <your password>" + - add "HashedControlPassword <hashed password>" to your torrc + - either restart Tor or run "pkill -sighup tor" + - when starting up arm will prompt you for this password + For full functionality this also needs: - To be ran with the same user as tor to avoid permission issues with connection resolution and reading the torrc. @@ -111,7 +127,6 @@ ChangeLog - revision history LICENSE - copy of the gpl v3 README - um... guess you figured this one out - TODO - known issues, future plans, etc setup.py - distutils installation script for arm src/ @@ -125,6 +140,13 @@ uninstall - removal script interface/ + connections/ + __init__.py + connPanel.py - (page 2) lists the active tor connections + circEntry.py - circuit entries in the connection panel + connEntry.py - individual connections to or from the system + entries.py - common parent for connPanel display entries + graphing/ __init__.py graphPanel.py - (page 1) presents graphs for data instances @@ -139,7 +161,7 @@ logPanel.py - (page 1) displays tor, arm, and torctl events fileDescriptorPopup.py - (popup) displays file descriptors used by tor - connPanel.py - (page 2) displays information on tor connections + connPanel.py - (page 2) deprecated counterpart for connections/* descriptorPopup.py - (popup) displays connection descriptor data configPanel.py - (page 3) editor panel for the tor configuration @@ -149,6 +171,7 @@ __init__.py conf.py - loading and persistence for user configuration connections.py - service providing periodic connection lookups + enum.py - enumerations for ordered collections hostnames.py - service providing nonblocking reverse dns lookups log.py - aggregator for application events panel.py - wrapper for safely working with curses subwindows Deleted: arm/release/TODO =================================================================== --- arm/release/TODO 2011-04-04 15:15:20 UTC (rev 24554) +++ arm/release/TODO 2011-04-04 15:22:31 UTC (rev 24555) @@ -1,281 +0,0 @@ -TODO - -- Roadmap and completed work for next release (1.4.2) - [ ] refactor panels - Currently the interface is a bit of a rat's nest (especially the - controller). The goal is to use better modularization to both simplify - the codebase and make it possible to use smarter caching to improve - performance (far too much is done in the ui logic). This work is in - progress - /init and /util are done and /interface is partly done. Known - bugs are being fixed while refactoring. - - * conn panel - - expand client connections and note location in circuit (entry-exit) - - for clients give an option to list all connections, to tell which are - going through tor and which might be leaking - - check family members to see if they're alive (VERSION cell - handshake?) - - fallback when pid or connection querying via pid is unavailable - List all connections listed both by netstat and the consensus - - note when connection times are estimates (color?), ie connection - was established before arm - - connection uptime to associate inbound/outbound connections? - - identify controller connections (if it's arm, vidalia, etc) with - special detail page for them - - provide bridge / client country / exiting port statistics - Include bridge related data via GETINFO option (feature request - by waltman and ioerror). - - note the common port usage along with the exit statistics - - show the port used in scrubbed exit connections - - pick apart applications like iftop and pktstat to see how they get - per-connection bandwidth usage. Forum thread discussing it: - https://bbs.archlinux.org/viewtopic.php?pid=715906 - - include an option to show both the internal and external ips for the - local connection, ie: - myInternal --> myExternal --> foreign - idea and initial patch by Fabian Keil - - give a warning if family relays don't name us - * classify config options as useful (defaultly shown), standard, and - deprecated (configured to be hidden by default) - * check tor source for deprecated options like 'group' (are they - ignored? idea is thanks to NightMonkey) - * elaborate on the password prompt (suggestion by weasel) - * release prep - * pylint --indent-string=" " --disable=C,R interface/foo.py | less - * double check __init__.py and README for added or removed files - * wait a week, then bump package versions - * Debian - Contact: weasel (Peter Palfrader) - Initial Release: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=603056 - Update Instructions: - * TBD - - * Gentoo - Contact: NightMonkey (Jesse Adelman) - Initial Release: https://bugs.gentoo.org/show_bug.cgi?id=341731 - Update Instructions: - * go to https://bugs.gentoo.org - * make a generic bug with "net-misc/arm-X.X.X version bump, please" - - * ArchLinux - Contact: Spider.007 - Initial Release: http://aur.archlinux.org/packages.php?ID=44172 - Update Instructions: - * go to aur.archlinux.org - * select "Out-of-date" for the package - -- Roadmap for version 1.4.3 - [ ] refactor panels - [ ] controller and popup panels - [ ] attempt to clear controller password from memory - - http://www.codexon.com/posts/clearing-passwords-in-memory-with-python - * release prep - * pylint --indent-string=" " --disable=C,R interface/foo.py | less - * double check __init__.py and README for changes - -- Roadmap for version 1.3.9 - [ ] refactor panels - [ ] controller and popup panels - - allow arm to resume after restarting tor - This requires a full move to the torTools controller. - - improve on performance bottlenecks for startup time and cpu usage - - intermittent concurrency bugs during shutdown, one possible source: - https://trac.torproject.org/projects/tor/ticket/2144 - [ ] setup scripts for arm - [ ] updater (checks for a new tarball and installs it automatically) - - attempt to verify download signature, providing a warning if unable - to do so - [ ] look into CAPs to get around permission issues for connection - listing sudo wrapper for arm to help arm run as the same user as - tor? Irc suggestions: - - man capabilities - - http://www.linuxjournal.com/article/5737 - -- Bugs - * The default resolver isn't configurable. - * When saving the config the Log entry should be filtered out if unnecessary. - * The config write dialog (ie, the one for saving the config) has its a - misaligned border when it's smaller than the top detail section. - * The arm header panel doesn't properly reflect when the ip address - changes. This provides a notice event saying: - "Our IP Address has changed from X to Y; rebuilding descriptor (source Z)." - * The cpu usage spikes for scrollable content when the key's held. Try - coalescing the events. - * The manpage layout is system dependent, so the scraper needs to be more - resilient against being confused by whitespace. Another improvement is - including fallback results if the man page can't be parsed (suggested by - rransom, issue caught by NightMonkey). - * Log deduplication is currently an n^2 operation. Hence it can't handle - large logs (for instance, when at the DEBUG runlevel). Currently we're - timing out the function if it takes too long, but a more efficient method - for deduplication would be preferable. - * when in client mode and tor stops the header panel doesn't say so - * util/torTools.py: effective bandwidth rate/burst measurements don't take - SETCONF into consideration, blocked on: - https://trac.torproject.org/projects/tor/ticket/1692 - * log prepopulation fails to limit entries to the current tor instance if - the file isn't logged to at the NOTICE level. A fix is to use the - timestamps to see if it belongs to this tor instance. This requires - tor's uptime - blocked on implementation of the following proposal: - https://gitweb.torproject.org/tor.git/blob/HEAD:/doc/spec/proposals/173-get… - * the STATUS_SERVER event may not be supported - 18:52 < mikeperry> atagar: I believe there is no event parsing for STATUS_SERVER - 18:53 < mikeperry> atagar: see TorCtl.EventSink and classes that inherit from it - 18:54 < mikeperry> specifically, TorCtl.EventHandler._decode1, _handle1, and _map1 - - * conn panel: - * *never* do reverse dns lookups for first hops (could be resolving via - tor and hence leaking to the exit) - * If there's duplicate family entries (and harder case: both nickname and - fingerprint entries for the same relay) then the duplicate should be - removed. This is also causing a bad scrolling bug where the cursor can't - get past the pair of duplicate entries. - * revise multikey sort of connections - Currently using a pretty ugly hack. Look at: - http://www.velocityreviews.com/forums/ - t356461-sorting-a-list-of-objects-by-multiple-attributes.html - and check for performance difference. - * replace checks against exit policy with Mike's torctl version - My version still isn't handling all inputs anyway (still need to handle - masks, private keyword, and prepended policy). Parse it from the rest - of the router if too heavy ("TorCtl.Router.will_exit_to instead"). - * avoid hostname lookups of private connections - Stripped most of them but suspect there might be others (have assertions - check for this in a debug mode?) - * connection uptimes shouldn't show fractions of a second - * connections aren't cleared when control port closes - -- Packaging - * OpenWrt - OpenWrt uses the opkg packaging format which could make use of - arm's current deb packages. Packaging for this platform would help with - the Torouter project: - https://trac.torproject.org/projects/tor/wiki/TheOnionRouter/Torouter - * Mac - Couple of options include macport and dmg... - * macport (http://guide.macports.org/#development) - Build-from-source distribution method (like BSD portinstall). This has - been suggested by several people. - - * dmg (http://en.wikipedia.org/wiki/Apple_Disk_Image) - Most conventional method of software distribution on mac. This is just - a container (no updating/removal support), but could contain an icon - for the dock that starts a terminal with arm. This might include a pkg - installer. - - * mpkg (http://pypi.python.org/pypi/bdist_mpkg/) - Plugin for distutils. Like most mac packaging, this can only run on a - mac. It also requires setuptools: - http://www.errorhelp.com/search/details/74034/importerror-no-module-named-s… - -- Future Features - * client mode use cases - * not sure what sort of information would be useful in the header (to - replace the orport, fingerprint, flags, etc) - * one idea by velope: - "whether you configured a dnsport, transport, etc. and whether they - were successfully opened. might be nice to know this after the log - messages might be gone." - [notice] Opening Socks listener on 127.0.0.1:9050 - [notice] Opening Transparent pf/netfilter listener on 127.0.0.1:9040 - [notice] Opening DNS listener on 127.0.0.1:53 - * rdns and whois lookups (to find ISP, country, and jurisdiction, etc) - To avoid disclosing connection data to third parties this needs to be - an all-or-nothing operation (ie, needs to fetch information on all - relays or none of them). Plan is something like: - * add resolving/caching capabilities to fetch information on all relays - and distil whois entries to just what we care about (hosting provider - or ISP), by default updating the cache on a daily basis - * construct tarball and make this available for download rather than - fetching everything at each client - * possibly make these archives downloadable from peer relays (this is a - no-go for clients) via torrents or some dirport like scheme - * look at Vidalia and TorK for ideas - * need to solicit for ideas on what would be most helpful to clients - * dialog with bridge statuses (idea by mikeperry) - https://trac.vidalia-project.net/ticket/570 - https://trac.torproject.org/projects/tor/ticket/2068 - * menus - * http://gnosis.cx/publish/programming/charming_python_6.html ? - * additional options: - * make update rates configurable via the ui - * dialog with flag descriptions and other help - * menu with all torrc options (making them editable/toggleable) - * control port interpreter (interactive prompt) - Panel and startup option (-t maybe?) for providing raw control port - access along with usability improvements (piggybacking on the arm - connection): - * irc like help (ex "/help GETINFO" could provide a summary of - getinfo commands, partly using the results from - "GETINFO info/names") - * tab completion and up/down for previous commands - * warn and get confirmation if command would disrupt arm (for - instance 'SETEVENTS') - * 'safe' option that restricts to read-only access (start with this) - * issue sighup reset - * make use of the new process/* GETINFO options - They'll be available in the next tor release, as per: - https://trac.torproject.org/projects/tor/ticket/2291 - * feature parity for arm's config values (armrc entries) - * editability - * parse descriptions from the man page? autogeneration of the man page from - something storing the descriptions - * handle mutiple tor instances - * screen style (dialog for switching between instances) - * extra window with whatever stats can be aggregated over all instances, - or a config option to aggregate stats for bw, resource usage, etc - * option to save the current settings to the config - * provide warning at startup if the armrc doesn't exist, with instructions - for generating it - * email alerts for changes to the relay's status, similar to tor-weather - * simple alert if tor shuts down - * accounting and alerts for if the bandwidth drops to zero - * daily/weekly/etc alerts for basic status (log output, bandwidth history, - etc), borrowing from the consensus tracker for some of the formatting - * tab completion for input fields that expect a filesystem path - * look through vidalia's tickets for more ideas - https://trac.vidalia-project.net/ - * look into additions to the used apis - * curses (python 2.6 extended?): http://docs.python.org/library/curses.html - * new control options (like "desc-annotations/id/<OR identity>")? - * look into better supporting hidden services (what could be useful here?) - * provide option for a consensus page - Shows full consensus with an interface similar to the connection panel. - For this Mike's ConsensusTracker would be helpful (though boost the - startup time by several seconds) - * show qos stats - Take a look at 'linux-tor-prio.sh' to see if any of the stats are - available and interesting. - * escaping function for uiTools' formatted strings - * switch check of ip address validity to regex? - match = re.match("(\d*)\.(\d*)\.(\d*)\.(\d*)", ip) - http://wang.yuxuan.org/blog/2009/4/2/python_script_to_convert_from_ip_range… - * setup wizard for new relays - Setting the password and such for torrc generation. Maybe a netinstaller - that fetches the right package for the plagform, verifies signatures, etc? - Another alternative would be that when arm is started and tor isn't - running offer to start tor as a client, relay, or bridge. (idea by ioerror) - * audit what tor does - * Provide warnings if tor connections misbehaves, for instance: - * ensuring ExitPolicyRejectPrivate is being obeyed - * check that ExitPolicy violations don't occur (not possible yet since - not all relays aren't identified) - * check that all connections are properly related to a circuit, for - instance no outbound connections without a corresponding inbound (not - possible yet due to being unable to correlate connections to circuits) - * check file descriptors being accessed by tor to see if they're outside a - known pattern - * script that dumps relay stats to stdout - Derived from an idea by StrangeCharm. Django has a small terminal coloring - module that could be nice for formatting. Could possibly include: - * desc / ns information for our relay - * ps / netstat stats like load, uptime, and connection counts, etc - * implement control-spec proposals: - * https://gitweb.torproject.org/tor.git/blob/HEAD:/doc/spec/proposals/172-cir… - * https://gitweb.torproject.org/tor.git/blob/HEAD:/doc/spec/proposals/173-get… - * gui frontend (gtk?) - Look into if the arm utilities and codebase would fit nicely for a gui - controller like Vidalia and TorK. - * unit tests - Primarily for util, for instance 'addfstr' would be a good candidate. - * python 3 compatibility - Currently blocked on TorCtl support. - Modified: arm/release/armrc.sample =================================================================== --- arm/release/armrc.sample 2011-04-04 15:15:20 UTC (rev 24554) +++ arm/release/armrc.sample 2011-04-04 15:22:31 UTC (rev 24555) @@ -62,12 +62,11 @@ # Paremters for the config panel # --------------------------- -# type -# 0 -> tor state, 1 -> torrc, 2 -> arm state, 3 -> armrc # order # three comma separated configuration attributes, options including: -# 0 -> Category, 1 -> Option Name, 2 -> Value, 3 -> Arg Type, -# 4 -> Arg Usage, 5 -> Description, 6 -> Man Entry, 7 -> Is Default +# 0 -> Category, 1 -> Option Name, 2 -> Value, 3 -> Arg Type, +# 4 -> Arg Usage, 5 -> Summary, 6 -> Description, 7 -> Man Entry, +# 8 -> Is Default # selectionDetails.height # rows of data for the panel showing details on the current selection, this # is disabled entirely if zero @@ -87,12 +86,11 @@ # file.maxLinesPerEntry # max number of lines to display for a single entry in the torrc -features.config.type 0 -features.config.order 0, 6, 7 +features.config.order 7, 1, 8 features.config.selectionDetails.height 6 features.config.prepopulateEditValues true features.config.state.colWidth.option 25 -features.config.state.colWidth.value 10 +features.config.state.colWidth.value 15 features.config.state.showPrivateOptions false features.config.state.showVirtualOptions false features.config.file.showScrollbars true @@ -104,12 +102,11 @@ # --------------------------- # enabled # allows the descriptions to be fetched from the man page if true -# persistPath -# location descriptions should be loaded from and saved to (this feature is -# disabled if unset) +# persist +# caches the descriptions (substantially saving on future startup times) features.config.descriptions.enabled true -features.config.descriptions.persistPath /tmp/arm/torConfigDescriptions.txt +features.config.descriptions.persist true # General graph parameters # ------------------------ @@ -139,6 +136,9 @@ # prepopulate # attempts to use tor's state file to prepopulate the bandwidth graph at the # 15-minute interval (this requires the minimum of a day's worth of uptime) +# prepopulateTotal +# populates the total stat from the state file if true (this only contains +# the last day's worth of information, so this metric isn't the true total) # transferInBystes # shows rate measurments in bytes if true, bits otherwise # accounting.show @@ -149,11 +149,57 @@ # provides verbose measurements of time if true features.graph.bw.prepopulate true +features.graph.bw.prepopulateTotal false features.graph.bw.transferInBytes false features.graph.bw.accounting.show true features.graph.bw.accounting.rate 10 features.graph.bw.accounting.isTimeLong false +# Parameters for connection display +# --------------------------------- +# oldPanel +# includes the old connection panel in the interface +# newPanel +# includes the new connection panel in the interface +# listingType +# the primary category of information shown by default, options including: +# 0 -> IP Address / Port 1 -> Hostname +# 2 -> Fingerprint 3 -> Nickname +# order +# three comma separated configuration attributes, options including: +# 0 -> Category, 1 -> Uptime, 2 -> Listing, 3 -> IP Address, +# 4 -> Port, 5 -> Hostname, 6 -> Fingerprint, 7 -> Nickname, +# 8 -> Country +# refreshRate +# rate at which the connection panel contents is redrawn (if higher than the +# connection resolution rate then reducing this won't casue new data to +# appear more frequently - just increase the rate at which the uptime field +# is updated) +# resolveApps +# issues lsof queries to determining the applications involved in local +# SOCKS and CONTROL connections +# markInitialConnections +# if true, the uptime of the initial connections when we start are marked +# with a '+' (these uptimes are estimates since arm can only track a +# connection's duration while it runs) +# showExitPort +# shows port related information of exit connections we relay if true +# showColumn.* +# toggles the visability of the connection table columns + +features.connection.oldPanel false +features.connection.newPanel true +features.connection.listingType 0 +features.connection.order 0, 2, 1 +features.connection.refreshRate 5 +features.connection.resolveApps true +features.connection.markInitialConnections true +features.connection.showExitPort true +features.connection.showColumn.fingerprint true +features.connection.showColumn.nickname true +features.connection.showColumn.destination true +features.connection.showColumn.expandedIp true + # Thread pool size for hostname resolutions # Determines the maximum number of concurrent requests. Upping this to around # thirty or so seems to be problematic, causing intermittently seizing. @@ -187,6 +233,7 @@ log.torGetInfo DEBUG log.torGetInfoCache NONE log.torGetConf DEBUG +log.torGetConfCache NONE log.torSetConf INFO log.torEventTypeUnrecognized NOTICE log.torPrefixPathInvalid NOTICE @@ -224,6 +271,7 @@ log.cursesColorSupport INFO log.bsdJailFound INFO log.unknownBsdJailId WARN +log.geoipUnavailable WARN log.stats.failedProcResolution DEBUG log.stats.procResolutionFailover INFO log.stats.failedPsResolution INFO Modified: arm/release/src/interface/configPanel.py =================================================================== --- arm/release/src/interface/configPanel.py 2011-04-04 15:15:20 UTC (rev 24554) +++ arm/release/src/interface/configPanel.py 2011-04-04 15:22:31 UTC (rev 24555) @@ -6,65 +6,76 @@ import curses import threading -from util import conf, panel, torTools, torConfig, uiTools +from util import conf, enum, panel, torTools, torConfig, uiTools DEFAULT_CONFIG = {"features.config.selectionDetails.height": 6, "features.config.state.showPrivateOptions": False, "features.config.state.showVirtualOptions": False, "features.config.state.colWidth.option": 25, - "features.config.state.colWidth.value": 10} + "features.config.state.colWidth.value": 15} # TODO: The arm use cases are incomplete since they currently can't be # modified, have their descriptions fetched, or even get a complete listing # of what's available. -TOR_STATE, ARM_STATE = range(1, 3) # state to be presented +State = enum.Enum("TOR", "ARM") # state to be presented # mappings of option categories to the color for their entries -CATEGORY_COLOR = {torConfig.GENERAL: "green", - torConfig.CLIENT: "blue", - torConfig.SERVER: "yellow", - torConfig.DIRECTORY: "magenta", - torConfig.AUTHORITY: "red", - torConfig.HIDDEN_SERVICE: "cyan", - torConfig.TESTING: "white", - torConfig.UNKNOWN: "white"} +CATEGORY_COLOR = {torConfig.Category.GENERAL: "green", + torConfig.Category.CLIENT: "blue", + torConfig.Category.RELAY: "yellow", + torConfig.Category.DIRECTORY: "magenta", + torConfig.Category.AUTHORITY: "red", + torConfig.Category.HIDDEN_SERVICE: "cyan", + torConfig.Category.TESTING: "white", + torConfig.Category.UNKNOWN: "white"} # attributes of a ConfigEntry -FIELD_CATEGORY, FIELD_OPTION, FIELD_VALUE, FIELD_TYPE, FIELD_ARG_USAGE, FIELD_SUMMARY, FIELD_DESCRIPTION, FIELD_MAN_ENTRY, FIELD_IS_DEFAULT = range(9) -DEFAULT_SORT_ORDER = (FIELD_CATEGORY, FIELD_MAN_ENTRY, FIELD_IS_DEFAULT) -FIELD_ATTR = {FIELD_CATEGORY: ("Category", "red"), - FIELD_OPTION: ("Option Name", "blue"), - FIELD_VALUE: ("Value", "cyan"), - FIELD_TYPE: ("Arg Type", "green"), - FIELD_ARG_USAGE: ("Arg Usage", "yellow"), - FIELD_SUMMARY: ("Summary", "green"), - FIELD_DESCRIPTION: ("Description", "white"), - FIELD_MAN_ENTRY: ("Man Page Entry", "blue"), - FIELD_IS_DEFAULT: ("Is Default", "magenta")} +Field = enum.Enum("CATEGORY", "OPTION", "VALUE", "TYPE", "ARG_USAGE", + "SUMMARY", "DESCRIPTION", "MAN_ENTRY", "IS_DEFAULT") +DEFAULT_SORT_ORDER = (Field.MAN_ENTRY, Field.OPTION, Field.IS_DEFAULT) +FIELD_ATTR = {Field.CATEGORY: ("Category", "red"), + Field.OPTION: ("Option Name", "blue"), + Field.VALUE: ("Value", "cyan"), + Field.TYPE: ("Arg Type", "green"), + Field.ARG_USAGE: ("Arg Usage", "yellow"), + Field.SUMMARY: ("Summary", "green"), + Field.DESCRIPTION: ("Description", "white"), + Field.MAN_ENTRY: ("Man Page Entry", "blue"), + Field.IS_DEFAULT: ("Is Default", "magenta")} class ConfigEntry(): """ Configuration option in the panel. """ - def __init__(self, option, type, isDefault, summary, manEntry): + def __init__(self, option, type, isDefault): self.fields = {} - self.fields[FIELD_OPTION] = option - self.fields[FIELD_TYPE] = type - self.fields[FIELD_IS_DEFAULT] = isDefault + self.fields[Field.OPTION] = option + self.fields[Field.TYPE] = type + self.fields[Field.IS_DEFAULT] = isDefault + # Fetches extra infromation from external sources (the arm config and tor + # man page). These are None if unavailable for this config option. + summary = torConfig.getConfigSummary(option) + manEntry = torConfig.getConfigDescription(option) + if manEntry: - self.fields[FIELD_MAN_ENTRY] = manEntry.index - self.fields[FIELD_CATEGORY] = manEntry.category - self.fields[FIELD_ARG_USAGE] = manEntry.argUsage - self.fields[FIELD_DESCRIPTION] = manEntry.description + self.fields[Field.MAN_ENTRY] = manEntry.index + self.fields[Field.CATEGORY] = manEntry.category + self.fields[Field.ARG_USAGE] = manEntry.argUsage + self.fields[Field.DESCRIPTION] = manEntry.description else: - self.fields[FIELD_MAN_ENTRY] = 99999 # sorts non-man entries last - self.fields[FIELD_CATEGORY] = torConfig.UNKNOWN - self.fields[FIELD_ARG_USAGE] = "" - self.fields[FIELD_DESCRIPTION] = "" + self.fields[Field.MAN_ENTRY] = 99999 # sorts non-man entries last + self.fields[Field.CATEGORY] = torConfig.Category.UNKNOWN + self.fields[Field.ARG_USAGE] = "" + self.fields[Field.DESCRIPTION] = "" - self.fields[FIELD_SUMMARY] = summary if summary != None else self.fields[FIELD_DESCRIPTION] + # uses the full man page description if a summary is unavailable + self.fields[Field.SUMMARY] = summary if summary != None else self.fields[Field.DESCRIPTION] + + # cache of what's displayed for this configuration option + self.labelCache = None + self.labelCacheArgs = None def get(self, field): """ @@ -74,9 +85,44 @@ field - enum for the field to be provided back """ - if field == FIELD_VALUE: return self._getValue() + if field == Field.VALUE: return self._getValue() else: return self.fields[field] + def getAll(self, fields): + """ + Provides back a list with the given field values. + + Arguments: + field - enums for the fields to be provided back + """ + + return [self.get(field) for field in fields] + + def getLabel(self, optionWidth, valueWidth, summaryWidth): + """ + Provides display string of the configuration entry with the given + constraints on the width of the contents. + + Arguments: + optionWidth - width of the option column + valueWidth - width of the value column + summaryWidth - width of the summary column + """ + + # Fetching the display entries is very common so this caches the values. + # Doing this substantially drops cpu usage when scrolling (by around 40%). + + argSet = (optionWidth, valueWidth, summaryWidth) + if not self.labelCache or self.labelCacheArgs != argSet: + optionLabel = uiTools.cropStr(self.get(Field.OPTION), optionWidth) + valueLabel = uiTools.cropStr(self.get(Field.VALUE), valueWidth) + summaryLabel = uiTools.cropStr(self.get(Field.SUMMARY), summaryWidth, None) + lineTextLayout = "%%-%is %%-%is %%-%is" % (optionWidth, valueWidth, summaryWidth) + self.labelCache = lineTextLayout % (optionLabel, valueLabel, summaryLabel) + self.labelCacheArgs = argSet + + return self.labelCache + def _getValue(self): """ Provides the current value of the configuration entry, taking advantage of @@ -84,28 +130,18 @@ value's type to provide a user friendly representation if able. """ - confValue = ", ".join(torTools.getConn().getOption(self.get(FIELD_OPTION), [], True)) + confValue = ", ".join(torTools.getConn().getOption(self.get(Field.OPTION), [], True)) # provides nicer values for recognized types if not confValue: confValue = "<none>" - elif self.get(FIELD_TYPE) == "Boolean" and confValue in ("0", "1"): + elif self.get(Field.TYPE) == "Boolean" and confValue in ("0", "1"): confValue = "False" if confValue == "0" else "True" - elif self.get(FIELD_TYPE) == "DataSize" and confValue.isdigit(): + elif self.get(Field.TYPE) == "DataSize" and confValue.isdigit(): confValue = uiTools.getSizeLabel(int(confValue)) - elif self.get(FIELD_TYPE) == "TimeInterval" and confValue.isdigit(): + elif self.get(Field.TYPE) == "TimeInterval" and confValue.isdigit(): confValue = uiTools.getTimeLabel(int(confValue), isLong = True) return confValue - - def getAttr(self, argTypes): - """ - Provides back a list with the given parameters. - - Arguments: - argTypes - list of enums for the arguments to be provided back - """ - - return [self.get(field) for field in argTypes] class ConfigPanel(panel.Panel): """ @@ -124,14 +160,22 @@ "features.config.state.colWidth.option": 5, "features.config.state.colWidth.value": 5}) - self.sortOrdering = config.getIntCSV("features.config.order", self.sortOrdering, 3, 0, 6) + sortFields = Field.values() + customOrdering = config.getIntCSV("features.config.order", None, 3, 0, len(sortFields)) + + if customOrdering: + self.sortOrdering = [sortFields[i] for i in customOrdering] self.configType = configType self.confContents = [] self.scroller = uiTools.Scroller(True) self.valsLock = threading.RLock() - if self.configType == TOR_STATE: + # shows all configuration options if true, otherwise only the ones with + # the 'important' flag are shown + self.showAll = False + + if self.configType == State.TOR: conn = torTools.getConn() customOptions = torConfig.getCustomOptions() configOptionLines = conn.getInfo("config/names", "").strip().split("\n") @@ -141,29 +185,37 @@ # UseEntryGuards Boolean confOption, confType = line.strip().split(" ", 1) - # skips private and virtual entries if not set to show them + # skips private and virtual entries if not configured to show them if not self._config["features.config.state.showPrivateOptions"] and confOption.startswith("__"): continue elif not self._config["features.config.state.showVirtualOptions"] and confType == "Virtual": continue - summary = torConfig.getConfigSummary(confOption) - manEntry = torConfig.getConfigDescription(confOption) - self.confContents.append(ConfigEntry(confOption, confType, not confOption in customOptions, summary, manEntry)) - - self.setSortOrder() # initial sorting of the contents - elif self.configType == ARM_STATE: + self.confContents.append(ConfigEntry(confOption, confType, not confOption in customOptions)) + elif self.configType == State.ARM: # loaded via the conf utility armConf = conf.getConfig("arm") for key in armConf.getKeys(): pass # TODO: implement + + # mirror listing with only the important configuration options + self.confImportantContents = [] + for entry in self.confContents: + if torConfig.isImportant(entry.get(Field.OPTION)): + self.confImportantContents.append(entry) + + # if there aren't any important options then show everything + if not self.confImportantContents: + self.confImportantContents = self.confContents + + self.setSortOrder() # initial sorting of the contents def getSelection(self): """ Provides the currently selected entry. """ - return self.scroller.getCursorSelection(self.confContents) + return self.scroller.getCursorSelection(self._getConfigOptions()) def setSortOrder(self, ordering = None): """ @@ -177,7 +229,8 @@ self.valsLock.acquire() if ordering: self.sortOrdering = ordering - self.confContents.sort(key=lambda i: (i.getAttr(self.sortOrdering))) + self.confContents.sort(key=lambda i: (i.getAll(self.sortOrdering))) + self.confImportantContents.sort(key=lambda i: (i.getAll(self.sortOrdering))) self.valsLock.release() def handleKey(self, key): @@ -188,107 +241,104 @@ if detailPanelHeight > 0 and detailPanelHeight + 2 <= pageHeight: pageHeight -= (detailPanelHeight + 1) - isChanged = self.scroller.handleKey(key, self.confContents, pageHeight) + isChanged = self.scroller.handleKey(key, self._getConfigOptions(), pageHeight) if isChanged: self.redraw(True) + elif key == ord('a') or key == ord('A'): + self.showAll = not self.showAll + self.redraw(True) self.valsLock.release() - def draw(self, subwindow, width, height): + def draw(self, width, height): self.valsLock.acquire() # draws the top label - titleLabel = "%s Configuration:" % ("Tor" if self.configType == TOR_STATE else "Arm") - self.addstr(0, 0, titleLabel, curses.A_STANDOUT) + configType = "Tor" if self.configType == State.TOR else "Arm" + hiddenMsg = "press 'a' to hide most options" if self.showAll else "press 'a' to show all options" # panel with details for the current selection detailPanelHeight = self._config["features.config.selectionDetails.height"] + isScrollbarVisible = False if detailPanelHeight == 0 or detailPanelHeight + 2 >= height: # no detail panel detailPanelHeight = 0 - scrollLoc = self.scroller.getScrollLoc(self.confContents, height - 1) + scrollLoc = self.scroller.getScrollLoc(self._getConfigOptions(), height - 1) cursorSelection = self.getSelection() + isScrollbarVisible = len(self._getConfigOptions()) > height - 1 else: # Shrink detail panel if there isn't sufficient room for the whole # thing. The extra line is for the bottom border. detailPanelHeight = min(height - 1, detailPanelHeight + 1) - scrollLoc = self.scroller.getScrollLoc(self.confContents, height - 1 - detailPanelHeight) + scrollLoc = self.scroller.getScrollLoc(self._getConfigOptions(), height - 1 - detailPanelHeight) cursorSelection = self.getSelection() + isScrollbarVisible = len(self._getConfigOptions()) > height - detailPanelHeight - 1 - self._drawSelectionPanel(cursorSelection, width, detailPanelHeight, titleLabel) + self._drawSelectionPanel(cursorSelection, width, detailPanelHeight, isScrollbarVisible) + titleLabel = "%s Configuration (%s):" % (configType, hiddenMsg) + self.addstr(0, 0, titleLabel, curses.A_STANDOUT) + # draws left-hand scroll bar if content's longer than the height - scrollOffset = 0 - if len(self.confContents) > height - detailPanelHeight - 1: + scrollOffset = 1 + if isScrollbarVisible: scrollOffset = 3 - self.addScrollBar(scrollLoc, scrollLoc + height - detailPanelHeight - 1, len(self.confContents), 1 + detailPanelHeight) + self.addScrollBar(scrollLoc, scrollLoc + height - detailPanelHeight - 1, len(self._getConfigOptions()), 1 + detailPanelHeight) optionWidth = self._config["features.config.state.colWidth.option"] valueWidth = self._config["features.config.state.colWidth.value"] descriptionWidth = max(0, width - scrollOffset - optionWidth - valueWidth - 2) - for lineNum in range(scrollLoc, len(self.confContents)): - entry = self.confContents[lineNum] + for lineNum in range(scrollLoc, len(self._getConfigOptions())): + entry = self._getConfigOptions()[lineNum] drawLine = lineNum + detailPanelHeight + 1 - scrollLoc - optionLabel = uiTools.cropStr(entry.get(FIELD_OPTION), optionWidth) - valueLabel = uiTools.cropStr(entry.get(FIELD_VALUE), valueWidth) - summaryLabel = uiTools.cropStr(entry.get(FIELD_SUMMARY), descriptionWidth, None) - - lineFormat = curses.A_NORMAL if entry.get(FIELD_IS_DEFAULT) else curses.A_BOLD - if entry.get(FIELD_CATEGORY): lineFormat |= uiTools.getColor(CATEGORY_COLOR[entry.get(FIELD_CATEGORY)]) + lineFormat = curses.A_NORMAL if entry.get(Field.IS_DEFAULT) else curses.A_BOLD + if entry.get(Field.CATEGORY): lineFormat |= uiTools.getColor(CATEGORY_COLOR[entry.get(Field.CATEGORY)]) if entry == cursorSelection: lineFormat |= curses.A_STANDOUT - lineTextLayout = "%%-%is %%-%is %%-%is" % (optionWidth, valueWidth, descriptionWidth) - lineText = lineTextLayout % (optionLabel, valueLabel, summaryLabel) + lineText = entry.getLabel(optionWidth, valueWidth, descriptionWidth) self.addstr(drawLine, scrollOffset, lineText, lineFormat) if drawLine >= height: break self.valsLock.release() - def _drawSelectionPanel(self, cursorSelection, width, detailPanelHeight, titleLabel): + def _getConfigOptions(self): + return self.confContents if self.showAll else self.confImportantContents + + def _drawSelectionPanel(self, selection, width, detailPanelHeight, isScrollbarVisible): """ Renders a panel for the selected configuration option. """ - # border (top) - if width >= len(titleLabel): - self.win.hline(0, len(titleLabel), curses.ACS_HLINE, width - len(titleLabel)) - self.win.addch(0, width, curses.ACS_URCORNER) + # This is a solid border unless the scrollbar is visible, in which case a + # 'T' pipe connects the border to the bar. + uiTools.drawBox(self, 0, 0, width, detailPanelHeight + 1) + if isScrollbarVisible: self.addch(detailPanelHeight, 1, curses.ACS_TTEE) - # border (sides) - self.win.vline(1, 0, curses.ACS_VLINE, detailPanelHeight - 1) - self.win.vline(1, width, curses.ACS_VLINE, detailPanelHeight - 1) + selectionFormat = curses.A_BOLD | uiTools.getColor(CATEGORY_COLOR[selection.get(Field.CATEGORY)]) - # border (bottom) - self.win.addch(detailPanelHeight, 0, curses.ACS_LLCORNER) - if width >= 2: self.win.addch(detailPanelHeight, 1, curses.ACS_TTEE) - if width >= 3: self.win.hline(detailPanelHeight, 2, curses.ACS_HLINE, width - 2) - self.win.addch(detailPanelHeight, width, curses.ACS_LRCORNER) - - selectionFormat = curses.A_BOLD | uiTools.getColor(CATEGORY_COLOR[cursorSelection.get(FIELD_CATEGORY)]) - # first entry: # <option> (<category> Option) - optionLabel =" (%s Option)" % torConfig.OPTION_CATEGORY_STR[cursorSelection.get(FIELD_CATEGORY)] - self.addstr(1, 2, cursorSelection.get(FIELD_OPTION) + optionLabel, selectionFormat) + optionLabel =" (%s Option)" % selection.get(Field.CATEGORY) + self.addstr(1, 2, selection.get(Field.OPTION) + optionLabel, selectionFormat) # second entry: # Value: <value> ([default|custom], <type>, usage: <argument usage>) if detailPanelHeight >= 3: valueAttr = [] - valueAttr.append("default" if cursorSelection.get(FIELD_IS_DEFAULT) else "custom") - valueAttr.append(cursorSelection.get(FIELD_TYPE)) - valueAttr.append("usage: %s" % (cursorSelection.get(FIELD_ARG_USAGE))) + valueAttr.append("default" if selection.get(Field.IS_DEFAULT) else "custom") + valueAttr.append(selection.get(Field.TYPE)) + valueAttr.append("usage: %s" % (selection.get(Field.ARG_USAGE))) valueAttrLabel = ", ".join(valueAttr) valueLabelWidth = width - 12 - len(valueAttrLabel) - valueLabel = uiTools.cropStr(cursorSelection.get(FIELD_VALUE), valueLabelWidth) + valueLabel = uiTools.cropStr(selection.get(Field.VALUE), valueLabelWidth) self.addstr(2, 2, "Value: %s (%s)" % (valueLabel, valueAttrLabel), selectionFormat) # remainder is filled with the man page description descriptionHeight = max(0, detailPanelHeight - 3) - descriptionContent = "Description: " + cursorSelection.get(FIELD_DESCRIPTION) + descriptionContent = "Description: " + selection.get(Field.DESCRIPTION) for i in range(descriptionHeight): # checks if we're done writing the description @@ -304,7 +354,7 @@ if i != descriptionHeight - 1: # there's more lines to display - msg, remainder = uiTools.cropStr(lineContent, width - 2, 4, 4, uiTools.END_WITH_HYPHEN, True) + msg, remainder = uiTools.cropStr(lineContent, width - 2, 4, 4, uiTools.Ending.HYPHEN, True) descriptionContent = remainder.strip() + descriptionContent else: # this is the last line, end it with an ellipse Modified: arm/release/src/interface/connPanel.py =================================================================== --- arm/release/src/interface/connPanel.py 2011-04-04 15:15:20 UTC (rev 24554) +++ arm/release/src/interface/connPanel.py 2011-04-04 15:22:31 UTC (rev 24555) @@ -508,7 +508,7 @@ else: return # skip following redraw self.redraw(True) - def draw(self, subwindow, width, height): + def draw(self, width, height): self.connectionsLock.acquire() try: # hostnames frequently get updated so frequent sorting needed @@ -529,7 +529,7 @@ if self.showingDetails: listingHeight -= 8 isScrollBarVisible = len(self.connections) > height - 9 - if width > 80: subwindow.hline(8, 80, curses.ACS_HLINE, width - 81) + if width > 80: self.win.hline(8, 80, curses.ACS_HLINE, width - 81) else: isScrollBarVisible = len(self.connections) > height - 1 xOffset = 3 if isScrollBarVisible else 0 # content offset for scroll bar @@ -878,6 +878,8 @@ self.familyFingerprints = {} for familyEntry in self.family: + if not familyEntry: continue + if familyEntry[0] == "$": # relay identified by fingerprint self.familyFingerprints[familyEntry] = familyEntry[1:] Deleted: arm/release/src/interface/connections/__init__.py =================================================================== --- arm/trunk/src/interface/connections/__init__.py 2011-04-04 15:15:20 UTC (rev 24554) +++ arm/release/src/interface/connections/__init__.py 2011-04-04 15:22:31 UTC (rev 24555) @@ -1,6 +0,0 @@ -""" -Panels, popups, and handlers comprising the arm user interface. -""" - -__all__ = ["circEntry", "connEntry", "connPanel", "entries"] - Copied: arm/release/src/interface/connections/__init__.py (from rev 24554, arm/trunk/src/interface/connections/__init__.py) =================================================================== --- arm/release/src/interface/connections/__init__.py (rev 0) +++ arm/release/src/interface/connections/__init__.py 2011-04-04 15:22:31 UTC (rev 24555) @@ -0,0 +1,6 @@ +""" +Panels, popups, and handlers comprising the arm user interface. +""" + +__all__ = ["circEntry", "connEntry", "connPanel", "entries"] + Deleted: arm/release/src/interface/connections/circEntry.py =================================================================== --- arm/trunk/src/interface/connections/circEntry.py 2011-04-04 15:15:20 UTC (rev 24554) +++ arm/release/src/interface/connections/circEntry.py 2011-04-04 15:22:31 UTC (rev 24555) @@ -1,216 +0,0 @@ -""" -Connection panel entries for client circuits. This includes a header entry -followed by an entry for each hop in the circuit. For instance: - -89.188.20.246:42667 --> 217.172.182.26 (de) General / Built 8.6m (CIRCUIT) -| 85.8.28.4 (se) 98FBC3B2B93897A78CDD797EF549E6B62C9A8523 1 / Guard -| 91.121.204.76 (fr) 546387D93F8D40CFF8842BB9D3A8EC477CEDA984 2 / Middle -+- 217.172.182.26 (de) 5CFA9EA136C0EA0AC096E5CEA7EB674F1207CF86 3 / Exit -""" - -import curses - -from interface.connections import entries, connEntry -from util import torTools, uiTools - -# cached fingerprint -> (IP Address, ORPort) results -RELAY_INFO = {} - -def getRelayInfo(fingerprint): - """ - Provides the (IP Address, ORPort) tuple for the given relay. If the lookup - fails then this returns ("192.168.0.1", "0"). - - Arguments: - fingerprint - relay to look up - """ - - if not fingerprint in RELAY_INFO: - conn = torTools.getConn() - failureResult = ("192.168.0.1", "0") - - nsEntry = conn.getConsensusEntry(fingerprint) - if not nsEntry: return failureResult - - nsLineComp = nsEntry.split("\n")[0].split(" ") - if len(nsLineComp) < 8: return failureResult - - RELAY_INFO[fingerprint] = (nsLineComp[6], nsLineComp[7]) - - return RELAY_INFO[fingerprint] - -class CircEntry(connEntry.ConnectionEntry): - def __init__(self, circuitID, status, purpose, path): - connEntry.ConnectionEntry.__init__(self, "127.0.0.1", "0", "127.0.0.1", "0") - - self.circuitID = circuitID - self.status = status - - # drops to lowercase except the first letter - if len(purpose) >= 2: - purpose = purpose[0].upper() + purpose[1:].lower() - - self.lines = [CircHeaderLine(self.circuitID, purpose)] - - # Overwrites attributes of the initial line to make it more fitting as the - # header for our listing. - - self.lines[0].baseType = connEntry.Category.CIRCUIT - - self.update(status, path) - - def update(self, status, path): - """ - Our status and path can change over time if the circuit is still in the - process of being built. Updates these attributes of our relay. - - Arguments: - status - new status of the circuit - path - list of fingerprints for the series of relays involved in the - circuit - """ - - self.status = status - self.lines = [self.lines[0]] - - if status == "BUILT" and not self.lines[0].isBuilt: - exitIp, exitORPort = getRelayInfo(path[-1]) - self.lines[0].setExit(exitIp, exitORPort, path[-1]) - - for i in range(len(path)): - relayFingerprint = path[i] - relayIp, relayOrPort = getRelayInfo(relayFingerprint) - - if i == len(path) - 1: - if status == "BUILT": placementType = "Exit" - else: placementType = "Extending" - elif i == 0: placementType = "Guard" - else: placementType = "Middle" - - placementLabel = "%i / %s" % (i + 1, placementType) - - self.lines.append(CircLine(relayIp, relayOrPort, relayFingerprint, placementLabel)) - - self.lines[-1].isLast = True - -class CircHeaderLine(connEntry.ConnectionLine): - """ - Initial line of a client entry. This has the same basic format as connection - lines except that its etc field has circuit attributes. - """ - - def __init__(self, circuitID, purpose): - connEntry.ConnectionLine.__init__(self, "127.0.0.1", "0", "0.0.0.0", "0", False, False) - self.circuitID = circuitID - self.purpose = purpose - self.isBuilt = False - - def setExit(self, exitIpAddr, exitPort, exitFingerprint): - connEntry.ConnectionLine.__init__(self, "127.0.0.1", "0", exitIpAddr, exitPort, False, False) - self.isBuilt = True - self.foreign.fingerprintOverwrite = exitFingerprint - - def getType(self): - return connEntry.Category.CIRCUIT - - def getDestinationLabel(self, maxLength, includeLocale=False, includeHostname=False): - if not self.isBuilt: return "Building..." - return connEntry.ConnectionLine.getDestinationLabel(self, maxLength, includeLocale, includeHostname) - - def getEtcContent(self, width, listingType): - """ - Attempts to provide all circuit related stats. Anything that can't be - shown completely (not enough room) is dropped. - """ - - etcAttr = ["Purpose: %s" % self.purpose, "Circuit ID: %i" % self.circuitID] - - for i in range(len(etcAttr), -1, -1): - etcLabel = ", ".join(etcAttr[:i]) - if len(etcLabel) <= width: - return ("%%-%is" % width) % etcLabel - - return "" - - def getDetails(self, width): - if not self.isBuilt: - detailFormat = curses.A_BOLD | uiTools.getColor(connEntry.CATEGORY_COLOR[self.getType()]) - return [uiTools.DrawEntry("Building Circuit...", detailFormat)] - else: return connEntry.ConnectionLine.getDetails(self, width) - -class CircLine(connEntry.ConnectionLine): - """ - An individual hop in a circuit. This overwrites the displayed listing, but - otherwise makes use of the ConnectionLine attributes (for the detail display, - caching, etc). - """ - - def __init__(self, fIpAddr, fPort, fFingerprint, placementLabel): - connEntry.ConnectionLine.__init__(self, "127.0.0.1", "0", fIpAddr, fPort) - self.foreign.fingerprintOverwrite = fFingerprint - self.placementLabel = placementLabel - self.includePort = False - - # determines the sort of left hand bracketing we use - self.isLast = False - - def getType(self): - return connEntry.Category.CIRCUIT - - def getListingEntry(self, width, currentTime, listingType): - """ - Provides the DrawEntry for this relay in the circuilt listing. Lines are - composed of the following components: - <bracket> <dst> <etc> <placement label> - - The dst and etc entries largely match their ConnectionEntry counterparts. - - Arguments: - width - maximum length of the line - currentTime - the current unix time (ignored) - listingType - primary attribute we're listing connections by - """ - - return entries.ConnectionPanelLine.getListingEntry(self, width, currentTime, listingType) - - def _getListingEntry(self, width, currentTime, listingType): - lineFormat = uiTools.getColor(connEntry.CATEGORY_COLOR[self.getType()]) - - # The required widths are the sum of the following: - # bracketing (3 characters) - # placementLabel (14 characters) - # gap between etc and placement label (5 characters) - - if self.isLast: bracket = (curses.ACS_LLCORNER, curses.ACS_HLINE, ord(' ')) - else: bracket = (curses.ACS_VLINE, ord(' '), ord(' ')) - baselineSpace = len(bracket) + 14 + 5 - - dst, etc = "", "" - if listingType == entries.ListingType.IP_ADDRESS: - # TODO: include hostname when that's available - # dst width is derived as: - # src (21) + dst (26) + divider (7) + right gap (2) - bracket (3) = 53 char - dst = "%-53s" % self.getDestinationLabel(53, includeLocale = True) - etc = self.getEtcContent(width - baselineSpace - len(dst), listingType) - elif listingType == entries.ListingType.HOSTNAME: - # min space for the hostname is 40 characters - etc = self.getEtcContent(width - baselineSpace - 40, listingType) - dstLayout = "%%-%is" % (width - baselineSpace - len(etc)) - dst = dstLayout % self.foreign.getHostname(self.foreign.getIpAddr()) - elif listingType == entries.ListingType.FINGERPRINT: - # dst width is derived as: - # src (9) + dst (40) + divider (7) + right gap (2) - bracket (3) = 55 char - dst = "%-55s" % self.foreign.getFingerprint() - etc = self.getEtcContent(width - baselineSpace - len(dst), listingType) - else: - # min space for the nickname is 56 characters - etc = self.getEtcContent(width - baselineSpace - 56, listingType) - dstLayout = "%%-%is" % (width - baselineSpace - len(etc)) - dst = dstLayout % self.foreign.getNickname() - - drawEntry = uiTools.DrawEntry("%-14s" % self.placementLabel, lineFormat) - drawEntry = uiTools.DrawEntry(" " * (width - baselineSpace - len(dst) - len(etc) + 5), lineFormat, drawEntry) - drawEntry = uiTools.DrawEntry(dst + etc, lineFormat, drawEntry) - drawEntry = uiTools.DrawEntry(bracket, curses.A_NORMAL, drawEntry, lockFormat = True) - return drawEntry - Copied: arm/release/src/interface/connections/circEntry.py (from rev 24554, arm/trunk/src/interface/connections/circEntry.py) =================================================================== --- arm/release/src/interface/connections/circEntry.py (rev 0) +++ arm/release/src/interface/connections/circEntry.py 2011-04-04 15:22:31 UTC (rev 24555) @@ -0,0 +1,216 @@ +""" +Connection panel entries for client circuits. This includes a header entry +followed by an entry for each hop in the circuit. For instance: + +89.188.20.246:42667 --> 217.172.182.26 (de) General / Built 8.6m (CIRCUIT) +| 85.8.28.4 (se) 98FBC3B2B93897A78CDD797EF549E6B62C9A8523 1 / Guard +| 91.121.204.76 (fr) 546387D93F8D40CFF8842BB9D3A8EC477CEDA984 2 / Middle ++- 217.172.182.26 (de) 5CFA9EA136C0EA0AC096E5CEA7EB674F1207CF86 3 / Exit +""" + +import curses + +from interface.connections import entries, connEntry +from util import torTools, uiTools + +# cached fingerprint -> (IP Address, ORPort) results +RELAY_INFO = {} + +def getRelayInfo(fingerprint): + """ + Provides the (IP Address, ORPort) tuple for the given relay. If the lookup + fails then this returns ("192.168.0.1", "0"). + + Arguments: + fingerprint - relay to look up + """ + + if not fingerprint in RELAY_INFO: + conn = torTools.getConn() + failureResult = ("192.168.0.1", "0") + + nsEntry = conn.getConsensusEntry(fingerprint) + if not nsEntry: return failureResult + + nsLineComp = nsEntry.split("\n")[0].split(" ") + if len(nsLineComp) < 8: return failureResult + + RELAY_INFO[fingerprint] = (nsLineComp[6], nsLineComp[7]) + + return RELAY_INFO[fingerprint] + +class CircEntry(connEntry.ConnectionEntry): + def __init__(self, circuitID, status, purpose, path): + connEntry.ConnectionEntry.__init__(self, "127.0.0.1", "0", "127.0.0.1", "0") + + self.circuitID = circuitID + self.status = status + + # drops to lowercase except the first letter + if len(purpose) >= 2: + purpose = purpose[0].upper() + purpose[1:].lower() + + self.lines = [CircHeaderLine(self.circuitID, purpose)] + + # Overwrites attributes of the initial line to make it more fitting as the + # header for our listing. + + self.lines[0].baseType = connEntry.Category.CIRCUIT + + self.update(status, path) + + def update(self, status, path): + """ + Our status and path can change over time if the circuit is still in the + process of being built. Updates these attributes of our relay. + + Arguments: + status - new status of the circuit + path - list of fingerprints for the series of relays involved in the + circuit + """ + + self.status = status + self.lines = [self.lines[0]] + + if status == "BUILT" and not self.lines[0].isBuilt: + exitIp, exitORPort = getRelayInfo(path[-1]) + self.lines[0].setExit(exitIp, exitORPort, path[-1]) + + for i in range(len(path)): + relayFingerprint = path[i] + relayIp, relayOrPort = getRelayInfo(relayFingerprint) + + if i == len(path) - 1: + if status == "BUILT": placementType = "Exit" + else: placementType = "Extending" + elif i == 0: placementType = "Guard" + else: placementType = "Middle" + + placementLabel = "%i / %s" % (i + 1, placementType) + + self.lines.append(CircLine(relayIp, relayOrPort, relayFingerprint, placementLabel)) + + self.lines[-1].isLast = True + +class CircHeaderLine(connEntry.ConnectionLine): + """ + Initial line of a client entry. This has the same basic format as connection + lines except that its etc field has circuit attributes. + """ + + def __init__(self, circuitID, purpose): + connEntry.ConnectionLine.__init__(self, "127.0.0.1", "0", "0.0.0.0", "0", False, False) + self.circuitID = circuitID + self.purpose = purpose + self.isBuilt = False + + def setExit(self, exitIpAddr, exitPort, exitFingerprint): + connEntry.ConnectionLine.__init__(self, "127.0.0.1", "0", exitIpAddr, exitPort, False, False) + self.isBuilt = True + self.foreign.fingerprintOverwrite = exitFingerprint + + def getType(self): + return connEntry.Category.CIRCUIT + + def getDestinationLabel(self, maxLength, includeLocale=False, includeHostname=False): + if not self.isBuilt: return "Building..." + return connEntry.ConnectionLine.getDestinationLabel(self, maxLength, includeLocale, includeHostname) + + def getEtcContent(self, width, listingType): + """ + Attempts to provide all circuit related stats. Anything that can't be + shown completely (not enough room) is dropped. + """ + + etcAttr = ["Purpose: %s" % self.purpose, "Circuit ID: %i" % self.circuitID] + + for i in range(len(etcAttr), -1, -1): + etcLabel = ", ".join(etcAttr[:i]) + if len(etcLabel) <= width: + return ("%%-%is" % width) % etcLabel + + return "" + + def getDetails(self, width): + if not self.isBuilt: + detailFormat = curses.A_BOLD | uiTools.getColor(connEntry.CATEGORY_COLOR[self.getType()]) + return [uiTools.DrawEntry("Building Circuit...", detailFormat)] + else: return connEntry.ConnectionLine.getDetails(self, width) + +class CircLine(connEntry.ConnectionLine): + """ + An individual hop in a circuit. This overwrites the displayed listing, but + otherwise makes use of the ConnectionLine attributes (for the detail display, + caching, etc). + """ + + def __init__(self, fIpAddr, fPort, fFingerprint, placementLabel): + connEntry.ConnectionLine.__init__(self, "127.0.0.1", "0", fIpAddr, fPort) + self.foreign.fingerprintOverwrite = fFingerprint + self.placementLabel = placementLabel + self.includePort = False + + # determines the sort of left hand bracketing we use + self.isLast = False + + def getType(self): + return connEntry.Category.CIRCUIT + + def getListingEntry(self, width, currentTime, listingType): + """ + Provides the DrawEntry for this relay in the circuilt listing. Lines are + composed of the following components: + <bracket> <dst> <etc> <placement label> + + The dst and etc entries largely match their ConnectionEntry counterparts. + + Arguments: + width - maximum length of the line + currentTime - the current unix time (ignored) + listingType - primary attribute we're listing connections by + """ + + return entries.ConnectionPanelLine.getListingEntry(self, width, currentTime, listingType) + + def _getListingEntry(self, width, currentTime, listingType): + lineFormat = uiTools.getColor(connEntry.CATEGORY_COLOR[self.getType()]) + + # The required widths are the sum of the following: + # bracketing (3 characters) + # placementLabel (14 characters) + # gap between etc and placement label (5 characters) + + if self.isLast: bracket = (curses.ACS_LLCORNER, curses.ACS_HLINE, ord(' ')) + else: bracket = (curses.ACS_VLINE, ord(' '), ord(' ')) + baselineSpace = len(bracket) + 14 + 5 + + dst, etc = "", "" + if listingType == entries.ListingType.IP_ADDRESS: + # TODO: include hostname when that's available + # dst width is derived as: + # src (21) + dst (26) + divider (7) + right gap (2) - bracket (3) = 53 char + dst = "%-53s" % self.getDestinationLabel(53, includeLocale = True) + etc = self.getEtcContent(width - baselineSpace - len(dst), listingType) + elif listingType == entries.ListingType.HOSTNAME: + # min space for the hostname is 40 characters + etc = self.getEtcContent(width - baselineSpace - 40, listingType) + dstLayout = "%%-%is" % (width - baselineSpace - len(etc)) + dst = dstLayout % self.foreign.getHostname(self.foreign.getIpAddr()) + elif listingType == entries.ListingType.FINGERPRINT: + # dst width is derived as: + # src (9) + dst (40) + divider (7) + right gap (2) - bracket (3) = 55 char + dst = "%-55s" % self.foreign.getFingerprint() + etc = self.getEtcContent(width - baselineSpace - len(dst), listingType) + else: + # min space for the nickname is 56 characters + etc = self.getEtcContent(width - baselineSpace - 56, listingType) + dstLayout = "%%-%is" % (width - baselineSpace - len(etc)) + dst = dstLayout % self.foreign.getNickname() + + drawEntry = uiTools.DrawEntry("%-14s" % self.placementLabel, lineFormat) + drawEntry = uiTools.DrawEntry(" " * (width - baselineSpace - len(dst) - len(etc) + 5), lineFormat, drawEntry) + drawEntry = uiTools.DrawEntry(dst + etc, lineFormat, drawEntry) + drawEntry = uiTools.DrawEntry(bracket, curses.A_NORMAL, drawEntry, lockFormat = True) + return drawEntry + Deleted: arm/release/src/interface/connections/connEntry.py =================================================================== --- arm/trunk/src/interface/connections/connEntry.py 2011-04-04 15:15:20 UTC (rev 24554) +++ arm/release/src/interface/connections/connEntry.py 2011-04-04 15:22:31 UTC (rev 24555) @@ -1,864 +0,0 @@ -""" -Connection panel entries related to actual connections to or from the system -(ie, results seen by netstat, lsof, etc). -""" - -import time -import curses - -from util import connections, enum, torTools, uiTools -from interface.connections import entries - -# Connection Categories: -# Inbound Relay connection, coming to us. -# Outbound Relay connection, leaving us. -# Exit Outbound relay connection leaving the Tor network. -# Hidden Connections to a hidden service we're providing. -# Socks Socks connections for applications using Tor. -# Circuit Circuits our tor client has created. -# Directory Fetching tor consensus information. -# Control Tor controller (arm, vidalia, etc). - -Category = enum.Enum("INBOUND", "OUTBOUND", "EXIT", "HIDDEN", "SOCKS", "CIRCUIT", "DIRECTORY", "CONTROL") -CATEGORY_COLOR = {Category.INBOUND: "green", Category.OUTBOUND: "blue", - Category.EXIT: "red", Category.HIDDEN: "magenta", - Category.SOCKS: "yellow", Category.CIRCUIT: "cyan", - Category.DIRECTORY: "magenta", Category.CONTROL: "red"} - -# static data for listing format -# <src> --> <dst> <etc><padding> -LABEL_FORMAT = "%s --> %s %s%s" -LABEL_MIN_PADDING = 2 # min space between listing label and following data - -# sort value for scrubbed ip addresses -SCRUBBED_IP_VAL = 255 ** 4 - -CONFIG = {"features.connection.markInitialConnections": True, - "features.connection.showExitPort": True, - "features.connection.showColumn.fingerprint": True, - "features.connection.showColumn.nickname": True, - "features.connection.showColumn.destination": True, - "features.connection.showColumn.expandedIp": True} - -def loadConfig(config): - config.update(CONFIG) - -class Endpoint: - """ - Collection of attributes associated with a connection endpoint. This is a - thin wrapper for torUtil functions, making use of its caching for - performance. - """ - - def __init__(self, ipAddr, port): - self.ipAddr = ipAddr - self.port = port - - # if true, we treat the port as an ORPort when searching for matching - # fingerprints (otherwise the ORPort is assumed to be unknown) - self.isORPort = False - - # if set then this overwrites fingerprint lookups - self.fingerprintOverwrite = None - - def getIpAddr(self): - """ - Provides the IP address of the endpoint. - """ - - return self.ipAddr - - def getPort(self): - """ - Provides the port of the endpoint. - """ - - return self.port - - def getHostname(self, default = None): - """ - Provides the hostname associated with the relay's address. This is a - non-blocking call and returns None if the address either can't be resolved - or hasn't been resolved yet. - - Arguments: - default - return value if no hostname is available - """ - - # TODO: skipping all hostname resolution to be safe for now - #try: - # myHostname = hostnames.resolve(self.ipAddr) - #except: - # # either a ValueError or IOError depending on the source of the lookup failure - # myHostname = None - # - #if not myHostname: return default - #else: return myHostname - - return default - - def getLocale(self, default=None): - """ - Provides the two letter country code for the IP address' locale. - - Arguments: - default - return value if no locale information is available - """ - - conn = torTools.getConn() - return conn.getInfo("ip-to-country/%s" % self.ipAddr, default) - - def getFingerprint(self): - """ - Provides the fingerprint of the relay, returning "UNKNOWN" if it can't be - determined. - """ - - if self.fingerprintOverwrite: - return self.fingerprintOverwrite - - conn = torTools.getConn() - orPort = self.port if self.isORPort else None - myFingerprint = conn.getRelayFingerprint(self.ipAddr, orPort) - - if myFingerprint: return myFingerprint - else: return "UNKNOWN" - - def getNickname(self): - """ - Provides the nickname of the relay, retuning "UNKNOWN" if it can't be - determined. - """ - - myFingerprint = self.getFingerprint() - - if myFingerprint != "UNKNOWN": - conn = torTools.getConn() - myNickname = conn.getRelayNickname(myFingerprint) - - if myNickname: return myNickname - else: return "UNKNOWN" - else: return "UNKNOWN" - -class ConnectionEntry(entries.ConnectionPanelEntry): - """ - Represents a connection being made to or from this system. These only - concern real connections so it includes the inbound, outbound, directory, - application, and controller categories. - """ - - def __init__(self, lIpAddr, lPort, fIpAddr, fPort): - entries.ConnectionPanelEntry.__init__(self) - self.lines = [ConnectionLine(lIpAddr, lPort, fIpAddr, fPort)] - - def getSortValue(self, attr, listingType): - """ - Provides the value of a single attribute used for sorting purposes. - """ - - connLine = self.lines[0] - if attr == entries.SortAttr.IP_ADDRESS: - if connLine.isPrivate(): return SCRUBBED_IP_VAL # orders at the end - return connLine.sortIpAddr - elif attr == entries.SortAttr.PORT: - return connLine.sortPort - elif attr == entries.SortAttr.HOSTNAME: - if connLine.isPrivate(): return "" - return connLine.foreign.getHostname("") - elif attr == entries.SortAttr.FINGERPRINT: - return connLine.foreign.getFingerprint() - elif attr == entries.SortAttr.NICKNAME: - myNickname = connLine.foreign.getNickname() - if myNickname == "UNKNOWN": return "z" * 20 # orders at the end - else: return myNickname.lower() - elif attr == entries.SortAttr.CATEGORY: - return Category.indexOf(connLine.getType()) - elif attr == entries.SortAttr.UPTIME: - return connLine.startTime - elif attr == entries.SortAttr.COUNTRY: - if connections.isIpAddressPrivate(self.lines[0].foreign.getIpAddr()): return "" - else: return connLine.foreign.getLocale("") - else: - return entries.ConnectionPanelEntry.getSortValue(self, attr, listingType) - -class ConnectionLine(entries.ConnectionPanelLine): - """ - Display component of the ConnectionEntry. - """ - - def __init__(self, lIpAddr, lPort, fIpAddr, fPort, includePort=True, includeExpandedIpAddr=True): - entries.ConnectionPanelLine.__init__(self) - - self.local = Endpoint(lIpAddr, lPort) - self.foreign = Endpoint(fIpAddr, fPort) - self.startTime = time.time() - self.isInitialConnection = False - - # overwrite the local fingerprint with ours - conn = torTools.getConn() - self.local.fingerprintOverwrite = conn.getInfo("fingerprint") - - # True if the connection has matched the properties of a client/directory - # connection every time we've checked. The criteria we check is... - # client - first hop in an established circuit - # directory - matches an established single-hop circuit (probably a - # directory mirror) - - self._possibleClient = True - self._possibleDirectory = True - - # attributes for SOCKS, HIDDEN, and CONTROL connections - self.appName = None - self.appPid = None - self.isAppResolving = False - - myOrPort = conn.getOption("ORPort") - myDirPort = conn.getOption("DirPort") - mySocksPort = conn.getOption("SocksPort", "9050") - myCtlPort = conn.getOption("ControlPort") - myHiddenServicePorts = conn.getHiddenServicePorts() - - # the ORListenAddress can overwrite the ORPort - listenAddr = conn.getOption("ORListenAddress") - if listenAddr and ":" in listenAddr: - myOrPort = listenAddr[listenAddr.find(":") + 1:] - - if lPort in (myOrPort, myDirPort): - self.baseType = Category.INBOUND - self.local.isORPort = True - elif lPort == mySocksPort: - self.baseType = Category.SOCKS - elif fPort in myHiddenServicePorts: - self.baseType = Category.HIDDEN - elif lPort == myCtlPort: - self.baseType = Category.CONTROL - else: - self.baseType = Category.OUTBOUND - self.foreign.isORPort = True - - self.cachedType = None - - # includes the port or expanded ip address field when displaying listing - # information if true - self.includePort = includePort - self.includeExpandedIpAddr = includeExpandedIpAddr - - # cached immutable values used for sorting - self.sortIpAddr = connections.ipToInt(self.foreign.getIpAddr()) - self.sortPort = int(self.foreign.getPort()) - - def getListingEntry(self, width, currentTime, listingType): - """ - Provides the DrawEntry for this connection's listing. Lines are composed - of the following components: - <src> --> <dst> <etc> <uptime> (<type>) - - ListingType.IP_ADDRESS: - src - <internal addr:port> --> <external addr:port> - dst - <destination addr:port> - etc - <fingerprint> <nickname> - - ListingType.HOSTNAME: - src - localhost:<port> - dst - <destination hostname:port> - etc - <destination addr:port> <fingerprint> <nickname> - - ListingType.FINGERPRINT: - src - localhost - dst - <destination fingerprint> - etc - <nickname> <destination addr:port> - - ListingType.NICKNAME: - src - <source nickname> - dst - <destination nickname> - etc - <fingerprint> <destination addr:port> - - Arguments: - width - maximum length of the line - currentTime - unix timestamp for what the results should consider to be - the current time - listingType - primary attribute we're listing connections by - """ - - # fetch our (most likely cached) display entry for the listing - myListing = entries.ConnectionPanelLine.getListingEntry(self, width, currentTime, listingType) - - # fill in the current uptime and return the results - if CONFIG["features.connection.markInitialConnections"]: - timePrefix = "+" if self.isInitialConnection else " " - else: timePrefix = "" - - timeEntry = myListing.getNext() - timeEntry.text = timePrefix + "%5s" % uiTools.getTimeLabel(currentTime - self.startTime, 1) - - return myListing - - def isUnresolvedApp(self): - """ - True if our display uses application information that hasn't yet been resolved. - """ - - return self.appName == None and self.getType() in (Category.SOCKS, Category.HIDDEN, Category.CONTROL) - - def _getListingEntry(self, width, currentTime, listingType): - entryType = self.getType() - - # Lines are split into the following components in reverse: - # content - "<src> --> <dst> <etc> " - # time - "<uptime>" - # preType - " (" - # category - "<type>" - # postType - ") " - - lineFormat = uiTools.getColor(CATEGORY_COLOR[entryType]) - timeWidth = 6 if CONFIG["features.connection.markInitialConnections"] else 5 - - drawEntry = uiTools.DrawEntry(")" + " " * (9 - len(entryType)), lineFormat) - drawEntry = uiTools.DrawEntry(entryType.upper(), lineFormat | curses.A_BOLD, drawEntry) - drawEntry = uiTools.DrawEntry(" (", lineFormat, drawEntry) - drawEntry = uiTools.DrawEntry(" " * timeWidth, lineFormat, drawEntry) - drawEntry = uiTools.DrawEntry(self._getListingContent(width - (12 + timeWidth), listingType), lineFormat, drawEntry) - return drawEntry - - def _getDetails(self, width): - """ - Provides details on the connection, correlated against available consensus - data. - - Arguments: - width - available space to display in - """ - - detailFormat = curses.A_BOLD | uiTools.getColor(CATEGORY_COLOR[self.getType()]) - return [uiTools.DrawEntry(line, detailFormat) for line in self._getDetailContent(width)] - - def _getDescriptors(self, width): - """ - Provides raw descriptor information for the relay. - - Arguments: - width - available space to display in - """ - - # TODO: Porting and refactoring the descriptorPopup.py functionality is - # gonna take quite a bit of work. This is a very rarely used feature and - # not worth delaying the 1.4.2 release any further, so this will be a part - # of 1.4.3. - - return [] - - def resetDisplay(self): - entries.ConnectionPanelLine.resetDisplay(self) - self.cachedType = None - - def isPrivate(self): - """ - Returns true if the endpoint is private, possibly belonging to a client - connection or exit traffic. - """ - - # This is used to scrub private information from the interface. Relaying - # etiquette (and wiretapping laws) say these are bad things to look at so - # DON'T CHANGE THIS UNLESS YOU HAVE A DAMN GOOD REASON! - - myType = self.getType() - - if myType == Category.INBOUND: - # if we're a guard or bridge and the connection doesn't belong to a - # known relay then it might be client traffic - - conn = torTools.getConn() - if "Guard" in conn.getMyFlags() or conn.getOption("BridgeRelay") == "1": - allMatches = conn.getRelayFingerprint(self.foreign.getIpAddr(), getAllMatches = True) - return allMatches == [] - elif myType == Category.EXIT: - # DNS connections exiting us aren't private (since they're hitting our - # resolvers). Everything else, however, is. - - # TODO: Ideally this would also double check that it's a UDP connection - # (since DNS is the only UDP connections Tor will relay), however this - # will take a bit more work to propagate the information up from the - # connection resolver. - return self.foreign.getPort() != "53" - - # for everything else this isn't a concern - return False - - def getType(self): - """ - Provides our best guess at the current type of the connection. This - depends on consensus results, our current client circuits, etc. Results - are cached until this entry's display is reset. - """ - - # caches both to simplify the calls and to keep the type consistent until - # we want to reflect changes - if not self.cachedType: - if self.baseType == Category.OUTBOUND: - # Currently the only non-static categories are OUTBOUND vs... - # - EXIT since this depends on the current consensus - # - CIRCUIT if this is likely to belong to our guard usage - # - DIRECTORY if this is a single-hop circuit (directory mirror?) - # - # The exitability, circuits, and fingerprints are all cached by the - # torTools util keeping this a quick lookup. - - conn = torTools.getConn() - destFingerprint = self.foreign.getFingerprint() - - if destFingerprint == "UNKNOWN": - # Not a known relay. This might be an exit connection. - - if conn.isExitingAllowed(self.foreign.getIpAddr(), self.foreign.getPort()): - self.cachedType = Category.EXIT - elif self._possibleClient or self._possibleDirectory: - # This belongs to a known relay. If we haven't eliminated ourselves as - # a possible client or directory connection then check if it still - # holds true. - - myCircuits = conn.getCircuits() - - if self._possibleClient: - # Checks that this belongs to the first hop in a circuit that's - # either unestablished or longer than a single hop (ie, anything but - # a built 1-hop connection since those are most likely a directory - # mirror). - - for _, status, _, path in myCircuits: - if path[0] == destFingerprint and (status != "BUILT" or len(path) > 1): - self.cachedType = Category.CIRCUIT # matched a probable guard connection - - # if we fell through, we can eliminate ourselves as a guard in the future - if not self.cachedType: - self._possibleClient = False - - if self._possibleDirectory: - # Checks if we match a built, single hop circuit. - - for _, status, _, path in myCircuits: - if path[0] == destFingerprint and status == "BUILT" and len(path) == 1: - self.cachedType = Category.DIRECTORY - - # if we fell through, eliminate ourselves as a directory connection - if not self.cachedType: - self._possibleDirectory = False - - if not self.cachedType: - self.cachedType = self.baseType - - return self.cachedType - - def getEtcContent(self, width, listingType): - """ - Provides the optional content for the connection. - - Arguments: - width - maximum length of the line - listingType - primary attribute we're listing connections by - """ - - # for applications show the command/pid - if self.getType() in (Category.SOCKS, Category.HIDDEN, Category.CONTROL): - displayLabel = "" - - if self.appName: - if self.appPid: displayLabel = "%s (%s)" % (self.appName, self.appPid) - else: displayLabel = self.appName - elif self.isAppResolving: - displayLabel = "resolving..." - else: displayLabel = "UNKNOWN" - - if len(displayLabel) < width: - return ("%%-%is" % width) % displayLabel - else: return "" - - # for everything else display connection/consensus information - dstAddress = self.getDestinationLabel(26, includeLocale = True) - etc, usedSpace = "", 0 - if listingType == entries.ListingType.IP_ADDRESS: - if width > usedSpace + 42 and CONFIG["features.connection.showColumn.fingerprint"]: - # show fingerprint (column width: 42 characters) - etc += "%-40s " % self.foreign.getFingerprint() - usedSpace += 42 - - if width > usedSpace + 10 and CONFIG["features.connection.showColumn.nickname"]: - # show nickname (column width: remainder) - nicknameSpace = width - usedSpace - nicknameLabel = uiTools.cropStr(self.foreign.getNickname(), nicknameSpace, 0) - etc += ("%%-%is " % nicknameSpace) % nicknameLabel - usedSpace += nicknameSpace + 2 - elif listingType == entries.ListingType.HOSTNAME: - if width > usedSpace + 28 and CONFIG["features.connection.showColumn.destination"]: - # show destination ip/port/locale (column width: 28 characters) - etc += "%-26s " % dstAddress - usedSpace += 28 - - if width > usedSpace + 42 and CONFIG["features.connection.showColumn.fingerprint"]: - # show fingerprint (column width: 42 characters) - etc += "%-40s " % self.foreign.getFingerprint() - usedSpace += 42 - - if width > usedSpace + 17 and CONFIG["features.connection.showColumn.nickname"]: - # show nickname (column width: min 17 characters, uses half of the remainder) - nicknameSpace = 15 + (width - (usedSpace + 17)) / 2 - nicknameLabel = uiTools.cropStr(self.foreign.getNickname(), nicknameSpace, 0) - etc += ("%%-%is " % nicknameSpace) % nicknameLabel - usedSpace += (nicknameSpace + 2) - elif listingType == entries.ListingType.FINGERPRINT: - if width > usedSpace + 17: - # show nickname (column width: min 17 characters, consumes any remaining space) - nicknameSpace = width - usedSpace - 2 - - # if there's room then also show a column with the destination - # ip/port/locale (column width: 28 characters) - isIpLocaleIncluded = width > usedSpace + 45 - isIpLocaleIncluded &= CONFIG["features.connection.showColumn.destination"] - if isIpLocaleIncluded: nicknameSpace -= 28 - - if CONFIG["features.connection.showColumn.nickname"]: - nicknameLabel = uiTools.cropStr(self.foreign.getNickname(), nicknameSpace, 0) - etc += ("%%-%is " % nicknameSpace) % nicknameLabel - usedSpace += nicknameSpace + 2 - - if isIpLocaleIncluded: - etc += "%-26s " % dstAddress - usedSpace += 28 - else: - if width > usedSpace + 42 and CONFIG["features.connection.showColumn.fingerprint"]: - # show fingerprint (column width: 42 characters) - etc += "%-40s " % self.foreign.getFingerprint() - usedSpace += 42 - - if width > usedSpace + 28 and CONFIG["features.connection.showColumn.destination"]: - # show destination ip/port/locale (column width: 28 characters) - etc += "%-26s " % dstAddress - usedSpace += 28 - - return ("%%-%is" % width) % etc - - def _getListingContent(self, width, listingType): - """ - Provides the source, destination, and extra info for our listing. - - Arguments: - width - maximum length of the line - listingType - primary attribute we're listing connections by - """ - - conn = torTools.getConn() - myType = self.getType() - dstAddress = self.getDestinationLabel(26, includeLocale = True) - - # The required widths are the sum of the following: - # - room for LABEL_FORMAT and LABEL_MIN_PADDING (11 characters) - # - base data for the listing - # - that extra field plus any previous - - usedSpace = len(LABEL_FORMAT % tuple([""] * 4)) + LABEL_MIN_PADDING - localPort = ":%s" % self.local.getPort() if self.includePort else "" - - src, dst, etc = "", "", "" - if listingType == entries.ListingType.IP_ADDRESS: - myExternalIpAddr = conn.getInfo("address", self.local.getIpAddr()) - addrDiffer = myExternalIpAddr != self.local.getIpAddr() - - # Expanding doesn't make sense, if the connection isn't actually - # going through Tor's external IP address. As there isn't a known - # method for checking if it is, we're checking the type instead. - # - # This isn't entirely correct. It might be a better idea to check if - # the source and destination addresses are both private, but that might - # not be perfectly reliable either. - - isExpansionType = not myType in (Category.SOCKS, Category.HIDDEN, Category.CONTROL) - - if isExpansionType: srcAddress = myExternalIpAddr + localPort - else: srcAddress = self.local.getIpAddr() + localPort - - if myType in (Category.SOCKS, Category.CONTROL): - # Like inbound connections these need their source and destination to - # be swapped. However, this only applies when listing by IP or hostname - # (their fingerprint and nickname are both for us). Reversing the - # fields here to keep the same column alignments. - - src = "%-21s" % dstAddress - dst = "%-26s" % srcAddress - else: - src = "%-21s" % srcAddress # ip:port = max of 21 characters - dst = "%-26s" % dstAddress # ip:port (xx) = max of 26 characters - - usedSpace += len(src) + len(dst) # base data requires 47 characters - - # Showing the fingerprint (which has the width of 42) has priority over - # an expanded address field. Hence check if we either have space for - # both or wouldn't be showing the fingerprint regardless. - - isExpandedAddrVisible = width > usedSpace + 28 - if isExpandedAddrVisible and CONFIG["features.connection.showColumn.fingerprint"]: - isExpandedAddrVisible = width < usedSpace + 42 or width > usedSpace + 70 - - if addrDiffer and isExpansionType and isExpandedAddrVisible and self.includeExpandedIpAddr and CONFIG["features.connection.showColumn.expandedIp"]: - # include the internal address in the src (extra 28 characters) - internalAddress = self.local.getIpAddr() + localPort - - # If this is an inbound connection then reverse ordering so it's: - # <foreign> --> <external> --> <internal> - # when the src and dst are swapped later - - if myType == Category.INBOUND: src = "%-21s --> %s" % (src, internalAddress) - else: src = "%-21s --> %s" % (internalAddress, src) - - usedSpace += 28 - - etc = self.getEtcContent(width - usedSpace, listingType) - usedSpace += len(etc) - elif listingType == entries.ListingType.HOSTNAME: - # 15 characters for source, and a min of 40 reserved for the destination - # TODO: when actually functional the src and dst need to be swapped for - # SOCKS and CONTROL connections - src = "localhost%-6s" % localPort - usedSpace += len(src) - minHostnameSpace = 40 - - etc = self.getEtcContent(width - usedSpace - minHostnameSpace, listingType) - usedSpace += len(etc) - - hostnameSpace = width - usedSpace - usedSpace = width # prevents padding at the end - if self.isPrivate(): - dst = ("%%-%is" % hostnameSpace) % "<scrubbed>" - else: - hostname = self.foreign.getHostname(self.foreign.getIpAddr()) - portLabel = ":%-5s" % self.foreign.getPort() if self.includePort else "" - - # truncates long hostnames and sets dst to <hostname>:<port> - hostname = uiTools.cropStr(hostname, hostnameSpace, 0) - dst = ("%%-%is" % hostnameSpace) % (hostname + portLabel) - elif listingType == entries.ListingType.FINGERPRINT: - src = "localhost" - if myType == Category.CONTROL: dst = "localhost" - else: dst = self.foreign.getFingerprint() - dst = "%-40s" % dst - - usedSpace += len(src) + len(dst) # base data requires 49 characters - - etc = self.getEtcContent(width - usedSpace, listingType) - usedSpace += len(etc) - else: - # base data requires 50 min characters - src = self.local.getNickname() - if myType == Category.CONTROL: dst = self.local.getNickname() - else: dst = self.foreign.getNickname() - minBaseSpace = 50 - - etc = self.getEtcContent(width - usedSpace - minBaseSpace, listingType) - usedSpace += len(etc) - - baseSpace = width - usedSpace - usedSpace = width # prevents padding at the end - - if len(src) + len(dst) > baseSpace: - src = uiTools.cropStr(src, baseSpace / 3) - dst = uiTools.cropStr(dst, baseSpace - len(src)) - - # pads dst entry to its max space - dst = ("%%-%is" % (baseSpace - len(src))) % dst - - if myType == Category.INBOUND: src, dst = dst, src - padding = " " * (width - usedSpace + LABEL_MIN_PADDING) - return LABEL_FORMAT % (src, dst, etc, padding) - - def _getDetailContent(self, width): - """ - Provides a list with detailed information for this connection. - - Arguments: - width - max length of lines - """ - - lines = [""] * 7 - lines[0] = "address: %s" % self.getDestinationLabel(width - 11) - lines[1] = "locale: %s" % ("??" if self.isPrivate() else self.foreign.getLocale("??")) - - # Remaining data concerns the consensus results, with three possible cases: - # - if there's a single match then display its details - # - if there's multiple potential relays then list all of the combinations - # of ORPorts / Fingerprints - # - if no consensus data is available then say so (probably a client or - # exit connection) - - fingerprint = self.foreign.getFingerprint() - conn = torTools.getConn() - - if fingerprint != "UNKNOWN": - # single match - display information available about it - nsEntry = conn.getConsensusEntry(fingerprint) - descEntry = conn.getDescriptorEntry(fingerprint) - - # append the fingerprint to the second line - lines[1] = "%-13sfingerprint: %s" % (lines[1], fingerprint) - - if nsEntry: - # example consensus entry: - # r murble R8sCM1ar1sS2GulQYFVmvN95xsk RJr6q+wkTFG+ng5v2bdCbVVFfA4 2011-02-21 00:25:32 195.43.157.85 443 0 - # s Exit Fast Guard Named Running Stable Valid - # w Bandwidth=2540 - # p accept 20-23,43,53,79-81,88,110,143,194,443 - - nsLines = nsEntry.split("\n") - - firstLineComp = nsLines[0].split(" ") - if len(firstLineComp) >= 9: - _, nickname, _, _, pubDate, pubTime, _, orPort, dirPort = firstLineComp[:9] - else: nickname, pubDate, pubTime, orPort, dirPort = "", "", "", "", "" - - flags = "unknown" - if len(nsLines) >= 2 and nsLines[1].startswith("s "): - flags = nsLines[1][2:] - - # The network status exit policy doesn't exist for older tor versions. - # If unavailable we'll need the full exit policy which is on the - # descriptor (if that's available). - - exitPolicy = "unknown" - if len(nsLines) >= 4 and nsLines[3].startswith("p "): - exitPolicy = nsLines[3][2:].replace(",", ", ") - elif descEntry: - # the descriptor has an individual line for each entry in the exit policy - exitPolicyEntries = [] - - for line in descEntry.split("\n"): - if line.startswith("accept") or line.startswith("reject"): - exitPolicyEntries.append(line.strip()) - - exitPolicy = ", ".join(exitPolicyEntries) - - dirPortLabel = "" if dirPort == "0" else "dirport: %s" % dirPort - lines[2] = "nickname: %-25s orport: %-10s %s" % (nickname, orPort, dirPortLabel) - lines[3] = "published: %s %s" % (pubDate, pubTime) - lines[4] = "flags: %s" % flags.replace(" ", ", ") - lines[5] = "exit policy: %s" % exitPolicy - - if descEntry: - torVersion, platform, contact = "", "", "" - - for descLine in descEntry.split("\n"): - if descLine.startswith("platform"): - # has the tor version and platform, ex: - # platform Tor 0.2.1.29 (r318f470bc5f2ad43) on Linux x86_64 - - torVersion = descLine[13:descLine.find(" ", 13)] - platform = descLine[descLine.rfind(" on ") + 4:] - elif descLine.startswith("contact"): - contact = descLine[8:] - - # clears up some highly common obscuring - for alias in (" at ", " AT "): contact = contact.replace(alias, "@") - for alias in (" dot ", " DOT "): contact = contact.replace(alias, ".") - - break # contact lines come after the platform - - lines[3] = "%-35s os: %-14s version: %s" % (lines[3], platform, torVersion) - - # contact information is an optional field - if contact: lines[6] = "contact: %s" % contact - else: - allMatches = conn.getRelayFingerprint(self.foreign.getIpAddr(), getAllMatches = True) - - if allMatches: - # multiple matches - lines[2] = "Multiple matches, possible fingerprints are:" - - for i in range(len(allMatches)): - isLastLine = i == 3 - - relayPort, relayFingerprint = allMatches[i] - lineText = "%i. or port: %-5s fingerprint: %s" % (i, relayPort, relayFingerprint) - - # if there's multiple lines remaining at the end then give a count - remainingRelays = len(allMatches) - i - if isLastLine and remainingRelays > 1: - lineText = "... %i more" % remainingRelays - - lines[3 + i] = lineText - - if isLastLine: break - else: - # no consensus entry for this ip address - lines[2] = "No consensus data found" - - # crops any lines that are too long - for i in range(len(lines)): - lines[i] = uiTools.cropStr(lines[i], width - 2) - - return lines - - def getDestinationLabel(self, maxLength, includeLocale=False, includeHostname=False): - """ - Provides a short description of the destination. This is made up of two - components, the base <ip addr>:<port> and an extra piece of information in - parentheses. The IP address is scrubbed from private connections. - - Extra information is... - - the port's purpose for exit connections - - the locale and/or hostname if set to do so, the address isn't private, - and isn't on the local network - - nothing otherwise - - Arguments: - maxLength - maximum length of the string returned - includeLocale - possibly includes the locale - includeHostname - possibly includes the hostname - """ - - # the port and port derived data can be hidden by config or without includePort - includePort = self.includePort and (CONFIG["features.connection.showExitPort"] or self.getType() != Category.EXIT) - - # destination of the connection - ipLabel = "<scrubbed>" if self.isPrivate() else self.foreign.getIpAddr() - portLabel = ":%s" % self.foreign.getPort() if includePort else "" - dstAddress = ipLabel + portLabel - - # Only append the extra info if there's at least a couple characters of - # space (this is what's needed for the country codes). - if len(dstAddress) + 5 <= maxLength: - spaceAvailable = maxLength - len(dstAddress) - 3 - - if self.getType() == Category.EXIT and includePort: - purpose = connections.getPortUsage(self.foreign.getPort()) - - if purpose: - # BitTorrent is a common protocol to truncate, so just use "Torrent" - # if there's not enough room. - if len(purpose) > spaceAvailable and purpose == "BitTorrent": - purpose = "Torrent" - - # crops with a hyphen if too long - purpose = uiTools.cropStr(purpose, spaceAvailable, endType = uiTools.Ending.HYPHEN) - - dstAddress += " (%s)" % purpose - elif not connections.isIpAddressPrivate(self.foreign.getIpAddr()): - extraInfo = [] - - if includeLocale: - foreignLocale = self.foreign.getLocale("??") - extraInfo.append(foreignLocale) - spaceAvailable -= len(foreignLocale) + 2 - - if includeHostname: - dstHostname = self.foreign.getHostname() - - if dstHostname: - # determines the full space available, taking into account the ", " - # dividers if there's multiple pieces of extra data - - maxHostnameSpace = spaceAvailable - 2 * len(extraInfo) - dstHostname = uiTools.cropStr(dstHostname, maxHostnameSpace) - extraInfo.append(dstHostname) - spaceAvailable -= len(dstHostname) - - if extraInfo: - dstAddress += " (%s)" % ", ".join(extraInfo) - - return dstAddress[:maxLength] - Copied: arm/release/src/interface/connections/connEntry.py (from rev 24554, arm/trunk/src/interface/connections/connEntry.py) =================================================================== --- arm/release/src/interface/connections/connEntry.py (rev 0) +++ arm/release/src/interface/connections/connEntry.py 2011-04-04 15:22:31 UTC (rev 24555) @@ -0,0 +1,864 @@ +""" +Connection panel entries related to actual connections to or from the system +(ie, results seen by netstat, lsof, etc). +""" + +import time +import curses + +from util import connections, enum, torTools, uiTools +from interface.connections import entries + +# Connection Categories: +# Inbound Relay connection, coming to us. +# Outbound Relay connection, leaving us. +# Exit Outbound relay connection leaving the Tor network. +# Hidden Connections to a hidden service we're providing. +# Socks Socks connections for applications using Tor. +# Circuit Circuits our tor client has created. +# Directory Fetching tor consensus information. +# Control Tor controller (arm, vidalia, etc). + +Category = enum.Enum("INBOUND", "OUTBOUND", "EXIT", "HIDDEN", "SOCKS", "CIRCUIT", "DIRECTORY", "CONTROL") +CATEGORY_COLOR = {Category.INBOUND: "green", Category.OUTBOUND: "blue", + Category.EXIT: "red", Category.HIDDEN: "magenta", + Category.SOCKS: "yellow", Category.CIRCUIT: "cyan", + Category.DIRECTORY: "magenta", Category.CONTROL: "red"} + +# static data for listing format +# <src> --> <dst> <etc><padding> +LABEL_FORMAT = "%s --> %s %s%s" +LABEL_MIN_PADDING = 2 # min space between listing label and following data + +# sort value for scrubbed ip addresses +SCRUBBED_IP_VAL = 255 ** 4 + +CONFIG = {"features.connection.markInitialConnections": True, + "features.connection.showExitPort": True, + "features.connection.showColumn.fingerprint": True, + "features.connection.showColumn.nickname": True, + "features.connection.showColumn.destination": True, + "features.connection.showColumn.expandedIp": True} + +def loadConfig(config): + config.update(CONFIG) + +class Endpoint: + """ + Collection of attributes associated with a connection endpoint. This is a + thin wrapper for torUtil functions, making use of its caching for + performance. + """ + + def __init__(self, ipAddr, port): + self.ipAddr = ipAddr + self.port = port + + # if true, we treat the port as an ORPort when searching for matching + # fingerprints (otherwise the ORPort is assumed to be unknown) + self.isORPort = False + + # if set then this overwrites fingerprint lookups + self.fingerprintOverwrite = None + + def getIpAddr(self): + """ + Provides the IP address of the endpoint. + """ + + return self.ipAddr + + def getPort(self): + """ + Provides the port of the endpoint. + """ + + return self.port + + def getHostname(self, default = None): + """ + Provides the hostname associated with the relay's address. This is a + non-blocking call and returns None if the address either can't be resolved + or hasn't been resolved yet. + + Arguments: + default - return value if no hostname is available + """ + + # TODO: skipping all hostname resolution to be safe for now + #try: + # myHostname = hostnames.resolve(self.ipAddr) + #except: + # # either a ValueError or IOError depending on the source of the lookup failure + # myHostname = None + # + #if not myHostname: return default + #else: return myHostname + + return default + + def getLocale(self, default=None): + """ + Provides the two letter country code for the IP address' locale. + + Arguments: + default - return value if no locale information is available + """ + + conn = torTools.getConn() + return conn.getInfo("ip-to-country/%s" % self.ipAddr, default) + + def getFingerprint(self): + """ + Provides the fingerprint of the relay, returning "UNKNOWN" if it can't be + determined. + """ + + if self.fingerprintOverwrite: + return self.fingerprintOverwrite + + conn = torTools.getConn() + orPort = self.port if self.isORPort else None + myFingerprint = conn.getRelayFingerprint(self.ipAddr, orPort) + + if myFingerprint: return myFingerprint + else: return "UNKNOWN" + + def getNickname(self): + """ + Provides the nickname of the relay, retuning "UNKNOWN" if it can't be + determined. + """ + + myFingerprint = self.getFingerprint() + + if myFingerprint != "UNKNOWN": + conn = torTools.getConn() + myNickname = conn.getRelayNickname(myFingerprint) + + if myNickname: return myNickname + else: return "UNKNOWN" + else: return "UNKNOWN" + +class ConnectionEntry(entries.ConnectionPanelEntry): + """ + Represents a connection being made to or from this system. These only + concern real connections so it includes the inbound, outbound, directory, + application, and controller categories. + """ + + def __init__(self, lIpAddr, lPort, fIpAddr, fPort): + entries.ConnectionPanelEntry.__init__(self) + self.lines = [ConnectionLine(lIpAddr, lPort, fIpAddr, fPort)] + + def getSortValue(self, attr, listingType): + """ + Provides the value of a single attribute used for sorting purposes. + """ + + connLine = self.lines[0] + if attr == entries.SortAttr.IP_ADDRESS: + if connLine.isPrivate(): return SCRUBBED_IP_VAL # orders at the end + return connLine.sortIpAddr + elif attr == entries.SortAttr.PORT: + return connLine.sortPort + elif attr == entries.SortAttr.HOSTNAME: + if connLine.isPrivate(): return "" + return connLine.foreign.getHostname("") + elif attr == entries.SortAttr.FINGERPRINT: + return connLine.foreign.getFingerprint() + elif attr == entries.SortAttr.NICKNAME: + myNickname = connLine.foreign.getNickname() + if myNickname == "UNKNOWN": return "z" * 20 # orders at the end + else: return myNickname.lower() + elif attr == entries.SortAttr.CATEGORY: + return Category.indexOf(connLine.getType()) + elif attr == entries.SortAttr.UPTIME: + return connLine.startTime + elif attr == entries.SortAttr.COUNTRY: + if connections.isIpAddressPrivate(self.lines[0].foreign.getIpAddr()): return "" + else: return connLine.foreign.getLocale("") + else: + return entries.ConnectionPanelEntry.getSortValue(self, attr, listingType) + +class ConnectionLine(entries.ConnectionPanelLine): + """ + Display component of the ConnectionEntry. + """ + + def __init__(self, lIpAddr, lPort, fIpAddr, fPort, includePort=True, includeExpandedIpAddr=True): + entries.ConnectionPanelLine.__init__(self) + + self.local = Endpoint(lIpAddr, lPort) + self.foreign = Endpoint(fIpAddr, fPort) + self.startTime = time.time() + self.isInitialConnection = False + + # overwrite the local fingerprint with ours + conn = torTools.getConn() + self.local.fingerprintOverwrite = conn.getInfo("fingerprint") + + # True if the connection has matched the properties of a client/directory + # connection every time we've checked. The criteria we check is... + # client - first hop in an established circuit + # directory - matches an established single-hop circuit (probably a + # directory mirror) + + self._possibleClient = True + self._possibleDirectory = True + + # attributes for SOCKS, HIDDEN, and CONTROL connections + self.appName = None + self.appPid = None + self.isAppResolving = False + + myOrPort = conn.getOption("ORPort") + myDirPort = conn.getOption("DirPort") + mySocksPort = conn.getOption("SocksPort", "9050") + myCtlPort = conn.getOption("ControlPort") + myHiddenServicePorts = conn.getHiddenServicePorts() + + # the ORListenAddress can overwrite the ORPort + listenAddr = conn.getOption("ORListenAddress") + if listenAddr and ":" in listenAddr: + myOrPort = listenAddr[listenAddr.find(":") + 1:] + + if lPort in (myOrPort, myDirPort): + self.baseType = Category.INBOUND + self.local.isORPort = True + elif lPort == mySocksPort: + self.baseType = Category.SOCKS + elif fPort in myHiddenServicePorts: + self.baseType = Category.HIDDEN + elif lPort == myCtlPort: + self.baseType = Category.CONTROL + else: + self.baseType = Category.OUTBOUND + self.foreign.isORPort = True + + self.cachedType = None + + # includes the port or expanded ip address field when displaying listing + # information if true + self.includePort = includePort + self.includeExpandedIpAddr = includeExpandedIpAddr + + # cached immutable values used for sorting + self.sortIpAddr = connections.ipToInt(self.foreign.getIpAddr()) + self.sortPort = int(self.foreign.getPort()) + + def getListingEntry(self, width, currentTime, listingType): + """ + Provides the DrawEntry for this connection's listing. Lines are composed + of the following components: + <src> --> <dst> <etc> <uptime> (<type>) + + ListingType.IP_ADDRESS: + src - <internal addr:port> --> <external addr:port> + dst - <destination addr:port> + etc - <fingerprint> <nickname> + + ListingType.HOSTNAME: + src - localhost:<port> + dst - <destination hostname:port> + etc - <destination addr:port> <fingerprint> <nickname> + + ListingType.FINGERPRINT: + src - localhost + dst - <destination fingerprint> + etc - <nickname> <destination addr:port> + + ListingType.NICKNAME: + src - <source nickname> + dst - <destination nickname> + etc - <fingerprint> <destination addr:port> + + Arguments: + width - maximum length of the line + currentTime - unix timestamp for what the results should consider to be + the current time + listingType - primary attribute we're listing connections by + """ + + # fetch our (most likely cached) display entry for the listing + myListing = entries.ConnectionPanelLine.getListingEntry(self, width, currentTime, listingType) + + # fill in the current uptime and return the results + if CONFIG["features.connection.markInitialConnections"]: + timePrefix = "+" if self.isInitialConnection else " " + else: timePrefix = "" + + timeEntry = myListing.getNext() + timeEntry.text = timePrefix + "%5s" % uiTools.getTimeLabel(currentTime - self.startTime, 1) + + return myListing + + def isUnresolvedApp(self): + """ + True if our display uses application information that hasn't yet been resolved. + """ + + return self.appName == None and self.getType() in (Category.SOCKS, Category.HIDDEN, Category.CONTROL) + + def _getListingEntry(self, width, currentTime, listingType): + entryType = self.getType() + + # Lines are split into the following components in reverse: + # content - "<src> --> <dst> <etc> " + # time - "<uptime>" + # preType - " (" + # category - "<type>" + # postType - ") " + + lineFormat = uiTools.getColor(CATEGORY_COLOR[entryType]) + timeWidth = 6 if CONFIG["features.connection.markInitialConnections"] else 5 + + drawEntry = uiTools.DrawEntry(")" + " " * (9 - len(entryType)), lineFormat) + drawEntry = uiTools.DrawEntry(entryType.upper(), lineFormat | curses.A_BOLD, drawEntry) + drawEntry = uiTools.DrawEntry(" (", lineFormat, drawEntry) + drawEntry = uiTools.DrawEntry(" " * timeWidth, lineFormat, drawEntry) + drawEntry = uiTools.DrawEntry(self._getListingContent(width - (12 + timeWidth), listingType), lineFormat, drawEntry) + return drawEntry + + def _getDetails(self, width): + """ + Provides details on the connection, correlated against available consensus + data. + + Arguments: + width - available space to display in + """ + + detailFormat = curses.A_BOLD | uiTools.getColor(CATEGORY_COLOR[self.getType()]) + return [uiTools.DrawEntry(line, detailFormat) for line in self._getDetailContent(width)] + + def _getDescriptors(self, width): + """ + Provides raw descriptor information for the relay. + + Arguments: + width - available space to display in + """ + + # TODO: Porting and refactoring the descriptorPopup.py functionality is + # gonna take quite a bit of work. This is a very rarely used feature and + # not worth delaying the 1.4.2 release any further, so this will be a part + # of 1.4.3. + + return [] + + def resetDisplay(self): + entries.ConnectionPanelLine.resetDisplay(self) + self.cachedType = None + + def isPrivate(self): + """ + Returns true if the endpoint is private, possibly belonging to a client + connection or exit traffic. + """ + + # This is used to scrub private information from the interface. Relaying + # etiquette (and wiretapping laws) say these are bad things to look at so + # DON'T CHANGE THIS UNLESS YOU HAVE A DAMN GOOD REASON! + + myType = self.getType() + + if myType == Category.INBOUND: + # if we're a guard or bridge and the connection doesn't belong to a + # known relay then it might be client traffic + + conn = torTools.getConn() + if "Guard" in conn.getMyFlags() or conn.getOption("BridgeRelay") == "1": + allMatches = conn.getRelayFingerprint(self.foreign.getIpAddr(), getAllMatches = True) + return allMatches == [] + elif myType == Category.EXIT: + # DNS connections exiting us aren't private (since they're hitting our + # resolvers). Everything else, however, is. + + # TODO: Ideally this would also double check that it's a UDP connection + # (since DNS is the only UDP connections Tor will relay), however this + # will take a bit more work to propagate the information up from the + # connection resolver. + return self.foreign.getPort() != "53" + + # for everything else this isn't a concern + return False + + def getType(self): + """ + Provides our best guess at the current type of the connection. This + depends on consensus results, our current client circuits, etc. Results + are cached until this entry's display is reset. + """ + + # caches both to simplify the calls and to keep the type consistent until + # we want to reflect changes + if not self.cachedType: + if self.baseType == Category.OUTBOUND: + # Currently the only non-static categories are OUTBOUND vs... + # - EXIT since this depends on the current consensus + # - CIRCUIT if this is likely to belong to our guard usage + # - DIRECTORY if this is a single-hop circuit (directory mirror?) + # + # The exitability, circuits, and fingerprints are all cached by the + # torTools util keeping this a quick lookup. + + conn = torTools.getConn() + destFingerprint = self.foreign.getFingerprint() + + if destFingerprint == "UNKNOWN": + # Not a known relay. This might be an exit connection. + + if conn.isExitingAllowed(self.foreign.getIpAddr(), self.foreign.getPort()): + self.cachedType = Category.EXIT + elif self._possibleClient or self._possibleDirectory: + # This belongs to a known relay. If we haven't eliminated ourselves as + # a possible client or directory connection then check if it still + # holds true. + + myCircuits = conn.getCircuits() + + if self._possibleClient: + # Checks that this belongs to the first hop in a circuit that's + # either unestablished or longer than a single hop (ie, anything but + # a built 1-hop connection since those are most likely a directory + # mirror). + + for _, status, _, path in myCircuits: + if path[0] == destFingerprint and (status != "BUILT" or len(path) > 1): + self.cachedType = Category.CIRCUIT # matched a probable guard connection + + # if we fell through, we can eliminate ourselves as a guard in the future + if not self.cachedType: + self._possibleClient = False + + if self._possibleDirectory: + # Checks if we match a built, single hop circuit. + + for _, status, _, path in myCircuits: + if path[0] == destFingerprint and status == "BUILT" and len(path) == 1: + self.cachedType = Category.DIRECTORY + + # if we fell through, eliminate ourselves as a directory connection + if not self.cachedType: + self._possibleDirectory = False + + if not self.cachedType: + self.cachedType = self.baseType + + return self.cachedType + + def getEtcContent(self, width, listingType): + """ + Provides the optional content for the connection. + + Arguments: + width - maximum length of the line + listingType - primary attribute we're listing connections by + """ + + # for applications show the command/pid + if self.getType() in (Category.SOCKS, Category.HIDDEN, Category.CONTROL): + displayLabel = "" + + if self.appName: + if self.appPid: displayLabel = "%s (%s)" % (self.appName, self.appPid) + else: displayLabel = self.appName + elif self.isAppResolving: + displayLabel = "resolving..." + else: displayLabel = "UNKNOWN" + + if len(displayLabel) < width: + return ("%%-%is" % width) % displayLabel + else: return "" + + # for everything else display connection/consensus information + dstAddress = self.getDestinationLabel(26, includeLocale = True) + etc, usedSpace = "", 0 + if listingType == entries.ListingType.IP_ADDRESS: + if width > usedSpace + 42 and CONFIG["features.connection.showColumn.fingerprint"]: + # show fingerprint (column width: 42 characters) + etc += "%-40s " % self.foreign.getFingerprint() + usedSpace += 42 + + if width > usedSpace + 10 and CONFIG["features.connection.showColumn.nickname"]: + # show nickname (column width: remainder) + nicknameSpace = width - usedSpace + nicknameLabel = uiTools.cropStr(self.foreign.getNickname(), nicknameSpace, 0) + etc += ("%%-%is " % nicknameSpace) % nicknameLabel + usedSpace += nicknameSpace + 2 + elif listingType == entries.ListingType.HOSTNAME: + if width > usedSpace + 28 and CONFIG["features.connection.showColumn.destination"]: + # show destination ip/port/locale (column width: 28 characters) + etc += "%-26s " % dstAddress + usedSpace += 28 + + if width > usedSpace + 42 and CONFIG["features.connection.showColumn.fingerprint"]: + # show fingerprint (column width: 42 characters) + etc += "%-40s " % self.foreign.getFingerprint() + usedSpace += 42 + + if width > usedSpace + 17 and CONFIG["features.connection.showColumn.nickname"]: + # show nickname (column width: min 17 characters, uses half of the remainder) + nicknameSpace = 15 + (width - (usedSpace + 17)) / 2 + nicknameLabel = uiTools.cropStr(self.foreign.getNickname(), nicknameSpace, 0) + etc += ("%%-%is " % nicknameSpace) % nicknameLabel + usedSpace += (nicknameSpace + 2) + elif listingType == entries.ListingType.FINGERPRINT: + if width > usedSpace + 17: + # show nickname (column width: min 17 characters, consumes any remaining space) + nicknameSpace = width - usedSpace - 2 + + # if there's room then also show a column with the destination + # ip/port/locale (column width: 28 characters) + isIpLocaleIncluded = width > usedSpace + 45 + isIpLocaleIncluded &= CONFIG["features.connection.showColumn.destination"] + if isIpLocaleIncluded: nicknameSpace -= 28 + + if CONFIG["features.connection.showColumn.nickname"]: + nicknameLabel = uiTools.cropStr(self.foreign.getNickname(), nicknameSpace, 0) + etc += ("%%-%is " % nicknameSpace) % nicknameLabel + usedSpace += nicknameSpace + 2 + + if isIpLocaleIncluded: + etc += "%-26s " % dstAddress + usedSpace += 28 + else: + if width > usedSpace + 42 and CONFIG["features.connection.showColumn.fingerprint"]: + # show fingerprint (column width: 42 characters) + etc += "%-40s " % self.foreign.getFingerprint() + usedSpace += 42 + + if width > usedSpace + 28 and CONFIG["features.connection.showColumn.destination"]: + # show destination ip/port/locale (column width: 28 characters) + etc += "%-26s " % dstAddress + usedSpace += 28 + + return ("%%-%is" % width) % etc + + def _getListingContent(self, width, listingType): + """ + Provides the source, destination, and extra info for our listing. + + Arguments: + width - maximum length of the line + listingType - primary attribute we're listing connections by + """ + + conn = torTools.getConn() + myType = self.getType() + dstAddress = self.getDestinationLabel(26, includeLocale = True) + + # The required widths are the sum of the following: + # - room for LABEL_FORMAT and LABEL_MIN_PADDING (11 characters) + # - base data for the listing + # - that extra field plus any previous + + usedSpace = len(LABEL_FORMAT % tuple([""] * 4)) + LABEL_MIN_PADDING + localPort = ":%s" % self.local.getPort() if self.includePort else "" + + src, dst, etc = "", "", "" + if listingType == entries.ListingType.IP_ADDRESS: + myExternalIpAddr = conn.getInfo("address", self.local.getIpAddr()) + addrDiffer = myExternalIpAddr != self.local.getIpAddr() + + # Expanding doesn't make sense, if the connection isn't actually + # going through Tor's external IP address. As there isn't a known + # method for checking if it is, we're checking the type instead. + # + # This isn't entirely correct. It might be a better idea to check if + # the source and destination addresses are both private, but that might + # not be perfectly reliable either. + + isExpansionType = not myType in (Category.SOCKS, Category.HIDDEN, Category.CONTROL) + + if isExpansionType: srcAddress = myExternalIpAddr + localPort + else: srcAddress = self.local.getIpAddr() + localPort + + if myType in (Category.SOCKS, Category.CONTROL): + # Like inbound connections these need their source and destination to + # be swapped. However, this only applies when listing by IP or hostname + # (their fingerprint and nickname are both for us). Reversing the + # fields here to keep the same column alignments. + + src = "%-21s" % dstAddress + dst = "%-26s" % srcAddress + else: + src = "%-21s" % srcAddress # ip:port = max of 21 characters + dst = "%-26s" % dstAddress # ip:port (xx) = max of 26 characters + + usedSpace += len(src) + len(dst) # base data requires 47 characters + + # Showing the fingerprint (which has the width of 42) has priority over + # an expanded address field. Hence check if we either have space for + # both or wouldn't be showing the fingerprint regardless. + + isExpandedAddrVisible = width > usedSpace + 28 + if isExpandedAddrVisible and CONFIG["features.connection.showColumn.fingerprint"]: + isExpandedAddrVisible = width < usedSpace + 42 or width > usedSpace + 70 + + if addrDiffer and isExpansionType and isExpandedAddrVisible and self.includeExpandedIpAddr and CONFIG["features.connection.showColumn.expandedIp"]: + # include the internal address in the src (extra 28 characters) + internalAddress = self.local.getIpAddr() + localPort + + # If this is an inbound connection then reverse ordering so it's: + # <foreign> --> <external> --> <internal> + # when the src and dst are swapped later + + if myType == Category.INBOUND: src = "%-21s --> %s" % (src, internalAddress) + else: src = "%-21s --> %s" % (internalAddress, src) + + usedSpace += 28 + + etc = self.getEtcContent(width - usedSpace, listingType) + usedSpace += len(etc) + elif listingType == entries.ListingType.HOSTNAME: + # 15 characters for source, and a min of 40 reserved for the destination + # TODO: when actually functional the src and dst need to be swapped for + # SOCKS and CONTROL connections + src = "localhost%-6s" % localPort + usedSpace += len(src) + minHostnameSpace = 40 + + etc = self.getEtcContent(width - usedSpace - minHostnameSpace, listingType) + usedSpace += len(etc) + + hostnameSpace = width - usedSpace + usedSpace = width # prevents padding at the end + if self.isPrivate(): + dst = ("%%-%is" % hostnameSpace) % "<scrubbed>" + else: + hostname = self.foreign.getHostname(self.foreign.getIpAddr()) + portLabel = ":%-5s" % self.foreign.getPort() if self.includePort else "" + + # truncates long hostnames and sets dst to <hostname>:<port> + hostname = uiTools.cropStr(hostname, hostnameSpace, 0) + dst = ("%%-%is" % hostnameSpace) % (hostname + portLabel) + elif listingType == entries.ListingType.FINGERPRINT: + src = "localhost" + if myType == Category.CONTROL: dst = "localhost" + else: dst = self.foreign.getFingerprint() + dst = "%-40s" % dst + + usedSpace += len(src) + len(dst) # base data requires 49 characters + + etc = self.getEtcContent(width - usedSpace, listingType) + usedSpace += len(etc) + else: + # base data requires 50 min characters + src = self.local.getNickname() + if myType == Category.CONTROL: dst = self.local.getNickname() + else: dst = self.foreign.getNickname() + minBaseSpace = 50 + + etc = self.getEtcContent(width - usedSpace - minBaseSpace, listingType) + usedSpace += len(etc) + + baseSpace = width - usedSpace + usedSpace = width # prevents padding at the end + + if len(src) + len(dst) > baseSpace: + src = uiTools.cropStr(src, baseSpace / 3) + dst = uiTools.cropStr(dst, baseSpace - len(src)) + + # pads dst entry to its max space + dst = ("%%-%is" % (baseSpace - len(src))) % dst + + if myType == Category.INBOUND: src, dst = dst, src + padding = " " * (width - usedSpace + LABEL_MIN_PADDING) + return LABEL_FORMAT % (src, dst, etc, padding) + + def _getDetailContent(self, width): + """ + Provides a list with detailed information for this connection. + + Arguments: + width - max length of lines + """ + + lines = [""] * 7 + lines[0] = "address: %s" % self.getDestinationLabel(width - 11) + lines[1] = "locale: %s" % ("??" if self.isPrivate() else self.foreign.getLocale("??")) + + # Remaining data concerns the consensus results, with three possible cases: + # - if there's a single match then display its details + # - if there's multiple potential relays then list all of the combinations + # of ORPorts / Fingerprints + # - if no consensus data is available then say so (probably a client or + # exit connection) + + fingerprint = self.foreign.getFingerprint() + conn = torTools.getConn() + + if fingerprint != "UNKNOWN": + # single match - display information available about it + nsEntry = conn.getConsensusEntry(fingerprint) + descEntry = conn.getDescriptorEntry(fingerprint) + + # append the fingerprint to the second line + lines[1] = "%-13sfingerprint: %s" % (lines[1], fingerprint) + + if nsEntry: + # example consensus entry: + # r murble R8sCM1ar1sS2GulQYFVmvN95xsk RJr6q+wkTFG+ng5v2bdCbVVFfA4 2011-02-21 00:25:32 195.43.157.85 443 0 + # s Exit Fast Guard Named Running Stable Valid + # w Bandwidth=2540 + # p accept 20-23,43,53,79-81,88,110,143,194,443 + + nsLines = nsEntry.split("\n") + + firstLineComp = nsLines[0].split(" ") + if len(firstLineComp) >= 9: + _, nickname, _, _, pubDate, pubTime, _, orPort, dirPort = firstLineComp[:9] + else: nickname, pubDate, pubTime, orPort, dirPort = "", "", "", "", "" + + flags = "unknown" + if len(nsLines) >= 2 and nsLines[1].startswith("s "): + flags = nsLines[1][2:] + + # The network status exit policy doesn't exist for older tor versions. + # If unavailable we'll need the full exit policy which is on the + # descriptor (if that's available). + + exitPolicy = "unknown" + if len(nsLines) >= 4 and nsLines[3].startswith("p "): + exitPolicy = nsLines[3][2:].replace(",", ", ") + elif descEntry: + # the descriptor has an individual line for each entry in the exit policy + exitPolicyEntries = [] + + for line in descEntry.split("\n"): + if line.startswith("accept") or line.startswith("reject"): + exitPolicyEntries.append(line.strip()) + + exitPolicy = ", ".join(exitPolicyEntries) + + dirPortLabel = "" if dirPort == "0" else "dirport: %s" % dirPort + lines[2] = "nickname: %-25s orport: %-10s %s" % (nickname, orPort, dirPortLabel) + lines[3] = "published: %s %s" % (pubDate, pubTime) + lines[4] = "flags: %s" % flags.replace(" ", ", ") + lines[5] = "exit policy: %s" % exitPolicy + + if descEntry: + torVersion, platform, contact = "", "", "" + + for descLine in descEntry.split("\n"): + if descLine.startswith("platform"): + # has the tor version and platform, ex: + # platform Tor 0.2.1.29 (r318f470bc5f2ad43) on Linux x86_64 + + torVersion = descLine[13:descLine.find(" ", 13)] + platform = descLine[descLine.rfind(" on ") + 4:] + elif descLine.startswith("contact"): + contact = descLine[8:] + + # clears up some highly common obscuring + for alias in (" at ", " AT "): contact = contact.replace(alias, "@") + for alias in (" dot ", " DOT "): contact = contact.replace(alias, ".") + + break # contact lines come after the platform + + lines[3] = "%-35s os: %-14s version: %s" % (lines[3], platform, torVersion) + + # contact information is an optional field + if contact: lines[6] = "contact: %s" % contact + else: + allMatches = conn.getRelayFingerprint(self.foreign.getIpAddr(), getAllMatches = True) + + if allMatches: + # multiple matches + lines[2] = "Multiple matches, possible fingerprints are:" + + for i in range(len(allMatches)): + isLastLine = i == 3 + + relayPort, relayFingerprint = allMatches[i] + lineText = "%i. or port: %-5s fingerprint: %s" % (i, relayPort, relayFingerprint) + + # if there's multiple lines remaining at the end then give a count + remainingRelays = len(allMatches) - i + if isLastLine and remainingRelays > 1: + lineText = "... %i more" % remainingRelays + + lines[3 + i] = lineText + + if isLastLine: break + else: + # no consensus entry for this ip address + lines[2] = "No consensus data found" + + # crops any lines that are too long + for i in range(len(lines)): + lines[i] = uiTools.cropStr(lines[i], width - 2) + + return lines + + def getDestinationLabel(self, maxLength, includeLocale=False, includeHostname=False): + """ + Provides a short description of the destination. This is made up of two + components, the base <ip addr>:<port> and an extra piece of information in + parentheses. The IP address is scrubbed from private connections. + + Extra information is... + - the port's purpose for exit connections + - the locale and/or hostname if set to do so, the address isn't private, + and isn't on the local network + - nothing otherwise + + Arguments: + maxLength - maximum length of the string returned + includeLocale - possibly includes the locale + includeHostname - possibly includes the hostname + """ + + # the port and port derived data can be hidden by config or without includePort + includePort = self.includePort and (CONFIG["features.connection.showExitPort"] or self.getType() != Category.EXIT) + + # destination of the connection + ipLabel = "<scrubbed>" if self.isPrivate() else self.foreign.getIpAddr() + portLabel = ":%s" % self.foreign.getPort() if includePort else "" + dstAddress = ipLabel + portLabel + + # Only append the extra info if there's at least a couple characters of + # space (this is what's needed for the country codes). + if len(dstAddress) + 5 <= maxLength: + spaceAvailable = maxLength - len(dstAddress) - 3 + + if self.getType() == Category.EXIT and includePort: + purpose = connections.getPortUsage(self.foreign.getPort()) + + if purpose: + # BitTorrent is a common protocol to truncate, so just use "Torrent" + # if there's not enough room. + if len(purpose) > spaceAvailable and purpose == "BitTorrent": + purpose = "Torrent" + + # crops with a hyphen if too long + purpose = uiTools.cropStr(purpose, spaceAvailable, endType = uiTools.Ending.HYPHEN) + + dstAddress += " (%s)" % purpose + elif not connections.isIpAddressPrivate(self.foreign.getIpAddr()): + extraInfo = [] + + if includeLocale: + foreignLocale = self.foreign.getLocale("??") + extraInfo.append(foreignLocale) + spaceAvailable -= len(foreignLocale) + 2 + + if includeHostname: + dstHostname = self.foreign.getHostname() + + if dstHostname: + # determines the full space available, taking into account the ", " + # dividers if there's multiple pieces of extra data + + maxHostnameSpace = spaceAvailable - 2 * len(extraInfo) + dstHostname = uiTools.cropStr(dstHostname, maxHostnameSpace) + extraInfo.append(dstHostname) + spaceAvailable -= len(dstHostname) + + if extraInfo: + dstAddress += " (%s)" % ", ".join(extraInfo) + + return dstAddress[:maxLength] + Deleted: arm/release/src/interface/connections/connPanel.py =================================================================== --- arm/trunk/src/interface/connections/connPanel.py 2011-04-04 15:15:20 UTC (rev 24554) +++ arm/release/src/interface/connections/connPanel.py 2011-04-04 15:22:31 UTC (rev 24555) @@ -1,398 +0,0 @@ -""" -Listing of the currently established connections tor has made. -""" - -import time -import curses -import threading - -from interface.connections import entries, connEntry, circEntry -from util import connections, enum, panel, torTools, uiTools - -DEFAULT_CONFIG = {"features.connection.resolveApps": True, - "features.connection.listingType": 0, - "features.connection.refreshRate": 5} - -# height of the detail panel content, not counting top and bottom border -DETAILS_HEIGHT = 7 - -# listing types -Listing = enum.Enum(("IP_ADDRESS", "IP Address"), "HOSTNAME", "FINGERPRINT", "NICKNAME") - -DEFAULT_SORT_ORDER = (entries.SortAttr.CATEGORY, entries.SortAttr.LISTING, entries.SortAttr.UPTIME) - -class ConnectionPanel(panel.Panel, threading.Thread): - """ - Listing of connections tor is making, with information correlated against - the current consensus and other data sources. - """ - - def __init__(self, stdscr, config=None): - panel.Panel.__init__(self, stdscr, "conn", 0) - threading.Thread.__init__(self) - self.setDaemon(True) - - self._sortOrdering = DEFAULT_SORT_ORDER - self._config = dict(DEFAULT_CONFIG) - - if config: - config.update(self._config, { - "features.connection.listingType": (0, len(Listing.values()) - 1), - "features.connection.refreshRate": 1}) - - sortFields = entries.SortAttr.values() - customOrdering = config.getIntCSV("features.connection.order", None, 3, 0, len(sortFields)) - - if customOrdering: - self._sortOrdering = [sortFields[i] for i in customOrdering] - - self._listingType = Listing.values()[self._config["features.connection.listingType"]] - self._scroller = uiTools.Scroller(True) - self._title = "Connections:" # title line of the panel - self._entries = [] # last fetched display entries - self._entryLines = [] # individual lines rendered from the entries listing - self._showDetails = False # presents the details panel if true - - self._lastUpdate = -1 # time the content was last revised - self._isTorRunning = True # indicates if tor is currently running or not - self._isPaused = True # prevents updates if true - self._pauseTime = None # time when the panel was paused - self._halt = False # terminates thread if true - self._cond = threading.Condition() # used for pausing the thread - self.valsLock = threading.RLock() - - # Last sampling received from the ConnectionResolver, used to detect when - # it changes. - self._lastResourceFetch = -1 - - # resolver for the command/pid associated with SOCKS, HIDDEN, and CONTROL connections - self._appResolver = connections.AppResolver("arm") - - # rate limits appResolver queries to once per update - self.appResolveSinceUpdate = False - - self._update() # populates initial entries - self._resolveApps(False) # resolves initial applications - - # mark the initially exitsing connection uptimes as being estimates - for entry in self._entries: - if isinstance(entry, connEntry.ConnectionEntry): - entry.getLines()[0].isInitialConnection = True - - # listens for when tor stops so we know to stop reflecting changes - torTools.getConn().addStatusListener(self.torStateListener) - - def torStateListener(self, conn, eventType): - """ - Freezes the connection contents when Tor stops. - - Arguments: - conn - tor controller - eventType - type of event detected - """ - - self._isTorRunning = eventType == torTools.State.INIT - - if self._isPaused or not self._isTorRunning: - if not self._pauseTime: self._pauseTime = time.time() - else: self._pauseTime = None - - self.redraw(True) - - def setPaused(self, isPause): - """ - If true, prevents the panel from updating. - """ - - if not self._isPaused == isPause: - self._isPaused = isPause - - if isPause or not self._isTorRunning: - if not self._pauseTime: self._pauseTime = time.time() - else: self._pauseTime = None - - # redraws so the display reflects any changes between the last update - # and being paused - self.redraw(True) - - def setSortOrder(self, ordering = None): - """ - Sets the connection attributes we're sorting by and resorts the contents. - - Arguments: - ordering - new ordering, if undefined then this resorts with the last - set ordering - """ - - self.valsLock.acquire() - if ordering: self._sortOrdering = ordering - self._entries.sort(key=lambda i: (i.getSortValues(self._sortOrdering, self._listingType))) - - self._entryLines = [] - for entry in self._entries: - self._entryLines += entry.getLines() - self.valsLock.release() - - def setListingType(self, listingType): - """ - Sets the priority information presented by the panel. - - Arguments: - listingType - Listing instance for the primary information to be shown - """ - - self.valsLock.acquire() - self._listingType = listingType - - # if we're sorting by the listing then we need to resort - if entries.SortAttr.LISTING in self._sortOrdering: - self.setSortOrder() - - self.valsLock.release() - - def handleKey(self, key): - self.valsLock.acquire() - - if uiTools.isScrollKey(key): - pageHeight = self.getPreferredSize()[0] - 1 - if self._showDetails: pageHeight -= (DETAILS_HEIGHT + 1) - isChanged = self._scroller.handleKey(key, self._entryLines, pageHeight) - if isChanged: self.redraw(True) - elif uiTools.isSelectionKey(key): - self._showDetails = not self._showDetails - self.redraw(True) - - self.valsLock.release() - - def run(self): - """ - Keeps connections listing updated, checking for new entries at a set rate. - """ - - lastDraw = time.time() - 1 - while not self._halt: - currentTime = time.time() - - if self._isPaused or not self._isTorRunning or currentTime - lastDraw < self._config["features.connection.refreshRate"]: - self._cond.acquire() - if not self._halt: self._cond.wait(0.2) - self._cond.release() - else: - # updates content if their's new results, otherwise just redraws - self._update() - self.redraw(True) - - # we may have missed multiple updates due to being paused, showing - # another panel, etc so lastDraw might need to jump multiple ticks - drawTicks = (time.time() - lastDraw) / self._config["features.connection.refreshRate"] - lastDraw += self._config["features.connection.refreshRate"] * drawTicks - - def draw(self, width, height): - self.valsLock.acquire() - - # extra line when showing the detail panel is for the bottom border - detailPanelOffset = DETAILS_HEIGHT + 1 if self._showDetails else 0 - isScrollbarVisible = len(self._entryLines) > height - detailPanelOffset - 1 - - scrollLoc = self._scroller.getScrollLoc(self._entryLines, height - detailPanelOffset - 1) - cursorSelection = self._scroller.getCursorSelection(self._entryLines) - - # draws the detail panel if currently displaying it - if self._showDetails: - # This is a solid border unless the scrollbar is visible, in which case a - # 'T' pipe connects the border to the bar. - uiTools.drawBox(self, 0, 0, width, DETAILS_HEIGHT + 2) - if isScrollbarVisible: self.addch(DETAILS_HEIGHT + 1, 1, curses.ACS_TTEE) - - drawEntries = cursorSelection.getDetails(width) - for i in range(min(len(drawEntries), DETAILS_HEIGHT)): - drawEntries[i].render(self, 1 + i, 2) - - # title label with connection counts - title = "Connection Details:" if self._showDetails else self._title - self.addstr(0, 0, title, curses.A_STANDOUT) - - scrollOffset = 1 - if isScrollbarVisible: - scrollOffset = 3 - self.addScrollBar(scrollLoc, scrollLoc + height - detailPanelOffset - 1, len(self._entryLines), 1 + detailPanelOffset) - - currentTime = self._pauseTime if self._pauseTime else time.time() - for lineNum in range(scrollLoc, len(self._entryLines)): - entryLine = self._entryLines[lineNum] - - # if this is an unresolved SOCKS, HIDDEN, or CONTROL entry then queue up - # resolution for the applicaitions they belong to - if isinstance(entryLine, connEntry.ConnectionLine) and entryLine.isUnresolvedApp(): - self._resolveApps() - - # hilighting if this is the selected line - extraFormat = curses.A_STANDOUT if entryLine == cursorSelection else curses.A_NORMAL - - drawEntry = entryLine.getListingEntry(width - scrollOffset, currentTime, self._listingType) - drawLine = lineNum + detailPanelOffset + 1 - scrollLoc - drawEntry.render(self, drawLine, scrollOffset, extraFormat) - if drawLine >= height: break - - self.valsLock.release() - - def stop(self): - """ - Halts further resolutions and terminates the thread. - """ - - self._cond.acquire() - self._halt = True - self._cond.notifyAll() - self._cond.release() - - def _update(self): - """ - Fetches the newest resolved connections. - """ - - connResolver = connections.getResolver("tor") - currentResolutionCount = connResolver.getResolutionCount() - self.appResolveSinceUpdate = False - - if self._lastResourceFetch != currentResolutionCount: - self.valsLock.acquire() - - newEntries = [] # the new results we'll display - - # Fetches new connections and client circuits... - # newConnections [(local ip, local port, foreign ip, foreign port)...] - # newCircuits {circuitID => (status, purpose, path)...} - - newConnections = connResolver.getConnections() - newCircuits = {} - - for circuitID, status, purpose, path in torTools.getConn().getCircuits(): - # Skips established single-hop circuits (these are for directory - # fetches, not client circuits) - if not (status == "BUILT" and len(path) == 1): - newCircuits[circuitID] = (status, purpose, path) - - # Populates newEntries with any of our old entries that still exist. - # This is both for performance and to keep from resetting the uptime - # attributes. Note that CircEntries are a ConnectionEntry subclass so - # we need to check for them first. - - for oldEntry in self._entries: - if isinstance(oldEntry, circEntry.CircEntry): - newEntry = newCircuits.get(oldEntry.circuitID) - - if newEntry: - oldEntry.update(newEntry[0], newEntry[2]) - newEntries.append(oldEntry) - del newCircuits[oldEntry.circuitID] - elif isinstance(oldEntry, connEntry.ConnectionEntry): - connLine = oldEntry.getLines()[0] - connAttr = (connLine.local.getIpAddr(), connLine.local.getPort(), - connLine.foreign.getIpAddr(), connLine.foreign.getPort()) - - if connAttr in newConnections: - newEntries.append(oldEntry) - newConnections.remove(connAttr) - - # Reset any display attributes for the entries we're keeping - for entry in newEntries: entry.resetDisplay() - - # Adds any new connection and circuit entries. - for lIp, lPort, fIp, fPort in newConnections: - newConnEntry = connEntry.ConnectionEntry(lIp, lPort, fIp, fPort) - if newConnEntry.getLines()[0].getType() != connEntry.Category.CIRCUIT: - newEntries.append(newConnEntry) - - for circuitID in newCircuits: - status, purpose, path = newCircuits[circuitID] - newEntries.append(circEntry.CircEntry(circuitID, status, purpose, path)) - - # Counts the relays in each of the categories. This also flushes the - # type cache for all of the connections (in case its changed since last - # fetched). - - categoryTypes = connEntry.Category.values() - typeCounts = dict((type, 0) for type in categoryTypes) - for entry in newEntries: - if isinstance(entry, connEntry.ConnectionEntry): - typeCounts[entry.getLines()[0].getType()] += 1 - elif isinstance(entry, circEntry.CircEntry): - typeCounts[connEntry.Category.CIRCUIT] += 1 - - # makes labels for all the categories with connections (ie, - # "21 outbound", "1 control", etc) - countLabels = [] - - for category in categoryTypes: - if typeCounts[category] > 0: - countLabels.append("%i %s" % (typeCounts[category], category.lower())) - - if countLabels: self._title = "Connections (%s):" % ", ".join(countLabels) - else: self._title = "Connections:" - - self._entries = newEntries - - self._entryLines = [] - for entry in self._entries: - self._entryLines += entry.getLines() - - self.setSortOrder() - self._lastResourceFetch = currentResolutionCount - self.valsLock.release() - - def _resolveApps(self, flagQuery = True): - """ - Triggers an asynchronous query for all unresolved SOCKS, HIDDEN, and - CONTROL entries. - - Arguments: - flagQuery - sets a flag to prevent further call from being respected - until the next update if true - """ - - if self.appResolveSinceUpdate or not self._config["features.connection.resolveApps"]: return - unresolvedLines = [l for l in self._entryLines if isinstance(l, connEntry.ConnectionLine) and l.isUnresolvedApp()] - - # get the ports used for unresolved applications - appPorts = [] - - for line in unresolvedLines: - appConn = line.local if line.getType() == connEntry.Category.HIDDEN else line.foreign - appPorts.append(appConn.getPort()) - - # Queue up resolution for the unresolved ports (skips if it's still working - # on the last query). - if appPorts and not self._appResolver.isResolving: - self._appResolver.resolve(appPorts) - - # Fetches results. If the query finishes quickly then this is what we just - # asked for, otherwise these belong to an earlier resolution. - # - # The application resolver might have given up querying (for instance, if - # the lsof lookups aren't working on this platform or lacks permissions). - # The isAppResolving flag lets the unresolved entries indicate if there's - # a lookup in progress for them or not. - - appResults = self._appResolver.getResults(0.2) - - for line in unresolvedLines: - isLocal = line.getType() == connEntry.Category.HIDDEN - linePort = line.local.getPort() if isLocal else line.foreign.getPort() - - if linePort in appResults: - # sets application attributes if there's a result with this as the - # inbound port - for inboundPort, outboundPort, cmd, pid in appResults[linePort]: - appPort = outboundPort if isLocal else inboundPort - - if linePort == appPort: - line.appName = cmd - line.appPid = pid - line.isAppResolving = False - else: - line.isAppResolving = self._appResolver.isResolving - - if flagQuery: - self.appResolveSinceUpdate = True - Copied: arm/release/src/interface/connections/connPanel.py (from rev 24554, arm/trunk/src/interface/connections/connPanel.py) =================================================================== --- arm/release/src/interface/connections/connPanel.py (rev 0) +++ arm/release/src/interface/connections/connPanel.py 2011-04-04 15:22:31 UTC (rev 24555) @@ -0,0 +1,398 @@ +""" +Listing of the currently established connections tor has made. +""" + +import time +import curses +import threading + +from interface.connections import entries, connEntry, circEntry +from util import connections, enum, panel, torTools, uiTools + +DEFAULT_CONFIG = {"features.connection.resolveApps": True, + "features.connection.listingType": 0, + "features.connection.refreshRate": 5} + +# height of the detail panel content, not counting top and bottom border +DETAILS_HEIGHT = 7 + +# listing types +Listing = enum.Enum(("IP_ADDRESS", "IP Address"), "HOSTNAME", "FINGERPRINT", "NICKNAME") + +DEFAULT_SORT_ORDER = (entries.SortAttr.CATEGORY, entries.SortAttr.LISTING, entries.SortAttr.UPTIME) + +class ConnectionPanel(panel.Panel, threading.Thread): + """ + Listing of connections tor is making, with information correlated against + the current consensus and other data sources. + """ + + def __init__(self, stdscr, config=None): + panel.Panel.__init__(self, stdscr, "conn", 0) + threading.Thread.__init__(self) + self.setDaemon(True) + + self._sortOrdering = DEFAULT_SORT_ORDER + self._config = dict(DEFAULT_CONFIG) + + if config: + config.update(self._config, { + "features.connection.listingType": (0, len(Listing.values()) - 1), + "features.connection.refreshRate": 1}) + + sortFields = entries.SortAttr.values() + customOrdering = config.getIntCSV("features.connection.order", None, 3, 0, len(sortFields)) + + if customOrdering: + self._sortOrdering = [sortFields[i] for i in customOrdering] + + self._listingType = Listing.values()[self._config["features.connection.listingType"]] + self._scroller = uiTools.Scroller(True) + self._title = "Connections:" # title line of the panel + self._entries = [] # last fetched display entries + self._entryLines = [] # individual lines rendered from the entries listing + self._showDetails = False # presents the details panel if true + + self._lastUpdate = -1 # time the content was last revised + self._isTorRunning = True # indicates if tor is currently running or not + self._isPaused = True # prevents updates if true + self._pauseTime = None # time when the panel was paused + self._halt = False # terminates thread if true + self._cond = threading.Condition() # used for pausing the thread + self.valsLock = threading.RLock() + + # Last sampling received from the ConnectionResolver, used to detect when + # it changes. + self._lastResourceFetch = -1 + + # resolver for the command/pid associated with SOCKS, HIDDEN, and CONTROL connections + self._appResolver = connections.AppResolver("arm") + + # rate limits appResolver queries to once per update + self.appResolveSinceUpdate = False + + self._update() # populates initial entries + self._resolveApps(False) # resolves initial applications + + # mark the initially exitsing connection uptimes as being estimates + for entry in self._entries: + if isinstance(entry, connEntry.ConnectionEntry): + entry.getLines()[0].isInitialConnection = True + + # listens for when tor stops so we know to stop reflecting changes + torTools.getConn().addStatusListener(self.torStateListener) + + def torStateListener(self, conn, eventType): + """ + Freezes the connection contents when Tor stops. + + Arguments: + conn - tor controller + eventType - type of event detected + """ + + self._isTorRunning = eventType == torTools.State.INIT + + if self._isPaused or not self._isTorRunning: + if not self._pauseTime: self._pauseTime = time.time() + else: self._pauseTime = None + + self.redraw(True) + + def setPaused(self, isPause): + """ + If true, prevents the panel from updating. + """ + + if not self._isPaused == isPause: + self._isPaused = isPause + + if isPause or not self._isTorRunning: + if not self._pauseTime: self._pauseTime = time.time() + else: self._pauseTime = None + + # redraws so the display reflects any changes between the last update + # and being paused + self.redraw(True) + + def setSortOrder(self, ordering = None): + """ + Sets the connection attributes we're sorting by and resorts the contents. + + Arguments: + ordering - new ordering, if undefined then this resorts with the last + set ordering + """ + + self.valsLock.acquire() + if ordering: self._sortOrdering = ordering + self._entries.sort(key=lambda i: (i.getSortValues(self._sortOrdering, self._listingType))) + + self._entryLines = [] + for entry in self._entries: + self._entryLines += entry.getLines() + self.valsLock.release() + + def setListingType(self, listingType): + """ + Sets the priority information presented by the panel. + + Arguments: + listingType - Listing instance for the primary information to be shown + """ + + self.valsLock.acquire() + self._listingType = listingType + + # if we're sorting by the listing then we need to resort + if entries.SortAttr.LISTING in self._sortOrdering: + self.setSortOrder() + + self.valsLock.release() + + def handleKey(self, key): + self.valsLock.acquire() + + if uiTools.isScrollKey(key): + pageHeight = self.getPreferredSize()[0] - 1 + if self._showDetails: pageHeight -= (DETAILS_HEIGHT + 1) + isChanged = self._scroller.handleKey(key, self._entryLines, pageHeight) + if isChanged: self.redraw(True) + elif uiTools.isSelectionKey(key): + self._showDetails = not self._showDetails + self.redraw(True) + + self.valsLock.release() + + def run(self): + """ + Keeps connections listing updated, checking for new entries at a set rate. + """ + + lastDraw = time.time() - 1 + while not self._halt: + currentTime = time.time() + + if self._isPaused or not self._isTorRunning or currentTime - lastDraw < self._config["features.connection.refreshRate"]: + self._cond.acquire() + if not self._halt: self._cond.wait(0.2) + self._cond.release() + else: + # updates content if their's new results, otherwise just redraws + self._update() + self.redraw(True) + + # we may have missed multiple updates due to being paused, showing + # another panel, etc so lastDraw might need to jump multiple ticks + drawTicks = (time.time() - lastDraw) / self._config["features.connection.refreshRate"] + lastDraw += self._config["features.connection.refreshRate"] * drawTicks + + def draw(self, width, height): + self.valsLock.acquire() + + # extra line when showing the detail panel is for the bottom border + detailPanelOffset = DETAILS_HEIGHT + 1 if self._showDetails else 0 + isScrollbarVisible = len(self._entryLines) > height - detailPanelOffset - 1 + + scrollLoc = self._scroller.getScrollLoc(self._entryLines, height - detailPanelOffset - 1) + cursorSelection = self._scroller.getCursorSelection(self._entryLines) + + # draws the detail panel if currently displaying it + if self._showDetails: + # This is a solid border unless the scrollbar is visible, in which case a + # 'T' pipe connects the border to the bar. + uiTools.drawBox(self, 0, 0, width, DETAILS_HEIGHT + 2) + if isScrollbarVisible: self.addch(DETAILS_HEIGHT + 1, 1, curses.ACS_TTEE) + + drawEntries = cursorSelection.getDetails(width) + for i in range(min(len(drawEntries), DETAILS_HEIGHT)): + drawEntries[i].render(self, 1 + i, 2) + + # title label with connection counts + title = "Connection Details:" if self._showDetails else self._title + self.addstr(0, 0, title, curses.A_STANDOUT) + + scrollOffset = 1 + if isScrollbarVisible: + scrollOffset = 3 + self.addScrollBar(scrollLoc, scrollLoc + height - detailPanelOffset - 1, len(self._entryLines), 1 + detailPanelOffset) + + currentTime = self._pauseTime if self._pauseTime else time.time() + for lineNum in range(scrollLoc, len(self._entryLines)): + entryLine = self._entryLines[lineNum] + + # if this is an unresolved SOCKS, HIDDEN, or CONTROL entry then queue up + # resolution for the applicaitions they belong to + if isinstance(entryLine, connEntry.ConnectionLine) and entryLine.isUnresolvedApp(): + self._resolveApps() + + # hilighting if this is the selected line + extraFormat = curses.A_STANDOUT if entryLine == cursorSelection else curses.A_NORMAL + + drawEntry = entryLine.getListingEntry(width - scrollOffset, currentTime, self._listingType) + drawLine = lineNum + detailPanelOffset + 1 - scrollLoc + drawEntry.render(self, drawLine, scrollOffset, extraFormat) + if drawLine >= height: break + + self.valsLock.release() + + def stop(self): + """ + Halts further resolutions and terminates the thread. + """ + + self._cond.acquire() + self._halt = True + self._cond.notifyAll() + self._cond.release() + + def _update(self): + """ + Fetches the newest resolved connections. + """ + + connResolver = connections.getResolver("tor") + currentResolutionCount = connResolver.getResolutionCount() + self.appResolveSinceUpdate = False + + if self._lastResourceFetch != currentResolutionCount: + self.valsLock.acquire() + + newEntries = [] # the new results we'll display + + # Fetches new connections and client circuits... + # newConnections [(local ip, local port, foreign ip, foreign port)...] + # newCircuits {circuitID => (status, purpose, path)...} + + newConnections = connResolver.getConnections() + newCircuits = {} + + for circuitID, status, purpose, path in torTools.getConn().getCircuits(): + # Skips established single-hop circuits (these are for directory + # fetches, not client circuits) + if not (status == "BUILT" and len(path) == 1): + newCircuits[circuitID] = (status, purpose, path) + + # Populates newEntries with any of our old entries that still exist. + # This is both for performance and to keep from resetting the uptime + # attributes. Note that CircEntries are a ConnectionEntry subclass so + # we need to check for them first. + + for oldEntry in self._entries: + if isinstance(oldEntry, circEntry.CircEntry): + newEntry = newCircuits.get(oldEntry.circuitID) + + if newEntry: + oldEntry.update(newEntry[0], newEntry[2]) + newEntries.append(oldEntry) + del newCircuits[oldEntry.circuitID] + elif isinstance(oldEntry, connEntry.ConnectionEntry): + connLine = oldEntry.getLines()[0] + connAttr = (connLine.local.getIpAddr(), connLine.local.getPort(), + connLine.foreign.getIpAddr(), connLine.foreign.getPort()) + + if connAttr in newConnections: + newEntries.append(oldEntry) + newConnections.remove(connAttr) + + # Reset any display attributes for the entries we're keeping + for entry in newEntries: entry.resetDisplay() + + # Adds any new connection and circuit entries. + for lIp, lPort, fIp, fPort in newConnections: + newConnEntry = connEntry.ConnectionEntry(lIp, lPort, fIp, fPort) + if newConnEntry.getLines()[0].getType() != connEntry.Category.CIRCUIT: + newEntries.append(newConnEntry) + + for circuitID in newCircuits: + status, purpose, path = newCircuits[circuitID] + newEntries.append(circEntry.CircEntry(circuitID, status, purpose, path)) + + # Counts the relays in each of the categories. This also flushes the + # type cache for all of the connections (in case its changed since last + # fetched). + + categoryTypes = connEntry.Category.values() + typeCounts = dict((type, 0) for type in categoryTypes) + for entry in newEntries: + if isinstance(entry, connEntry.ConnectionEntry): + typeCounts[entry.getLines()[0].getType()] += 1 + elif isinstance(entry, circEntry.CircEntry): + typeCounts[connEntry.Category.CIRCUIT] += 1 + + # makes labels for all the categories with connections (ie, + # "21 outbound", "1 control", etc) + countLabels = [] + + for category in categoryTypes: + if typeCounts[category] > 0: + countLabels.append("%i %s" % (typeCounts[category], category.lower())) + + if countLabels: self._title = "Connections (%s):" % ", ".join(countLabels) + else: self._title = "Connections:" + + self._entries = newEntries + + self._entryLines = [] + for entry in self._entries: + self._entryLines += entry.getLines() + + self.setSortOrder() + self._lastResourceFetch = currentResolutionCount + self.valsLock.release() + + def _resolveApps(self, flagQuery = True): + """ + Triggers an asynchronous query for all unresolved SOCKS, HIDDEN, and + CONTROL entries. + + Arguments: + flagQuery - sets a flag to prevent further call from being respected + until the next update if true + """ + + if self.appResolveSinceUpdate or not self._config["features.connection.resolveApps"]: return + unresolvedLines = [l for l in self._entryLines if isinstance(l, connEntry.ConnectionLine) and l.isUnresolvedApp()] + + # get the ports used for unresolved applications + appPorts = [] + + for line in unresolvedLines: + appConn = line.local if line.getType() == connEntry.Category.HIDDEN else line.foreign + appPorts.append(appConn.getPort()) + + # Queue up resolution for the unresolved ports (skips if it's still working + # on the last query). + if appPorts and not self._appResolver.isResolving: + self._appResolver.resolve(appPorts) + + # Fetches results. If the query finishes quickly then this is what we just + # asked for, otherwise these belong to an earlier resolution. + # + # The application resolver might have given up querying (for instance, if + # the lsof lookups aren't working on this platform or lacks permissions). + # The isAppResolving flag lets the unresolved entries indicate if there's + # a lookup in progress for them or not. + + appResults = self._appResolver.getResults(0.2) + + for line in unresolvedLines: + isLocal = line.getType() == connEntry.Category.HIDDEN + linePort = line.local.getPort() if isLocal else line.foreign.getPort() + + if linePort in appResults: + # sets application attributes if there's a result with this as the + # inbound port + for inboundPort, outboundPort, cmd, pid in appResults[linePort]: + appPort = outboundPort if isLocal else inboundPort + + if linePort == appPort: + line.appName = cmd + line.appPid = pid + line.isAppResolving = False + else: + line.isAppResolving = self._appResolver.isResolving + + if flagQuery: + self.appResolveSinceUpdate = True + Deleted: arm/release/src/interface/connections/entries.py =================================================================== --- arm/trunk/src/interface/connections/entries.py 2011-04-04 15:15:20 UTC (rev 24554) +++ arm/release/src/interface/connections/entries.py 2011-04-04 15:22:31 UTC (rev 24555) @@ -1,183 +0,0 @@ -""" -Interface for entries in the connection panel. These consist of two parts: the -entry itself (ie, Tor connection, client circuit, etc) and the lines it -consists of in the listing. -""" - -from util import enum - -# attributes we can list entries by -ListingType = enum.Enum(("IP_ADDRESS", "IP Address"), "HOSTNAME", "FINGERPRINT", "NICKNAME") - -SortAttr = enum.Enum("CATEGORY", "UPTIME", "LISTING", "IP_ADDRESS", "PORT", - "HOSTNAME", "FINGERPRINT", "NICKNAME", "COUNTRY") - -SORT_COLORS = {SortAttr.CATEGORY: "red", SortAttr.UPTIME: "yellow", - SortAttr.LISTING: "green", SortAttr.IP_ADDRESS: "blue", - SortAttr.PORT: "blue", SortAttr.HOSTNAME: "magenta", - SortAttr.FINGERPRINT: "cyan", SortAttr.NICKNAME: "cyan", - SortAttr.COUNTRY: "blue"} - -# maximum number of ports a system can have -PORT_COUNT = 65536 - -class ConnectionPanelEntry: - """ - Common parent for connection panel entries. This consists of a list of lines - in the panel listing. This caches results until the display indicates that - they should be flushed. - """ - - def __init__(self): - self.lines = [] - self.flushCache = True - - def getLines(self): - """ - Provides the individual lines in the connection listing. - """ - - if self.flushCache: - self.lines = self._getLines(self.lines) - self.flushCache = False - - return self.lines - - def _getLines(self, oldResults): - # implementation of getLines - - for line in oldResults: - line.resetDisplay() - - return oldResults - - def getSortValues(self, sortAttrs, listingType): - """ - Provides the value used in comparisons to sort based on the given - attribute. - - Arguments: - sortAttrs - list of SortAttr values for the field being sorted on - listingType - ListingType enumeration for the attribute we're listing - entries by - """ - - return [self.getSortValue(attr, listingType) for attr in sortAttrs] - - def getSortValue(self, attr, listingType): - """ - Provides the value of a single attribute used for sorting purposes. - - Arguments: - attr - list of SortAttr values for the field being sorted on - listingType - ListingType enumeration for the attribute we're listing - entries by - """ - - if attr == SortAttr.LISTING: - if listingType == ListingType.IP_ADDRESS: - # uses the IP address as the primary value, and port as secondary - sortValue = self.getSortValue(SortAttr.IP_ADDRESS, listingType) * PORT_COUNT - sortValue += self.getSortValue(SortAttr.PORT, listingType) - return sortValue - elif listingType == ListingType.HOSTNAME: - return self.getSortValue(SortAttr.HOSTNAME, listingType) - elif listingType == ListingType.FINGERPRINT: - return self.getSortValue(SortAttr.FINGERPRINT, listingType) - elif listingType == ListingType.NICKNAME: - return self.getSortValue(SortAttr.NICKNAME, listingType) - - return "" - - def resetDisplay(self): - """ - Flushes cached display results. - """ - - self.flushCache = True - -class ConnectionPanelLine: - """ - Individual line in the connection panel listing. - """ - - def __init__(self): - # cache for displayed information - self._listingCache = None - self._listingCacheArgs = (None, None) - - self._detailsCache = None - self._detailsCacheArgs = None - - self._descriptorCache = None - self._descriptorCacheArgs = None - - def getListingEntry(self, width, currentTime, listingType): - """ - Provides a DrawEntry instance for contents to be displayed in the - connection panel listing. - - Arguments: - width - available space to display in - currentTime - unix timestamp for what the results should consider to be - the current time (this may be ignored due to caching) - listingType - ListingType enumeration for the highest priority content - to be displayed - """ - - if self._listingCacheArgs != (width, listingType): - self._listingCache = self._getListingEntry(width, currentTime, listingType) - self._listingCacheArgs = (width, listingType) - - return self._listingCache - - def _getListingEntry(self, width, currentTime, listingType): - # implementation of getListingEntry - return None - - def getDetails(self, width): - """ - Provides a list of DrawEntry instances with detailed information for this - connection. - - Arguments: - width - available space to display in - """ - - if self._detailsCacheArgs != width: - self._detailsCache = self._getDetails(width) - self._detailsCacheArgs = width - - return self._detailsCache - - def _getDetails(self, width): - # implementation of getDetails - return [] - - def getDescriptor(self, width): - """ - Provides a list of DrawEntry instances with descriptor information for - this connection. - - Arguments: - width - available space to display in - """ - - if self._descriptorCacheArgs != width: - self._descriptorCache = self._getDescriptor(width) - self._descriptorCacheArgs = width - - return self._descriptorCache - - def _getDescriptor(self, width): - # implementation of getDescriptor - return [] - - def resetDisplay(self): - """ - Flushes cached display results. - """ - - self._listingCacheArgs = (None, None) - self._detailsCacheArgs = None - Copied: arm/release/src/interface/connections/entries.py (from rev 24554, arm/trunk/src/interface/connections/entries.py) =================================================================== --- arm/release/src/interface/connections/entries.py (rev 0) +++ arm/release/src/interface/connections/entries.py 2011-04-04 15:22:31 UTC (rev 24555) @@ -0,0 +1,183 @@ +""" +Interface for entries in the connection panel. These consist of two parts: the +entry itself (ie, Tor connection, client circuit, etc) and the lines it +consists of in the listing. +""" + +from util import enum + +# attributes we can list entries by +ListingType = enum.Enum(("IP_ADDRESS", "IP Address"), "HOSTNAME", "FINGERPRINT", "NICKNAME") + +SortAttr = enum.Enum("CATEGORY", "UPTIME", "LISTING", "IP_ADDRESS", "PORT", + "HOSTNAME", "FINGERPRINT", "NICKNAME", "COUNTRY") + +SORT_COLORS = {SortAttr.CATEGORY: "red", SortAttr.UPTIME: "yellow", + SortAttr.LISTING: "green", SortAttr.IP_ADDRESS: "blue", + SortAttr.PORT: "blue", SortAttr.HOSTNAME: "magenta", + SortAttr.FINGERPRINT: "cyan", SortAttr.NICKNAME: "cyan", + SortAttr.COUNTRY: "blue"} + +# maximum number of ports a system can have +PORT_COUNT = 65536 + +class ConnectionPanelEntry: + """ + Common parent for connection panel entries. This consists of a list of lines + in the panel listing. This caches results until the display indicates that + they should be flushed. + """ + + def __init__(self): + self.lines = [] + self.flushCache = True + + def getLines(self): + """ + Provides the individual lines in the connection listing. + """ + + if self.flushCache: + self.lines = self._getLines(self.lines) + self.flushCache = False + + return self.lines + + def _getLines(self, oldResults): + # implementation of getLines + + for line in oldResults: + line.resetDisplay() + + return oldResults + + def getSortValues(self, sortAttrs, listingType): + """ + Provides the value used in comparisons to sort based on the given + attribute. + + Arguments: + sortAttrs - list of SortAttr values for the field being sorted on + listingType - ListingType enumeration for the attribute we're listing + entries by + """ + + return [self.getSortValue(attr, listingType) for attr in sortAttrs] + + def getSortValue(self, attr, listingType): + """ + Provides the value of a single attribute used for sorting purposes. + + Arguments: + attr - list of SortAttr values for the field being sorted on + listingType - ListingType enumeration for the attribute we're listing + entries by + """ + + if attr == SortAttr.LISTING: + if listingType == ListingType.IP_ADDRESS: + # uses the IP address as the primary value, and port as secondary + sortValue = self.getSortValue(SortAttr.IP_ADDRESS, listingType) * PORT_COUNT + sortValue += self.getSortValue(SortAttr.PORT, listingType) + return sortValue + elif listingType == ListingType.HOSTNAME: + return self.getSortValue(SortAttr.HOSTNAME, listingType) + elif listingType == ListingType.FINGERPRINT: + return self.getSortValue(SortAttr.FINGERPRINT, listingType) + elif listingType == ListingType.NICKNAME: + return self.getSortValue(SortAttr.NICKNAME, listingType) + + return "" + + def resetDisplay(self): + """ + Flushes cached display results. + """ + + self.flushCache = True + +class ConnectionPanelLine: + """ + Individual line in the connection panel listing. + """ + + def __init__(self): + # cache for displayed information + self._listingCache = None + self._listingCacheArgs = (None, None) + + self._detailsCache = None + self._detailsCacheArgs = None + + self._descriptorCache = None + self._descriptorCacheArgs = None + + def getListingEntry(self, width, currentTime, listingType): + """ + Provides a DrawEntry instance for contents to be displayed in the + connection panel listing. + + Arguments: + width - available space to display in + currentTime - unix timestamp for what the results should consider to be + the current time (this may be ignored due to caching) + listingType - ListingType enumeration for the highest priority content + to be displayed + """ + + if self._listingCacheArgs != (width, listingType): + self._listingCache = self._getListingEntry(width, currentTime, listingType) + self._listingCacheArgs = (width, listingType) + + return self._listingCache + + def _getListingEntry(self, width, currentTime, listingType): + # implementation of getListingEntry + return None + + def getDetails(self, width): + """ + Provides a list of DrawEntry instances with detailed information for this + connection. + + Arguments: + width - available space to display in + """ + + if self._detailsCacheArgs != width: + self._detailsCache = self._getDetails(width) + self._detailsCacheArgs = width + + return self._detailsCache + + def _getDetails(self, width): + # implementation of getDetails + return [] + + def getDescriptor(self, width): + """ + Provides a list of DrawEntry instances with descriptor information for + this connection. + + Arguments: + width - available space to display in + """ + + if self._descriptorCacheArgs != width: + self._descriptorCache = self._getDescriptor(width) + self._descriptorCacheArgs = width + + return self._descriptorCache + + def _getDescriptor(self, width): + # implementation of getDescriptor + return [] + + def resetDisplay(self): + """ + Flushes cached display results. + """ + + self._listingCacheArgs = (None, None) + self._detailsCacheArgs = None + Modified: arm/release/src/interface/controller.py =================================================================== --- arm/release/src/interface/controller.py 2011-04-04 15:15:20 UTC (rev 24554) +++ arm/release/src/interface/controller.py 2011-04-04 15:22:31 UTC (rev 24555) @@ -24,6 +24,9 @@ import descriptorPopup import fileDescriptorPopup +import interface.connections.connPanel +import interface.connections.connEntry +import interface.connections.entries from util import conf, log, connections, hostnames, panel, sysTools, torConfig, torTools, uiTools import graphing.bandwidthStats import graphing.connStats @@ -41,13 +44,17 @@ PAGES = [ ["graph", "log"], ["conn"], + ["conn2"], ["config"], ["torrc"]] -PAUSEABLE = ["header", "graph", "log", "conn"] +PAUSEABLE = ["header", "graph", "log", "conn", "conn2"] + CONFIG = {"log.torrc.readFailed": log.WARN, "features.graph.type": 1, "features.config.prepopulateEditValues": True, + "features.connection.oldPanel": False, + "features.connection.newPanel": True, "queries.refreshRate.rate": 5, "log.torEventTypeUnrecognized": log.NOTICE, "features.graph.bw.prepopulate": True, @@ -78,7 +85,7 @@ self.msgText = msgText self.msgAttr = msgAttr - def draw(self, subwindow, width, height): + def draw(self, width, height): msgText = self.msgText msgAttr = self.msgAttr barTab = 2 # space between msgText and progress bar @@ -113,7 +120,11 @@ currentPage = self.page pageCount = len(PAGES) - if self.isBlindMode: + if not CONFIG["features.connection.newPanel"]: + if currentPage >= 3: currentPage -= 1 + pageCount -= 1 + + if self.isBlindMode or not CONFIG["features.connection.oldPanel"]: if currentPage >= 2: currentPage -= 1 pageCount -= 1 @@ -240,7 +251,7 @@ popup.recreate(stdscr, newWidth) key = 0 - while key not in (curses.KEY_ENTER, 10, ord(' ')): + while not uiTools.isSelectionKey(key): popup.clear() popup.win.box() popup.addstr(0, 0, title, curses.A_STANDOUT) @@ -341,7 +352,7 @@ elif key == curses.KEY_RIGHT: cursorLoc = min(len(selectionOptions) - 1, cursorLoc + 1) elif key == curses.KEY_UP: cursorLoc = max(0, cursorLoc - 4) elif key == curses.KEY_DOWN: cursorLoc = min(len(selectionOptions) - 1, cursorLoc + 4) - elif key in (curses.KEY_ENTER, 10, ord(' ')): + elif uiTools.isSelectionKey(key): # selected entry (the ord of '10' seems needed to pick up enter) selection = selectionOptions[cursorLoc] if selection == "Cancel": break @@ -394,7 +405,7 @@ if connections.isResolverAlive("tor"): resolver = connections.getResolver("tor") - resolver.setPaused(eventType == torTools.TOR_CLOSED) + resolver.setPaused(eventType == torTools.State.CLOSED) def selectiveRefresh(panels, page): """ @@ -419,6 +430,7 @@ config = conf.getConfig("arm") config.update(CONFIG) graphing.graphPanel.loadConfig(config) + interface.connections.connEntry.loadConfig(config) # adds events needed for arm functionality to the torTools REQ_EVENTS mapping # (they're then included with any setControllerEvents call, and log a more @@ -471,12 +483,12 @@ duplicateOptions, defaultOptions, mismatchLines, missingOptions = [], [], [], [] for lineNum, issue, msg in corrections: - if issue == torConfig.VAL_DUPLICATE: - duplicateOptions.append("%s (line %i)" % (msg, lineNum)) - elif issue == torConfig.VAL_IS_DEFAULT: - defaultOptions.append("%s (line %i)" % (msg, lineNum)) - elif issue == torConfig.VAL_MISMATCH: mismatchLines.append(lineNum) - elif issue == torConfig.VAL_MISSING: missingOptions.append(msg) + if issue == torConfig.ValidationError.DUPLICATE: + duplicateOptions.append("%s (line %i)" % (msg, lineNum + 1)) + elif issue == torConfig.ValidationError.IS_DEFAULT: + defaultOptions.append("%s (line %i)" % (msg, lineNum + 1)) + elif issue == torConfig.ValidationError.MISMATCH: mismatchLines.append(lineNum + 1) + elif issue == torConfig.ValidationError.MISSING: missingOptions.append(msg) if duplicateOptions or defaultOptions: msg = "Unneeded torrc entries found. They've been highlighted in blue on the torrc page." @@ -551,10 +563,21 @@ # before being positioned - the following is a quick hack til rewritten panels["log"].setPaused(True) - panels["conn"] = connPanel.ConnPanel(stdscr, conn, isBlindMode) + if CONFIG["features.connection.oldPanel"]: + panels["conn"] = connPanel.ConnPanel(stdscr, conn, isBlindMode) + else: + panels["conn"] = panel.Panel(stdscr, "blank", 0, 0, 0) + PAUSEABLE.remove("conn") + + if CONFIG["features.connection.newPanel"]: + panels["conn2"] = interface.connections.connPanel.ConnectionPanel(stdscr, config) + else: + panels["conn2"] = panel.Panel(stdscr, "blank", 0, 0, 0) + PAUSEABLE.remove("conn2") + panels["control"] = ControlPanel(stdscr, isBlindMode) - panels["config"] = configPanel.ConfigPanel(stdscr, configPanel.TOR_STATE, config) - panels["torrc"] = torrcPanel.TorrcPanel(stdscr, torrcPanel.TORRC, config) + panels["config"] = configPanel.ConfigPanel(stdscr, configPanel.State.TOR, config) + panels["torrc"] = torrcPanel.TorrcPanel(stdscr, torrcPanel.Config.TORRC, config) # provides error if pid coulnd't be determined (hopefully shouldn't happen...) if not torPid: log.log(log.WARN, "Unable to resolve tor pid, abandoning connection listing") @@ -577,7 +600,8 @@ conn.add_event_listener(panels["graph"].stats["bandwidth"]) conn.add_event_listener(panels["graph"].stats["system resources"]) if not isBlindMode: conn.add_event_listener(panels["graph"].stats["connections"]) - conn.add_event_listener(panels["conn"]) + if CONFIG["features.connection.oldPanel"]: + conn.add_event_listener(panels["conn"]) conn.add_event_listener(sighupTracker) # prepopulates bandwidth values from state file @@ -604,16 +628,18 @@ # tells revised panels to run as daemons panels["header"].start() panels["log"].start() + if CONFIG["features.connection.newPanel"]: + panels["conn2"].start() # warns if tor isn't updating descriptors - try: - if conn.get_option("FetchUselessDescriptors")[0][1] == "0" and conn.get_option("DirPort")[0][1] == "0": - warning = """Descriptors won't be updated (causing some connection information to be stale) unless: - a. 'FetchUselessDescriptors 1' is set in your torrc - b. the directory service is provided ('DirPort' defined) - c. or tor is used as a client""" - log.log(log.WARN, warning) - except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed): pass + #try: + # if conn.get_option("FetchUselessDescriptors")[0][1] == "0" and conn.get_option("DirPort")[0][1] == "0": + # warning = """Descriptors won't be updated (causing some connection information to be stale) unless: + #a. 'FetchUselessDescriptors 1' is set in your torrc + #b. the directory service is provided ('DirPort' defined) + #c. or tor is used as a client""" + # log.log(log.WARN, warning) + #except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed): pass isUnresponsive = False # true if it's been over ten seconds since the last BW event (probably due to Tor closing) isPaused = False # if true updates are frozen @@ -641,6 +667,11 @@ lastSize = None + # sets initial visiblity for the pages + for i in range(len(PAGES)): + isVisible = i == page + for entry in PAGES[i]: panels[entry].setVisible(isVisible) + # TODO: come up with a nice, clean method for other threads to immediately # terminate the draw loop and provide a stacktrace while True: @@ -657,7 +688,8 @@ #panels["header"]._updateParams(True) # other panels that use torrc data - panels["conn"].resetOptions() + if CONFIG["features.connection.oldPanel"]: + panels["conn"].resetOptions() #if not isBlindMode: panels["graph"].stats["connections"].resetOptions(conn) #panels["graph"].stats["bandwidth"].resetOptions() @@ -718,7 +750,8 @@ isUnresponsive = False log.log(log.NOTICE, "Relay resumed") - panels["conn"].reset() + if CONFIG["features.connection.oldPanel"]: + panels["conn"].reset() # TODO: part two of hack to prevent premature drawing by log panel if page == 0 and not isPaused: panels["log"].setPaused(False) @@ -731,7 +764,7 @@ isResize = lastSize != newSize lastSize = newSize - if panelKey in ("header", "graph", "log", "config", "torrc"): + if panelKey in ("header", "graph", "log", "config", "torrc", "conn2"): # revised panel (manages its own content refreshing) panels[panelKey].redraw(isResize) else: @@ -808,11 +841,15 @@ # this appears to be a python bug: http://bugs.python.org/issue3014 # (haven't seen this is quite some time... mysteriously resolved?) + torTools.NO_SPAWN = True # prevents further worker threads from being spawned + # stops panel daemons panels["header"].stop() + if CONFIG["features.connection.newPanel"]: panels["conn2"].stop() panels["log"].stop() panels["header"].join() + if CONFIG["features.connection.newPanel"]: panels["conn2"].join() panels["log"].join() # joins on utility daemon threads - this might take a moment since @@ -835,15 +872,25 @@ if key == curses.KEY_LEFT: page = (page - 1) % len(PAGES) else: page = (page + 1) % len(PAGES) - # skip connections listing if it's disabled - if page == 1 and isBlindMode: - if key == curses.KEY_LEFT: page = (page - 1) % len(PAGES) - else: page = (page + 1) % len(PAGES) + # skip connections listings if it's disabled + while True: + if page == 1 and (isBlindMode or not CONFIG["features.connection.oldPanel"]): + if key == curses.KEY_LEFT: page = (page - 1) % len(PAGES) + else: page = (page + 1) % len(PAGES) + elif page == 2 and (isBlindMode or not CONFIG["features.connection.newPanel"]): + if key == curses.KEY_LEFT: page = (page - 1) % len(PAGES) + else: page = (page + 1) % len(PAGES) + else: break # pauses panels that aren't visible to prevent events from accumilating # (otherwise they'll wait on the curses lock which might get demanding) setPauseState(panels, isPaused, page) + # prevents panels on other pages from redrawing + for i in range(len(PAGES)): + isVisible = i == page + for entry in PAGES[i]: panels[entry].setVisible(isVisible) + panels["control"].page = page + 1 # TODO: this redraw doesn't seem necessary (redraws anyway after this @@ -914,7 +961,7 @@ popup.addfstr(2, 41, "n: decrease graph size") popup.addfstr(3, 2, "s: graphed stats (%s)" % graphedStats) popup.addfstr(3, 41, "i: graph update interval (%s)" % graphing.graphPanel.UPDATE_INTERVALS[panels["graph"].updateInterval][0]) - popup.addfstr(4, 2, "b: graph bounds (%s)" % graphing.graphPanel.BOUND_LABELS[panels["graph"].bounds]) + popup.addfstr(4, 2, "b: graph bounds (%s)" % panels["graph"].bounds.lower()) popup.addfstr(4, 41, "d: file descriptors") popup.addfstr(5, 2, "e: change logged events") @@ -940,10 +987,11 @@ resolverUtil = connections.getResolver("tor").overwriteResolver if resolverUtil == None: resolverUtil = "auto" - else: resolverUtil = connections.CMD_STR[resolverUtil] popup.addfstr(4, 41, "u: resolving utility (%s)" % resolverUtil) - allowDnsLabel = "allow" if panels["conn"].allowDNS else "disallow" + if CONFIG["features.connection.oldPanel"]: + allowDnsLabel = "allow" if panels["conn"].allowDNS else "disallow" + else: allowDnsLabel = "disallow" popup.addfstr(5, 2, "r: permit DNS resolution (%s)" % allowDnsLabel) popup.addfstr(5, 41, "s: sort ordering") @@ -959,8 +1007,18 @@ popup.addfstr(2, 41, "page down: scroll down a page") popup.addfstr(3, 2, "enter: edit configuration option") - popup.addfstr(3, 41, "w: save current configuration") - popup.addfstr(4, 2, "s: sort ordering") + #popup.addfstr(3, 41, "d: raw consensus descriptor") + + listingType = panels["conn2"]._listingType.lower() + popup.addfstr(4, 2, "l: listed identity (%s)" % listingType) + + popup.addfstr(4, 41, "s: sort ordering") + + resolverUtil = connections.getResolver("tor").overwriteResolver + if resolverUtil == None: resolverUtil = "auto" + popup.addfstr(3, 41, "u: resolving utility (%s)" % resolverUtil) + + pageOverrideKeys = (ord('l'), ord('s'), ord('u')) elif page == 3: popup.addfstr(1, 2, "up arrow: scroll up a line") popup.addfstr(1, 41, "down arrow: scroll down a line") @@ -975,6 +1033,12 @@ popup.addfstr(4, 2, "r: reload torrc") popup.addfstr(4, 41, "x: reset tor (issue sighup)") + elif page == 4: + popup.addfstr(1, 2, "up arrow: scroll up a line") + popup.addfstr(1, 41, "down arrow: scroll down a line") + popup.addfstr(2, 2, "page up: scroll up a page") + popup.addfstr(2, 41, "page down: scroll down a page") + popup.addfstr(3, 2, "enter: connection details") popup.addstr(7, 2, "Press any key...") popup.refresh() @@ -1054,7 +1118,7 @@ selectiveRefresh(panels, page) elif page == 0 and (key == ord('b') or key == ord('B')): # uses the next boundary type for graph - panels["graph"].bounds = (panels["graph"].bounds + 1) % 3 + panels["graph"].bounds = graphing.graphPanel.Bounds.next(panels["graph"].bounds) selectiveRefresh(panels, page) elif page == 0 and key in (ord('d'), ord('D')): @@ -1245,13 +1309,16 @@ setPauseState(panels, isPaused, page) finally: panel.CURSES_LOCK.release() - elif key == 27 and panels["conn"].listingType == connPanel.LIST_HOSTNAME and panels["control"].resolvingCounter != -1: + elif CONFIG["features.connection.oldPanel"] and key == 27 and panels["conn"].listingType == connPanel.LIST_HOSTNAME and panels["control"].resolvingCounter != -1: # canceling hostname resolution (esc on any page) panels["conn"].listingType = connPanel.LIST_IP panels["control"].resolvingCounter = -1 hostnames.setPaused(True) panels["conn"].sortConnections() - elif page == 1 and panels["conn"].isCursorEnabled and key in (curses.KEY_ENTER, 10, ord(' ')): + elif page == 1 and panels["conn"].isCursorEnabled and uiTools.isSelectionKey(key): + # TODO: deprecated when migrated to the new connection panel, thought as + # well keep around until there's a counterpart for hostname fetching + # provides details on selected connection panel.CURSES_LOCK.acquire() try: @@ -1269,7 +1336,7 @@ curses.cbreak() # wait indefinitely for key presses (no timeout) key = 0 - while key not in (curses.KEY_ENTER, 10, ord(' ')): + while not uiTools.isSelectionKey(key): popup.clear() popup.win.box() popup.addstr(0, 0, "Connection Details:", curses.A_STANDOUT) @@ -1455,13 +1522,13 @@ hostnames.setPaused(True) panels["conn"].sortConnections() - elif page == 1 and (key == ord('u') or key == ord('U')): + elif page in (1, 2) and (key == ord('u') or key == ord('U')): # provides menu to pick identification resolving utility - optionTypes = [None, connections.CMD_PROC, connections.CMD_NETSTAT, connections.CMD_SOCKSTAT, connections.CMD_LSOF, connections.CMD_SS, connections.CMD_BSD_SOCKSTAT, connections.CMD_BSD_PROCSTAT] - options = ["auto"] + [connections.CMD_STR[util] for util in optionTypes[1:]] + options = ["auto"] + connections.Resolver.values() - initialSelection = connections.getResolver("tor").overwriteResolver # enums correspond to indices - if initialSelection == None: initialSelection = 0 + currentOverwrite = connections.getResolver("tor").overwriteResolver # enums correspond to indices + if currentOverwrite == None: initialSelection = 0 + else: initialSelection = options.index(currentOverwrite) # hides top label of conn panel and pauses panels panels["conn"].showLabel = False @@ -1469,14 +1536,15 @@ setPauseState(panels, isPaused, page, True) selection = showMenu(stdscr, panels["popup"], "Resolver Util:", options, initialSelection) + selectedOption = options[selection] if selection != "auto" else None # reverts changes made for popup panels["conn"].showLabel = True setPauseState(panels, isPaused, page) # applies new setting - if selection != -1 and optionTypes[selection] != connections.getResolver("tor").overwriteResolver: - connections.getResolver("tor").overwriteResolver = optionTypes[selection] + if selection != -1 and selectedOption != connections.getResolver("tor").overwriteResolver: + connections.getResolver("tor").overwriteResolver = selectedOption elif page == 1 and (key == ord('s') or key == ord('S')): # set ordering for connection listing titleLabel = "Connection Ordering:" @@ -1543,7 +1611,44 @@ setPauseState(panels, isPaused, page) finally: panel.CURSES_LOCK.release() - elif page == 2 and (key == ord('c') or key == ord('C')) and False: + elif page == 2 and (key == ord('l') or key == ord('L')): + # provides a menu to pick the primary information we list connections by + options = interface.connections.entries.ListingType.values() + + # dropping the HOSTNAME listing type until we support displaying that content + options.remove(interface.connections.entries.ListingType.HOSTNAME) + + initialSelection = options.index(panels["conn2"]._listingType) + + # hides top label of connection panel and pauses the display + panelTitle = panels["conn2"]._title + panels["conn2"]._title = "" + panels["conn2"].redraw(True) + setPauseState(panels, isPaused, page, True) + + selection = showMenu(stdscr, panels["popup"], "List By:", options, initialSelection) + + # reverts changes made for popup + panels["conn2"]._title = panelTitle + setPauseState(panels, isPaused, page) + + # applies new setting + if selection != -1 and options[selection] != panels["conn2"]._listingType: + panels["conn2"].setListingType(options[selection]) + panels["conn2"].redraw(True) + elif page == 2 and (key == ord('s') or key == ord('S')): + # set ordering for connection options + titleLabel = "Connection Ordering:" + options = interface.connections.entries.SortAttr.values() + oldSelection = panels["conn2"]._sortOrdering + optionColors = dict([(attr, interface.connections.entries.SORT_COLORS[attr]) for attr in options]) + results = showSortDialog(stdscr, panels, isPaused, page, titleLabel, options, oldSelection, optionColors) + + if results: + panels["conn2"].setSortOrder(results) + + panels["conn2"].redraw(True) + elif page == 3 and (key == ord('c') or key == ord('C')) and False: # TODO: disabled for now (probably gonna be going with separate pages # rather than popup menu) # provides menu to pick config being displayed @@ -1566,12 +1671,11 @@ if selection != -1: panels["torrc"].setConfigType(selection) selectiveRefresh(panels, page) - elif page == 2 and (key == ord('w') or key == ord('W')): + elif page == 3 and (key == ord('w') or key == ord('W')): # display a popup for saving the current configuration panel.CURSES_LOCK.acquire() try: - configText = torTools.getConn().getInfo("config-text", "").strip() - configLines = configText.split("\n") + configLines = torConfig.getCustomOptions(True) # lists event types popup = panels["popup"] @@ -1594,7 +1698,7 @@ popup.recreate(stdscr) key, selection = 0, 2 - while key not in (curses.KEY_ENTER, 10, ord(' ')): + while not uiTools.isSelectionKey(key): # if the popup has been resized then recreate it (needed for the # proper border height) newHeight, newWidth = panels["popup"].getPreferredSize() @@ -1669,7 +1773,7 @@ # saves the configuration to the file configFile = open(configLocation, "w") - configFile.write(configText) + configFile.write("\n".join(configLines)) configFile.close() # reloads the cached torrc if overwriting it @@ -1696,12 +1800,12 @@ panel.CURSES_LOCK.release() panels["config"].redraw(True) - elif page == 2 and (key == ord('s') or key == ord('S')): + elif page == 3 and (key == ord('s') or key == ord('S')): # set ordering for config options titleLabel = "Config Option Ordering:" - options = [configPanel.FIELD_ATTR[i][0] for i in range(8)] - oldSelection = [configPanel.FIELD_ATTR[entry][0] for entry in panels["config"].sortOrdering] - optionColors = dict([configPanel.FIELD_ATTR[i] for i in range(8)]) + options = [configPanel.FIELD_ATTR[field][0] for field in configPanel.Field.values()] + oldSelection = [configPanel.FIELD_ATTR[field][0] for field in panels["config"].sortOrdering] + optionColors = dict([configPanel.FIELD_ATTR[field] for field in configPanel.Field.values()]) results = showSortDialog(stdscr, panels, isPaused, page, titleLabel, options, oldSelection, optionColors) if results: @@ -1717,7 +1821,7 @@ panels["config"].setSortOrder(resultEnums) panels["config"].redraw(True) - elif page == 2 and key in (curses.KEY_ENTER, 10, ord(' ')): + elif page == 3 and uiTools.isSelectionKey(key): # let the user edit the configuration value, unchanged if left blank panel.CURSES_LOCK.acquire() try: @@ -1725,13 +1829,13 @@ # provides prompt selection = panels["config"].getSelection() - configOption = selection.get(configPanel.FIELD_OPTION) + configOption = selection.get(configPanel.Field.OPTION) titleMsg = "%s Value (esc to cancel): " % configOption panels["control"].setMsg(titleMsg) panels["control"].redraw(True) displayWidth = panels["control"].getPreferredSize()[1] - initialValue = selection.get(configPanel.FIELD_VALUE) + initialValue = selection.get(configPanel.Field.VALUE) # initial input for the text field initialText = "" @@ -1745,19 +1849,19 @@ conn = torTools.getConn() # if the value's a boolean then allow for 'true' and 'false' inputs - if selection.get(configPanel.FIELD_TYPE) == "Boolean": + if selection.get(configPanel.Field.TYPE) == "Boolean": if newConfigValue.lower() == "true": newConfigValue = "1" elif newConfigValue.lower() == "false": newConfigValue = "0" try: - if selection.get(configPanel.FIELD_TYPE) == "LineList": + if selection.get(configPanel.Field.TYPE) == "LineList": newConfigValue = newConfigValue.split(",") conn.setOption(configOption, newConfigValue) # resets the isDefault flag customOptions = torConfig.getCustomOptions() - selection.fields[configPanel.FIELD_IS_DEFAULT] = not configOption in customOptions + selection.fields[configPanel.Field.IS_DEFAULT] = not configOption in customOptions panels["config"].redraw(True) except Exception, exc: @@ -1773,7 +1877,7 @@ setPauseState(panels, isPaused, page) finally: panel.CURSES_LOCK.release() - elif page == 3 and key == ord('r') or key == ord('R'): + elif page == 4 and key == ord('r') or key == ord('R'): # reloads torrc, providing a notice if successful or not loadedTorrc = torConfig.getTorrc() loadedTorrc.getLock().acquire() @@ -1803,8 +1907,10 @@ elif page == 1: panels["conn"].handleKey(key) elif page == 2: + panels["conn2"].handleKey(key) + elif page == 3: panels["config"].handleKey(key) - elif page == 3: + elif page == 4: panels["torrc"].handleKey(key) def startTorMonitor(startTime, loggedEvents, isBlindMode): Modified: arm/release/src/interface/descriptorPopup.py =================================================================== --- arm/release/src/interface/descriptorPopup.py 2011-04-04 15:15:20 UTC (rev 24554) +++ arm/release/src/interface/descriptorPopup.py 2011-04-04 15:22:31 UTC (rev 24555) @@ -106,7 +106,7 @@ draw(popup, properties) key = stdscr.getch() - if key in (curses.KEY_ENTER, 10, ord(' '), ord('d'), ord('D')): + if uiTools.isSelectionKey(key) or key in (ord('d'), ord('D')): # closes popup isVisible = False elif key in (curses.KEY_LEFT, curses.KEY_RIGHT): Modified: arm/release/src/interface/graphing/__init__.py =================================================================== --- arm/release/src/interface/graphing/__init__.py 2011-04-04 15:15:20 UTC (rev 24554) +++ arm/release/src/interface/graphing/__init__.py 2011-04-04 15:22:31 UTC (rev 24555) @@ -2,5 +2,5 @@ Panels, popups, and handlers comprising the arm user interface. """ -__all__ = ["graphPanel.py", "bandwidthStats", "connStats", "resourceStats"] +__all__ = ["graphPanel", "bandwidthStats", "connStats", "resourceStats"] Modified: arm/release/src/interface/graphing/bandwidthStats.py =================================================================== --- arm/release/src/interface/graphing/bandwidthStats.py 2011-04-04 15:15:20 UTC (rev 24554) +++ arm/release/src/interface/graphing/bandwidthStats.py 2011-04-04 15:22:31 UTC (rev 24555) @@ -20,7 +20,8 @@ PREPOPULATE_SUCCESS_MSG = "Read the last day of bandwidth history from the state file" PREPOPULATE_FAILURE_MSG = "Unable to prepopulate bandwidth information (%s)" -DEFAULT_CONFIG = {"features.graph.bw.transferInBytes": False, +DEFAULT_CONFIG = {"features.graph.bw.prepopulateTotal": False, + "features.graph.bw.transferInBytes": False, "features.graph.bw.accounting.show": True, "features.graph.bw.accounting.rate": 10, "features.graph.bw.accounting.isTimeLong": False, @@ -39,6 +40,11 @@ if config: config.update(self._config, {"features.graph.bw.accounting.rate": 1}) + # stats prepopulated from tor's state file + self.prepopulatePrimaryTotal = 0 + self.prepopulateSecondaryTotal = 0 + self.prepopulateTicks = 0 + # accounting data (set by _updateAccountingInfo method) self.accountingLastUpdated = 0 self.accountingInfo = dict([(arg, "") for arg in ACCOUNTING_ARGS]) @@ -47,7 +53,7 @@ # rate/burst and if tor's using accounting conn = torTools.getConn() self._titleStats, self.isAccounting = [], False - self.resetListener(conn, torTools.TOR_INIT) # initializes values + self.resetListener(conn, torTools.State.INIT) # initializes values conn.addStatusListener(self.resetListener) def resetListener(self, conn, eventType): @@ -55,7 +61,7 @@ self._titleStats = [] # force reset of title self.new_desc_event(None) # updates title params - if eventType == torTools.TOR_INIT and self._config["features.graph.bw.accounting.show"]: + if eventType == torTools.State.INIT and self._config["features.graph.bw.accounting.show"]: self.isAccounting = conn.getInfo('accounting/enabled') == '1' def prepopulateFromState(self): @@ -164,10 +170,11 @@ readVal, writeVal = bwReadEntries[i], bwWriteEntries[i] self.lastPrimary, self.lastSecondary = readVal, writeVal - self.primaryTotal += readVal * 900 - self.secondaryTotal += writeVal * 900 - self.tick += 900 + self.prepopulatePrimaryTotal += readVal * 900 + self.prepopulateSecondaryTotal += writeVal * 900 + self.prepopulateTicks += 900 + self.primaryCounts[intervalIndex].insert(0, readVal) self.secondaryCounts[intervalIndex].insert(0, writeVal) @@ -316,10 +323,15 @@ def _getAvgLabel(self, isPrimary): total = self.primaryTotal if isPrimary else self.secondaryTotal - return "avg: %s/sec" % uiTools.getSizeLabel((total / max(1, self.tick)) * 1024, 1, False, self._config["features.graph.bw.transferInBytes"]) + total += self.prepopulatePrimaryTotal if isPrimary else self.prepopulateSecondaryTotal + return "avg: %s/sec" % uiTools.getSizeLabel((total / max(1, self.tick + self.prepopulateTicks)) * 1024, 1, False, self._config["features.graph.bw.transferInBytes"]) def _getTotalLabel(self, isPrimary): total = self.primaryTotal if isPrimary else self.secondaryTotal + + if self._config["features.graph.bw.prepopulateTotal"]: + total += self.prepopulatePrimaryTotal if isPrimary else self.prepopulateSecondaryTotal + return "total: %s" % uiTools.getSizeLabel(total * 1024, 1) def _updateAccountingInfo(self): Modified: arm/release/src/interface/graphing/connStats.py =================================================================== --- arm/release/src/interface/graphing/connStats.py 2011-04-04 15:15:20 UTC (rev 24554) +++ arm/release/src/interface/graphing/connStats.py 2011-04-04 15:22:31 UTC (rev 24555) @@ -17,11 +17,11 @@ # listens for tor reload (sighup) events which can reset the ports tor uses conn = torTools.getConn() self.orPort, self.dirPort, self.controlPort = "0", "0", "0" - self.resetListener(conn, torTools.TOR_INIT) # initialize port values + self.resetListener(conn, torTools.State.INIT) # initialize port values conn.addStatusListener(self.resetListener) def resetListener(self, conn, eventType): - if eventType == torTools.TOR_INIT: + if eventType == torTools.State.INIT: self.orPort = conn.getOption("ORPort", "0") self.dirPort = conn.getOption("DirPort", "0") self.controlPort = conn.getOption("ControlPort", "0") Modified: arm/release/src/interface/graphing/graphPanel.py =================================================================== --- arm/release/src/interface/graphing/graphPanel.py 2011-04-04 15:15:20 UTC (rev 24554) +++ arm/release/src/interface/graphing/graphPanel.py 2011-04-04 15:22:31 UTC (rev 24555) @@ -20,7 +20,7 @@ import curses from TorCtl import TorCtl -from util import panel, uiTools +from util import enum, panel, uiTools # time intervals at which graphs can be updated UPDATE_INTERVALS = [("each second", 1), ("5 seconds", 5), ("30 seconds", 30), @@ -32,11 +32,10 @@ MIN_GRAPH_HEIGHT = 1 # enums for graph bounds: -# BOUNDS_GLOBAL_MAX - global maximum (highest value ever seen) -# BOUNDS_LOCAL_MAX - local maximum (highest value currently on the graph) -# BOUNDS_TIGHT - local maximum and minimum -BOUNDS_GLOBAL_MAX, BOUNDS_LOCAL_MAX, BOUNDS_TIGHT = range(3) -BOUND_LABELS = {BOUNDS_GLOBAL_MAX: "global max", BOUNDS_LOCAL_MAX: "local max", BOUNDS_TIGHT: "tight"} +# Bounds.GLOBAL_MAX - global maximum (highest value ever seen) +# Bounds.LOCAL_MAX - local maximum (highest value currently on the graph) +# Bounds.TIGHT - local maximum and minimum +Bounds = enum.Enum("GLOBAL_MAX", "LOCAL_MAX", "TIGHT") WIDE_LABELING_GRAPH_COL = 50 # minimum graph columns to use wide spacing for x-axis labels @@ -248,7 +247,7 @@ def __init__(self, stdscr): panel.Panel.__init__(self, stdscr, "graph", 0) self.updateInterval = CONFIG["features.graph.interval"] - self.bounds = CONFIG["features.graph.bound"] + self.bounds = Bounds.values()[CONFIG["features.graph.bound"]] self.graphHeight = CONFIG["features.graph.height"] self.currentDisplay = None # label of the stats currently being displayed self.stats = {} # available stats (mappings of label -> instance) @@ -276,7 +275,7 @@ self.graphHeight = max(MIN_GRAPH_HEIGHT, newGraphHeight) - def draw(self, subwindow, width, height): + def draw(self, width, height): """ Redraws graph panel """ if self.currentDisplay: @@ -294,11 +293,11 @@ if right: self.addstr(1, graphCol + 5, right, curses.A_BOLD | secondaryColor) # determines max/min value on the graph - if self.bounds == BOUNDS_GLOBAL_MAX: + if self.bounds == Bounds.GLOBAL_MAX: primaryMaxBound = int(param.maxPrimary[self.updateInterval]) secondaryMaxBound = int(param.maxSecondary[self.updateInterval]) else: - # both BOUNDS_LOCAL_MAX and BOUNDS_TIGHT use local maxima + # both Bounds.LOCAL_MAX and Bounds.TIGHT use local maxima if graphCol < 2: # nothing being displayed primaryMaxBound, secondaryMaxBound = 0, 0 @@ -307,7 +306,7 @@ secondaryMaxBound = int(max(param.secondaryCounts[self.updateInterval][1:graphCol + 1])) primaryMinBound = secondaryMinBound = 0 - if self.bounds == BOUNDS_TIGHT: + if self.bounds == Bounds.TIGHT: primaryMinBound = int(min(param.primaryCounts[self.updateInterval][1:graphCol + 1])) secondaryMinBound = int(min(param.secondaryCounts[self.updateInterval][1:graphCol + 1])) Modified: arm/release/src/interface/headerPanel.py =================================================================== --- arm/release/src/interface/headerPanel.py 2011-04-04 15:15:20 UTC (rev 24554) +++ arm/release/src/interface/headerPanel.py 2011-04-04 15:22:31 UTC (rev 24555) @@ -37,8 +37,8 @@ Top area contenting tor settings and system information. Stats are stored in the vals mapping, keys including: tor/ version, versionStatus, nickname, orPort, dirPort, controlPort, - exitPolicy, isAuthPassword (bool), isAuthCookie (bool) - *address, *fingerprint, *flags, pid, startTime + exitPolicy, isAuthPassword (bool), isAuthCookie (bool), + orListenAddr, *address, *fingerprint, *flags, pid, startTime sys/ hostname, os, version stat/ *%torCpu, *%armCpu, *rss, *%mem @@ -95,7 +95,7 @@ if self.vals["tor/orPort"]: return 4 if isWide else 6 else: return 3 if isWide else 4 - def draw(self, subwindow, width, height): + def draw(self, width, height): self.valsLock.acquire() isWide = width + 1 >= MIN_DUAL_COL_WIDTH @@ -127,10 +127,14 @@ # Line 2 / Line 2 Left (tor ip/port information) if self.vals["tor/orPort"]: + myAddress = "Unknown" + if self.vals["tor/orListenAddr"]: myAddress = self.vals["tor/orListenAddr"] + elif self.vals["tor/address"]: myAddress = self.vals["tor/address"] + # acting as a relay (we can assume certain parameters are set entry = "" dirPortLabel = ", Dir Port: %s" % self.vals["tor/dirPort"] if self.vals["tor/dirPort"] != "0" else "" - for label in (self.vals["tor/nickname"], " - " + self.vals["tor/address"], ":" + self.vals["tor/orPort"], dirPortLabel): + for label in (self.vals["tor/nickname"], " - " + myAddress, ":" + self.vals["tor/orPort"], dirPortLabel): if len(entry) + len(label) <= leftWidth: entry += label else: break else: @@ -239,7 +243,7 @@ def run(self): """ - Keeps stats updated, querying new information at a set rate. + Keeps stats updated, checking for new information at a set rate. """ lastDraw = time.time() - 1 @@ -288,14 +292,14 @@ eventType - type of event detected """ - if eventType == torTools.TOR_INIT: + if eventType == torTools.State.INIT: self._isTorConnected = True if self._isPaused: self._haltTime = time.time() else: self._haltTime = None self._update(True) self.redraw(True) - elif eventType == torTools.TOR_CLOSED: + elif eventType == torTools.State.CLOSED: self._isTorConnected = False self._haltTime = time.time() self._update() @@ -329,15 +333,15 @@ if self.vals["tor/orPort"] == "0": self.vals["tor/orPort"] = "" # overwrite address if ORListenAddress is set (and possibly orPort too) - self.vals["tor/address"] = "Unknown" + self.vals["tor/orListenAddr"] = "" listenAddr = conn.getOption("ORListenAddress") if listenAddr: if ":" in listenAddr: # both ip and port overwritten - self.vals["tor/address"] = listenAddr[:listenAddr.find(":")] + self.vals["tor/orListenAddr"] = listenAddr[:listenAddr.find(":")] self.vals["tor/orPort"] = listenAddr[listenAddr.find(":") + 1:] else: - self.vals["tor/address"] = listenAddr + self.vals["tor/orListenAddr"] = listenAddr # fetch exit policy (might span over multiple lines) policyEntries = [] @@ -368,8 +372,7 @@ # sets volatile parameters # TODO: This can change, being reported by STATUS_SERVER -> EXTERNAL_ADDRESS # events. Introduce caching via torTools? - if self.vals["tor/address"] == "Unknown": - self.vals["tor/address"] = conn.getInfo("address", self.vals["tor/address"]) + self.vals["tor/address"] = conn.getInfo("address", "") self.vals["tor/fingerprint"] = conn.getInfo("fingerprint", self.vals["tor/fingerprint"]) self.vals["tor/flags"] = conn.getMyFlags(self.vals["tor/flags"]) Modified: arm/release/src/interface/logPanel.py =================================================================== --- arm/release/src/interface/logPanel.py 2011-04-04 15:15:20 UTC (rev 24554) +++ arm/release/src/interface/logPanel.py 2011-04-04 15:22:31 UTC (rev 24555) @@ -32,8 +32,8 @@ 12345 arm runlevel+ X No Events 67890 torctl runlevel+ U Unknown Events""" -RUNLEVELS = ["DEBUG", "INFO", "NOTICE", "WARN", "ERR"] -RUNLEVEL_EVENT_COLOR = {"DEBUG": "magenta", "INFO": "blue", "NOTICE": "green", "WARN": "yellow", "ERR": "red"} +RUNLEVEL_EVENT_COLOR = {log.DEBUG: "magenta", log.INFO: "blue", log.NOTICE: "green", + log.WARN: "yellow", log.ERR: "red"} DAYBREAK_EVENT = "DAYBREAK" # special event for marking when the date changes TIMEZONE_OFFSET = time.altzone if time.localtime()[8] else time.timezone @@ -112,8 +112,8 @@ for flag in eventAbbr: if flag == "A": - armRunlevels = ["ARM_" + runlevel for runlevel in RUNLEVELS] - torctlRunlevels = ["TORCTL_" + runlevel for runlevel in RUNLEVELS] + armRunlevels = ["ARM_" + runlevel for runlevel in log.Runlevel.values()] + torctlRunlevels = ["TORCTL_" + runlevel for runlevel in log.Runlevel.values()] expandedEvents = set(TOR_EVENT_TYPES.values() + armRunlevels + torctlRunlevels + ["UNKNOWN"]) break elif flag == "X": @@ -131,7 +131,7 @@ elif flag in "W49": runlevelIndex = 3 elif flag in "E50": runlevelIndex = 4 - runlevelSet = [typePrefix + runlevel for runlevel in RUNLEVELS[runlevelIndex:]] + runlevelSet = [typePrefix + runlevel for runlevel in log.Runlevel.values()[runlevelIndex:]] expandedEvents = expandedEvents.union(set(runlevelSet)) elif flag == "U": expandedEvents.add("UNKNOWN") @@ -210,16 +210,17 @@ # if the runlevels argument is a superset of the log file then we can # limit the read contents to the addLimit + runlevels = log.Runlevel.values() loggingTypes = loggingTypes.upper() if addLimit and (not readLimit or readLimit > addLimit): if "-" in loggingTypes: divIndex = loggingTypes.find("-") - sIndex = RUNLEVELS.index(loggingTypes[:divIndex]) - eIndex = RUNLEVELS.index(loggingTypes[divIndex+1:]) - logFileRunlevels = RUNLEVELS[sIndex:eIndex+1] + sIndex = runlevels.index(loggingTypes[:divIndex]) + eIndex = runlevels.index(loggingTypes[divIndex+1:]) + logFileRunlevels = runlevels[sIndex:eIndex+1] else: - sIndex = RUNLEVELS.index(loggingTypes) - logFileRunlevels = RUNLEVELS[sIndex:] + sIndex = runlevels.index(loggingTypes) + logFileRunlevels = runlevels[sIndex:] # checks if runlevels we're reporting are a superset of the file's contents isFileSubset = True @@ -570,7 +571,7 @@ # fetches past tor events from log file, if available torEventBacklog = [] if self._config["features.log.prepopulate"]: - setRunlevels = list(set.intersection(set(self.loggedEvents), set(RUNLEVELS))) + setRunlevels = list(set.intersection(set(self.loggedEvents), set(log.Runlevel.values()))) readLimit = self._config["features.log.prepopulateReadLimit"] addLimit = self._config["cache.logPanel.size"] torEventBacklog = getLogFileEntries(setRunlevels, readLimit, addLimit, self._config) @@ -584,13 +585,12 @@ # gets the set of arm events we're logging setRunlevels = [] for i in range(len(armRunlevels)): - if "ARM_" + RUNLEVELS[i] in self.loggedEvents: + if "ARM_" + log.Runlevel.values()[i] in self.loggedEvents: setRunlevels.append(armRunlevels[i]) armEventBacklog = [] for level, msg, eventTime in log._getEntries(setRunlevels): - runlevelStr = log.RUNLEVEL_STR[level] - armEventEntry = LogEntry(eventTime, "ARM_" + runlevelStr, msg, RUNLEVEL_EVENT_COLOR[runlevelStr]) + armEventEntry = LogEntry(eventTime, "ARM_" + level, msg, RUNLEVEL_EVENT_COLOR[level]) armEventBacklog.insert(0, armEventEntry) # joins armEventBacklog and torEventBacklog chronologically into msgLog @@ -621,14 +621,14 @@ if self._config["features.logFile"]: logPath = self._config["features.logFile"] - # make dir if the path doesn't already exist - baseDir = os.path.dirname(logPath) - if not os.path.exists(baseDir): os.makedirs(baseDir) - try: + # make dir if the path doesn't already exist + baseDir = os.path.dirname(logPath) + if not os.path.exists(baseDir): os.makedirs(baseDir) + self.logFile = open(logPath, "a") log.log(self._config["log.logPanel.logFileOpened"], "arm %s opening log file (%s)" % (VERSION, logPath)) - except IOError, exc: + except (IOError, OSError), exc: log.log(self._config["log.logPanel.logFileWriteFailed"], "Unable to write to log file: %s" % sysTools.getFileErrorMsg(exc)) self.logFile = None @@ -778,7 +778,7 @@ self.redraw(True) self.valsLock.release() - def draw(self, subwindow, width, height): + def draw(self, width, height): """ Redraws message log. Entries stretch to use available space and may contain up to two lines. Starts with newest entries. @@ -831,23 +831,22 @@ # bottom of the divider if seenFirstDateDivider: if lineCount >= 1 and lineCount < height and showDaybreaks: - self.win.vline(lineCount, dividerIndent, curses.ACS_LLCORNER | dividerAttr, 1) - self.win.hline(lineCount, dividerIndent + 1, curses.ACS_HLINE | dividerAttr, width - dividerIndent - 1) - self.win.vline(lineCount, width, curses.ACS_LRCORNER | dividerAttr, 1) + self.addch(lineCount, dividerIndent, curses.ACS_LLCORNER, dividerAttr) + self.hline(lineCount, dividerIndent + 1, width - dividerIndent - 1, dividerAttr) + self.addch(lineCount, width, curses.ACS_LRCORNER, dividerAttr) lineCount += 1 # top of the divider if lineCount >= 1 and lineCount < height and showDaybreaks: timeLabel = time.strftime(" %B %d, %Y ", time.localtime(entry.timestamp)) - self.win.vline(lineCount, dividerIndent, curses.ACS_ULCORNER | dividerAttr, 1) - self.win.hline(lineCount, dividerIndent + 1, curses.ACS_HLINE | dividerAttr, 1) + self.addch(lineCount, dividerIndent, curses.ACS_ULCORNER, dividerAttr) + self.addch(lineCount, dividerIndent + 1, curses.ACS_HLINE, dividerAttr) self.addstr(lineCount, dividerIndent + 2, timeLabel, curses.A_BOLD | dividerAttr) - if dividerIndent + len(timeLabel) + 2 <= width: - lineLength = width - dividerIndent - len(timeLabel) - 2 - self.win.hline(lineCount, dividerIndent + len(timeLabel) + 2, curses.ACS_HLINE | dividerAttr, lineLength) - self.win.vline(lineCount, dividerIndent + len(timeLabel) + 2 + lineLength, curses.ACS_URCORNER | dividerAttr, 1) + lineLength = width - dividerIndent - len(timeLabel) - 2 + self.hline(lineCount, dividerIndent + len(timeLabel) + 2, lineLength, dividerAttr) + self.addch(lineCount, dividerIndent + len(timeLabel) + 2 + lineLength, curses.ACS_URCORNER, dividerAttr) seenFirstDateDivider = True lineCount += 1 @@ -879,15 +878,15 @@ if lineOffset == maxEntriesPerLine - 1: msg = uiTools.cropStr(msg, maxMsgSize) else: - msg, remainder = uiTools.cropStr(msg, maxMsgSize, 4, 4, uiTools.END_WITH_HYPHEN, True) + msg, remainder = uiTools.cropStr(msg, maxMsgSize, 4, 4, uiTools.Ending.HYPHEN, True) displayQueue.insert(0, (remainder.strip(), format, includeBreak)) includeBreak = True if drawLine < height and drawLine >= 1: if seenFirstDateDivider and width - dividerIndent >= 3 and showDaybreaks: - self.win.vline(drawLine, dividerIndent, curses.ACS_VLINE | dividerAttr, 1) - self.win.vline(drawLine, width, curses.ACS_VLINE | dividerAttr, 1) + self.addch(drawLine, dividerIndent, curses.ACS_VLINE, dividerAttr) + self.addch(drawLine, width, curses.ACS_VLINE, dividerAttr) self.addstr(drawLine, cursorLoc, msg, format) @@ -902,13 +901,9 @@ # if this is the last line and there's room, then draw the bottom of the divider if not deduplicatedLog and seenFirstDateDivider: if lineCount < height and showDaybreaks: - # when resizing with a small width the following entries can be - # problematc (though I'm not sure why) - try: - self.win.vline(lineCount, dividerIndent, curses.ACS_LLCORNER | dividerAttr, 1) - self.win.hline(lineCount, dividerIndent + 1, curses.ACS_HLINE | dividerAttr, width - dividerIndent - 1) - self.win.vline(lineCount, width, curses.ACS_LRCORNER | dividerAttr, 1) - except: pass + self.addch(lineCount, dividerIndent, curses.ACS_LLCORNER, dividerAttr) + self.hline(lineCount, dividerIndent + 1, width - dividerIndent - 1, dividerAttr) + self.addch(lineCount, width, curses.ACS_LRCORNER, dividerAttr) lineCount += 1 @@ -1019,7 +1014,7 @@ runlevelRanges = [] # tuple of type, startLevel, endLevel for ranges to be consensed # reverses runlevels and types so they're appended in the right order - reversedRunlevels = list(RUNLEVELS) + reversedRunlevels = log.Runlevel.values() reversedRunlevels.reverse() for prefix in ("TORCTL_", "ARM_", ""): # blank ending runlevel forces the break condition to be reached at the end Modified: arm/release/src/interface/torrcPanel.py =================================================================== --- arm/release/src/interface/torrcPanel.py 2011-04-04 15:15:20 UTC (rev 24554) +++ arm/release/src/interface/torrcPanel.py 2011-04-04 15:22:31 UTC (rev 24555) @@ -6,14 +6,14 @@ import curses import threading -from util import conf, panel, torConfig, uiTools +from util import conf, enum, panel, torConfig, uiTools DEFAULT_CONFIG = {"features.config.file.showScrollbars": True, "features.config.file.maxLinesPerEntry": 8} # TODO: The armrc use case is incomplete. There should be equivilant reloading # and validation capabilities to the torrc. -TORRC, ARMRC = range(1, 3) # configuration file types that can be displayed +Config = enum.Enum("TORRC", "ARMRC") # configuration file types that can be displayed class TorrcPanel(panel.Panel): """ @@ -60,7 +60,7 @@ self.valsLock.release() - def draw(self, subwindow, width, height): + def draw(self, width, height): self.valsLock.acquire() # If true, we assume that the cached value in self._lastContentHeight is @@ -74,7 +74,7 @@ self.scroll = max(0, min(self.scroll, self._lastContentHeight - height + 1)) renderedContents, corrections, confLocation = None, {}, None - if self.configType == TORRC: + if self.configType == Config.TORRC: loadedTorrc = torConfig.getTorrc() loadedTorrc.getLock().acquire() confLocation = loadedTorrc.getConfigLocation() @@ -109,7 +109,7 @@ # draws the top label if self.showLabel: - sourceLabel = "Tor" if self.configType == TORRC else "Arm" + sourceLabel = "Tor" if self.configType == Config.TORRC else "Arm" locationLabel = " (%s)" % confLocation if confLocation else "" self.addstr(0, 0, "%s Configuration File%s:" % (sourceLabel, locationLabel), curses.A_STANDOUT) @@ -157,10 +157,10 @@ if lineNumber in corrections: lineIssue, lineIssueMsg = corrections[lineNumber] - if lineIssue in (torConfig.VAL_DUPLICATE, torConfig.VAL_IS_DEFAULT): + if lineIssue in (torConfig.ValidationError.DUPLICATE, torConfig.ValidationError.IS_DEFAULT): lineComp["option"][1] = curses.A_BOLD | uiTools.getColor("blue") lineComp["argument"][1] = curses.A_BOLD | uiTools.getColor("blue") - elif lineIssue == torConfig.VAL_MISMATCH: + elif lineIssue == torConfig.ValidationError.MISMATCH: lineComp["argument"][1] = curses.A_BOLD | uiTools.getColor("red") lineComp["correction"][0] = " (%s)" % lineIssueMsg else: @@ -189,7 +189,7 @@ msg = uiTools.cropStr(msg, maxMsgSize) else: includeBreak = True - msg, remainder = uiTools.cropStr(msg, maxMsgSize, 4, 4, uiTools.END_WITH_HYPHEN, True) + msg, remainder = uiTools.cropStr(msg, maxMsgSize, 4, 4, uiTools.Ending.HYPHEN, True) displayQueue.insert(0, (remainder.strip(), format)) drawLine = displayLine + lineOffset Modified: arm/release/src/settings.cfg =================================================================== --- arm/release/src/settings.cfg 2011-04-04 15:15:20 UTC (rev 24554) +++ arm/release/src/settings.cfg 2011-04-04 15:22:31 UTC (rev 24555) @@ -1,3 +1,32 @@ +# Important tor configuration options (shown by default) +config.important BandwidthRate +config.important BandwidthBurst +config.important RelayBandwidthRate +config.important RelayBandwidthBurst +config.important ControlPort +config.important HashedControlPassword +config.important CookieAuthentication +config.important DataDirectory +config.important Log +config.important RunAsDaemon +config.important User +config.important Bridge +config.important ExcludeNodes +config.important SocksPort +config.important UseBridges +config.important BridgeRelay +config.important ContactInfo +config.important ExitPolicy +config.important MyFamily +config.important Nickname +config.important ORPort +config.important AccountingMax +config.important AccountingStart +config.important DirPortFrontPage +config.important DirPort +config.important HiddenServiceDir +config.important HiddenServicePort + # Summary descriptions for Tor configuration options # General Config Options config.summary.BandwidthRate Average bandwidth usage limit @@ -145,6 +174,58 @@ config.summary.ExitPortStatistics Toggles storing traffic and port usage data to disk config.summary.ExtraInfoStatistics Publishes statistic data in the extra-info documents +# Directory Server Options +config.summary.AuthoritativeDirectory act as a directory authority +config.summary.DirPortFrontPage publish this html file on the DirPort +config.summary.V1AuthoritativeDirectory generates a version 1 consensus +config.summary.V2AuthoritativeDirectory generates a version 2 consensus +config.summary.V3AuthoritativeDirectory generates a version 3 consensus +config.summary.VersioningAuthoritativeDirectory provides opinions on recommended versions of tor +config.summary.NamingAuthoritativeDirectory provides opinions on fingerprint to nickname bindings +config.summary.HSAuthoritativeDir toggles accepting hidden service descriptors +config.summary.HidServDirectoryV2 toggles accepting version 2 hidden service descriptors +config.summary.BridgeAuthoritativeDir acts as a bridge authority +config.summary.MinUptimeHidServDirectoryV2 required uptime before accepting hidden service directory +config.summary.DirPort port for directory connections +config.summary.DirListenAddress address the directory service is bound to +config.summary.DirPolicy access policy for the DirPort + +# Directory Authority Server Options +config.summary.RecommendedVersions tor versions believed to be safe +config.summary.RecommendedClientVersions tor versions believed to be safe for clients +config.summary.RecommendedServerVersions tor versions believed to be safe for relays +config.summary.ConsensusParams params entry of the networkstatus vote +config.summary.DirAllowPrivateAddresses toggles allowing arbitrary input or non-public IPs in descriptors +config.summary.AuthDirBadDir relays to be flagged as bad directory caches +config.summary.AuthDirBadExit relays to be flagged as bad exits +config.summary.AuthDirInvalid relays from which the valid flag is withheld +config.summary.AuthDirReject relays to be dropped from the consensus +config.summary.AuthDirListBadDirs toggles if we provide an opinion on bad directory caches +config.summary.AuthDirListBadExits toggles if we provide an opinion on bad exits +config.summary.AuthDirRejectUnlisted rejects further relay descriptors +config.summary.AuthDirMaxServersPerAddr limit on the number of relays accepted per ip +config.summary.AuthDirMaxServersPerAuthAddr limit on the number of relays accepted per an authority's ip +config.summary.V3AuthVotingInterval consensus voting interval +config.summary.V3AuthVoteDelay wait time to collect votes of other authorities +config.summary.V3AuthDistDelay wait time to collect the signatures of other authorities +config.summary.V3AuthNIntervalsValid number of voting intervals a consensus is valid for +config.summary.V3BandwidthsFile path to a file containing measured relay bandwidths + +# Hidden Service Options +config.summary.HiddenServiceDir directory contents for the hidden service +config.summary.HiddenServicePort port the hidden service is provided on +config.summary.PublishHidServDescriptors toggles automated publishing of the hidden service to the rendezvous directory +config.summary.HiddenServiceAuthorizeClient restricts access to the hidden service +config.summary.RendPostPeriod period at which the rendezvous service descriptors are refreshed + +# Testing Network Options +config.summary.TestingTorNetwork overrides other options to be a testing network +config.summary.TestingV3AuthInitialVotingInterval overrides V3AuthVotingInterval for the first consensus +config.summary.TestingV3AuthInitialVoteDelay overrides TestingV3AuthInitialVoteDelay for the first consensus +config.summary.TestingV3AuthInitialDistDelay overrides TestingV3AuthInitialDistDelay for the first consensus +config.summary.TestingAuthDirTimeToLearnReachability delay until opinions are given about which relays are running or not +config.summary.TestingEstimatedDescriptorPropagationTime delay before clients attempt to fetch descriptors from directory caches + # Snippets from common log messages # These are static bits of log messages, used to determine when entries with # dynamic content (hostnames, numbers, etc) are the same. If this matches the @@ -285,3 +366,287 @@ torrc.label.time.day day, days torrc.label.time.week week, weeks +# Common usages for ports based on: +# https://secure.wikimedia.org/wikipedia/en/wiki/List_of_TCP_and_UDP_port_num… +# http://isc.sans.edu/services.html +# +# Including all the official low ports (< 1024), and higher ones I recognize. + +port.label.1 TCPMUX +port.label.2 CompressNET +port.label.3 CompressNET +port.label.5 RJE +port.label.7 Echo +port.label.9 Discard +port.label.11 SYSTAT +port.label.13 Daytime +port.label.15 netstat +port.label.17 QOTD +port.label.18 MSP +port.label.19 CHARGEN +port.label.20 FTP +port.label.21 FTP +port.label.22 SSH +port.label.23 Telnet +port.label.24 Priv-mail +port.label.25 SMTP +port.label.34 RF +port.label.35 Printer +port.label.37 TIME +port.label.39 RLP +port.label.41 Graphics +port.label.42 WINS +port.label.43 WHOIS +port.label.47 NI FTP +port.label.49 TACACS +port.label.50 Remote Mail +port.label.51 IMP +port.label.52 XNS +port.label.53 DNS +port.label.54 XNS +port.label.55 ISI-GL +port.label.56 RAP +port.label.57 MTP +port.label.58 XNS +port.label.67 BOOTP +port.label.68 BOOTP +port.label.69 TFTP +port.label.70 Gopher +port.label.79 Finger +port.label.80 HTTP +port.label.81 Torpark +port.label.82 Torpark +port.label.83 MIT ML +port.label.88 Kerberos +port.label.90 dnsix +port.label.99 WIP +port.label.101 NIC +port.label.102 ISO-TSAP +port.label.104 ACR/NEMA +port.label.105 CCSO +port.label.107 Telnet +port.label.108 SNA +port.label.109 POP2 +port.label.110 POP3 +port.label.111 ONC RPC +port.label.113 ident +port.label.115 SFTP +port.label.117 UUCP +port.label.118 SQL +port.label.119 NNTP +port.label.123 NTP +port.label.135 DCE +port.label.137 NetBIOS +port.label.138 NetBIOS +port.label.139 NetBIOS +port.label.143 IMAP +port.label.152 BFTP +port.label.153 SGMP +port.label.156 SQL +port.label.158 DMSP +port.label.161 SNMP +port.label.162 SNMPTRAP +port.label.170 Print-srv +port.label.177 XDMCP +port.label.179 BGP +port.label.194 IRC +port.label.199 SMUX +port.label.201 AppleTalk +port.label.209 QMTP +port.label.210 ANSI +port.label.213 IPX +port.label.218 MPP +port.label.220 IMAP +port.label.256 2DEV +port.label.259 ESRO +port.label.264 BGMP +port.label.308 Novastor +port.label.311 OSX Admin +port.label.318 PKIX TSP +port.label.319 PTP +port.label.320 PTP +port.label.323 IMMP +port.label.350 MATIP +port.label.351 MATIP +port.label.366 ODMR +port.label.369 Rpc2portmap +port.label.370 codaauth2 +port.label.371 ClearCase +port.label.383 HP Alarm Mgr +port.label.384 ARNS +port.label.387 AURP +port.label.389 LDAP +port.label.401 UPS +port.label.402 Altiris +port.label.427 SLP +port.label.443 HTTPS +port.label.444 SNPP +port.label.445 SMB +port.label.464 Kerberos +port.label.465 SMTP +port.label.475 tcpnethaspsrv +port.label.497 Retrospect +port.label.500 ISAKMP +port.label.501 STMF +port.label.502 Modbus +port.label.504 Citadel +port.label.510 FirstClass +port.label.512 Rexec +port.label.513 rlogin +port.label.514 rsh +port.label.515 LPD +port.label.517 Talk +port.label.518 NTalk +port.label.520 efs +port.label.524 NCP +port.label.530 RPC +port.label.531 AIM/IRC +port.label.532 netnews +port.label.533 netwall +port.label.540 UUCP +port.label.542 commerce +port.label.543 klogin +port.label.544 klogin +port.label.545 OSISoft PI +port.label.546 DHCPv6 +port.label.547 DHCPv6 +port.label.548 AFP +port.label.550 new-who +port.label.554 RTSP +port.label.556 RFS +port.label.560 rmonitor +port.label.561 monitor +port.label.563 NNTPS +port.label.587 SMTP +port.label.591 FileMaker +port.label.593 HTTP RPC +port.label.604 TUNNEL +port.label.623 ASF-RMCP +port.label.631 CUPS +port.label.635 RLZ DBase +port.label.636 LDAPS +port.label.639 MSDP +port.label.641 SupportSoft +port.label.646 LDP +port.label.647 DHCP +port.label.648 RRP +port.label.651 IEEE-MMS +port.label.652 DTCP +port.label.653 SupportSoft +port.label.654 MMS/MMP +port.label.657 RMC +port.label.660 OSX Admin +port.label.665 sun-dr +port.label.666 Doom +port.label.674 ACAP +port.label.691 MS Exchange +port.label.692 Hyperwave-ISP +port.label.694 Linux-HA +port.label.695 IEEE-MMS-SSL +port.label.698 OLSR +port.label.699 Access Network +port.label.700 EPP +port.label.701 LMP +port.label.702 IRIS +port.label.706 SILC +port.label.711 MPLS +port.label.712 TBRPF +port.label.720 SMQP +port.label.749 Kerberos +port.label.750 rfile +port.label.751 pump +port.label.752 qrh +port.label.753 rrh +port.label.754 tell send +port.label.760 ns +port.label.782 Conserver +port.label.783 spamd +port.label.829 CMP +port.label.843 Flash +port.label.847 DHCP +port.label.860 iSCSI +port.label.873 rsync +port.label.888 CDDB +port.label.901 SWAT +port.label.902 VMware +port.label.903 VMware +port.label.904 VMware +port.label.911 NCA +port.label.953 DNS RNDC +port.label.981 SofaWare +port.label.989 FTPS +port.label.990 FTPS +port.label.991 NAS +port.label.992 Telnet +port.label.993 IMAPS +port.label.995 POP3S +port.label.999 ScimoreDB +port.label.1001 JtoMB +port.label.1002 cogbot + +port.label.1080 SOCKS +port.label.1085 WebObjects +port.label.1109 KPOP +port.label.1169 Tripwire +port.label.1194 OpenVPN +port.label.1214 Kazaa +port.label.1220 QuickTime +port.label.1234 VLC +port.label.1241 Nessus +port.label.1270 SCOM +port.label.1293 IPSec +port.label.1433 MSSQL +port.label.1434 MSSQL +port.label.1503 MSN +port.label.1512 WINS +port.label.1521 Oracle +port.label.1526 Oracle +port.label.1666 Perforce +port.label.1725 Steam +port.label.1863 MSNP +port.label.2049 NFS +port.label.2086 GNUnet +port.label.2401 CVS +port.label.2525 SMTP +port.label.2710 BitTorrent +port.label.3074 XBox LIVE +port.label.3101 BlackBerry +port.label.3306 MySQL +port.label.3690 SVN +port.label.3723 Battle.net +port.label.3724 WoW +port.label.4662 eMule +port.label.5003 FileMaker +port.label.5050 Yahoo IM +port.label.5060 SIP +port.label.5061 SIP +port.label.5190 AIM/ICQ +port.label.5222 Jabber +port.label.5223 Jabber +port.label.5269 Jabber +port.label.5298 Jabber +port.label.5432 PostgreSQL +port.label.5500 VNC +port.label.5556 Freeciv +port.label.5666 NRPE +port.label.5667 NSCA +port.label.5800 VNC +port.label.5900 VNC +port.label.6346 gnutella +port.label.6347 gnutella +port.label.6660-6669 IRC +port.label.6679 IRC +port.label.6697 IRC +port.label.6881-6999 BitTorrent +port.label.8008 HTTP +port.label.8010 XMPP +port.label.8080 Tomcat +port.label.8118 Privoxy +port.label.8123 Polipo +port.label.9030 Tor +port.label.9050 Tor +port.label.9051 Tor +port.label.23399 Skype +port.label.30301 BitTorrent +port.label.33434 traceroute + Modified: arm/release/src/starter.py =================================================================== --- arm/release/src/starter.py 2011-04-04 15:15:20 UTC (rev 24554) +++ arm/release/src/starter.py 2011-04-04 15:22:31 UTC (rev 24555) @@ -36,8 +36,9 @@ "startup.interface.port": 9051, "startup.blindModeEnabled": False, "startup.events": "N3", - "data.cache.path": "~/.arm/cache", + "startup.dataDirectory": "~/.arm", "features.config.descriptions.enabled": True, + "features.config.descriptions.persist": True, "log.configDescriptions.readManPageSuccess": util.log.INFO, "log.configDescriptions.readManPageFailed": util.log.NOTICE, "log.configDescriptions.internalLoadSuccess": util.log.NOTICE, @@ -88,29 +89,6 @@ # torrc entries that are scrubbed when dumping PRIVATE_TORRC_ENTRIES = ["HashedControlPassword", "Bridge", "HiddenServiceDir"] -def isValidIpAddr(ipStr): - """ - Returns true if input is a valid IPv4 address, false otherwise. - """ - - for i in range(4): - if i < 3: - divIndex = ipStr.find(".") - if divIndex == -1: return False # expected a period to be valid - octetStr = ipStr[:divIndex] - ipStr = ipStr[divIndex + 1:] - else: - octetStr = ipStr - - try: - octet = int(octetStr) - if not octet >= 0 or not octet <= 255: return False - except ValueError: - # address value isn't an integer - return False - - return True - def _loadConfigurationDescriptions(pathPrefix): """ Attempts to load descriptions for tor's configuration options, fetching them @@ -125,13 +103,14 @@ isConfigDescriptionsLoaded = False # determines the path where cached descriptions should be persisted (left - # undefined of arm caching is disabled) - cachePath, descriptorPath = CONFIG["data.cache.path"], None + # undefined if caching is disabled) + descriptorPath = None + if CONFIG["features.config.descriptions.persist"]: + dataDir = CONFIG["startup.dataDirectory"] + if not dataDir.endswith("/"): dataDir += "/" + + descriptorPath = os.path.expanduser(dataDir + "cache/") + CONFIG_DESC_FILENAME - if cachePath: - if not cachePath.endswith("/"): cachePath += "/" - descriptorPath = os.path.expanduser(cachePath) + CONFIG_DESC_FILENAME - # attempts to load configuration descriptions cached in the data directory if descriptorPath: try: @@ -166,7 +145,7 @@ msg = DESC_SAVE_SUCCESS_MSG % (descriptorPath, time.time() - loadStartTime) util.log.log(CONFIG["log.configDescriptions.persistance.loadSuccess"], msg) - except IOError, exc: + except (IOError, OSError), exc: msg = DESC_SAVE_FAILED_MSG % util.sysTools.getFileErrorMsg(exc) util.log.log(CONFIG["log.configDescriptions.persistance.saveFailed"], msg) @@ -324,7 +303,7 @@ controlAddr = param["startup.interface.ipAddress"] controlPort = param["startup.interface.port"] - if not isValidIpAddr(controlAddr): + if not util.connections.isValidIpAddress(controlAddr): print "'%s' isn't a valid IP address" % controlAddr sys.exit() elif controlPort < 0 or controlPort > 65535: @@ -409,5 +388,4 @@ _dumpConfig() interface.controller.startTorMonitor(time.time() - initTime, expandedEvents, param["startup.blindModeEnabled"]) - conn.close() Modified: arm/release/src/test.py =================================================================== --- arm/release/src/test.py 2011-04-04 15:15:20 UTC (rev 24554) +++ arm/release/src/test.py 2011-04-04 15:22:31 UTC (rev 24555) @@ -11,6 +11,7 @@ 1. Resolver Performance Test 2. Resolver Dump 3. Glyph Demo + 4. Exit Policy Check q. Quit Selection: """ @@ -23,7 +24,7 @@ userInput = raw_input(MENU) # initiate the TorCtl connection if the test needs it - if userInput in ("1", "2") and not conn: + if userInput in ("1", "2", "4") and not conn: conn = torTools.getConn() conn.init() @@ -43,7 +44,7 @@ connectionResults.sort() allConnectionResults.append(connectionResults) - resolverLabel = "%-10s" % connections.CMD_STR[resolver] + resolverLabel = "%-10s" % resolver countLabel = "%4i results" % len(connectionResults) timeLabel = "%0.4f seconds" % (time.time() - startTime) print "%s %s %s" % (resolverLabel, countLabel, timeLabel) @@ -67,8 +68,9 @@ # provide the selection options printDivider() print("Select a resolver:") - for i in range(1, 8): - print(" %i. %s" % (i, connections.CMD_STR[i])) + availableResolvers = connections.Resolver.values() + for i in range(len(availableResolvers)): + print(" %i. %s" % (i, availableResolvers[i])) print(" q. Go back to the main menu") userSelection = raw_input("\nSelection: ") @@ -101,6 +103,42 @@ # Switching to a curses context and back repeatedly seems to screw up the # terminal. Just to be safe this ends the process after the demo. break + elif userInput == "4": + # display the current exit policy and query if destinations are allowed by it + exitPolicy = conn.getExitPolicy() + print("Exit Policy: %s" % exitPolicy) + printDivider() + + while True: + # provide the selection options + userSelection = raw_input("\nCheck if destination is allowed (q to go back): ") + userSelection = userSelection.replace(" ", "").strip() # removes all whitespace + + isValidQuery, isExitAllowed = True, False + if userSelection == "q": + printDivider() + break + elif connections.isValidIpAddress(userSelection): + # just an ip address (use port 80) + isExitAllowed = exitPolicy.check(userSelection, 80) + elif userSelection.isdigit(): + # just a port (use a common ip like 4.2.2.2) + isExitAllowed = exitPolicy.check("4.2.2.2", userSelection) + elif ":" in userSelection: + # ip/port combination + ipAddr, port = userSelection.split(":", 1) + + if connections.isValidIpAddress(ipAddr) and port.isdigit(): + isExitAllowed = exitPolicy.check(ipAddr, port) + else: isValidQuery = False + else: isValidQuery = False # invalid input + + if isValidQuery: + resultStr = "is" if isExitAllowed else "is *not*" + print("Exiting %s allowed to that destination" % resultStr) + else: + print("'%s' isn't a valid destination (should be an ip, port, or ip:port)\n" % userSelection) + else: print("'%s' isn't a valid selection\n" % userInput) Modified: arm/release/src/util/__init__.py =================================================================== --- arm/release/src/util/__init__.py 2011-04-04 15:15:20 UTC (rev 24554) +++ arm/release/src/util/__init__.py 2011-04-04 15:22:31 UTC (rev 24555) @@ -4,5 +4,5 @@ and safely working with curses (hiding some of the gory details). """ -__all__ = ["conf", "connections", "hostnames", "log", "panel", "procTools", "sysTools", "torConfig", "torTools", "uiTools"] +__all__ = ["conf", "connections", "enum", "hostnames", "log", "panel", "procTools", "sysTools", "torConfig", "torTools", "uiTools"] Modified: arm/release/src/util/conf.py =================================================================== --- arm/release/src/util/conf.py 2011-04-04 15:15:20 UTC (rev 24554) +++ arm/release/src/util/conf.py 2011-04-04 15:22:31 UTC (rev 24555) @@ -105,36 +105,35 @@ default - value provided if no such key exists """ - callDefault = log.runlevelToStr(default) if key.startswith("log.") else default isMultivalue = isinstance(default, list) or isinstance(default, dict) - val = self.getValue(key, callDefault, isMultivalue) + val = self.getValue(key, default, isMultivalue) if val == default: return val if key.startswith("log."): - if val.lower() in ("none", "debug", "info", "notice", "warn", "err"): - val = log.strToRunlevel(val) + if val.upper() == "NONE": val = None + elif val.upper() in log.Runlevel.values(): val = val.upper() else: - msg = "config entry '%s' is expected to be a runlevel" % key - if default != None: msg += ", defaulting to '%s'" % callDefault + msg = "Config entry '%s' is expected to be a runlevel" % key + if default != None: msg += ", defaulting to '%s'" % default log.log(CONFIG["log.configEntryTypeError"], msg) val = default elif isinstance(default, bool): if val.lower() == "true": val = True elif val.lower() == "false": val = False else: - msg = "config entry '%s' is expected to be a boolean, defaulting to '%s'" % (key, str(default)) + msg = "Config entry '%s' is expected to be a boolean, defaulting to '%s'" % (key, str(default)) log.log(CONFIG["log.configEntryTypeError"], msg) val = default elif isinstance(default, int): try: val = int(val) except ValueError: - msg = "config entry '%s' is expected to be an integer, defaulting to '%i'" % (key, default) + msg = "Config entry '%s' is expected to be an integer, defaulting to '%i'" % (key, default) log.log(CONFIG["log.configEntryTypeError"], msg) val = default elif isinstance(default, float): try: val = float(val) except ValueError: - msg = "config entry '%s' is expected to be a float, defaulting to '%f'" % (key, default) + msg = "Config entry '%s' is expected to be a float, defaulting to '%f'" % (key, default) log.log(CONFIG["log.configEntryTypeError"], msg) val = default elif isinstance(default, list): @@ -146,7 +145,7 @@ entryKey, entryVal = entry.split("=>", 1) valMap[entryKey.strip()] = entryVal.strip() else: - msg = "ignoring invalid %s config entry (expected a mapping, but \"%s\" was missing \"=>\")" % (key, entry) + msg = "Ignoring invalid %s config entry (expected a mapping, but \"%s\" was missing \"=>\")" % (key, entry) log.log(CONFIG["log.configEntryTypeError"], msg) val = valMap @@ -171,7 +170,7 @@ # check if the count doesn't match if count != None and len(confComp) != count: - msg = "config entry '%s' is expected to be %i comma separated values" % (key, count) + msg = "Config entry '%s' is expected to be %i comma separated values" % (key, count) if default != None and (isinstance(default, list) or isinstance(default, tuple)): defaultStr = ", ".join([str(i) for i in default]) msg += ", defaulting to '%s'" % defaultStr @@ -201,7 +200,7 @@ # validates the input, setting the errorMsg if there's a problem errorMsg = None - baseErrorMsg = "config entry '%s' is expected to %%s" % key + baseErrorMsg = "Config entry '%s' is expected to %%s" % key if default != None and (isinstance(default, list) or isinstance(default, tuple)): defaultStr = ", ".join([str(i) for i in default]) baseErrorMsg += ", defaulting to '%s'" % defaultStr Modified: arm/release/src/util/connections.py =================================================================== --- arm/release/src/util/connections.py 2011-04-04 15:15:20 UTC (rev 24554) +++ arm/release/src/util/connections.py 2011-04-04 15:22:31 UTC (rev 24555) @@ -21,17 +21,16 @@ import time import threading -from util import log, procTools, sysTools +from util import enum, log, procTools, sysTools # enums for connection resolution utilities -CMD_PROC, CMD_NETSTAT, CMD_SOCKSTAT, CMD_LSOF, CMD_SS, CMD_BSD_SOCKSTAT, CMD_BSD_PROCSTAT = range(1, 8) -CMD_STR = {CMD_PROC: "proc", - CMD_NETSTAT: "netstat", - CMD_SS: "ss", - CMD_LSOF: "lsof", - CMD_SOCKSTAT: "sockstat", - CMD_BSD_SOCKSTAT: "sockstat (bsd)", - CMD_BSD_PROCSTAT: "procstat (bsd)"} +Resolver = enum.Enum(("PROC", "proc"), + ("NETSTAT", "netstat"), + ("SS", "ss"), + ("LSOF", "lsof"), + ("SOCKSTAT", "sockstat"), + ("BSD_SOCKSTAT", "sockstat (bsd)"), + ("BSD_PROCSTAT", "procstat (bsd)")) # If true this provides new instantiations for resolvers if the old one has # been stopped. This can make it difficult ensure all threads are terminated @@ -80,11 +79,107 @@ "log.connLookupFailed": log.INFO, "log.connLookupFailover": log.NOTICE, "log.connLookupAbandon": log.WARN, - "log.connLookupRateGrowing": None} + "log.connLookupRateGrowing": None, + "log.configEntryTypeError": log.NOTICE} +PORT_USAGE = {} + def loadConfig(config): config.update(CONFIG) + + for configKey in config.getKeys(): + # fetches any port.label.* values + if configKey.startswith("port.label."): + portEntry = configKey[11:] + purpose = config.get(configKey) + + divIndex = portEntry.find("-") + if divIndex == -1: + # single port + if portEntry.isdigit(): + PORT_USAGE[portEntry] = purpose + else: + msg = "Port value isn't numeric for entry: %s" % configKey + log.log(CONFIG["log.configEntryTypeError"], msg) + else: + try: + # range of ports (inclusive) + minPort = int(portEntry[:divIndex]) + maxPort = int(portEntry[divIndex + 1:]) + if minPort > maxPort: raise ValueError() + + for port in range(minPort, maxPort + 1): + PORT_USAGE[str(port)] = purpose + except ValueError: + msg = "Unable to parse port range for entry: %s" % configKey + log.log(CONFIG["log.configEntryTypeError"], msg) +def isValidIpAddress(ipStr): + """ + Returns true if input is a valid IPv4 address, false otherwise. + """ + + # checks if theres four period separated values + if not ipStr.count(".") == 3: return False + + # checks that each value in the octet are decimal values between 0-255 + for ipComp in ipStr.split("."): + if not ipComp.isdigit() or int(ipComp) < 0 or int(ipComp) > 255: + return False + + return True + +def isIpAddressPrivate(ipAddr): + """ + Provides true if the IP address belongs on the local network or belongs to + loopback, false otherwise. These include: + Private ranges: 10.*, 172.16.* - 172.31.*, 192.168.* + Loopback: 127.* + + Arguments: + ipAddr - IP address to be checked + """ + + # checks for any of the simple wildcard ranges + if ipAddr.startswith("10.") or ipAddr.startswith("192.168.") or ipAddr.startswith("127."): + return True + + # checks for the 172.16.* - 172.31.* range + if ipAddr.startswith("172.") and ipAddr.count(".") == 3: + secondOctet = ipAddr[4:ipAddr.find(".", 4)] + + if secondOctet.isdigit() and int(secondOctet) >= 16 and int(secondOctet) <= 31: + return True + + return False + +def ipToInt(ipAddr): + """ + Provides an integer representation of the ip address, suitable for sorting. + + Arguments: + ipAddr - ip address to be converted + """ + + total = 0 + + for comp in ipAddr.split("."): + total *= 255 + total += int(comp) + + return total + +def getPortUsage(port): + """ + Provides the common use of a given port. If no useage is known then this + provides None. + + Arguments: + port - port number to look up + """ + + return PORT_USAGE.get(port) + def getResolverCommand(resolutionCmd, processName, processPid = ""): """ Provides the command that would be processed for the given resolver type. @@ -99,19 +194,19 @@ if not processPid: # the pid is required for procstat resolution - if resolutionCmd == CMD_BSD_PROCSTAT: + if resolutionCmd == Resolver.BSD_PROCSTAT: raise ValueError("procstat resolution requires a pid") # if the pid was undefined then match any in that field processPid = "[0-9]*" - if resolutionCmd == CMD_PROC: return "" - elif resolutionCmd == CMD_NETSTAT: return RUN_NETSTAT % (processPid, processName) - elif resolutionCmd == CMD_SS: return RUN_SS % (processName, processPid) - elif resolutionCmd == CMD_LSOF: return RUN_LSOF % (processName, processPid) - elif resolutionCmd == CMD_SOCKSTAT: return RUN_SOCKSTAT % (processName, processPid) - elif resolutionCmd == CMD_BSD_SOCKSTAT: return RUN_BSD_SOCKSTAT % (processName, processPid) - elif resolutionCmd == CMD_BSD_PROCSTAT: return RUN_BSD_PROCSTAT % processPid + if resolutionCmd == Resolver.PROC: return "" + elif resolutionCmd == Resolver.NETSTAT: return RUN_NETSTAT % (processPid, processName) + elif resolutionCmd == Resolver.SS: return RUN_SS % (processName, processPid) + elif resolutionCmd == Resolver.LSOF: return RUN_LSOF % (processName, processPid) + elif resolutionCmd == Resolver.SOCKSTAT: return RUN_SOCKSTAT % (processName, processPid) + elif resolutionCmd == Resolver.BSD_SOCKSTAT: return RUN_BSD_SOCKSTAT % (processName, processPid) + elif resolutionCmd == Resolver.BSD_PROCSTAT: return RUN_BSD_PROCSTAT % processPid else: raise ValueError("Unrecognized resolution type: %s" % resolutionCmd) def getConnections(resolutionCmd, processName, processPid = ""): @@ -131,7 +226,7 @@ processPid - process ID (this helps improve accuracy) """ - if resolutionCmd == CMD_PROC: + if resolutionCmd == Resolver.PROC: # Attempts resolution via checking the proc contents. if not processPid: raise ValueError("proc resolution requires a pid") @@ -151,30 +246,30 @@ # parses results for the resolution command conn = [] for line in results: - if resolutionCmd == CMD_LSOF: + if resolutionCmd == Resolver.LSOF: # Different versions of lsof have different numbers of columns, so # stripping off the optional 'established' entry so we can just use # the last one. comp = line.replace("(ESTABLISHED)", "").strip().split() else: comp = line.split() - if resolutionCmd == CMD_NETSTAT: + if resolutionCmd == Resolver.NETSTAT: localIp, localPort = comp[3].split(":") foreignIp, foreignPort = comp[4].split(":") - elif resolutionCmd == CMD_SS: + elif resolutionCmd == Resolver.SS: localIp, localPort = comp[4].split(":") foreignIp, foreignPort = comp[5].split(":") - elif resolutionCmd == CMD_LSOF: + elif resolutionCmd == Resolver.LSOF: local, foreign = comp[-1].split("->") localIp, localPort = local.split(":") foreignIp, foreignPort = foreign.split(":") - elif resolutionCmd == CMD_SOCKSTAT: + elif resolutionCmd == Resolver.SOCKSTAT: localIp, localPort = comp[4].split(":") foreignIp, foreignPort = comp[5].split(":") - elif resolutionCmd == CMD_BSD_SOCKSTAT: + elif resolutionCmd == Resolver.BSD_SOCKSTAT: localIp, localPort = comp[5].split(":") foreignIp, foreignPort = comp[6].split(":") - elif resolutionCmd == CMD_BSD_PROCSTAT: + elif resolutionCmd == Resolver.BSD_PROCSTAT: localIp, localPort = comp[9].split(":") foreignIp, foreignPort = comp[10].split(":") @@ -241,13 +336,13 @@ if osType == None: osType = os.uname()[0] if osType == "FreeBSD": - resolvers = [CMD_BSD_SOCKSTAT, CMD_BSD_PROCSTAT, CMD_LSOF] + resolvers = [Resolver.BSD_SOCKSTAT, Resolver.BSD_PROCSTAT, Resolver.LSOF] else: - resolvers = [CMD_NETSTAT, CMD_SOCKSTAT, CMD_LSOF, CMD_SS] + resolvers = [Resolver.NETSTAT, Resolver.SOCKSTAT, Resolver.LSOF, Resolver.SS] # proc resolution, by far, outperforms the others so defaults to this is able if procTools.isProcAvailable(): - resolvers = [CMD_PROC] + resolvers + resolvers = [Resolver.PROC] + resolvers return resolvers @@ -317,22 +412,26 @@ self.defaultRate = CONFIG["queries.connections.minRate"] self.lastLookup = -1 self.overwriteResolver = None - self.defaultResolver = CMD_PROC + self.defaultResolver = Resolver.PROC osType = os.uname()[0] self.resolverOptions = getSystemResolvers(osType) - resolverLabels = ", ".join([CMD_STR[option] for option in self.resolverOptions]) - log.log(CONFIG["log.connResolverOptions"], "Operating System: %s, Connection Resolvers: %s" % (osType, resolverLabels)) + log.log(CONFIG["log.connResolverOptions"], "Operating System: %s, Connection Resolvers: %s" % (osType, ", ".join(self.resolverOptions))) # sets the default resolver to be the first found in the system's PATH # (left as netstat if none are found) for resolver in self.resolverOptions: - if resolver == CMD_PROC or sysTools.isAvailable(CMD_STR[resolver]): + # Resolver strings correspond to their command with the exception of bsd + # resolvers. + resolverCmd = resolver.replace(" (bsd)", "") + + if resolver == Resolver.PROC or sysTools.isAvailable(resolverCmd): self.defaultResolver = resolver break self._connections = [] # connection cache (latest results) + self._resolutionCounter = 0 # number of successful connection resolutions self._isPaused = False self._halt = False # terminates thread if true self._cond = threading.Condition() # used for pausing the thread @@ -371,6 +470,7 @@ lookupTime = time.time() - resolveStart self._connections = connResults + self._resolutionCounter += 1 newMinDefaultRate = 100 * lookupTime if self.defaultRate < newMinDefaultRate: @@ -409,7 +509,7 @@ if newResolver: # provide notice that failures have occurred and resolver is changing - msg = RESOLVER_SERIAL_FAILURE_MSG % (CMD_STR[resolver], CMD_STR[newResolver]) + msg = RESOLVER_SERIAL_FAILURE_MSG % (resolver, newResolver) log.log(CONFIG["log.connLookupFailover"], msg) else: # exhausted all resolvers, give warning @@ -428,6 +528,14 @@ if self._halt: return [] else: return list(self._connections) + def getResolutionCount(self): + """ + Provides the number of successful resolutions so far. This can be used to + determine if the connection results are new for the caller or not. + """ + + return self._resolutionCounter + def setPaused(self, isPause): """ Allows or prevents further connection resolutions (this still makes use of @@ -451,3 +559,168 @@ self._cond.notifyAll() self._cond.release() +class AppResolver: + """ + Provides the names and pids of appliations attached to the given ports. This + stops attempting to query if it fails three times without successfully + getting lsof results. + """ + + def __init__(self, scriptName = "python"): + """ + Constructs a resolver instance. + + Arguments: + scriptName - name by which to all our own entries + """ + + self.scriptName = scriptName + self.queryResults = {} + self.resultsLock = threading.RLock() + self._cond = threading.Condition() # used for pausing when waiting for results + self.isResolving = False # flag set if we're in the process of making a query + self.failureCount = 0 # -1 if we've made a successful query + + def getResults(self, maxWait=0): + """ + Provides the last queried results. If we're in the process of making a + query then we can optionally block for a time to see if it finishes. + + Arguments: + maxWait - maximum second duration to block on getting results before + returning + """ + + self._cond.acquire() + if self.isResolving and maxWait > 0: + self._cond.wait(maxWait) + self._cond.release() + + self.resultsLock.acquire() + results = dict(self.queryResults) + self.resultsLock.release() + + return results + + def resolve(self, ports): + """ + Queues the given listing of ports to be resolved. This clears the last set + of results when completed. + + Arguments: + ports - list of ports to be resolved to applications + """ + + if self.failureCount < 3: + self.isResolving = True + t = threading.Thread(target = self._queryApplications, kwargs = {"ports": ports}) + t.setDaemon(True) + t.start() + + def _queryApplications(self, ports=[]): + """ + Performs an lsof lookup on the given ports to get the command/pid tuples. + + Arguments: + ports - list of ports to be resolved to applications + """ + + # atagar@fenrir:~/Desktop/arm$ lsof -i tcp:51849 -i tcp:37277 + # COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME + # tor 2001 atagar 14u IPv4 14048 0t0 TCP localhost:9051->localhost:37277 (ESTABLISHED) + # tor 2001 atagar 15u IPv4 22024 0t0 TCP localhost:9051->localhost:51849 (ESTABLISHED) + # python 2462 atagar 3u IPv4 14047 0t0 TCP localhost:37277->localhost:9051 (ESTABLISHED) + # python 3444 atagar 3u IPv4 22023 0t0 TCP localhost:51849->localhost:9051 (ESTABLISHED) + + if not ports: + self.resultsLock.acquire() + self.queryResults = {} + self.isResolving = False + self.resultsLock.release() + + # wakes threads waiting on results + self._cond.acquire() + self._cond.notifyAll() + self._cond.release() + + return + + results = {} + lsofArgs = [] + + # Uses results from the last query if we have any, otherwise appends the + # port to the lsof command. This has the potential for persisting dirty + # results but if we're querying by the dynamic port on the local tcp + # connections then this should be very rare (and definitely worth the + # chance of being able to skip an lsof query altogether). + for port in ports: + if port in self.queryResults: + results[port] = self.queryResults[port] + else: lsofArgs.append("-i tcp:%s" % port) + + if lsofArgs: + lsofResults = sysTools.call("lsof -nP " + " ".join(lsofArgs)) + else: lsofResults = None + + if not lsofResults and self.failureCount != -1: + # lsof query failed and we aren't yet sure if it's possible to + # successfully get results on this platform + self.failureCount += 1 + self.isResolving = False + return + elif lsofResults: + # (iPort, oPort) tuple for our own process, if it was fetched + ourConnection = None + + for line in lsofResults: + lineComp = line.split() + + if len(lineComp) == 10 and lineComp[9] == "(ESTABLISHED)": + cmd, pid, _, _, _, _, _, _, portMap, _ = lineComp + + if "->" in portMap: + iPort, oPort = portMap.split("->") + iPort = iPort.split(":")[1] + oPort = oPort.split(":")[1] + + # entry belongs to our own process + if pid == str(os.getpid()): + cmd = self.scriptName + ourConnection = (iPort, oPort) + + if iPort.isdigit() and oPort.isdigit(): + newEntry = (iPort, oPort, cmd, pid) + + # adds the entry under the key of whatever we queried it with + # (this might be both the inbound _and_ outbound ports) + for portMatch in (iPort, oPort): + if portMatch in ports: + if portMatch in results: + results[portMatch].append(newEntry) + else: results[portMatch] = [newEntry] + + # making the lsof call generated an extraneous sh entry for our own connection + if ourConnection: + for ourPort in ourConnection: + if ourPort in results: + shIndex = None + + for i in range(len(results[ourPort])): + if results[ourPort][i][2] == "sh": + shIndex = i + break + + if shIndex != None: + del results[ourPort][shIndex] + + self.resultsLock.acquire() + self.failureCount = -1 + self.queryResults = results + self.isResolving = False + self.resultsLock.release() + + # wakes threads waiting on results + self._cond.acquire() + self._cond.notifyAll() + self._cond.release() + Copied: arm/release/src/util/enum.py (from rev 24554, arm/trunk/src/util/enum.py) =================================================================== --- arm/release/src/util/enum.py (rev 0) +++ arm/release/src/util/enum.py 2011-04-04 15:22:31 UTC (rev 24555) @@ -0,0 +1,116 @@ +""" +Basic enumeration, providing ordered types for collections. These can be +constructed as simple type listings, ie: +>>> insects = Enum("ANT", "WASP", "LADYBUG", "FIREFLY") +>>> insects.ANT +'Ant' +>>> insects.values() +['Ant', 'Wasp', 'Ladybug', 'Firefly'] + +with overwritten string counterparts: +>>> pets = Enum(("DOG", "Skippy"), "CAT", ("FISH", "Nemo")) +>>> pets.DOG +'Skippy' +>>> pets.CAT +'Cat' + +or with entirely custom string components as an unordered enum with: +>>> pets = LEnum(DOG="Skippy", CAT="Kitty", FISH="Nemo") +>>> pets.CAT +'Kitty' +""" + +def toCamelCase(label): + """ + Converts the given string to camel case, ie: + >>> toCamelCase("I_LIKE_PEPPERJACK!") + 'I Like Pepperjack!' + + Arguments: + label - input string to be converted + """ + + words = [] + for entry in label.split("_"): + if len(entry) == 0: words.append("") + elif len(entry) == 1: words.append(entry.upper()) + else: words.append(entry[0].upper() + entry[1:].lower()) + + return " ".join(words) + +class Enum: + """ + Basic enumeration. + """ + + def __init__(self, *args): + self.orderedValues = [] + + for entry in args: + if isinstance(entry, str): + key, val = entry, toCamelCase(entry) + elif isinstance(entry, tuple) and len(entry) == 2: + key, val = entry + else: raise ValueError("Unrecognized input: %s" % args) + + self.__dict__[key] = val + self.orderedValues.append(val) + + def values(self): + """ + Provides an ordered listing of the enumerations in this set. + """ + + return list(self.orderedValues) + + def indexOf(self, value): + """ + Provides the index of the given value in the collection. This raises a + ValueError if no such element exists. + + Arguments: + value - entry to be looked up + """ + + return self.orderedValues.index(value) + + def next(self, value): + """ + Provides the next enumeration after the given value, raising a ValueError + if no such enum exists. + + Arguments: + value - enumeration for which to get the next entry + """ + + if not value in self.orderedValues: + raise ValueError("No such enumeration exists: %s (options: %s)" % (value, ", ".join(self.orderedValues))) + + nextIndex = (self.orderedValues.index(value) + 1) % len(self.orderedValues) + return self.orderedValues[nextIndex] + + def previous(self, value): + """ + Provides the previous enumeration before the given value, raising a + ValueError if no such enum exists. + + Arguments: + value - enumeration for which to get the previous entry + """ + + if not value in self.orderedValues: + raise ValueError("No such enumeration exists: %s (options: %s)" % (value, ", ".join(self.orderedValues))) + + prevIndex = (self.orderedValues.index(value) - 1) % len(self.orderedValues) + return self.orderedValues[prevIndex] + +class LEnum(Enum): + """ + Enumeration that accepts custom string mappings. + """ + + def __init__(self, **args): + Enum.__init__(self) + self.__dict__.update(args) + self.orderedValues = sorted(args.values()) + Modified: arm/release/src/util/log.py =================================================================== --- arm/release/src/util/log.py 2011-04-04 15:15:20 UTC (rev 24554) +++ arm/release/src/util/log.py 2011-04-04 15:22:31 UTC (rev 24555) @@ -11,19 +11,23 @@ from sys import maxint from threading import RLock -# logging runlevels -DEBUG, INFO, NOTICE, WARN, ERR = range(1, 6) -RUNLEVEL_STR = {DEBUG: "DEBUG", INFO: "INFO", NOTICE: "NOTICE", WARN: "WARN", ERR: "ERR"} +from util import enum +# Logging runlevels. These are *very* commonly used so including shorter +# aliases (so they can be referenced as log.DEBUG, log.WARN, etc). +Runlevel = enum.Enum(("DEBUG", "DEBUG"), ("INFO", "INFO"), ("NOTICE", "NOTICE"), + ("WARN", "WARN"), ("ERR", "ERR")) +DEBUG, INFO, NOTICE, WARN, ERR = Runlevel.values() + # provides thread safety for logging operations LOG_LOCK = RLock() # chronologically ordered records of events for each runlevel, stored as tuples # consisting of: (time, message) -_backlog = dict([(level, []) for level in range(1, 6)]) +_backlog = dict([(level, []) for level in Runlevel.values()]) # mapping of runlevels to the listeners interested in receiving events from it -_listeners = dict([(level, []) for level in range(1, 6)]) +_listeners = dict([(level, []) for level in Runlevel.values()]) CONFIG = {"cache.armLog.size": 1000, "cache.armLog.trimSize": 200} @@ -55,36 +59,6 @@ DUMP_FILE = open(logPath, "w") -def strToRunlevel(runlevelStr): - """ - Converts runlevel strings ("DEBUG", "INFO", "NOTICE", etc) to their - corresponding enumeations. This isn't case sensitive and provides None if - unrecognized. - - Arguments: - runlevelStr - string to be converted to runlevel - """ - - if not runlevelStr: return None - - runlevelStr = runlevelStr.upper() - for enum, level in RUNLEVEL_STR.items(): - if level == runlevelStr: return enum - - return None - -def runlevelToStr(runlevelEnum): - """ - Converts runlevel enumerations to corresponding string. If unrecognized then - this provides "NONE". - - Arguments: - runlevelEnum - enumeration to be converted to string - """ - - if runlevelEnum in RUNLEVEL_STR: return RUNLEVEL_STR[runlevelEnum] - else: return "NONE" - def log(level, msg, eventTime = None): """ Registers an event, directing it to interested listeners and preserving it in @@ -128,7 +102,7 @@ try: entryTime = time.localtime(eventTime) timeLabel = "%i/%i/%i %02i:%02i:%02i" % (entryTime[1], entryTime[2], entryTime[0], entryTime[3], entryTime[4], entryTime[5]) - logEntry = "%s [%s] %s\n" % (timeLabel, runlevelToStr(level), msg) + logEntry = "%s [%s] %s\n" % (timeLabel, level, msg) DUMP_FILE.write(logEntry) DUMP_FILE.flush() except IOError, exc: @@ -137,7 +111,7 @@ # notifies listeners for callback in _listeners[level]: - callback(RUNLEVEL_STR[level], msg, eventTime) + callback(level, msg, eventTime) finally: LOG_LOCK.release() @@ -175,7 +149,7 @@ if dumpBacklog: for level, msg, eventTime in _getEntries(levels): - callback(RUNLEVEL_STR[level], msg, eventTime) + callback(level, msg, eventTime) finally: LOG_LOCK.release() Modified: arm/release/src/util/panel.py =================================================================== --- arm/release/src/util/panel.py 2011-04-04 15:15:20 UTC (rev 24554) +++ arm/release/src/util/panel.py 2011-04-04 15:22:31 UTC (rev 24555) @@ -56,8 +56,9 @@ # implementations aren't entirely deterministic (for instance panels # might chose their height based on its parent's current width). + self.panelName = name self.parent = parent - self.panelName = name + self.visible = True self.top = top self.height = height self.width = width @@ -99,6 +100,23 @@ self.parent = parent self.win = None + def isVisible(self): + """ + Provides if the panel's configured to be visible or not. + """ + + return self.visible + + def setVisible(self, isVisible): + """ + Toggles if the panel is visible or not. + + Arguments: + isVisible - panel is redrawn when requested if true, skipped otherwise + """ + + self.visible = isVisible + def getTop(self): """ Provides the position subwindows are placed at within its parent. @@ -170,7 +188,7 @@ if setWidth != -1: newWidth = min(newWidth, setWidth) return (newHeight, newWidth) - def draw(self, subwindow, width, height): + def draw(self, width, height): """ Draws display's content. This is meant to be overwritten by implementations and not called directly (use redraw() instead). The @@ -178,10 +196,8 @@ a column less than the actual space. Arguments: - sudwindow - panel's current subwindow instance, providing raw access to - its curses functions - width - horizontal space available for content - height - vertical space available for content + width - horizontal space available for content + height - vertical space available for content """ pass @@ -198,6 +214,9 @@ abandoned """ + # skipped if not currently visible + if not self.isVisible(): return + # if the panel's completely outside its parent then this is a no-op newHeight, newWidth = self.getPreferredSize() if newHeight == 0: @@ -222,11 +241,70 @@ try: if forceRedraw: self.win.erase() # clears any old contents - self.draw(self.win, self.maxX - 1, self.maxY) + self.draw(self.maxX - 1, self.maxY) self.win.refresh() finally: CURSES_LOCK.release() + def hline(self, y, x, length, attr=curses.A_NORMAL): + """ + Draws a horizontal line. This should only be called from the context of a + panel's draw method. + + Arguments: + y - vertical location + x - horizontal location + length - length the line spans + attr - text attributes + """ + + if self.win and self.maxX > x and self.maxY > y: + try: + drawLength = min(length, self.maxX - x) + self.win.hline(y, x, curses.ACS_HLINE | attr, drawLength) + except: + # in edge cases drawing could cause a _curses.error + pass + + def vline(self, y, x, length, attr=curses.A_NORMAL): + """ + Draws a vertical line. This should only be called from the context of a + panel's draw method. + + Arguments: + y - vertical location + x - horizontal location + length - length the line spans + attr - text attributes + """ + + if self.win and self.maxX > x and self.maxY > y: + try: + drawLength = min(length, self.maxY - y) + self.win.vline(y, x, curses.ACS_VLINE | attr, drawLength) + except: + # in edge cases drawing could cause a _curses.error + pass + + def addch(self, y, x, char, attr=curses.A_NORMAL): + """ + Draws a single character. This should only be called from the context of a + panel's draw method. + + Arguments: + y - vertical location + x - horizontal location + char - character to be drawn + attr - text attributes + """ + + if self.win and self.maxX > x and self.maxY > y: + try: + self.win.addch(y, x, char, attr) + except: + # in edge cases drawing could cause a _curses.error + pass + def addstr(self, y, x, msg, attr=curses.A_NORMAL): """ Writes string to subwindow if able. This takes into account screen bounds Modified: arm/release/src/util/procTools.py =================================================================== --- arm/release/src/util/procTools.py 2011-04-04 15:15:20 UTC (rev 24554) +++ arm/release/src/util/procTools.py 2011-04-04 15:22:31 UTC (rev 24555) @@ -20,12 +20,12 @@ import socket import base64 -from util import log +from util import enum, log # cached system values SYS_START_TIME, SYS_PHYSICAL_MEMORY = None, None CLOCK_TICKS = os.sysconf(os.sysconf_names["SC_CLK_TCK"]) -STAT_COMMAND, STAT_CPU_UTIME, STAT_CPU_STIME, STAT_START_TIME = range(4) +Stat = enum.Enum("COMMAND", "CPU_UTIME", "CPU_STIME", "START_TIME") CONFIG = {"queries.useProc": True, "log.procCallMade": log.DEBUG} @@ -128,10 +128,10 @@ def getStats(pid, *statTypes): """ Provides process specific information. Options are: - STAT_COMMAND command name under which the process is running - STAT_CPU_UTIME total user time spent on the process - STAT_CPU_STIME total system time spent on the process - STAT_START_TIME when this process began, in unix time + Stat.COMMAND command name under which the process is running + Stat.CPU_UTIME total user time spent on the process + Stat.CPU_STIME total system time spent on the process + Stat.START_TIME when this process began, in unix time Arguments: pid - queried process @@ -159,19 +159,19 @@ results, queriedStats = [], [] for statType in statTypes: - if statType == STAT_COMMAND: + if statType == Stat.COMMAND: queriedStats.append("command") if pid == 0: results.append("sched") else: results.append(statComp[1]) - elif statType == STAT_CPU_UTIME: + elif statType == Stat.CPU_UTIME: queriedStats.append("utime") if pid == 0: results.append("0") else: results.append(str(float(statComp[13]) / CLOCK_TICKS)) - elif statType == STAT_CPU_STIME: + elif statType == Stat.CPU_STIME: queriedStats.append("stime") if pid == 0: results.append("0") else: results.append(str(float(statComp[14]) / CLOCK_TICKS)) - elif statType == STAT_START_TIME: + elif statType == Stat.START_TIME: queriedStats.append("start time") if pid == 0: return getSystemStartTime() else: Modified: arm/release/src/util/sysTools.py =================================================================== --- arm/release/src/util/sysTools.py 2011-04-04 15:15:20 UTC (rev 24554) +++ arm/release/src/util/sysTools.py 2011-04-04 15:22:31 UTC (rev 24555) @@ -126,7 +126,7 @@ # fetch it from proc contents if available if procTools.isProcAvailable(): try: - processName = procTools.getStats(pid, procTools.STAT_COMMAND)[0] + processName = procTools.getStats(pid, procTools.Stat.COMMAND)[0] except IOError, exc: raisedExc = exc @@ -466,7 +466,7 @@ newValues = {} try: if self._useProc: - utime, stime, startTime = procTools.getStats(self.processPid, procTools.STAT_CPU_UTIME, procTools.STAT_CPU_STIME, procTools.STAT_START_TIME) + utime, stime, startTime = procTools.getStats(self.processPid, procTools.Stat.CPU_UTIME, procTools.Stat.CPU_STIME, procTools.Stat.START_TIME) totalCpuTime = float(utime) + float(stime) cpuDelta = totalCpuTime - self._lastCpuTotal newValues["cpuSampling"] = cpuDelta / timeSinceReset Modified: arm/release/src/util/torConfig.py =================================================================== --- arm/release/src/util/torConfig.py 2011-04-04 15:15:20 UTC (rev 24554) +++ arm/release/src/util/torConfig.py 2011-04-04 15:22:31 UTC (rev 24555) @@ -5,9 +5,10 @@ import os import threading -from util import log, sysTools, torTools, uiTools +from util import enum, log, sysTools, torTools, uiTools CONFIG = {"features.torrc.validate": True, + "config.important": [], "torrc.alias": {}, "torrc.label.size.b": [], "torrc.label.size.kb": [], @@ -22,27 +23,23 @@ "log.configDescriptions.unrecognizedCategory": log.NOTICE} # enums and values for numeric torrc entries -UNRECOGNIZED, SIZE_VALUE, TIME_VALUE = range(1, 4) +ValueType = enum.Enum("UNRECOGNIZED", "SIZE", "TIME") SIZE_MULT = {"b": 1, "kb": 1024, "mb": 1048576, "gb": 1073741824, "tb": 1099511627776} TIME_MULT = {"sec": 1, "min": 60, "hour": 3600, "day": 86400, "week": 604800} # enums for issues found during torrc validation: -# VAL_DUPLICATE - entry is ignored due to being a duplicate -# VAL_MISMATCH - the value doesn't match tor's current state -# VAL_MISSING - value differs from its default but is missing from the torrc -# VAL_IS_DEFAULT - the configuration option matches tor's default -VAL_DUPLICATE, VAL_MISMATCH, VAL_MISSING, VAL_IS_DEFAULT = range(1, 5) +# DUPLICATE - entry is ignored due to being a duplicate +# MISMATCH - the value doesn't match tor's current state +# MISSING - value differs from its default but is missing from the torrc +# IS_DEFAULT - the configuration option matches tor's default +ValidationError = enum.Enum("DUPLICATE", "MISMATCH", "MISSING", "IS_DEFAULT") # descriptions of tor's configuration options fetched from its man page CONFIG_DESCRIPTIONS_LOCK = threading.RLock() CONFIG_DESCRIPTIONS = {} # categories for tor configuration options -GENERAL, CLIENT, SERVER, DIRECTORY, AUTHORITY, HIDDEN_SERVICE, TESTING, UNKNOWN = range(1, 9) -OPTION_CATEGORY_STR = {GENERAL: "General", CLIENT: "Client", - SERVER: "Relay", DIRECTORY: "Directory", - AUTHORITY: "Authority", HIDDEN_SERVICE: "Hidden Service", - TESTING: "Testing", UNKNOWN: "Unknown"} +Category = enum.Enum("GENERAL", "CLIENT", "RELAY", "DIRECTORY", "AUTHORITY", "HIDDEN_SERVICE", "TESTING", "UNKNOWN") TORRC = None # singleton torrc instance MAN_OPT_INDENT = 7 # indentation before options in the man page @@ -51,10 +48,13 @@ MULTILINE_PARAM = None # cached multiline parameters (lazily loaded) def loadConfig(config): - CONFIG["torrc.alias"] = config.get("torrc.alias", {}) + config.update(CONFIG) - # fetches any config.summary.* values + # stores lowercase entries to drop case sensitivity + CONFIG["config.important"] = [entry.lower() for entry in CONFIG["config.important"]] + for configKey in config.getKeys(): + # fetches any config.summary.* values if configKey.startswith("config.summary."): CONFIG[configKey.lower()] = config.get(configKey) @@ -119,9 +119,6 @@ inputFileContents = inputFile.readlines() inputFile.close() - # constructs a reverse mapping for categories - strToCat = dict([(OPTION_CATEGORY_STR[cat], cat) for cat in OPTION_CATEGORY_STR]) - try: versionLine = inputFileContents.pop(0).rstrip() @@ -138,10 +135,8 @@ while inputFileContents: # gets category enum, failing if it doesn't exist - categoryStr = inputFileContents.pop(0).rstrip() - if categoryStr in strToCat: - category = strToCat[categoryStr] - else: + category = inputFileContents.pop(0).rstrip() + if not category in Category.values(): baseMsg = "invalid category in input file: '%s'" raise IOError(baseMsg % categoryStr) @@ -183,7 +178,7 @@ validOptions = [line[:line.find(" ")].lower() for line in configOptionQuery] optionCount, lastOption, lastArg = 0, None, None - lastCategory, lastDescription = GENERAL, "" + lastCategory, lastDescription = Category.GENERAL, "" for line in manCallResults: line = uiTools.getPrintable(line) strippedLine = line.strip() @@ -217,13 +212,13 @@ # if this is a category header then switch it if isCategoryLine: - if line.startswith("OPTIONS"): lastCategory = GENERAL - elif line.startswith("CLIENT"): lastCategory = CLIENT - elif line.startswith("SERVER"): lastCategory = SERVER - elif line.startswith("DIRECTORY SERVER"): lastCategory = DIRECTORY - elif line.startswith("DIRECTORY AUTHORITY SERVER"): lastCategory = AUTHORITY - elif line.startswith("HIDDEN SERVICE"): lastCategory = HIDDEN_SERVICE - elif line.startswith("TESTING NETWORK"): lastCategory = TESTING + if line.startswith("OPTIONS"): lastCategory = Category.GENERAL + elif line.startswith("CLIENT"): lastCategory = Category.CLIENT + elif line.startswith("SERVER"): lastCategory = Category.RELAY + elif line.startswith("DIRECTORY SERVER"): lastCategory = Category.DIRECTORY + elif line.startswith("DIRECTORY AUTHORITY SERVER"): lastCategory = Category.AUTHORITY + elif line.startswith("HIDDEN SERVICE"): lastCategory = Category.HIDDEN_SERVICE + elif line.startswith("TESTING NETWORK"): lastCategory = Category.TESTING else: msg = "Unrecognized category in the man page: %s" % line.strip() log.log(CONFIG["log.configDescriptions.unrecognizedCategory"], msg) @@ -249,7 +244,7 @@ def saveOptionDescriptions(path): """ Preserves the current configuration descriptors to the given path. This - raises an IOError if unable to do so. + raises an IOError or OSError if unable to do so. Arguments: path - location to persist configuration descriptors @@ -269,7 +264,7 @@ for i in range(len(sortedOptions)): option = sortedOptions[i] manEntry = getConfigDescription(option) - outputFile.write("%s\nindex: %i\n%s\n%s\n%s\n" % (OPTION_CATEGORY_STR[manEntry.category], manEntry.index, option, manEntry.argUsage, manEntry.description)) + outputFile.write("%s\nindex: %i\n%s\n%s\n%s\n" % (manEntry.category, manEntry.index, option, manEntry.argUsage, manEntry.description)) if i != len(sortedOptions) - 1: outputFile.write(PERSIST_ENTRY_DIVIDER) outputFile.close() @@ -286,6 +281,17 @@ return CONFIG.get("config.summary.%s" % option.lower()) +def isImportant(option): + """ + Provides True if the option has the 'important' flag in the configuration, + False otherwise. + + Arguments: + option - tor config option + """ + + return option.lower() in CONFIG["config.important"] + def getConfigDescription(option): """ Provides ManPageEntry instances populated with information fetched from the @@ -345,19 +351,31 @@ return tuple(MULTILINE_PARAM) -def getCustomOptions(): +def getCustomOptions(includeValue = False): """ - Provides the set of torrc parameters that differ from their defaults. + Provides the torrc parameters that differ from their defaults. + + Arguments: + includeValue - provides the current value with results if true, otherwise + this just contains the options """ - customOptions, conn = set(), torTools.getConn() - configTextQuery = conn.getInfo("config-text", "").strip().split("\n") + configText = torTools.getConn().getInfo("config-text", "").strip() + configLines = configText.split("\n") - for entry in configTextQuery: - # tor provides a Log entry even if it matches the default - if entry != "Log notice stdout": - customOptions.add(entry[:entry.find(" ")]) - return customOptions + # removes any duplicates + configLines = list(set(configLines)) + + # The "GETINFO config-text" query only provides options that differ + # from Tor's defaults with the exception of its Log entry which, even + # if undefined, returns "Log notice stdout" as per: + # https://trac.torproject.org/projects/tor/ticket/2362 + + try: configLines.remove("Log notice stdout") + except ValueError: pass + + if includeValue: return configLines + else: return [line[:line.find(" ")] for line in configLines] def validate(contents = None): """ @@ -403,13 +421,13 @@ # most parameters are overwritten if defined multiple times if option in seenOptions and not option in getMultilineParameters(): - issuesFound.append((lineNumber, VAL_DUPLICATE, option)) + issuesFound.append((lineNumber, ValidationError.DUPLICATE, option)) continue else: seenOptions.append(option) # checks if the value isn't necessary due to matching the defaults if not option in customOptions: - issuesFound.append((lineNumber, VAL_IS_DEFAULT, option)) + issuesFound.append((lineNumber, ValidationError.IS_DEFAULT, option)) # replace aliases with their recognized representation if option in CONFIG["torrc.alias"]: @@ -444,17 +462,17 @@ if not isBlankMatch and not val in torValues: # converts corrections to reader friedly size values displayValues = torValues - if valueType == SIZE_VALUE: + if valueType == ValueType.SIZE: displayValues = [uiTools.getSizeLabel(int(val)) for val in torValues] - elif valueType == TIME_VALUE: + elif valueType == ValueType.TIME: displayValues = [uiTools.getTimeLabel(int(val)) for val in torValues] - issuesFound.append((lineNumber, VAL_MISMATCH, ", ".join(displayValues))) + issuesFound.append((lineNumber, ValidationError.MISMATCH, ", ".join(displayValues))) # checks if any custom options are missing from the torrc for option in customOptions: if not option in seenOptions: - issuesFound.append((None, VAL_MISSING, option)) + issuesFound.append((None, ValidationError.MISSING, option)) return issuesFound @@ -470,13 +488,13 @@ if confArg.count(" ") == 1: val, unit = confArg.lower().split(" ", 1) - if not val.isdigit(): return confArg, UNRECOGNIZED + if not val.isdigit(): return confArg, ValueType.UNRECOGNIZED mult, multType = _getUnitType(unit) if mult != None: return str(int(val) * mult), multType - return confArg, UNRECOGNIZED + return confArg, ValueType.UNRECOGNIZED def _getUnitType(unit): """ @@ -489,13 +507,13 @@ for label in SIZE_MULT: if unit in CONFIG["torrc.label.size." + label]: - return SIZE_MULT[label], SIZE_VALUE + return SIZE_MULT[label], ValueType.SIZE for label in TIME_MULT: if unit in CONFIG["torrc.label.time." + label]: - return TIME_MULT[label], TIME_VALUE + return TIME_MULT[label], ValueType.TIME - return None, UNRECOGNIZED + return None, ValueType.UNRECOGNIZED def _stripComments(contents): """ @@ -622,13 +640,23 @@ self.valsLock.acquire() + # The torrc validation relies on 'GETINFO config-text' which was + # introduced in tor 0.2.2.7-alpha so if we're using an earlier version + # (or configured to skip torrc validation) then this is a no-op. For more + # information see: + # https://trac.torproject.org/projects/tor/ticket/2501 + if not self.isLoaded(): returnVal = None - elif not CONFIG["features.torrc.validate"]: returnVal = {} else: - if self.corrections == None: - self.corrections = validate(self.contents) + skipValidation = not CONFIG["features.torrc.validate"] + skipValidation |= not torTools.getConn().isVersion("0.2.2.7-alpha") - returnVal = list(self.corrections) + if skipValidation: returnVal = {} + else: + if self.corrections == None: + self.corrections = validate(self.contents) + + returnVal = list(self.corrections) self.valsLock.release() return returnVal Modified: arm/release/src/util/torTools.py =================================================================== --- arm/release/src/util/torTools.py 2011-04-04 15:15:20 UTC (rev 24554) +++ arm/release/src/util/torTools.py 2011-04-04 15:22:31 UTC (rev 24555) @@ -17,13 +17,25 @@ from TorCtl import TorCtl, TorUtil -from util import log, procTools, sysTools, uiTools +from util import enum, log, procTools, sysTools, uiTools # enums for tor's controller state: -# TOR_INIT - attached to a new controller or restart/sighup signal received -# TOR_CLOSED - control port closed -TOR_INIT, TOR_CLOSED = range(1, 3) +# INIT - attached to a new controller or restart/sighup signal received +# CLOSED - control port closed +State = enum.Enum("INIT", "CLOSED") +# Addresses of the default directory authorities for tor version 0.2.3.0-alpha +# (this comes from the dirservers array in src/or/config.c). +DIR_SERVERS = [("86.59.21.38", "80"), # tor26 + ("128.31.0.39", "9031"), # moria1 + ("216.224.124.114", "9030"), # ides + ("212.112.245.170", "80"), # gabelmoo + ("194.109.206.212", "80"), # dizum + ("193.23.244.244", "80"), # dannenberg + ("208.83.223.34", "443"), # urras + ("213.115.239.118", "443"), # maatuska + ("82.94.251.203", "80")] # Tonga + # message logged by default when a controller can't set an event type DEFAULT_FAILED_EVENT_MSG = "Unsupported event type: %s" @@ -41,9 +53,21 @@ # options (unchangable, even with a SETCONF) and other useful stats CACHE_ARGS = ("version", "config-file", "exit-policy/default", "fingerprint", "config/names", "info/names", "features/names", "events/names", - "nsEntry", "descEntry", "bwRate", "bwBurst", "bwObserved", - "bwMeasured", "flags", "pid", "pathPrefix", "startTime") + "nsEntry", "descEntry", "address", "bwRate", "bwBurst", + "bwObserved", "bwMeasured", "flags", "parsedVersion", "pid", + "pathPrefix", "startTime", "authorities", "circuits", "hsPorts") +CACHE_GETINFO_PREFIX_ARGS = ("ip-to-country/", ) +# Tor has a couple messages (in or/router.c) for when our ip address changes: +# "Our IP Address has changed from <previous> to <current>; rebuilding +# descriptor (source: <source>)." +# "Guessed our IP address as <current> (source: <source>)." +# +# It would probably be preferable to use the EXTERNAL_ADDRESS event, but I'm +# not quite sure why it's not provided by check_descriptor_ipaddress_changed +# so erring on the side of inclusiveness by using the notice event instead. +ADDR_CHANGED_MSG_PREFIX = ("Our IP Address has changed from", "Guessed our IP address as") + TOR_CTL_CLOSE_MSG = "Tor closed control connection. Exiting event thread." UNKNOWN = "UNKNOWN" # value used by cached information if undefined CONFIG = {"torrc.map": {}, @@ -52,6 +76,7 @@ "log.torGetInfo": log.DEBUG, "log.torGetInfoCache": None, "log.torGetConf": log.DEBUG, + "log.torGetConfCache": None, "log.torSetConf": log.INFO, "log.torPrefixPathInvalid": log.NOTICE, "log.bsdJailFound": log.INFO, @@ -65,9 +90,22 @@ "NS": "information related to the consensus will grow stale", "NEWCONSENSUS": "information related to the consensus will grow stale"} +# number of sequential attempts before we decide that the Tor geoip database +# is unavailable +GEOIP_FAILURE_THRESHOLD = 5 + # provides int -> str mappings for torctl event runlevels TORCTL_RUNLEVELS = dict([(val, key) for (key, val) in TorUtil.loglevels.items()]) +# ip address ranges substituted by the 'private' keyword +PRIVATE_IP_RANGES = ("0.0.0.0/8", "169.254.0.0/16", "127.0.0.0/8", "192.168.0.0/16", "10.0.0.0/8", "172.16.0.0/12") + +# This prevents controllers from spawning worker threads (and by extension +# notifying status listeners). This is important when shutting down to prevent +# rogue threads from being alive during shutdown. + +NO_SPAWN = False + def loadConfig(config): config.update(CONFIG) @@ -185,6 +223,41 @@ log.log(CONFIG["log.unknownBsdJailId"], "Failed to figure out the FreeBSD jail id. Assuming 0.") return 0 +def parseVersion(versionStr): + """ + Parses the given version string into its expected components, for instance... + '0.2.2.13-alpha (git-feb8c1b5f67f2c6f)' + + would provide: + (0, 2, 2, 13, 'alpha') + + If the input isn't recognized then this returns None. + + Arguments: + versionStr - version string to be parsed + """ + + # crops off extra arguments, for instance: + # '0.2.2.13-alpha (git-feb8c1b5f67f2c6f)' -> '0.2.2.13-alpha' + versionStr = versionStr.split()[0] + + result = None + if versionStr.count(".") in (2, 3): + # parses the optional suffix ('alpha', 'release', etc) + if versionStr.count("-") == 1: + versionStr, versionSuffix = versionStr.split("-") + else: versionSuffix = "" + + # Parses the numeric portion of the version. This can have three or four + # entries depending on if an optional patch level was provided. + try: + versionComp = [int(entry) for entry in versionStr.split(".")] + if len(versionComp) == 3: versionComp += [0] + result = tuple(versionComp + [versionSuffix]) + except ValueError: pass + + return result + def getConn(): """ Singleton constructor for a Controller. Be aware that this starts as being @@ -210,18 +283,29 @@ self.torctlListeners = [] # callback functions for TorCtl events self.statusListeners = [] # callback functions for tor's state changes self.controllerEvents = [] # list of successfully set controller events + self._fingerprintMappings = None # mappings of ip -> [(port, fingerprint), ...] + self._fingerprintLookupCache = {} # lookup cache with (ip, port) -> fingerprint mappings + self._fingerprintsAttachedCache = None # cache of relays we're connected to + self._nicknameLookupCache = {} # lookup cache with fingerprint -> nickname mappings + self._consensusLookupCache = {} # lookup cache with network status entries + self._descriptorLookupCache = {} # lookup cache with relay descriptors self._isReset = False # internal flag for tracking resets - self._status = TOR_CLOSED # current status of the attached control port + self._status = State.CLOSED # current status of the attached control port self._statusTime = 0 # unix time-stamp for the duration of the status self.lastHeartbeat = 0 # time of the last tor event + self._exitPolicyChecker = None + self._isExitingAllowed = False + self._exitPolicyLookupCache = {} # mappings of ip/port tuples to if they were accepted by the policy or not + # Logs issues and notices when fetching the path prefix if true. This is # only done once for the duration of the application to avoid pointless # messages. self._pathPrefixLogging = True - # cached GETINFO parameters (None if unset or possibly changed) - self._cachedParam = dict([(arg, "") for arg in CACHE_ARGS]) + # cached parameters for GETINFO and custom getters (None if unset or + # possibly changed) + self._cachedParam = {} # cached GETCONF parameters, entries consisting of: # (option, fetch_type) => value @@ -230,6 +314,9 @@ # directs TorCtl to notify us of events TorUtil.logger = self TorUtil.loglevel = "DEBUG" + + # tracks the number of sequential geoip lookup failures + self.geoipFailureCount = 0 def init(self, conn=None): """ @@ -254,17 +341,30 @@ self.conn.add_event_listener(self) for listener in self.eventListeners: self.conn.add_event_listener(listener) + # reset caches for ip -> fingerprint lookups + self._fingerprintMappings = None + self._fingerprintLookupCache = {} + self._fingerprintsAttachedCache = None + self._nicknameLookupCache = {} + self._consensusLookupCache = {} + self._descriptorLookupCache = {} + + self._exitPolicyChecker = self.getExitPolicy() + self._isExitingAllowed = self._exitPolicyChecker.isExitingAllowed() + self._exitPolicyLookupCache = {} + # sets the events listened for by the new controller (incompatible events # are dropped with a logged warning) self.setControllerEvents(self.controllerEvents) self.connLock.release() - self._status = TOR_INIT + self._status = State.INIT self._statusTime = time.time() # notifies listeners that a new controller is available - thread.start_new_thread(self._notifyStatusListeners, (TOR_INIT,)) + if not NO_SPAWN: + thread.start_new_thread(self._notifyStatusListeners, (State.INIT,)) def close(self): """ @@ -274,14 +374,29 @@ self.connLock.acquire() if self.conn: self.conn.close() + + # If we're closing due to an event from TorCtl (for instance, tor was + # stopped) then TorCtl is shutting itself down and there's no need to + # join on its thread (actually, this *is* the TorCtl thread in that + # case so joining on it causes deadlock). + # + # This poses a slight possability of shutting down with a live orphaned + # thread if Tor is shut down, then arm shuts down before TorCtl has a + # chance to terminate. However, I've never seen that occure so leaving + # that alone for now. + + if not threading.currentThread() == self.conn._thread: + self.conn._thread.join() + self.conn = None self.connLock.release() - self._status = TOR_CLOSED + self._status = State.CLOSED self._statusTime = time.time() # notifies listeners that the controller's been shut down - thread.start_new_thread(self._notifyStatusListeners, (TOR_CLOSED,)) + if not NO_SPAWN: + thread.start_new_thread(self._notifyStatusListeners, (State.CLOSED,)) else: self.connLock.release() def isAlive(self): @@ -337,21 +452,44 @@ self.connLock.acquire() + isGeoipRequest = param.startswith("ip-to-country/") + + # checks if this is an arg caching covers + isCacheArg = param in CACHE_ARGS + + if not isCacheArg: + for prefix in CACHE_GETINFO_PREFIX_ARGS: + if param.startswith(prefix): + isCacheArg = True + break + startTime = time.time() result, raisedExc, isFromCache = default, None, False if self.isAlive(): - if param in CACHE_ARGS and self._cachedParam[param]: - result = self._cachedParam[param] + cachedValue = self._cachedParam.get(param) + + if isCacheArg and cachedValue: + result = cachedValue isFromCache = True + elif isGeoipRequest and self.geoipFailureCount == GEOIP_FAILURE_THRESHOLD: + # the geoip database aleady looks to be unavailable - abort the request + raisedExc = TorCtl.ErrorReply("Tor geoip database is unavailable.") else: try: getInfoVal = self.conn.get_info(param)[param] if getInfoVal != None: result = getInfoVal + if isGeoipRequest: self.geoipFailureCount = 0 except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed), exc: if type(exc) == TorCtl.TorCtlClosed: self.close() raisedExc = exc + + if isGeoipRequest: + self.geoipFailureCount += 1 + + if self.geoipFailureCount == GEOIP_FAILURE_THRESHOLD: + log.log(CONFIG["log.geoipUnavailable"], "Tor geoip database is unavailable.") - if not isFromCache and result and param in CACHE_ARGS: + if isCacheArg and result and not isFromCache: self._cachedParam[param] = result if isFromCache: @@ -439,9 +577,9 @@ result = {} if fetchType == "map" else [] if self.isAlive(): - if (param, fetchType) in self._cachedConf: + if (param.lower(), fetchType) in self._cachedConf: isFromCache = True - result = self._cachedConf[(param, fetchType)] + result = self._cachedConf[(param.lower(), fetchType)] else: try: if fetchType == "str": @@ -458,15 +596,18 @@ if type(exc) == TorCtl.TorCtlClosed: self.close() result, raisedExc = default, exc - if not isFromCache and result: + if not isFromCache: cacheValue = result if fetchType == "list": cacheValue = list(result) elif fetchType == "map": cacheValue = dict(result) - self._cachedConf[(param, fetchType)] = cacheValue + self._cachedConf[(param.lower(), fetchType)] = cacheValue - runtimeLabel = "cache fetch" if isFromCache else "runtime: %0.4f" % (time.time() - startTime) - msg = "GETCONF %s (%s)" % (param, runtimeLabel) - log.log(CONFIG["log.torGetConf"], msg) + if isFromCache: + msg = "GETCONF %s (cache fetch)" % param + log.log(CONFIG["log.torGetConfCache"], msg) + else: + msg = "GETCONF %s (runtime: %0.4f)" % (param, time.time() - startTime) + log.log(CONFIG["log.torGetConf"], msg) self.connLock.release() @@ -496,10 +637,16 @@ # flushing cached values (needed until we can detect SETCONF calls) for fetchType in ("str", "list", "map"): - entry = (param, fetchType) + entry = (param.lower(), fetchType) if entry in self._cachedConf: del self._cachedConf[entry] + + # special caches for the exit policy + if param.lower() == "exitpolicy": + self._exitPolicyChecker = self.getExitPolicy() + self._isExitingAllowed = self._exitPolicyChecker.isExitingAllowed() + self._exitPolicyLookupCache = {} except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed), exc: if type(exc) == TorCtl.TorCtlClosed: self.close() elif type(exc) == TorCtl.ErrorReply: @@ -527,6 +674,27 @@ if raisedExc: raise raisedExc + def getCircuits(self, default = []): + """ + This provides a list with tuples of the form: + (circuitID, status, purpose, (fingerprint1, fingerprint2...)) + + Arguments: + default - value provided back if unable to query the circuit-status + """ + + return self._getRelayAttr("circuits", default) + + def getHiddenServicePorts(self, default = []): + """ + Provides the target ports hidden services are configured to use. + + Arguments: + default - value provided back if unable to query the hidden service ports + """ + + return self._getRelayAttr("hsPorts", default) + def getMyNetworkStatus(self, default = None): """ Provides the network status entry for this relay if available. This is @@ -615,6 +783,53 @@ return self._getRelayAttr("flags", default) + def isVersion(self, minVersionStr): + """ + Checks if we meet the given version. Recognized versions are of the form: + <major>.<minor>.<micro>[.<patch>][-<status_tag>] + + for instance, "0.2.2.13-alpha" or "0.2.1.5". This raises a ValueError if + the input isn't recognized, and returns False if unable to fetch our + instance's version. + + According to the spec the status_tag is purely informal, so it's ignored + in comparisons. + + Arguments: + minVersionStr - version to be compared against + """ + + minVersion = parseVersion(minVersionStr) + + if minVersion == None: + raise ValueError("unrecognized version: %s" % minVersionStr) + + self.connLock.acquire() + + result = False + if self.isAlive(): + myVersion = self._getRelayAttr("parsedVersion", None) + + if not myVersion: + result = False + elif myVersion[:4] == minVersion[:4]: + result = True # versions match + else: + # compares each of the numeric portions of the version + for i in range(4): + myVal, minVal = myVersion[i], minVersion[i] + + if myVal > minVal: + result = True + break + elif myVal < minVal: + result = False + break + + self.connLock.release() + + return result + def getMyPid(self): """ Provides the pid of the attached tor process (None if no controller exists @@ -623,6 +838,16 @@ return self._getRelayAttr("pid", None) + def getMyDirAuthorities(self): + """ + Provides a listing of IP/port tuples for the directory authorities we've + been configured to use. If set in the configuration then these are custom + authorities, otherwise its an estimate of what Tor has been hardcoded to + use (unfortunately, this might be out of date). + """ + + return self._getRelayAttr("authorities", []) + def getPathPrefix(self): """ Provides the path prefix that should be used for fetching tor resources. @@ -630,10 +855,7 @@ jail's path. """ - result = self._getRelayAttr("pathPrefix", "") - - if result == UNKNOWN: return "" - else: return result + return self._getRelayAttr("pathPrefix", "") def getStartTime(self): """ @@ -641,10 +863,7 @@ can't be determined then this provides None. """ - result = self._getRelayAttr("startTime", None) - - if result == UNKNOWN: return None - else: return result + return self._getRelayAttr("startTime", None) def getStatus(self): """ @@ -655,6 +874,199 @@ return (self._status, self._statusTime) + def isExitingAllowed(self, ipAddress, port): + """ + Checks if the given destination can be exited to by this relay, returning + True if so and False otherwise. + """ + + self.connLock.acquire() + + result = False + if self.isAlive(): + # query the policy if it isn't yet cached + if not (ipAddress, port) in self._exitPolicyLookupCache: + # If we allow any exiting then this could be relayed DNS queries, + # otherwise the policy is checked. Tor still makes DNS connections to + # test when exiting isn't allowed, but nothing is relayed over them. + # I'm registering these as non-exiting to avoid likely user confusion: + # https://trac.torproject.org/projects/tor/ticket/965 + + if self._isExitingAllowed and port == "53": isAccepted = True + else: isAccepted = self._exitPolicyChecker.check(ipAddress, port) + self._exitPolicyLookupCache[(ipAddress, port)] = isAccepted + + result = self._exitPolicyLookupCache[(ipAddress, port)] + + self.connLock.release() + + return result + + def getExitPolicy(self): + """ + Provides an ExitPolicy instance for the head of this relay's exit policy + chain. If there's no active connection then this provides None. + """ + + self.connLock.acquire() + + result = None + if self.isAlive(): + if self.getOption("ORPort"): + policyEntries = [] + for exitPolicy in self.getOption("ExitPolicy", [], True): + policyEntries += [policy.strip() for policy in exitPolicy.split(",")] + + # appends the default exit policy + defaultExitPolicy = self.getInfo("exit-policy/default") + + if defaultExitPolicy: + policyEntries += defaultExitPolicy.split(",") + + # construct the policy chain backwards + policyEntries.reverse() + + for entry in policyEntries: + result = ExitPolicy(entry, result) + + # Checks if we are rejecting private connections. If set, this appends + # 'reject private' and 'reject <my ip>' to the start of our policy chain. + isPrivateRejected = self.getOption("ExitPolicyRejectPrivate", True) + + if isPrivateRejected: + result = ExitPolicy("reject private", result) + + myAddress = self.getInfo("address") + if myAddress: result = ExitPolicy("reject %s" % myAddress, result) + else: + # no ORPort is set so all relaying is disabled + result = ExitPolicy("reject *:*") + + self.connLock.release() + + return result + + def getConsensusEntry(self, relayFingerprint): + """ + Provides the most recently available consensus information for the given + relay. This is none if no such information exists. + + Arguments: + relayFingerprint - fingerprint of the relay + """ + + self.connLock.acquire() + + result = None + if self.isAlive(): + if not relayFingerprint in self._consensusLookupCache: + nsEntry = self.getInfo("ns/id/%s" % relayFingerprint) + self._consensusLookupCache[relayFingerprint] = nsEntry + + result = self._consensusLookupCache[relayFingerprint] + + self.connLock.release() + + return result + + def getDescriptorEntry(self, relayFingerprint): + """ + Provides the most recently available descriptor information for the given + relay. Unless FetchUselessDescriptors is set this may frequently be + unavailable. If no such descriptor is available then this returns None. + + Arguments: + relayFingerprint - fingerprint of the relay + """ + + self.connLock.acquire() + + result = None + if self.isAlive(): + if not relayFingerprint in self._descriptorLookupCache: + descEntry = self.getInfo("desc/id/%s" % relayFingerprint) + self._descriptorLookupCache[relayFingerprint] = descEntry + + result = self._descriptorLookupCache[relayFingerprint] + + self.connLock.release() + + return result + + def getRelayFingerprint(self, relayAddress, relayPort = None, getAllMatches = False): + """ + Provides the fingerprint associated with the given address. If there's + multiple potential matches or the mapping is unknown then this returns + None. This disambiguates the fingerprint if there's multiple relays on + the same ip address by several methods, one of them being to pick relays + we have a connection with. + + Arguments: + relayAddress - address of relay to be returned + relayPort - orport of relay (to further narrow the results) + getAllMatches - ignores the relayPort and provides all of the + (port, fingerprint) tuples matching the given + address + """ + + self.connLock.acquire() + + result = None + if self.isAlive(): + if getAllMatches: + # populates the ip -> fingerprint mappings if not yet available + if self._fingerprintMappings == None: + self._fingerprintMappings = self._getFingerprintMappings() + + if relayAddress in self._fingerprintMappings: + result = self._fingerprintMappings[relayAddress] + else: result = [] + else: + # query the fingerprint if it isn't yet cached + if not (relayAddress, relayPort) in self._fingerprintLookupCache: + relayFingerprint = self._getRelayFingerprint(relayAddress, relayPort) + self._fingerprintLookupCache[(relayAddress, relayPort)] = relayFingerprint + + result = self._fingerprintLookupCache[(relayAddress, relayPort)] + + self.connLock.release() + + return result + + def getRelayNickname(self, relayFingerprint): + """ + Provides the nickname associated with the given relay. This provides None + if no such relay exists, and "Unnamed" if the name hasn't been set. + + Arguments: + relayFingerprint - fingerprint of the relay + """ + + self.connLock.acquire() + + result = None + if self.isAlive(): + # query the nickname if it isn't yet cached + if not relayFingerprint in self._nicknameLookupCache: + if relayFingerprint == self.getInfo("fingerprint"): + # this is us, simply check the config + myNickname = self.getOption("Nickname", "Unnamed") + self._nicknameLookupCache[relayFingerprint] = myNickname + else: + # check the consensus for the relay + nsEntry = self.getConsensusEntry(relayFingerprint) + + if nsEntry: relayNickname = nsEntry[2:nsEntry.find(" ", 2)] + else: relayNickname = None + + self._nicknameLookupCache[relayFingerprint] = relayNickname + + result = self._nicknameLookupCache[relayFingerprint] + + self.connLock.release() + + return result + def addEventListener(self, listener): """ Directs further tor controller events to callback functions of the @@ -825,7 +1237,7 @@ if not issueSighup: try: self.conn.send_signal("RELOAD") - self._cachedParam = dict([(arg, "") for arg in CACHE_ARGS]) + self._cachedParam = {} self._cachedConf = {} except Exception, exc: # new torrc parameters caused an error (tor's likely shut down) @@ -870,7 +1282,7 @@ if errorLine: raise IOError(" ".join(errorLine.split()[3:])) else: raise IOError("failed silently") - self._cachedParam = dict([(arg, "") for arg in CACHE_ARGS]) + self._cachedParam = {} self._cachedConf = {} except IOError, exc: raisedException = exc @@ -888,13 +1300,15 @@ if event.level == "NOTICE" and event.msg.startswith("Received reload signal (hup)"): self._isReset = True - self._status = TOR_INIT + self._status = State.INIT self._statusTime = time.time() - thread.start_new_thread(self._notifyStatusListeners, (TOR_INIT,)) + if not NO_SPAWN: + thread.start_new_thread(self._notifyStatusListeners, (State.INIT,)) def ns_event(self, event): self._updateHeartbeat() + self._consensusLookupCache = {} myFingerprint = self.getInfo("fingerprint") if myFingerprint: @@ -912,20 +1326,77 @@ def new_consensus_event(self, event): self._updateHeartbeat() + self.connLock.acquire() + self._cachedParam["nsEntry"] = None self._cachedParam["flags"] = None self._cachedParam["bwMeasured"] = None + + # reconstructs consensus based mappings + self._fingerprintLookupCache = {} + self._fingerprintsAttachedCache = None + self._nicknameLookupCache = {} + self._consensusLookupCache = {} + + if self._fingerprintMappings != None: + self._fingerprintMappings = self._getFingerprintMappings(event.nslist) + + self.connLock.release() def new_desc_event(self, event): self._updateHeartbeat() + self.connLock.acquire() + myFingerprint = self.getInfo("fingerprint") if not myFingerprint or myFingerprint in event.idlist: self._cachedParam["descEntry"] = None self._cachedParam["bwObserved"] = None + + # If we're tracking ip address -> fingerprint mappings then update with + # the new relays. + self._fingerprintLookupCache = {} + self._fingerprintsAttachedCache = None + self._descriptorLookupCache = {} + + if self._fingerprintMappings != None: + for fingerprint in event.idlist: + # gets consensus data for the new descriptor + try: nsLookup = self.conn.get_network_status("id/%s" % fingerprint) + except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed): continue + + if len(nsLookup) > 1: + # multiple records for fingerprint (shouldn't happen) + log.log(log.WARN, "Multiple consensus entries for fingerprint: %s" % fingerprint) + continue + + # updates fingerprintMappings with new data + newRelay = nsLookup[0] + if newRelay.ip in self._fingerprintMappings: + # if entry already exists with the same orport, remove it + orportMatch = None + for entryPort, entryFingerprint in self._fingerprintMappings[newRelay.ip]: + if entryPort == newRelay.orport: + orportMatch = (entryPort, entryFingerprint) + break + + if orportMatch: self._fingerprintMappings[newRelay.ip].remove(orportMatch) + + # add the new entry + self._fingerprintMappings[newRelay.ip].append((newRelay.orport, newRelay.idhex)) + else: + self._fingerprintMappings[newRelay.ip] = [(newRelay.orport, newRelay.idhex)] + + self.connLock.release() def circ_status_event(self, event): self._updateHeartbeat() + + # CIRC events aren't required, but if one's received then flush this cache + # since it uses circuit-status results. + self._fingerprintsAttachedCache = None + + self._cachedParam["circuits"] = None def buildtimeout_set_event(self, event): self._updateHeartbeat() @@ -959,6 +1430,13 @@ # checks if TorCtl is providing a notice that control port is closed if TOR_CTL_CLOSE_MSG in msg: self.close() + + # if the message is informing us of our ip address changing then clear + # its cached value + for prefix in ADDR_CHANGED_MSG_PREFIX: + if msg.startswith(prefix): + self._cachedParam["address"] = None + break def _updateHeartbeat(self): """ @@ -968,6 +1446,126 @@ # alternative is to use the event's timestamp (via event.arrived_at) self.lastHeartbeat = time.time() + def _getFingerprintMappings(self, nsList = None): + """ + Provides IP address to (port, fingerprint) tuple mappings for all of the + currently cached relays. + + Arguments: + nsList - network status listing (fetched if not provided) + """ + + results = {} + if self.isAlive(): + # fetch the current network status if not provided + if not nsList: + try: nsList = self.conn.get_network_status() + except (socket.error, TorCtl.TorCtlClosed, TorCtl.ErrorReply): nsList = [] + + # construct mappings of ips to relay data + for relay in nsList: + if relay.ip in results: results[relay.ip].append((relay.orport, relay.idhex)) + else: results[relay.ip] = [(relay.orport, relay.idhex)] + + return results + + def _getRelayFingerprint(self, relayAddress, relayPort): + """ + Provides the fingerprint associated with the address/port combination. + + Arguments: + relayAddress - address of relay to be returned + relayPort - orport of relay (to further narrow the results) + """ + + # If we were provided with a string port then convert to an int (so + # lookups won't mismatch based on type). + if isinstance(relayPort, str): relayPort = int(relayPort) + + # checks if this matches us + if relayAddress == self.getInfo("address"): + if not relayPort or relayPort == self.getOption("ORPort"): + return self.getInfo("fingerprint") + + # if we haven't yet populated the ip -> fingerprint mappings then do so + if self._fingerprintMappings == None: + self._fingerprintMappings = self._getFingerprintMappings() + + potentialMatches = self._fingerprintMappings.get(relayAddress) + if not potentialMatches: return None # no relay matches this ip address + + if len(potentialMatches) == 1: + # There's only one relay belonging to this ip address. If the port + # matches then we're done. + match = potentialMatches[0] + + if relayPort and match[0] != relayPort: return None + else: return match[1] + elif relayPort: + # Multiple potential matches, so trying to match based on the port. + for entryPort, entryFingerprint in potentialMatches: + if entryPort == relayPort: + return entryFingerprint + + # Disambiguates based on our orconn-status and circuit-status results. + # This only includes relays we're connected to, so chances are pretty + # slim that we'll still have a problem narrowing this down. Note that we + # aren't necessarily checking for events that can create new client + # circuits (so this cache might be a little dirty). + + # populates the cache + if self._fingerprintsAttachedCache == None: + self._fingerprintsAttachedCache = [] + + # orconn-status has entries of the form: + # $33173252B70A50FE3928C7453077936D71E45C52=shiven CONNECTED + orconnResults = self.getInfo("orconn-status") + if orconnResults: + for line in orconnResults.split("\n"): + self._fingerprintsAttachedCache.append(line[1:line.find("=")]) + + # circuit-status results (we only make connections to the first hop) + for _, _, _, path in self.getCircuits(): + self._fingerprintsAttachedCache.append(path[0]) + + # narrow to only relays we have a connection to + attachedMatches = [] + for _, entryFingerprint in potentialMatches: + if entryFingerprint in self._fingerprintsAttachedCache: + attachedMatches.append(entryFingerprint) + + if len(attachedMatches) == 1: + return attachedMatches[0] + + # Highly unlikely, but still haven't found it. Last we'll use some + # tricks from Mike's ConsensusTracker, excluding possiblities that + # have... + # - lost their Running flag + # - list a bandwidth of 0 + # - have 'opt hibernating' set + # + # This involves constructing a TorCtl Router and checking its 'down' + # flag (which is set by the three conditions above). This is the last + # resort since it involves a couple GETINFO queries. + + for entryPort, entryFingerprint in list(potentialMatches): + try: + nsCall = self.conn.get_network_status("id/%s" % entryFingerprint) + if not nsCall: raise TorCtl.ErrorReply() # network consensus couldn't be fetched + nsEntry = nsCall[0] + + descEntry = self.getInfo("desc/id/%s" % entryFingerprint) + if not descEntry: raise TorCtl.ErrorReply() # relay descriptor couldn't be fetched + descLines = descEntry.split("\n") + + isDown = TorCtl.Router.build_from_desc(descLines, nsEntry).down + if isDown: potentialMatches.remove((entryPort, entryFingerprint)) + except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed): pass + + if len(potentialMatches) == 1: + return potentialMatches[0][1] + else: return None + def _getRelayAttr(self, key, default, cacheUndefined = True): """ Provides information associated with this relay, using the cached value if @@ -980,15 +1578,15 @@ lookups if true """ - currentVal = self._cachedParam[key] - if currentVal: + currentVal = self._cachedParam.get(key) + if currentVal != None: if currentVal == UNKNOWN: return default else: return currentVal self.connLock.acquire() - currentVal, result = self._cachedParam[key], None - if not currentVal and self.isAlive(): + currentVal, result = self._cachedParam.get(key), None + if currentVal == None and self.isAlive(): # still unset - fetch value if key in ("nsEntry", "descEntry"): myFingerprint = self.getInfo("fingerprint") @@ -1043,6 +1641,8 @@ if line.startswith("s "): result = line[2:].split() break + elif key == "parsedVersion": + result = parseVersion(self.getInfo("version", "")) elif key == "pid": result = getPid(int(self.getOption("ControlPort", 9051)), self.getOption("PidFile")) elif key == "pathPrefix": @@ -1084,7 +1684,7 @@ if myPid: try: if procTools.isProcAvailable(): - result = float(procTools.getStats(myPid, procTools.STAT_START_TIME)[0]) + result = float(procTools.getStats(myPid, procTools.Stat.START_TIME)[0]) else: psCall = sysTools.call("ps -p %s -o etime" % myPid) @@ -1092,16 +1692,83 @@ etimeEntry = psCall[1].strip() result = time.time() - uiTools.parseShortTimeLabel(etimeEntry) except: pass + elif key == "authorities": + # There's two configuration options that can overwrite the default + # authorities: DirServer and AlternateDirAuthority. + + # TODO: Both options accept a set of flags to more precisely set what they + # overwrite. Ideally this would account for these flags to more accurately + # identify authority connections from relays. + + dirServerCfg = self.getOption("DirServer", [], True) + altDirAuthCfg = self.getOption("AlternateDirAuthority", [], True) + altAuthoritiesCfg = dirServerCfg + altDirAuthCfg + + if altAuthoritiesCfg: + result = [] + + # entries are of the form: + # [nickname] [flags] address:port fingerprint + for entry in altAuthoritiesCfg: + locationComp = entry.split()[-2] # address:port component + result.append(tuple(locationComp.split(":", 1))) + else: result = list(DIR_SERVERS) + elif key == "circuits": + # Parses our circuit-status results, for instance + # 91 BUILT $E4AE6E2FE320FBBD31924E8577F3289D4BE0B4AD=Qwerty PURPOSE=GENERAL + # would belong to a single hop circuit, most likely fetching the + # consensus via a directory mirror. + circStatusResults = self.getInfo("circuit-status") + + if circStatusResults == "": + result = [] # we don't have any circuits + elif circStatusResults != None: + result = [] + + for line in circStatusResults.split("\n"): + # appends a tuple with the (status, purpose, path) + lineComp = line.split(" ") + + # skips blank lines and circuits without a path, for instance: + # 5 LAUNCHED PURPOSE=TESTING + if len(lineComp) < 4: continue + + path = tuple([hopEntry[1:41] for hopEntry in lineComp[2].split(",")]) + result.append((int(lineComp[0]), lineComp[1], lineComp[3][8:], path)) + elif key == "hsPorts": + result = [] + hsOptions = self.getOptionMap("HiddenServiceOptions") + + if hsOptions and "HiddenServicePort" in hsOptions: + for hsEntry in hsOptions["HiddenServicePort"]: + # hidden service port entries are of the form: + # VIRTPORT [TARGET] + # with the TARGET being an IP, port, or IP:Port. If the target port + # isn't defined then uses the VIRTPORT. + + hsPort = None + + if " " in hsEntry: + # parses the target, checking if it's a port or IP:Port combination + hsTarget = hsEntry.split(" ")[1] + + if ":" in hsTarget: + hsPort = hsTarget.split(":")[1] # target is the IP:Port + elif hsTarget.isdigit(): + hsPort = hsTarget # target is just the port + else: hsPort = hsEntry # just has the virtual port + + if hsPort.isdigit(): + result.append(hsPort) # cache value - if result: self._cachedParam[key] = result + if result != None: self._cachedParam[key] = result elif cacheUndefined: self._cachedParam[key] = UNKNOWN - elif currentVal == UNKNOWN: result = currentVal self.connLock.release() - if result: return result - else: return default + if result == None or result == UNKNOWN: return default + else: return result def _notifyStatusListeners(self, eventType): """ @@ -1113,13 +1780,156 @@ """ # resets cached GETINFO and GETCONF parameters - self._cachedParam = dict([(arg, "") for arg in CACHE_ARGS]) + self._cachedParam = {} self._cachedConf = {} # gives a notice that the control port has closed - if eventType == TOR_CLOSED: + if eventType == State.CLOSED: log.log(CONFIG["log.torCtlPortClosed"], "Tor control port closed") for callback in self.statusListeners: callback(self, eventType) +class ExitPolicy: + """ + Single rule from the user's exit policy. These are chained together to form + complete policies. + """ + + def __init__(self, ruleEntry, nextRule): + """ + Exit policy rule constructor. + + Arguments: + ruleEntry - tor exit policy rule (for instance, "reject *:135-139") + nextRule - next rule to be checked when queries don't match this policy + """ + + # sanitize the input a bit, cleaning up tabs and stripping quotes + ruleEntry = ruleEntry.replace("\\t", " ").replace("\"", "") + + self.ruleEntry = ruleEntry + self.nextRule = nextRule + self.isAccept = ruleEntry.startswith("accept") + + # strips off "accept " or "reject " and extra spaces + ruleEntry = ruleEntry[7:].replace(" ", "") + + # split ip address (with mask if provided) and port + if ":" in ruleEntry: entryIp, entryPort = ruleEntry.split(":", 1) + else: entryIp, entryPort = ruleEntry, "*" + + # sets the ip address component + self.isIpWildcard = entryIp == "*" or entryIp.endswith("/0") + + # checks for the private alias (which expands this to a chain of entries) + if entryIp.lower() == "private": + entryIp = PRIVATE_IP_RANGES[0] + + # constructs the chain backwards (last first) + lastHop = self.nextRule + prefix = "accept " if self.isAccept else "reject " + suffix = ":" + entryPort + for addr in PRIVATE_IP_RANGES[-1:0:-1]: + lastHop = ExitPolicy(prefix + addr + suffix, lastHop) + + self.nextRule = lastHop # our next hop is the start of the chain + + if "/" in entryIp: + ipComp = entryIp.split("/", 1) + self.ipAddress = ipComp[0] + self.ipMask = int(ipComp[1]) + else: + self.ipAddress = entryIp + self.ipMask = 32 + + # constructs the binary address just in case of comparison with a mask + if self.ipAddress != "*": + self.ipAddressBin = "" + for octet in self.ipAddress.split("."): + # Converts the int to a binary string, padded with zeros. Source: + # http://www.daniweb.com/code/snippet216539.html + self.ipAddressBin += "".join([str((int(octet) >> y) & 1) for y in range(7, -1, -1)]) + else: + self.ipAddressBin = "0" * 32 + + # sets the port component + self.minPort, self.maxPort = 0, 0 + self.isPortWildcard = entryPort == "*" + + if entryPort != "*": + if "-" in entryPort: + portComp = entryPort.split("-", 1) + self.minPort = int(portComp[0]) + self.maxPort = int(portComp[1]) + else: + self.minPort = int(entryPort) + self.maxPort = int(entryPort) + + # if both the address and port are wildcards then we're effectively the + # last entry so cut off the remaining chain + if self.isIpWildcard and self.isPortWildcard: + self.nextRule = None + + def isExitingAllowed(self): + """ + Provides true if the policy allows exiting whatsoever, false otherwise. + """ + + if self.isAccept: return True + elif self.isIpWildcard and self.isPortWildcard: return False + elif not self.nextRule: return False # fell off policy (shouldn't happen) + else: return self.nextRule.isExitingAllowed() + + def check(self, ipAddress, port): + """ + Checks if the rule chain allows exiting to this address, returning true if + so and false otherwise. + """ + + port = int(port) + + # does the port check first since comparing ip masks is more work + isPortMatch = self.isPortWildcard or (port >= self.minPort and port <= self.maxPort) + + if isPortMatch: + isIpMatch = self.isIpWildcard or self.ipAddress == ipAddress + + # expands the check to include the mask if it has one + if not isIpMatch and self.ipMask != 32: + inputAddressBin = "" + for octet in ipAddress.split("."): + inputAddressBin += "".join([str((int(octet) >> y) & 1) for y in range(7, -1, -1)]) + + isIpMatch = self.ipAddressBin[:self.ipMask] == inputAddressBin[:self.ipMask] + + if isIpMatch: return self.isAccept + + # our policy doesn't concern this address, move on to the next one + if self.nextRule: return self.nextRule.check(ipAddress, port) + else: return True # fell off the chain without a conclusion (shouldn't happen...) + + def __str__(self): + # This provides the actual policy rather than the entry used to construct + # it so the 'private' keyword is expanded. + + acceptanceLabel = "accept" if self.isAccept else "reject" + + if self.isIpWildcard: + ipLabel = "*" + elif self.ipMask != 32: + ipLabel = "%s/%i" % (self.ipAddress, self.ipMask) + else: ipLabel = self.ipAddress + + if self.isPortWildcard: + portLabel = "*" + elif self.minPort != self.maxPort: + portLabel = "%i-%i" % (self.minPort, self.maxPort) + else: portLabel = str(self.minPort) + + myPolicy = "%s %s:%s" % (acceptanceLabel, ipLabel, portLabel) + + if self.nextRule: + return myPolicy + ", " + str(self.nextRule) + else: return myPolicy + Modified: arm/release/src/util/uiTools.py =================================================================== --- arm/release/src/util/uiTools.py 2011-04-04 15:15:20 UTC (rev 24554) +++ arm/release/src/util/uiTools.py 2011-04-04 15:22:31 UTC (rev 24555) @@ -9,7 +9,7 @@ import curses from curses.ascii import isprint -from util import log +from util import enum, log # colors curses can handle COLOR_LIST = {"red": curses.COLOR_RED, "green": curses.COLOR_GREEN, @@ -32,7 +32,7 @@ TIME_UNITS = [(86400.0, "d", " day"), (3600.0, "h", " hour"), (60.0, "m", " minute"), (1.0, "s", " second")] -END_WITH_ELLIPSE, END_WITH_HYPHEN = range(1, 3) +Ending = enum.Enum("ELLIPSE", "HYPHEN") SCROLL_KEYS = (curses.KEY_UP, curses.KEY_DOWN, curses.KEY_PPAGE, curses.KEY_NPAGE, curses.KEY_HOME, curses.KEY_END) CONFIG = {"features.colorInterface": True, "log.cursesColorSupport": log.INFO} @@ -117,7 +117,7 @@ if not COLOR_ATTR_INITIALIZED: _initColors() return COLOR_ATTR[color] -def cropStr(msg, size, minWordLen = 4, minCrop = 0, endType = END_WITH_ELLIPSE, getRemainder = False): +def cropStr(msg, size, minWordLen = 4, minCrop = 0, endType = Ending.ELLIPSE, getRemainder = False): """ Provides the msg constrained to the given length, truncating on word breaks. If the last words is long this truncates mid-word with an ellipse. If there @@ -143,8 +143,8 @@ minCrop - minimum characters that must be dropped if a word's cropped endType - type of ending used when truncating: None - blank ending - END_WITH_ELLIPSE - includes an ellipse - END_WITH_HYPHEN - adds hyphen when breaking words + Ending.ELLIPSE - includes an ellipse + Ending.HYPHEN - adds hyphen when breaking words getRemainder - returns a tuple instead, with the second part being the cropped portion of the message """ @@ -161,11 +161,12 @@ # since we're cropping, the effective space available is less with an # ellipse, and cropping words requires an extra space for hyphens - if endType == END_WITH_ELLIPSE: size -= 3 - elif endType == END_WITH_HYPHEN and minWordLen != None: minWordLen += 1 + if endType == Ending.ELLIPSE: size -= 3 + elif endType == Ending.HYPHEN and minWordLen != None: minWordLen += 1 # checks if there isn't the minimum space needed to include anything lastWordbreak = msg.rfind(" ", 0, size + 1) + lastWordbreak = len(msg[:lastWordbreak].rstrip()) # drops extra ending whitespaces if (minWordLen != None and size < minWordLen) or (minWordLen == None and lastWordbreak < 1): if getRemainder: return ("", msg) else: return "" @@ -181,23 +182,64 @@ if includeCrop: returnMsg, remainder = msg[:size], msg[size:] - if endType == END_WITH_HYPHEN: + if endType == Ending.HYPHEN: remainder = returnMsg[-1] + remainder - returnMsg = returnMsg[:-1] + "-" + returnMsg = returnMsg[:-1].rstrip() + "-" else: returnMsg, remainder = msg[:lastWordbreak], msg[lastWordbreak:] # if this is ending with a comma or period then strip it off if not getRemainder and returnMsg[-1] in (",", "."): returnMsg = returnMsg[:-1] - if endType == END_WITH_ELLIPSE: returnMsg += "..." + if endType == Ending.ELLIPSE: + returnMsg = returnMsg.rstrip() + "..." if getRemainder: return (returnMsg, remainder) else: return returnMsg +def drawBox(panel, top, left, width, height, attr=curses.A_NORMAL): + """ + Draws a box in the panel with the given bounds. + + Arguments: + panel - panel in which to draw + top - vertical position of the box's top + left - horizontal position of the box's left side + width - width of the drawn box + height - height of the drawn box + attr - text attributes + """ + + # draws the top and bottom + panel.hline(top, left + 1, width - 1, attr) + panel.hline(top + height - 1, left + 1, width - 1, attr) + + # draws the left and right sides + panel.vline(top + 1, left, height - 2, attr) + panel.vline(top + 1, left + width, height - 2, attr) + + # draws the corners + panel.addch(top, left, curses.ACS_ULCORNER, attr) + panel.addch(top, left + width, curses.ACS_URCORNER, attr) + panel.addch(top + height - 1, left, curses.ACS_LLCORNER, attr) + panel.addch(top + height - 1, left + width, curses.ACS_LRCORNER, attr) + +def isSelectionKey(key): + """ + Returns true if the keycode matches the enter or space keys. + + Argument: + key - keycode to be checked + """ + + return key in (curses.KEY_ENTER, 10, ord(' ')) + def isScrollKey(key): """ Returns true if the keycode is recognized by the getScrollPosition function for scrolling. + + Argument: + key - keycode to be checked """ return key in SCROLL_KEYS @@ -367,6 +409,73 @@ except ValueError: raise ValueError(errorMsg) +class DrawEntry: + """ + Renderable content, encapsulating the text and formatting. These can be + chained together to compose lines with multiple types of formatting. + """ + + def __init__(self, text, format=curses.A_NORMAL, nextEntry=None, lockFormat=False): + """ + Constructor for prepared draw entries. + + Arguments: + text - content to be drawn, this can either be a string or list of + integer character codes + format - properties to apply when drawing + nextEntry - entry to be drawn after this one + lockFormat - prevents extra formatting attributes from being applied + when rendered if true + """ + + self.text = text + self.format = format + self.nextEntry = nextEntry + self.lockFormat = lockFormat + + def getNext(self): + """ + Provides the next DrawEntry in the chain. + """ + + return self.nextEntry + + def setNext(self, nextEntry): + """ + Sets additional content to be drawn after this entry. If None then + rendering is terminated after this entry. + + Arguments: + nextEntry - DrawEntry instance to be rendered after this one + """ + + self.nextEntry = nextEntry + + def render(self, drawPanel, y, x, extraFormat=curses.A_NORMAL): + """ + Draws this content at the given position. + + Arguments: + drawPanel - context in which to be drawn + y - vertical location + x - horizontal location + extraFormat - additional formatting + """ + + if self.lockFormat: drawFormat = self.format + else: drawFormat = self.format | extraFormat + + if isinstance(self.text, str): + drawPanel.addstr(y, x, self.text, drawFormat) + else: + for i in range(len(self.text)): + drawChar = self.text[i] + drawPanel.addch(y, x + i, drawChar, drawFormat) + + # if there's additional content to show then render it too + if self.nextEntry: + self.nextEntry.render(drawPanel, y, x + len(self.text), extraFormat) + class Scroller: """ Tracks the scrolling position when there might be a visible cursor. This @@ -394,10 +503,16 @@ if self.isCursorEnabled: self.getCursorSelection(content) # resets the cursor location + # makes sure the cursor is visible if self.cursorLoc < self.scrollLoc: self.scrollLoc = self.cursorLoc elif self.cursorLoc > self.scrollLoc + pageHeight - 1: self.scrollLoc = self.cursorLoc - pageHeight + 1 + + # checks if the bottom would run off the content (this could be the + # case when the content's size is dynamic and entries are removed) + if len(content) > pageHeight: + self.scrollLoc = min(self.scrollLoc, len(content) - pageHeight) return self.scrollLoc Modified: arm/release/src/version.py =================================================================== --- arm/release/src/version.py 2011-04-04 15:15:20 UTC (rev 24554) +++ arm/release/src/version.py 2011-04-04 15:22:31 UTC (rev 24555) @@ -2,6 +2,6 @@ Provides arm's version and release date. """ -VERSION = '1.4.1.3' -LAST_MODIFIED = "January 15, 2011" +VERSION = '1.4.2' +LAST_MODIFIED = "April 4, 2011"

1 0

r24554: {arm} Release notes for version 1.4.2. (arm/trunk)
by Damian Johnson 04 Apr '11

04 Apr '11

Author: atagar Date: 2011-04-04 15:15:20 +0000 (Mon, 04 Apr 2011) New Revision: 24554 Modified: arm/trunk/ChangeLog Log: Release notes for version 1.4.2. Modified: arm/trunk/ChangeLog =================================================================== --- arm/trunk/ChangeLog 2011-04-03 19:44:18 UTC (rev 24553) +++ arm/trunk/ChangeLog 2011-04-04 15:15:20 UTC (rev 24554) @@ -1,5 +1,53 @@ CHANGE LOG +4/4/11 - version 1.4.2 +This release chiefly consists of a fully reimplemented connection panel. Besides being a sane, maintainable implementation this includes numerous new features and improvements like full circuit paths, applications involved for local connections, and better type identification. + + * added: full rewrite of the connection panel, providing: + o listing the full paths involved in active circuits + o identification of socks, hidden service, and controller applications (arm, vidalia, polipo, etc) + o identification of exit connections with the common usage for the port they're using + o display of the local -> internal -> external address when room is available (original patch by Fabian Keil) + o better accuracy and performance in identifying client and directory connections + o marking the uptimes for initial connections (arm only tracks connection uptimes since starting, so these entries are just minimum durations) + o lazily loading the initial IP -> fingerprint mappings to improve the startup time + o using the circuit-status to disambiguating multiple relays on the same IP address + o smarter space utilization, filling in smaller columns if there isn't room for higher priority but larger entries + o connection details popup changes: + + using the consensus exit policies rather than the longer descriptor versions when available + + displaying connection details no longer freezes the rest of the display + + detail panel uses the full screen width and is dynamically resizable + + more resilient to missing descriptors + * change: hiding most tor config values by default (idea by arma) + * change: dropping warning suggesting that users set the FetchUselessDescriptors option (suggestion by Sebastian and others) + * change: always starting the bandwidth field from zero rather than using the state file total, which only contains the last day's worth of data (thanks to guilhem) + * change: suggesting authentication and giving steps for it in the readme (suggestion by Sebastian) + * change: caching config display lines, which reduces the CPU usage when scrolling by around 40% + * change: added summaries for the remaining tor configuration options + * change: using a dedicated enum class rather than tuple sets + * fix: torrc validation requires 'GETINFO config-text' which was introduced in Tor verison 0.2.2.7 (caught by Sjon, talion, and torland, https://trac.torproject.org/projects/tor/ticket/2501) + * fix: off-by-one issue with the displayed line numbers for torrc errors (caught by Sjon) + * fix: bin function wasn't available before python 2.6 (caught by Paul Menzel) + * fix: mis-parsing family entries when there's no entry after the comma (caught by StrangeCharm, https://trac.torproject.org/projects/tor/ticket/2414) + * fix: preventing SOCKS and CONTROL connections from being expanded (patch by Fabian Keil) + * fix: disabling name resolution for application queries to avoid leaking to resolvers (patch by Fabian Keil) + * fix: reversing src and dst addresses of SOCKS and CONTROL connections (caught by Fabian Keil) + * fix: changing the 'APPLICATION' type to 'SOCKS' since the previous label was too long (caught by Fabian Keil) + * fix: crashing issue from unknown relay nicknames (caught by krkhan) + * fix: concurrency bug occasionally causing "syshook" stacktraces when shutting down + * fix: header panel displayed the wrong IP address if it changed since we first started (https://trac.torproject.org/projects/tor/ticket/2776) + * fix: unchecked OSError could cause us to crash when making directories (for instance if there was a permissions issue) + * fix: the availability check for bsd resolvers was broken, probably causing resolution to fail for a few seconds on that platform + * fix: dropping the pointless 'Log notice stdout' entry provided by config-text queries (https://trac.torproject.org/projects/tor/ticket/2362) + * fix: taking DirServer and AlternateDirAuthority into account when determining the directory authorities + * fix: consuming a little extra space in the connection panel when scrollbars aren't visible + * fix: dropping the deprecated 'features.config.descriptions.persistPath' config option + * fix: failed connection attempts to the control port were generating zombie connections (https://trac.torproject.org/projects/tor/ticket/2812) + * fix: concurrency bug in joining on the TorCtl thread when tor shut down + * fix: the 'startup.dataDirectory' config option was being ignored + * fix: recognizing the proper private ip ranges of the 172.* block + * fix: missing 'is default' option from config sort ordering + 1/7/11 - version 1.4.1 (r24054) Platform specific enhancements including BSD compatibility and vastly improved performance on Linux.

1 0