- tor-commits - lists.torproject.org

[arm/master] fix: Proc resolution without a pid gave stacktrace
by atagar＠torproject.org 27 Apr '11

27 Apr '11

commit f72aa4616ea605ac22fe012513d856aaeedb819e Author: Damian Johnson <atagar(a)torproject.org> Date: Wed Apr 27 08:52:54 2011 -0700 fix: Proc resolution without a pid gave stacktrace When the connection resolver hadn't been configured with a pid and we queried connections by proc resolution there was an uncaught exception. This didn't crash arm - it just resulted in a stacktrace for a split second at startup. This was most easily reproed by running arm on Linux in blind mode, with a sleep command between panel creation and redrawing the screen to keep the stack trace visible for a bit. --- src/util/connections.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/src/util/connections.py b/src/util/connections.py index f632306..c090893 100644 --- a/src/util/connections.py +++ b/src/util/connections.py @@ -486,7 +486,7 @@ class ConnectionResolver(threading.Thread): else: self._rateThresholdBroken = 0 if isDefault: self._subsiquentFailures = 0 - except IOError, exc: + except (ValueError, IOError), exc: # this logs in a couple of cases: # - special failures noted by getConnections (most cases are already # logged via sysTools)

1 0

[arm/packaging] fix: Dropping changelog entry for unreleased deb
by atagar＠torproject.org 27 Apr '11

27 Apr '11

commit cfdead81c5265e5025b55a82b83d9b8db025a0c6 Author: Damian Johnson <atagar(a)torproject.org> Date: Wed Apr 27 09:04:04 2011 -0700 fix: Dropping changelog entry for unreleased deb The 1.4.2.0 release had never been packaged as a deb, and having its changelog entry resulted in non-contiguous release versions. Caught by weasel --- build/debian/changelog | 6 ------ 1 files changed, 0 insertions(+), 6 deletions(-) diff --git a/build/debian/changelog b/build/debian/changelog index 1626da9..11e04bd 100644 --- a/build/debian/changelog +++ b/build/debian/changelog @@ -15,12 +15,6 @@ tor-arm (1.4.1.2-2) unstable; urgency=low -- Ulises Vitulli <dererk(a)debian.org> Wed, 20 Apr 2011 11:36:17 -0300 -tor-arm (1.4.2.0-1) unstable; urgency=low - - * Upstream release 1.4.2.0. - - -- Damian Johnson <atagar(a)torproject.org> Sat, 4 Apr 2011 1:30:43 -0700 - tor-arm (1.4.1.2-1) unstable; urgency=low * Upstream release 1.4.1.2.

1 0

r24685: {website} add bridge-by-default bundle link and some information about (in website/trunk: download/en include)
by Erinn Clark 27 Apr '11

27 Apr '11

Author: erinn Date: 2011-04-27 15:26:04 +0000 (Wed, 27 Apr 2011) New Revision: 24685 Modified: website/trunk/download/en/download.wml website/trunk/include/versions.wmi Log: add bridge-by-default bundle link and some information about it to download page Modified: website/trunk/download/en/download.wml =================================================================== --- website/trunk/download/en/download.wml 2011-04-27 06:57:24 UTC (rev 24684) +++ website/trunk/download/en/download.wml 2011-04-27 15:26:04 UTC (rev 24685) @@ -20,10 +20,11 @@ <td class="nopad"><div class="title"><a name="Windows">Microsoft Windows</a></div></td> </tr> <tr> -<td>The Tor Software for Windows comes bundled in three different ways: +<td>The Tor Software for Windows comes bundled in four different ways: <ul> <li>The Tor Browser Bundle contains everything you need to safely browse the Internet. This package requires no installation. Just extract it and run. <a href="<page projects/torbrowser>">Learn more »</a></li> <li>The Vidalia Bundle contains Tor, <a href="<page projects/vidalia>">Vidalia</a>, and Polipo for installation on your system. You need to configure your applications to use Tor.</li> +<li>The Bridge-by-Default Vidalia Bundle is a Vidalia Bundle which is configured to be a <a href="<page docs/bridges>">bridge</a> in order to help censored users reach the Tor network. <li>The Expert Package contains just Tor and nothing else. You'll need to configure Tor and all of your applications manually.</li> </ul> There are two versions of each package, a stable and alpha release. Stable packages are released when we believe the features and code will not change for many months. Alpha or unstable packages are released so you can help us test new features and bugfixes. Even though they have a higher version number than the stable versions listed above, there is a much higher chance of serious reliability and security bugs in these downloads. Please be prepared to <a href="https://bugs.torproject.org/">report bugs</a>. @@ -54,8 +55,11 @@ Stable Vidalia Bundle works with Windows 7, Vista, XP, <a href="<package-win32-bundle-stable>">Download Stable</a> (<a href="<package-win32-bundle-stable>.asc">sig</a>) </td> </tr> - <tr class="gray"><td>Unstable Vidalia Bundles works with Windows 7, Vista, XP, <a href="<package-win32-bundle-alpha>">Download Unstable</a> (<a href="<package-win32-bundle-alpha>.asc">sig</a>)</td> + <tr class="gray"><td>Unstable Vidalia Bundle works with Windows 7, Vista, XP, <a href="<package-win32-bundle-alpha>">Download Unstable</a> (<a href="<package-win32-bundle-alpha>.asc">sig</a>)</td> </tr> + <tr><td> + <tr class="gray"><td>Unstable Bridge-by-Default Vidalia Bundle works with Windows 7, Vista, XP, <a href="../dist/vidalia-bundles/vidalia-bridge-bundle-<version-win32-bridge-bundle-alpha>.exe">Download Unstable</a> (<a href="../dist/vidalia-bundles/vidalia-bridge-bundle-<version-win32-bridge-bundle-alpha>.asc">sig</a>)</td> + </tr> <tr> <td>Stable Expert Bundle works with Windows 98SE, ME, Windows 7, Vista, XP, 2000, 2003 Server, <a href="../dist/win32/tor-<version-win32-stable>-win32.exe">Download Stable</a> (<a href="../dist/win32/tor-<version-win32-stable>-win32.exe.asc">sig</a>)</td> </tr> Modified: website/trunk/include/versions.wmi =================================================================== --- website/trunk/include/versions.wmi 2011-04-27 06:57:24 UTC (rev 24684) +++ website/trunk/include/versions.wmi 2011-04-27 15:26:04 UTC (rev 24685) @@ -7,6 +7,8 @@ <define-tag version-win32-bundle-stable whitespace=delete>0.2.1.30</define-tag> <define-tag version-win32-bundle-alpha whitespace=delete>0.2.2.24-alpha</define-tag> +<define-tag version-win32-bridge-bundle-alpha whitespace=delete>0.2.2.24-alpha-<version-vidalia-stable></define-tag> + <define-tag version-osx-x86-bundle-stable whitespace=delete>0.2.1.30</define-tag> <define-tag version-osx-x86-bundle-alpha whitespace=delete>0.2.2.24-alpha</define-tag>

1 0

r24684: {projects} Minor tweaks. (projects/articles/browser-privacy)
by Mike Perry 27 Apr '11

27 Apr '11

Author: mikeperry Date: 2011-04-27 06:57:24 +0000 (Wed, 27 Apr 2011) New Revision: 24684 Modified: projects/articles/browser-privacy/W3CIdentity.bib projects/articles/browser-privacy/W3CIdentity.tex Log: Minor tweaks. Modified: projects/articles/browser-privacy/W3CIdentity.bib =================================================================== --- projects/articles/browser-privacy/W3CIdentity.bib 2011-04-27 06:11:22 UTC (rev 24683) +++ projects/articles/browser-privacy/W3CIdentity.bib 2011-04-27 06:57:24 UTC (rev 24684) @@ -85,9 +85,9 @@ } @Misc{not-to-toggle, - title = {To Toggle, or not to Toggle: The End of Torbutton}, - author={Mike Perry}, - note = {\url{https://lists.torproject.org/pipermail/tor-talk/2011-April/020077.html}} + title = {To Toggle, or not to Toggle: The End of Torbutton}, + author={Mike Perry}, + note = {\url{https://lists.torproject.org/pipermail/tor-talk/2011-April/020077.html}} } @Misc{firefox-personas, Modified: projects/articles/browser-privacy/W3CIdentity.tex =================================================================== --- projects/articles/browser-privacy/W3CIdentity.tex 2011-04-27 06:11:22 UTC (rev 24683) +++ projects/articles/browser-privacy/W3CIdentity.tex 2011-04-27 06:57:24 UTC (rev 24684) @@ -72,17 +72,16 @@ web actually functions with respect to user tracking. To this end, the rest of this document is structured as follows: First, we -examine how users perceive their privacy on the web, comparing the average -user's perspective to what actually is happening technically behind the -scenes, and noting the major disconnects. We then examine solutions that -bridge this disconnect from two different directions, corresponding to the two -major sources of disconnect\footnotemark. The first direction is improving -user cues and browser interface to suggest a coherent concept of identity to -users by accurately reflecting the set of unique identifiers they have -accumulated. The second direction is improving the linkability issues -inherent with the multi-origin model of the web itself. Both of these -directions must be pursued to provide users with the ability to properly use -the web in a privacy-preserving way. +compare the average user's understanding of web tracking to what actually is +happening technically behind the scenes, and note the major disconnects. We +then examine solutions that bridge this disconnect from two different +directions, corresponding to the two major sources of disconnect\footnotemark. +The first direction is improving user cues and browser interface to suggest a +coherent concept of identity to users by accurately reflecting the set of +unique identifiers they have accumulated. The second direction is improving +the linkability issues inherent in the multi-origin model of the web itself. +Both of these directions must be pursued to provide users with the ability to +properly use the web in a privacy-preserving way. \footnotetext{We only consider implementations that involve privacy-by-design. Privacy-by-policy approaches such as Do Not Track will not be discussed.} @@ -159,17 +158,15 @@ when it comes to user identity and private browsing, but they are not the whole story. -Next, we have long-term properties of the browser itself. These include the -User Agent string, the list of installed plugins, rendering capabilities, -window decoration size, and browser widget size. - -Then, we have properties of the computer. These include desktop size, IP +Next, we have long-term properties of the browser and the computer. These +include the User Agent string, the list of installed plugins, rendering +capabilities, window decoration size, browser widget size, desktop size, IP address, clock offset and timezone, and installed fonts. Finally, linkability also includes the properties of the multi-origin model of -the web that allow tracking due to partnerships. These include the implicit -cookie transmission model, and also explicit click referral and data exchange -partnerships. +the web that allow tracking due to partnerships and ubiquitous third-party +content elements. These include the implicit cookie transmission model, and +also explicit click referral and data exchange partnerships. \subsection{Developing a Threat Model} @@ -208,7 +205,7 @@ For users to have privacy, and for private browsing modes to function, the relationship between a user and a site must be understood by that user. -Users experience disconnects with the technical realities of the web on two +Users experience disconnect with the technical realities of the web on two major fronts: the average user is not given a clear concept of browser identity to grasp the privacy implications of the union of the linkable components of her browser, nor does she grasp the privacy implications of the @@ -221,26 +218,26 @@ \subsection{Conveying Identity to the User} The first major disconnect that prevents users from achieving true -privacy-by-design is that the user interface of most browsers does not provide -any clearly visible cues to the user to indicate that their current set of -accumulated linkable state comprise a single, trackable web identity. +privacy-by-design is that most browsers do not provide any cues to the user to +indicate that their current set of accumulated linkable state comprise a +single, trackable web identity that can be changed or cleared. We believe that the user interface of the browser should convey a sense of persistent identity prominently to the user in the form of a visual cue. This cue can either be an abstract image, graphic or theme (such as the user's choice of Firefox Persona~\cite{firefox-personas}), or it can be a text area -with the user's current favored pseudonym. This idea of identity should then -be integrated with the browsing experience. Users should be able to click a +with the user's current pseudonym. This idea of identity should then be +integrated with the browsing experience. Users should be able to click a button to get a clean slate for a new identity, and should be able to log into -and out of password-protected stored identities, which would -contain the entire state of the browser. +and out of password-protected stored identities, which would contain the +entire state of the browser. To this user, the Private Browsing Mode would be no more than a special case of this identity UI---a special identity that they can trust not to store browsing history information to disk. Such a UI also more explicitly captures what is going on with respect to the user's relationship to the web. -The Tor Project is heading in this direction with the Tor Browser -Bundle~\cite{not-to-toggle}. +%The Tor Project is heading in this direction with the Tor Browser +%Bundle~\cite{not-to-toggle}. Of the major private browsing modes, Google Chrome's Incognito Mode comes the closest to conveying this idea of ``identity'' to the user, and its @@ -262,10 +259,10 @@ Unfortunately, all current private browsing modes protect only against adversaries with access to the local computer and fail to deal with linkability against network adversaries (such as advertising -networks)~\cite{private-browsing}, claiming that it is outside their threat -model\footnotemark. If the user is given a new identity that is still linkable -to the previous one due to shortcomings of the browser, the approach has -failed as a privacy measure. +networks)~\cite{private-browsing}, claiming that the latter is outside their +threat model\footnotemark. If the user is given a new identity that is still +linkable to the previous one due to shortcomings of the browser, the approach +has failed as a privacy measure. \footnotetext{The primary reason given to abstain from addressing a network adversary is IP-address linkability. However, we believe this argument to be a red @@ -311,7 +308,7 @@ to make linkability less implicit and more consent-driven without the need for cumbersome interventionist user interface, and with minimal damage to existing content. Where explicit identifiers exist, they should be tied to the pair of -the top-level origin and the third-party content origin. Where linkability +the top-level origin and the third-party element origin. Where linkability attributes exist, they can be obfuscated on a per-origin basis. The work done by the Stanford Applied Crypto Group shows that it is relatively @@ -338,7 +335,7 @@ would only be transmitted if they match both the top-level origin and the third-party origin involved in their creation. Dan observed minimal breakage to popular sites, and where breakage did occur, alternative approaches that -did not violate the new model were readily available to web designers and +do not violate the new model were readily available to web designers and often already in use. Similarly, this two-level dual-keyed origin isolation can be deployed to @@ -352,11 +349,11 @@ intuitive control over site identifiers, and thus with more control over their actual relationship to particular sites. For example, the privacy settings window could have a user-intuitive way of representing the user's relationship -with different origins, perhaps by using only the `favicon' of that top level -origin to represent all of the browser state accumulated by that origin. The -user could delete the entire set of browser state (cookies, cache, storage, -cryptographic tokens) associated with a site simply by removing its favicon -from her privacy info panel. +with different top-level origins, perhaps by using only the `favicon' of that +top-level origin to represent all of the browser state accumulated by that +origin. The user could delete the entire set of browser state (cookies, cache, +storage, cryptographic tokens) associated with a site simply by removing its +favicon from her privacy info panel. Linkability based on fingerprintable browser properties is also amenable to improvement under this model. In particular, one can imagine per-origin plugin

1 0

r24683: {projects} Remove some header whitespace and tweak some wording. Still (projects/articles/browser-privacy)
by Mike Perry 27 Apr '11

27 Apr '11

Author: mikeperry Date: 2011-04-27 06:11:22 +0000 (Wed, 27 Apr 2011) New Revision: 24683 Modified: projects/articles/browser-privacy/W3CIdentity.tex projects/articles/browser-privacy/usenix.sty Log: Remove some header whitespace and tweak some wording. Still need to lose about a paragraph or two... Modified: projects/articles/browser-privacy/W3CIdentity.tex =================================================================== --- projects/articles/browser-privacy/W3CIdentity.tex 2011-04-27 05:43:55 UTC (rev 24682) +++ projects/articles/browser-privacy/W3CIdentity.tex 2011-04-27 06:11:22 UTC (rev 24683) @@ -19,7 +19,7 @@ \title{Bridging the Disconnect Between Web Privacy and User Perception} -\author{Mike Perry \\ The Internet \\ mikeperry(a)torproject.org} +\author{Mike Perry \\ mikeperry(a)torproject.org} %\institute{The Internet} @@ -52,28 +52,21 @@ to the current user. The cost of this incentive structure is that user privacy on the web is a -nightmare. There is -ubiquitous tracking, unseen partnership agreements and data exchange, and -surreptitious attempts to uncover users' identities against their will and -without their knowledge. This is not just happening in the dark, unseemly -corners of the web. It is happening everywhere~\cite{facebook-like}. +nightmare. There is ubiquitous tracking, unseen partnership agreements and +data exchange, and surreptitious attempts to uncover users' identities against +their will and without their knowledge. This is not just happening in the +dark, unseemly corners of the web. It is happening +everywhere~\cite{facebook-like}. -The problem is that the revenue model of the web has incentivized companies to -find ways to continue to track users against their will, even if those users -are attempting to protect themselves through currently available methods. -Starting with the infamous ``Flash cookies'', we have progressed through a -seemingly endless arms race of secondary identifiers and tracking information: -visited history, cache, font and system data, desktop resolution, keystroke -timing, and so on and so forth~\cite{wsj-fingerprinting}. - -These efforts have led to an even wider disconnect between users' -perception of their privacy and the reality of their privacy. Users simply -can't keep up with the ways they are being tracked. +The efforts towards ever increasing amounts of web tracking have led to a +growing disconnect between users' perception of their privacy and the reality +of their privacy. Users simply can't keep up with the ways they are being +tracked~\cite{wsj-fingerprinting}. % When users are being coerced into ceding data about themselves without clear understanding or consent (and in fact, in many cases despite their explicit attempts to decline to consent), serious moral issues begin to arise. -% + To understand and evaluate potential solutions and improvements to this status quo, we must explore the disconnect between user experience and the way the web actually functions with respect to user tracking. @@ -82,23 +75,23 @@ examine how users perceive their privacy on the web, comparing the average user's perspective to what actually is happening technically behind the scenes, and noting the major disconnects. We then examine solutions that -bridge this disconnect from two different directions, corresponding to the -two major sources of disconnect\footnotemark. The first direction is improving -the linkability issues inherent with the multi-origin model of the web itself. -The second direction is improving user cues and browser interface to suggest a -coherent concept of identity to users by accurately reflecting the -set of unique identifiers they have accumulated. Both of these directions must -be pursued to provide users with the ability to properly use the web in a -privacy-preserving way. +bridge this disconnect from two different directions, corresponding to the two +major sources of disconnect\footnotemark. The first direction is improving +user cues and browser interface to suggest a coherent concept of identity to +users by accurately reflecting the set of unique identifiers they have +accumulated. The second direction is improving the linkability issues +inherent with the multi-origin model of the web itself. Both of these +directions must be pursued to provide users with the ability to properly use +the web in a privacy-preserving way. \footnotetext{We only consider implementations that involve privacy-by-design. Privacy-by-policy approaches such as Do Not Track will not be discussed.} \section{User Privacy on the Web} -To properly examine the privacy problem, we must probe both the average users' -perception of what their ``web identity'' is, as well as the technical -realities of web authentication and tracking. +To properly examine the privacy problem, we must probe the average users' +perception of what their ``web identity'' is, and compare their perceptions to +the technical realities of web authentication and tracking. \subsection{User Perception of Privacy} @@ -268,7 +261,7 @@ Unfortunately, all current private browsing modes protect only against adversaries with access to the local computer and fail to deal with -linkability against a network adversary (such as advertising +linkability against network adversaries (such as advertising networks)~\cite{private-browsing}, claiming that it is outside their threat model\footnotemark. If the user is given a new identity that is still linkable to the previous one due to shortcomings of the browser, the approach has @@ -296,8 +289,8 @@ The other primary source of disconnect between user expectations and reality on the web is the origin model that governs cookie and other identifier transmission. The model allows unique, globally linkable identifiers to be -transmitted for arbitrary content elements on any page, and they can be -sourced from anywhere without user interaction or awareness. This property +transmitted for arbitrary content elements on any page, and such elements can +be sourced from anywhere without user interaction or awareness. This property enables popular advertising and content distribution networks to have near-omniscient visibility into all user activity retroactively after any level of authentication takes place with a cooperating partner site. @@ -330,14 +323,14 @@ crafted to include an identifier unique to each user, thus tracking even users who clear normal cookies. -The Stanford group correctly observed that the problem with origin model -improvements is that individually, they do not fully address the linkability -problem unless the same restriction is applied uniformly to all aspects of -stored browser state, and all other linkability issues are dealt with. -Behind-the-scenes partnerships can easily allow companies to continue to link -users to their identities through any linkable aspect of browser state that is -not properly compartmentalized to the top level origin and bound to the same -rules as all other linkable state. +The Stanford group correctly observed that individually, origin model +improvements do not fully address the linkability problem unless the same +restriction is applied uniformly to all aspects of stored browser state, and +all other linkability issues are dealt with. Behind-the-scenes partnerships +can easily allow companies to continue to link users to their activity +through any linkable aspect of browser state that is not properly +compartmentalized to the top level origin and bound to the same rules as all +other linkable attributes. Along these lines, the Mozilla development wiki describes an origin model improvement for cookie transmission written by Dan Witte~\cite{thirdparty}. He @@ -348,10 +341,11 @@ did not violate the new model were readily available to web designers and often already in use. -Similarly, one could imagine this two-level dual-keyed origin isolation being -deployed to improve similar issues with DOM Storage and cryptographic tokens. -This dual-origin policy should be considered a must for all future -origin-bound identifiers. +Similarly, this two-level dual-keyed origin isolation can be deployed to +improve similar issues with DOM Storage and cryptographic tokens, so that +these identifiers are sent only if both the top-level and the third-party +origins match. This dual-origin policy should be considered a must for all +future origin-bound identifiers. With a clear association between third-party cookies and their top-level origin due to double-keying, it becomes easier to provide the user with more Modified: projects/articles/browser-privacy/usenix.sty =================================================================== --- projects/articles/browser-privacy/usenix.sty 2011-04-27 05:43:55 UTC (rev 24682) +++ projects/articles/browser-privacy/usenix.sty 2011-04-27 06:11:22 UTC (rev 24683) @@ -66,7 +66,7 @@ \def\@maketitle{\newpage %\vbox to 0.5in{ - \vbox to 1.5in{ + \vbox to 0.9in{ %\vspace*{\fill} %\vskip 2em \begin{center}%

1 0

r24682: {projects} kill some more widows your footnote 1 is on the wrong page n (projects/articles/browser-privacy)
by Roger Dingledine 27 Apr '11

27 Apr '11

Author: arma Date: 2011-04-27 05:43:55 +0000 (Wed, 27 Apr 2011) New Revision: 24682 Modified: projects/articles/browser-privacy/W3CIdentity.tex Log: kill some more widows your footnote 1 is on the wrong page now btw Modified: projects/articles/browser-privacy/W3CIdentity.tex =================================================================== --- projects/articles/browser-privacy/W3CIdentity.tex 2011-04-27 05:09:51 UTC (rev 24681) +++ projects/articles/browser-privacy/W3CIdentity.tex 2011-04-27 05:43:55 UTC (rev 24682) @@ -69,11 +69,11 @@ These efforts have led to an even wider disconnect between users' perception of their privacy and the reality of their privacy. Users simply can't keep up with the ways they are being tracked. - +% When users are being coerced into ceding data about themselves without clear understanding or consent (and in fact, in many cases despite their explicit attempts to decline to consent), serious moral issues begin to arise. - +% To understand and evaluate potential solutions and improvements to this status quo, we must explore the disconnect between user experience and the way the web actually functions with respect to user tracking. @@ -305,8 +305,8 @@ This identifier transmission model is fundamentally flawed when viewed from the perspective of meeting the expectations of the user. -So far, industry has resisted changes to the identifier transmission model due -to both inertia and compatibility concerns. However, the disconnect is so +Industry has so far resisted changes to the identifier transmission model due +to compatibility concerns and inertia. However, the disconnect is so severe and the associated tracking is so pervasive that some level of temporary breakage must be tolerated to improve the status quo. Because of the retroactive nature of the linkability of cookies and other identifier storage, @@ -322,13 +322,13 @@ attributes exist, they can be obfuscated on a per-origin basis. The work done by the Stanford Applied Crypto Group shows that it is relatively -straight-forward to isolate the browser cache to specific top-level origins, +straightforward to isolate the browser cache to specific top-level origins, effectively binding identifiers hidden in cached elements to the pair of top-level and third-party origin~\cite{safecache}. Commonly sourced third-party content elements are then fetched and cached repeatedly, but this is necessary to prevent linkability: each of these content elements can be crafted to include an identifier unique to each user, thus tracking even users -who attempt to avoid tracking by clearing normal cookies. +who clear normal cookies. The Stanford group correctly observed that the problem with origin model improvements is that individually, they do not fully address the linkability @@ -373,9 +373,9 @@ of the web closer to what the user assumes is happening, they must be deployed uniformly, with a consistent top-level origin restriction model. Uniform deployment will take significant coordination and standardization efforts. -Furthermore, even an vastly improved origin model still cannot prevent +Furthermore, even a vastly improved origin model cannot prevent instances of explicit tracking partnerships between sites and third-party -content providers. Therefore, both origin improvements as well as +content providers. Therefore, both origin improvements and identity-isolation approaches are necessary. \section{Conclusions} @@ -395,11 +395,11 @@ In Section~6, it also strongly suggested that informed consent and user control should govern the interaction of users to tracking identifiers. -Without changes to both browser behavior and browser interface, such informed +Without changes to both browser behavior and browser interface, informed consent is simply not possible on today's web. The lack of informed consent makes it impossible to expect privacy-by-design approaches to function -properly. Users who do not even understand the basic properties of the -tracking mechanisms they are subjected to cannot be expected to effectively +properly. We cannot expect users who do not even understand the basic +properties of these tracking mechanisms to effectively use privacy mechanisms to avoid, opt out of, or decline such tracking. \bibliographystyle{plain} \bibliography{W3CIdentity}

1 0

r24681: {projects} Rework and reorganize to argue for both identity-isolation a (projects/articles/browser-privacy)
by Mike Perry 27 Apr '11

27 Apr '11

Author: mikeperry Date: 2011-04-27 05:09:51 +0000 (Wed, 27 Apr 2011) New Revision: 24681 Modified: projects/articles/browser-privacy/W3CIdentity.tex Log: Rework and reorganize to argue for both identity-isolation and origin model improvements. Modified: projects/articles/browser-privacy/W3CIdentity.tex =================================================================== --- projects/articles/browser-privacy/W3CIdentity.tex 2011-04-27 01:30:49 UTC (rev 24680) +++ projects/articles/browser-privacy/W3CIdentity.tex 2011-04-27 05:09:51 UTC (rev 24681) @@ -30,17 +30,16 @@ There is a huge disconnect between how users perceive their online presence and the reality of their relationship with the websites they visit. This -position paper explores this disconnect and provides some recommendations for -making the technical reality of the web match user perception. %, through both -%technical improvements as well as user interface cues. -We frame the core -technical problem as one of ``linkability''---the level of correlation +position paper explores this disconnect and provides recommendations for +making the technical reality of the web match user perception. We frame the +core technical problem as one of ``linkability''---the level of correlation between various online activities that the user naturally expects to be independent. We look to address the issue of unexpected linkability through -both technical improvements to the web's origin model, as well as through -user interface -cues about the set of accumulated identifiers that can be said to comprise -a user's online identity. +both technical improvements to the web's origin model, as well as through user +interface cues about the set of accumulated identifiers that can be said to +comprise a user's online identity. We argue that without approaching the +problem from both of these directions, true privacy-by-design is simply not a +possibility for average web users. \end{abstract} @@ -79,16 +78,6 @@ quo, we must explore the disconnect between user experience and the way the web actually functions with respect to user tracking. -% -% 20:16 < nickm> Not "identity-based", though. identity-separation, -% identity-isolation. "nym" and "pseudonym" are also fine words -% 20:18 < armadev> i'm still not entirely clear on what you mean by the -% identity model. i am guessing it's "the user thinks of his web -% experience in terms of whether the website can recognize -% him", but i think that's not it. i want clearer -% definitions up front, and then i can help with terms. :) - - To this end, the rest of this document is structured as follows: First, we examine how users perceive their privacy on the web, comparing the average user's perspective to what actually is happening technically behind the @@ -98,8 +87,9 @@ the linkability issues inherent with the multi-origin model of the web itself. The second direction is improving user cues and browser interface to suggest a coherent concept of identity to users by accurately reflecting the -set of unique identifiers they have accumulated. Both of these directions can -be pursued independently. +set of unique identifiers they have accumulated. Both of these directions must +be pursued to provide users with the ability to properly use the web in a +privacy-preserving way. \footnotetext{We only consider implementations that involve privacy-by-design. Privacy-by-policy approaches such as Do Not Track will not be discussed.} @@ -122,7 +112,7 @@ typically not aware that this relationship also extends to their activity on other, arbitrary sites that happen to include ``Like this on Facebook'' buttons or -Facebook-sourced advertising content. +Facebook-sourced advertising content~\cite{facebook-like}. Many, if not most, users expect that when they log out of a site, their relationship ends and that any associated tracking should be over. Even users @@ -224,119 +214,24 @@ For users to have privacy, and for private browsing modes to function, the relationship between a user and a site must be understood by that user. -% -%It is apparent that -Users experience disconnects with the technical -realities of the web on two major fronts: the average user does not grasp the -privacy implications of the multi-origin model, nor is she given a clear -concept of browser identity to grasp the privacy implications of the union -of the linkable components of her browser. -We will now examine examples of attempts at reducing this disconnect on each -of these two fronts. Note that these two fronts are orthogonal. Approaches from -them may be combined, or used independently. +Users experience disconnects with the technical realities of the web on two +major fronts: the average user is not given a clear concept of browser +identity to grasp the privacy implications of the union of the linkable +components of her browser, nor does she grasp the privacy implications of the +multi-origin model and how identifiers are transmitted under this model. -\subsection{Improving the Origin Model} +Both of these areas of disconnect must be fully addressed in order for the +average user to have the technical level of privacy that they intuitively +expect. -The current identifier origin model used by the web is fundamentally flawed -when viewed from the perspective of meeting the expectations of the user. -Unique, globally linkable identifiers can be transmitted for arbitrary content -elements on any page, and they can be sourced from anywhere without user -interaction or awareness. - -However, the behavior of identifiers and linkable attributes can be improved -to make linkability less implicit and more consent-driven without the need for -cumbersome interventionist user interface. Where explicit identifiers exist, -they should be tied to the pair of the top-level origin and the third-party -content origin. Where linkability attributes exist, they should be obfuscated -on a per-origin basis. - -An early relevant example of this idea is SafeCache~\cite{safecache}. -SafeCache seeks to reduce the ability for third-party content elements to use -the cache to store identifiers. It does this by limiting the scope of the -cache to the top-level origin in the URL bar. Commonly sourced content -elements are then fetched and cached repeatedly, but this -is necessary to prevent linkability: each of these content elements can be -crafted to include an identifier unique to each user, thus tracking even -users who -attempt to avoid tracking by clearing normal cookies. - -The Mozilla development wiki describes an origin model improvement for -cookie transmission -written by Dan Witte~\cite{thirdparty}. He describes a new -dual-keyed origin for cookies, so that cookies would only be transmitted if -they match both the top-level origin and the third-party origin involved in -their creation. This approach would go a long way towards preventing implicit -tracking across multiple websites, and has some interesting properties that -make user interaction with content elements more explicitly tied to the -current site. -% XXXX I can't tell what this paragraph is supposed to mean. --RR -% -Similarly, one could imagine this two-level dual-keyed origin isolation being -deployed to improve similar issues with DOM Storage and cryptographic tokens. - -Making the origin model for browser identifiers more closely match user -activity and user expectation has other advantages as well. With a clear -distinction between third-party and top-level cookies due to double-keying, the -privacy settings window could have a user-intuitive way of representing the -user's relationship with different origins, perhaps by using only the `favicon' -of that top level origin to represent all of the browser state accumulated by -that origin. The user could delete the entire set of browser state (cookies, -cache, storage, cryptographic tokens) associated with a site simply by -removing its favicon from her privacy info panel. - -The problem with origin model improvement approaches is that individually, -they do not fully address the linkability problem unless the same -restriction is applied uniformly to all aspects of stored browser state, and -all other linkability issues are dealt with. Behind-the-scenes partnerships -can easily allow companies to continue to link users to their identities -through any linkable aspect of browser state that is not properly -compartmentalized to the top level origin and bound to the same rules as all -other linkable state. - -However, linkability based on fingerprintable browser properties is also -amenable to improvement under this model. In particular, one can imagine -per-origin plugin loading permissions, per-origin limits on the number of -fonts that can be used, and randomized window-specific time offsets. -% -So, while these approaches are in fact useful for bringing the technical -realities of the web closer to what the user assumes is happening, they must -be deployed uniformly, with a consistent top-level origin restriction model. -Uniform deployment will take significant coordination and standardization -efforts. Until then, -it is necessary to fill the remaining linkability gaps by presenting -the user with a visual representation of her overall web identity. - \subsection{Conveying Identity to the User} -Even if the origin model of identifier transmission and other linkable -attributes is altered uniformly to be more in line with what users expect, -the average user can still experience privacy benefits if the -browser conveys the sum of all linkable information as a single storable, -mutable, and clearable user identity. -% -Providing this concept of identity to the user is also simpler than origin -improvements, as it does not require extensive compatibility testing or -standards coordination. +The first major disconnect that prevents users from achieving true +privacy-by-design is that the user interface of most browsers does not provide +any clearly visible cues to the user to indicate that their current set of +accumulated linkable state comprise a single, trackable web identity. -Of the major private browsing modes, Google Chrome's Incognito Mode comes the -closest to conveying the idea of ``identity'' to the user, and its -implementation is also simple as a result. The Incognito Mode window is a -separate, stylized window which clearly conveys that an alternate identity -is in use -in this window, which can be used concurrently with the non-private identity. -The better UI appears to lead to less mode error (in which the user forgets -whether -private browsing is enabled) than other browsers' private browsing -modes~\cite{private-browsing}. - -The Mozilla Weave project~\cite{weave-manager} appears to be proposing -an identity-oriented method -of managing, syncing, and storing authentication tokens, and also has use -cases described for multiple users of a single browser. It -is the closest idea on paper to what we envision as the way to bridge user -assumptions with reality. - We believe that the user interface of the browser should convey a sense of persistent identity prominently to the user in the form of a visual cue. This cue can either be an abstract image, graphic or theme (such as the user's @@ -346,7 +241,7 @@ button to get a clean slate for a new identity, and should be able to log into and out of password-protected stored identities, which would contain the entire state of the browser. -% + To this user, the Private Browsing Mode would be no more than a special case of this identity UI---a special identity that they can trust not to store browsing history information to disk. Such a UI also more explicitly captures @@ -354,30 +249,135 @@ The Tor Project is heading in this direction with the Tor Browser Bundle~\cite{not-to-toggle}. -Unfortunately, all current private browsing modes fall short of protecting -against a -network-level adversary and fail to deal with linkability against such an -adversary~\cite{private-browsing}, claiming that it is outside their threat +Of the major private browsing modes, Google Chrome's Incognito Mode comes the +closest to conveying this idea of ``identity'' to the user, and its +implementation is also simple as a result. The Incognito Mode window is a +separate, stylized window which clearly conveys that an alternate identity +is in use +in this window, which can be used concurrently with the non-private identity. +The better UI appears to lead to less mode error (in which the user forgets +whether +private browsing is enabled) than other browsers' private browsing +modes~\cite{private-browsing}. + +The Mozilla Weave project~\cite{weave-manager} appears to be proposing an +identity-oriented method of managing, syncing, and storing authentication +tokens, and also has use cases described for multiple users of a single +browser. It is the closest idea on paper to what we envision as the way to +bridge user assumptions with reality. + +Unfortunately, all current private browsing modes protect only against +adversaries with access to the local computer and fail to deal with +linkability against a network adversary (such as advertising +networks)~\cite{private-browsing}, claiming that it is outside their threat model\footnotemark. If the user is given a new identity that is still linkable -to the previous one due to shortcomings of the browser, this approach has +to the previous one due to shortcomings of the browser, the approach has failed as a privacy measure. -% XXXX Define network-level adversary. -% I agree. You could do that here or in the footnote. -RD -\footnotetext{The primary reason given to abstain from addressing a -network-level -adversary is IP-address linkability. However, we believe this to be a red +\footnotetext{The primary reason given to abstain from addressing a network +adversary is IP-address linkability. However, we believe this argument to be a red herring. Users are quite capable of using alternate Internet connections, and it is common practice for ISPs (especially cellular IP networks) -%in many parts of the world -to rotate user IP -addresses daily, to discourage users from operating servers and to impede the -spread of malware.} +to rotate user IP addresses daily, to discourage users from operating servers +and to impede the spread of malware.} -Linkability solutions within the identity framework would be similar to the -origin model solutions, except they would be properties of the entire browser -or browser profile, and would be obfuscated only once per identity switch. +Therefore, in addition to isolating explicit disk-based identifiers and +browser state, an attempt should be made to obfuscate or alter the biggest +culprits in terms of the entropy linkability metric mentioned in Section~2.4. +However, not all linkability sources have viable solutions under an +identity-isolation approach, and moreover, identity-isolation approaches fail +to protect the user against linkability due to ubiquitous third party content +elements that track them across nearly all sites as soon as they log into any +one site~\cite{facebook-like}. + +\subsection{Improving the Origin Model} + +The other primary source of disconnect between user expectations and reality +on the web is the origin model that governs cookie and other identifier +transmission. The model allows unique, globally linkable identifiers to be +transmitted for arbitrary content elements on any page, and they can be +sourced from anywhere without user interaction or awareness. This property +enables popular advertising and content distribution networks to have +near-omniscient visibility into all user activity retroactively after any +level of authentication takes place with a cooperating partner site. + +This identifier transmission model is fundamentally flawed when viewed from +the perspective of meeting the expectations of the user. + +So far, industry has resisted changes to the identifier transmission model due +to both inertia and compatibility concerns. However, the disconnect is so +severe and the associated tracking is so pervasive that some level of +temporary breakage must be tolerated to improve the status quo. Because of the +retroactive nature of the linkability of cookies and other identifier storage, +and because of the invisible and pervasive nature of these partnerships, +privacy-by-design is essentially impossible to provide to the average user +without addressing this issue. + +However, the behavior of identifiers and linkable attributes can be improved +to make linkability less implicit and more consent-driven without the need for +cumbersome interventionist user interface, and with minimal damage to existing +content. Where explicit identifiers exist, they should be tied to the pair of +the top-level origin and the third-party content origin. Where linkability +attributes exist, they can be obfuscated on a per-origin basis. + +The work done by the Stanford Applied Crypto Group shows that it is relatively +straight-forward to isolate the browser cache to specific top-level origins, +effectively binding identifiers hidden in cached elements to the pair of +top-level and third-party origin~\cite{safecache}. Commonly sourced +third-party content elements are then fetched and cached repeatedly, but this +is necessary to prevent linkability: each of these content elements can be +crafted to include an identifier unique to each user, thus tracking even users +who attempt to avoid tracking by clearing normal cookies. + +The Stanford group correctly observed that the problem with origin model +improvements is that individually, they do not fully address the linkability +problem unless the same restriction is applied uniformly to all aspects of +stored browser state, and all other linkability issues are dealt with. +Behind-the-scenes partnerships can easily allow companies to continue to link +users to their identities through any linkable aspect of browser state that is +not properly compartmentalized to the top level origin and bound to the same +rules as all other linkable state. + +Along these lines, the Mozilla development wiki describes an origin model +improvement for cookie transmission written by Dan Witte~\cite{thirdparty}. He +describes applying this same dual-keyed origin to cookies, so that cookies +would only be transmitted if they match both the top-level origin and the +third-party origin involved in their creation. Dan observed minimal breakage +to popular sites, and where breakage did occur, alternative approaches that +did not violate the new model were readily available to web designers and +often already in use. + +Similarly, one could imagine this two-level dual-keyed origin isolation being +deployed to improve similar issues with DOM Storage and cryptographic tokens. +This dual-origin policy should be considered a must for all future +origin-bound identifiers. + +With a clear association between third-party cookies and their top-level +origin due to double-keying, it becomes easier to provide the user with more +intuitive control over site identifiers, and thus with more control over their +actual relationship to particular sites. For example, the privacy settings +window could have a user-intuitive way of representing the user's relationship +with different origins, perhaps by using only the `favicon' of that top level +origin to represent all of the browser state accumulated by that origin. The +user could delete the entire set of browser state (cookies, cache, storage, +cryptographic tokens) associated with a site simply by removing its favicon +from her privacy info panel. + +Linkability based on fingerprintable browser properties is also amenable to +improvement under this model. In particular, one can imagine per-origin plugin +loading permissions, per-origin limits on the number of fonts that can be +used, and randomized window-specific time offsets. + +While these approaches are in fact useful for bringing the technical realities +of the web closer to what the user assumes is happening, they must be deployed +uniformly, with a consistent top-level origin restriction model. Uniform +deployment will take significant coordination and standardization efforts. +Furthermore, even an vastly improved origin model still cannot prevent +instances of explicit tracking partnerships between sites and third-party +content providers. Therefore, both origin improvements as well as +identity-isolation approaches are necessary. + \section{Conclusions} The appeal of the web's prevailing revenue model and the difficulties @@ -395,17 +395,13 @@ In Section~6, it also strongly suggested that informed consent and user control should govern the interaction of users to tracking identifiers. -Without changes to browser behavior, browser interface, or both, such informed -consent is simply not possible on today's web. Several examples from academia -and practice show that it is possible to bridge this disconnect by addressing -the linkability issues with the web's origin model with minimal breakage. -Additionally, the first steps towards providing the user with an explicit -representation of their web identity have been taken. -% Taken by who? Change to active tense. -RD +Without changes to both browser behavior and browser interface, such informed +consent is simply not possible on today's web. The lack of informed consent +makes it impossible to expect privacy-by-design approaches to function +properly. Users who do not even understand the basic properties of the +tracking mechanisms they are subjected to cannot be expected to effectively +use privacy mechanisms to avoid, opt out of, or decline such tracking. -The pieces are in place to build robust private browsing modes based on these -two approaches, and metrics exist to measure their success. - \bibliographystyle{plain} \bibliography{W3CIdentity} %\clearpage

1 0

[arm/master] Renaming interface directory to cli
by atagar＠torproject.org 27 Apr '11

27 Apr '11

commit adf453c2402b805582d448d17f62ac173f00ec61 Author: Damian Johnson <atagar(a)torproject.org> Date: Tue Apr 26 19:20:55 2011 -0700 Renaming interface directory to cli Kamran's working on an arm gui for GSoC, so renaming the interface directory to cli (and he'll introduce a /gui). --- src/cli/__init__.py | 6 + src/cli/configPanel.py | 364 +++++++ src/cli/connections/__init__.py | 6 + src/cli/connections/circEntry.py | 216 ++++ src/cli/connections/connEntry.py | 850 ++++++++++++++++ src/cli/connections/connPanel.py | 398 ++++++++ src/cli/connections/entries.py | 164 +++ src/cli/controller.py | 1584 ++++++++++++++++++++++++++++++ src/cli/descriptorPopup.py | 181 ++++ src/cli/graphing/__init__.py | 6 + src/cli/graphing/bandwidthStats.py | 398 ++++++++ src/cli/graphing/connStats.py | 54 + src/cli/graphing/graphPanel.py | 407 ++++++++ src/cli/graphing/resourceStats.py | 47 + src/cli/headerPanel.py | 474 +++++++++ src/cli/logPanel.py | 1100 +++++++++++++++++++++ src/cli/torrcPanel.py | 221 +++++ src/interface/__init__.py | 6 - src/interface/configPanel.py | 364 ------- src/interface/connections/__init__.py | 6 - src/interface/connections/circEntry.py | 216 ---- src/interface/connections/connEntry.py | 850 ---------------- src/interface/connections/connPanel.py | 398 -------- src/interface/connections/entries.py | 164 --- src/interface/controller.py | 1584 ------------------------------ src/interface/descriptorPopup.py | 181 ---- src/interface/graphing/__init__.py | 6 - src/interface/graphing/bandwidthStats.py | 398 -------- src/interface/graphing/connStats.py | 54 - src/interface/graphing/graphPanel.py | 407 -------- src/interface/graphing/resourceStats.py | 47 - src/interface/headerPanel.py | 474 --------- src/interface/logPanel.py | 1100 --------------------- src/interface/torrcPanel.py | 221 ----- src/starter.py | 10 +- 35 files changed, 6481 insertions(+), 6481 deletions(-) diff --git a/src/cli/__init__.py b/src/cli/__init__.py new file mode 100644 index 0000000..0f11fc1 --- /dev/null +++ b/src/cli/__init__.py @@ -0,0 +1,6 @@ +""" +Panels, popups, and handlers comprising the arm user interface. +""" + +__all__ = ["configPanel", "connPanel", "controller", "descriptorPopup", "headerPanel", "logPanel", "torrcPanel"] + diff --git a/src/cli/configPanel.py b/src/cli/configPanel.py new file mode 100644 index 0000000..fd6fb54 --- /dev/null +++ b/src/cli/configPanel.py @@ -0,0 +1,364 @@ +""" +Panel presenting the configuration state for tor or arm. Options can be edited +and the resulting configuration files saved. +""" + +import curses +import threading + +from util import conf, enum, panel, torTools, torConfig, uiTools + +DEFAULT_CONFIG = {"features.config.selectionDetails.height": 6, + "features.config.state.showPrivateOptions": False, + "features.config.state.showVirtualOptions": False, + "features.config.state.colWidth.option": 25, + "features.config.state.colWidth.value": 15} + +# TODO: The arm use cases are incomplete since they currently can't be +# modified, have their descriptions fetched, or even get a complete listing +# of what's available. +State = enum.Enum("TOR", "ARM") # state to be presented + +# mappings of option categories to the color for their entries +CATEGORY_COLOR = {torConfig.Category.GENERAL: "green", + torConfig.Category.CLIENT: "blue", + torConfig.Category.RELAY: "yellow", + torConfig.Category.DIRECTORY: "magenta", + torConfig.Category.AUTHORITY: "red", + torConfig.Category.HIDDEN_SERVICE: "cyan", + torConfig.Category.TESTING: "white", + torConfig.Category.UNKNOWN: "white"} + +# attributes of a ConfigEntry +Field = enum.Enum("CATEGORY", "OPTION", "VALUE", "TYPE", "ARG_USAGE", + "SUMMARY", "DESCRIPTION", "MAN_ENTRY", "IS_DEFAULT") +DEFAULT_SORT_ORDER = (Field.MAN_ENTRY, Field.OPTION, Field.IS_DEFAULT) +FIELD_ATTR = {Field.CATEGORY: ("Category", "red"), + Field.OPTION: ("Option Name", "blue"), + Field.VALUE: ("Value", "cyan"), + Field.TYPE: ("Arg Type", "green"), + Field.ARG_USAGE: ("Arg Usage", "yellow"), + Field.SUMMARY: ("Summary", "green"), + Field.DESCRIPTION: ("Description", "white"), + Field.MAN_ENTRY: ("Man Page Entry", "blue"), + Field.IS_DEFAULT: ("Is Default", "magenta")} + +class ConfigEntry(): + """ + Configuration option in the panel. + """ + + def __init__(self, option, type, isDefault): + self.fields = {} + self.fields[Field.OPTION] = option + self.fields[Field.TYPE] = type + self.fields[Field.IS_DEFAULT] = isDefault + + # Fetches extra infromation from external sources (the arm config and tor + # man page). These are None if unavailable for this config option. + summary = torConfig.getConfigSummary(option) + manEntry = torConfig.getConfigDescription(option) + + if manEntry: + self.fields[Field.MAN_ENTRY] = manEntry.index + self.fields[Field.CATEGORY] = manEntry.category + self.fields[Field.ARG_USAGE] = manEntry.argUsage + self.fields[Field.DESCRIPTION] = manEntry.description + else: + self.fields[Field.MAN_ENTRY] = 99999 # sorts non-man entries last + self.fields[Field.CATEGORY] = torConfig.Category.UNKNOWN + self.fields[Field.ARG_USAGE] = "" + self.fields[Field.DESCRIPTION] = "" + + # uses the full man page description if a summary is unavailable + self.fields[Field.SUMMARY] = summary if summary != None else self.fields[Field.DESCRIPTION] + + # cache of what's displayed for this configuration option + self.labelCache = None + self.labelCacheArgs = None + + def get(self, field): + """ + Provides back the value in the given field. + + Arguments: + field - enum for the field to be provided back + """ + + if field == Field.VALUE: return self._getValue() + else: return self.fields[field] + + def getAll(self, fields): + """ + Provides back a list with the given field values. + + Arguments: + field - enums for the fields to be provided back + """ + + return [self.get(field) for field in fields] + + def getLabel(self, optionWidth, valueWidth, summaryWidth): + """ + Provides display string of the configuration entry with the given + constraints on the width of the contents. + + Arguments: + optionWidth - width of the option column + valueWidth - width of the value column + summaryWidth - width of the summary column + """ + + # Fetching the display entries is very common so this caches the values. + # Doing this substantially drops cpu usage when scrolling (by around 40%). + + argSet = (optionWidth, valueWidth, summaryWidth) + if not self.labelCache or self.labelCacheArgs != argSet: + optionLabel = uiTools.cropStr(self.get(Field.OPTION), optionWidth) + valueLabel = uiTools.cropStr(self.get(Field.VALUE), valueWidth) + summaryLabel = uiTools.cropStr(self.get(Field.SUMMARY), summaryWidth, None) + lineTextLayout = "%%-%is %%-%is %%-%is" % (optionWidth, valueWidth, summaryWidth) + self.labelCache = lineTextLayout % (optionLabel, valueLabel, summaryLabel) + self.labelCacheArgs = argSet + + return self.labelCache + + def _getValue(self): + """ + Provides the current value of the configuration entry, taking advantage of + the torTools caching to effectively query the accurate value. This uses the + value's type to provide a user friendly representation if able. + """ + + confValue = ", ".join(torTools.getConn().getOption(self.get(Field.OPTION), [], True)) + + # provides nicer values for recognized types + if not confValue: confValue = "<none>" + elif self.get(Field.TYPE) == "Boolean" and confValue in ("0", "1"): + confValue = "False" if confValue == "0" else "True" + elif self.get(Field.TYPE) == "DataSize" and confValue.isdigit(): + confValue = uiTools.getSizeLabel(int(confValue)) + elif self.get(Field.TYPE) == "TimeInterval" and confValue.isdigit(): + confValue = uiTools.getTimeLabel(int(confValue), isLong = True) + + return confValue + +class ConfigPanel(panel.Panel): + """ + Renders a listing of the tor or arm configuration state, allowing options to + be selected and edited. + """ + + def __init__(self, stdscr, configType, config=None): + panel.Panel.__init__(self, stdscr, "configState", 0) + + self.sortOrdering = DEFAULT_SORT_ORDER + self._config = dict(DEFAULT_CONFIG) + if config: + config.update(self._config, { + "features.config.selectionDetails.height": 0, + "features.config.state.colWidth.option": 5, + "features.config.state.colWidth.value": 5}) + + sortFields = Field.values() + customOrdering = config.getIntCSV("features.config.order", None, 3, 0, len(sortFields)) + + if customOrdering: + self.sortOrdering = [sortFields[i] for i in customOrdering] + + self.configType = configType + self.confContents = [] + self.scroller = uiTools.Scroller(True) + self.valsLock = threading.RLock() + + # shows all configuration options if true, otherwise only the ones with + # the 'important' flag are shown + self.showAll = False + + if self.configType == State.TOR: + conn = torTools.getConn() + customOptions = torConfig.getCustomOptions() + configOptionLines = conn.getInfo("config/names", "").strip().split("\n") + + for line in configOptionLines: + # lines are of the form "<option> <type>", like: + # UseEntryGuards Boolean + confOption, confType = line.strip().split(" ", 1) + + # skips private and virtual entries if not configured to show them + if not self._config["features.config.state.showPrivateOptions"] and confOption.startswith("__"): + continue + elif not self._config["features.config.state.showVirtualOptions"] and confType == "Virtual": + continue + + self.confContents.append(ConfigEntry(confOption, confType, not confOption in customOptions)) + elif self.configType == State.ARM: + # loaded via the conf utility + armConf = conf.getConfig("arm") + for key in armConf.getKeys(): + pass # TODO: implement + + # mirror listing with only the important configuration options + self.confImportantContents = [] + for entry in self.confContents: + if torConfig.isImportant(entry.get(Field.OPTION)): + self.confImportantContents.append(entry) + + # if there aren't any important options then show everything + if not self.confImportantContents: + self.confImportantContents = self.confContents + + self.setSortOrder() # initial sorting of the contents + + def getSelection(self): + """ + Provides the currently selected entry. + """ + + return self.scroller.getCursorSelection(self._getConfigOptions()) + + def setSortOrder(self, ordering = None): + """ + Sets the configuration attributes we're sorting by and resorts the + contents. + + Arguments: + ordering - new ordering, if undefined then this resorts with the last + set ordering + """ + + self.valsLock.acquire() + if ordering: self.sortOrdering = ordering + self.confContents.sort(key=lambda i: (i.getAll(self.sortOrdering))) + self.confImportantContents.sort(key=lambda i: (i.getAll(self.sortOrdering))) + self.valsLock.release() + + def handleKey(self, key): + self.valsLock.acquire() + if uiTools.isScrollKey(key): + pageHeight = self.getPreferredSize()[0] - 1 + detailPanelHeight = self._config["features.config.selectionDetails.height"] + if detailPanelHeight > 0 and detailPanelHeight + 2 <= pageHeight: + pageHeight -= (detailPanelHeight + 1) + + isChanged = self.scroller.handleKey(key, self._getConfigOptions(), pageHeight) + if isChanged: self.redraw(True) + elif key == ord('a') or key == ord('A'): + self.showAll = not self.showAll + self.redraw(True) + self.valsLock.release() + + def draw(self, width, height): + self.valsLock.acquire() + + # draws the top label + configType = "Tor" if self.configType == State.TOR else "Arm" + hiddenMsg = "press 'a' to hide most options" if self.showAll else "press 'a' to show all options" + + # panel with details for the current selection + detailPanelHeight = self._config["features.config.selectionDetails.height"] + isScrollbarVisible = False + if detailPanelHeight == 0 or detailPanelHeight + 2 >= height: + # no detail panel + detailPanelHeight = 0 + scrollLoc = self.scroller.getScrollLoc(self._getConfigOptions(), height - 1) + cursorSelection = self.getSelection() + isScrollbarVisible = len(self._getConfigOptions()) > height - 1 + else: + # Shrink detail panel if there isn't sufficient room for the whole + # thing. The extra line is for the bottom border. + detailPanelHeight = min(height - 1, detailPanelHeight + 1) + scrollLoc = self.scroller.getScrollLoc(self._getConfigOptions(), height - 1 - detailPanelHeight) + cursorSelection = self.getSelection() + isScrollbarVisible = len(self._getConfigOptions()) > height - detailPanelHeight - 1 + + self._drawSelectionPanel(cursorSelection, width, detailPanelHeight, isScrollbarVisible) + + titleLabel = "%s Configuration (%s):" % (configType, hiddenMsg) + self.addstr(0, 0, titleLabel, curses.A_STANDOUT) + + # draws left-hand scroll bar if content's longer than the height + scrollOffset = 1 + if isScrollbarVisible: + scrollOffset = 3 + self.addScrollBar(scrollLoc, scrollLoc + height - detailPanelHeight - 1, len(self._getConfigOptions()), 1 + detailPanelHeight) + + optionWidth = self._config["features.config.state.colWidth.option"] + valueWidth = self._config["features.config.state.colWidth.value"] + descriptionWidth = max(0, width - scrollOffset - optionWidth - valueWidth - 2) + + for lineNum in range(scrollLoc, len(self._getConfigOptions())): + entry = self._getConfigOptions()[lineNum] + drawLine = lineNum + detailPanelHeight + 1 - scrollLoc + + lineFormat = curses.A_NORMAL if entry.get(Field.IS_DEFAULT) else curses.A_BOLD + if entry.get(Field.CATEGORY): lineFormat |= uiTools.getColor(CATEGORY_COLOR[entry.get(Field.CATEGORY)]) + if entry == cursorSelection: lineFormat |= curses.A_STANDOUT + + lineText = entry.getLabel(optionWidth, valueWidth, descriptionWidth) + self.addstr(drawLine, scrollOffset, lineText, lineFormat) + + if drawLine >= height: break + + self.valsLock.release() + + def _getConfigOptions(self): + return self.confContents if self.showAll else self.confImportantContents + + def _drawSelectionPanel(self, selection, width, detailPanelHeight, isScrollbarVisible): + """ + Renders a panel for the selected configuration option. + """ + + # This is a solid border unless the scrollbar is visible, in which case a + # 'T' pipe connects the border to the bar. + uiTools.drawBox(self, 0, 0, width, detailPanelHeight + 1) + if isScrollbarVisible: self.addch(detailPanelHeight, 1, curses.ACS_TTEE) + + selectionFormat = curses.A_BOLD | uiTools.getColor(CATEGORY_COLOR[selection.get(Field.CATEGORY)]) + + # first entry: + # <option> (<category> Option) + optionLabel =" (%s Option)" % selection.get(Field.CATEGORY) + self.addstr(1, 2, selection.get(Field.OPTION) + optionLabel, selectionFormat) + + # second entry: + # Value: <value> ([default|custom], <type>, usage: <argument usage>) + if detailPanelHeight >= 3: + valueAttr = [] + valueAttr.append("default" if selection.get(Field.IS_DEFAULT) else "custom") + valueAttr.append(selection.get(Field.TYPE)) + valueAttr.append("usage: %s" % (selection.get(Field.ARG_USAGE))) + valueAttrLabel = ", ".join(valueAttr) + + valueLabelWidth = width - 12 - len(valueAttrLabel) + valueLabel = uiTools.cropStr(selection.get(Field.VALUE), valueLabelWidth) + + self.addstr(2, 2, "Value: %s (%s)" % (valueLabel, valueAttrLabel), selectionFormat) + + # remainder is filled with the man page description + descriptionHeight = max(0, detailPanelHeight - 3) + descriptionContent = "Description: " + selection.get(Field.DESCRIPTION) + + for i in range(descriptionHeight): + # checks if we're done writing the description + if not descriptionContent: break + + # there's a leading indent after the first line + if i > 0: descriptionContent = " " + descriptionContent + + # we only want to work with content up until the next newline + if "\n" in descriptionContent: + lineContent, descriptionContent = descriptionContent.split("\n", 1) + else: lineContent, descriptionContent = descriptionContent, "" + + if i != descriptionHeight - 1: + # there's more lines to display + msg, remainder = uiTools.cropStr(lineContent, width - 2, 4, 4, uiTools.Ending.HYPHEN, True) + descriptionContent = remainder.strip() + descriptionContent + else: + # this is the last line, end it with an ellipse + msg = uiTools.cropStr(lineContent, width - 2, 4, 4) + + self.addstr(3 + i, 2, msg, selectionFormat) + diff --git a/src/cli/connections/__init__.py b/src/cli/connections/__init__.py new file mode 100644 index 0000000..5babdde --- /dev/null +++ b/src/cli/connections/__init__.py @@ -0,0 +1,6 @@ +""" +Connection panel related resources. +""" + +__all__ = ["circEntry", "connEntry", "connPanel", "entries"] + diff --git a/src/cli/connections/circEntry.py b/src/cli/connections/circEntry.py new file mode 100644 index 0000000..b15b26a --- /dev/null +++ b/src/cli/connections/circEntry.py @@ -0,0 +1,216 @@ +""" +Connection panel entries for client circuits. This includes a header entry +followed by an entry for each hop in the circuit. For instance: + +89.188.20.246:42667 --> 217.172.182.26 (de) General / Built 8.6m (CIRCUIT) +| 85.8.28.4 (se) 98FBC3B2B93897A78CDD797EF549E6B62C9A8523 1 / Guard +| 91.121.204.76 (fr) 546387D93F8D40CFF8842BB9D3A8EC477CEDA984 2 / Middle ++- 217.172.182.26 (de) 5CFA9EA136C0EA0AC096E5CEA7EB674F1207CF86 3 / Exit +""" + +import curses + +from cli.connections import entries, connEntry +from util import torTools, uiTools + +# cached fingerprint -> (IP Address, ORPort) results +RELAY_INFO = {} + +def getRelayInfo(fingerprint): + """ + Provides the (IP Address, ORPort) tuple for the given relay. If the lookup + fails then this returns ("192.168.0.1", "0"). + + Arguments: + fingerprint - relay to look up + """ + + if not fingerprint in RELAY_INFO: + conn = torTools.getConn() + failureResult = ("192.168.0.1", "0") + + nsEntry = conn.getConsensusEntry(fingerprint) + if not nsEntry: return failureResult + + nsLineComp = nsEntry.split("\n")[0].split(" ") + if len(nsLineComp) < 8: return failureResult + + RELAY_INFO[fingerprint] = (nsLineComp[6], nsLineComp[7]) + + return RELAY_INFO[fingerprint] + +class CircEntry(connEntry.ConnectionEntry): + def __init__(self, circuitID, status, purpose, path): + connEntry.ConnectionEntry.__init__(self, "127.0.0.1", "0", "127.0.0.1", "0") + + self.circuitID = circuitID + self.status = status + + # drops to lowercase except the first letter + if len(purpose) >= 2: + purpose = purpose[0].upper() + purpose[1:].lower() + + self.lines = [CircHeaderLine(self.circuitID, purpose)] + + # Overwrites attributes of the initial line to make it more fitting as the + # header for our listing. + + self.lines[0].baseType = connEntry.Category.CIRCUIT + + self.update(status, path) + + def update(self, status, path): + """ + Our status and path can change over time if the circuit is still in the + process of being built. Updates these attributes of our relay. + + Arguments: + status - new status of the circuit + path - list of fingerprints for the series of relays involved in the + circuit + """ + + self.status = status + self.lines = [self.lines[0]] + + if status == "BUILT" and not self.lines[0].isBuilt: + exitIp, exitORPort = getRelayInfo(path[-1]) + self.lines[0].setExit(exitIp, exitORPort, path[-1]) + + for i in range(len(path)): + relayFingerprint = path[i] + relayIp, relayOrPort = getRelayInfo(relayFingerprint) + + if i == len(path) - 1: + if status == "BUILT": placementType = "Exit" + else: placementType = "Extending" + elif i == 0: placementType = "Guard" + else: placementType = "Middle" + + placementLabel = "%i / %s" % (i + 1, placementType) + + self.lines.append(CircLine(relayIp, relayOrPort, relayFingerprint, placementLabel)) + + self.lines[-1].isLast = True + +class CircHeaderLine(connEntry.ConnectionLine): + """ + Initial line of a client entry. This has the same basic format as connection + lines except that its etc field has circuit attributes. + """ + + def __init__(self, circuitID, purpose): + connEntry.ConnectionLine.__init__(self, "127.0.0.1", "0", "0.0.0.0", "0", False, False) + self.circuitID = circuitID + self.purpose = purpose + self.isBuilt = False + + def setExit(self, exitIpAddr, exitPort, exitFingerprint): + connEntry.ConnectionLine.__init__(self, "127.0.0.1", "0", exitIpAddr, exitPort, False, False) + self.isBuilt = True + self.foreign.fingerprintOverwrite = exitFingerprint + + def getType(self): + return connEntry.Category.CIRCUIT + + def getDestinationLabel(self, maxLength, includeLocale=False, includeHostname=False): + if not self.isBuilt: return "Building..." + return connEntry.ConnectionLine.getDestinationLabel(self, maxLength, includeLocale, includeHostname) + + def getEtcContent(self, width, listingType): + """ + Attempts to provide all circuit related stats. Anything that can't be + shown completely (not enough room) is dropped. + """ + + etcAttr = ["Purpose: %s" % self.purpose, "Circuit ID: %i" % self.circuitID] + + for i in range(len(etcAttr), -1, -1): + etcLabel = ", ".join(etcAttr[:i]) + if len(etcLabel) <= width: + return ("%%-%is" % width) % etcLabel + + return "" + + def getDetails(self, width): + if not self.isBuilt: + detailFormat = curses.A_BOLD | uiTools.getColor(connEntry.CATEGORY_COLOR[self.getType()]) + return [uiTools.DrawEntry("Building Circuit...", detailFormat)] + else: return connEntry.ConnectionLine.getDetails(self, width) + +class CircLine(connEntry.ConnectionLine): + """ + An individual hop in a circuit. This overwrites the displayed listing, but + otherwise makes use of the ConnectionLine attributes (for the detail display, + caching, etc). + """ + + def __init__(self, fIpAddr, fPort, fFingerprint, placementLabel): + connEntry.ConnectionLine.__init__(self, "127.0.0.1", "0", fIpAddr, fPort) + self.foreign.fingerprintOverwrite = fFingerprint + self.placementLabel = placementLabel + self.includePort = False + + # determines the sort of left hand bracketing we use + self.isLast = False + + def getType(self): + return connEntry.Category.CIRCUIT + + def getListingEntry(self, width, currentTime, listingType): + """ + Provides the DrawEntry for this relay in the circuilt listing. Lines are + composed of the following components: + <bracket> <dst> <etc> <placement label> + + The dst and etc entries largely match their ConnectionEntry counterparts. + + Arguments: + width - maximum length of the line + currentTime - the current unix time (ignored) + listingType - primary attribute we're listing connections by + """ + + return entries.ConnectionPanelLine.getListingEntry(self, width, currentTime, listingType) + + def _getListingEntry(self, width, currentTime, listingType): + lineFormat = uiTools.getColor(connEntry.CATEGORY_COLOR[self.getType()]) + + # The required widths are the sum of the following: + # bracketing (3 characters) + # placementLabel (14 characters) + # gap between etc and placement label (5 characters) + + if self.isLast: bracket = (curses.ACS_LLCORNER, curses.ACS_HLINE, ord(' ')) + else: bracket = (curses.ACS_VLINE, ord(' '), ord(' ')) + baselineSpace = len(bracket) + 14 + 5 + + dst, etc = "", "" + if listingType == entries.ListingType.IP_ADDRESS: + # TODO: include hostname when that's available + # dst width is derived as: + # src (21) + dst (26) + divider (7) + right gap (2) - bracket (3) = 53 char + dst = "%-53s" % self.getDestinationLabel(53, includeLocale = True) + etc = self.getEtcContent(width - baselineSpace - len(dst), listingType) + elif listingType == entries.ListingType.HOSTNAME: + # min space for the hostname is 40 characters + etc = self.getEtcContent(width - baselineSpace - 40, listingType) + dstLayout = "%%-%is" % (width - baselineSpace - len(etc)) + dst = dstLayout % self.foreign.getHostname(self.foreign.getIpAddr()) + elif listingType == entries.ListingType.FINGERPRINT: + # dst width is derived as: + # src (9) + dst (40) + divider (7) + right gap (2) - bracket (3) = 55 char + dst = "%-55s" % self.foreign.getFingerprint() + etc = self.getEtcContent(width - baselineSpace - len(dst), listingType) + else: + # min space for the nickname is 56 characters + etc = self.getEtcContent(width - baselineSpace - 56, listingType) + dstLayout = "%%-%is" % (width - baselineSpace - len(etc)) + dst = dstLayout % self.foreign.getNickname() + + drawEntry = uiTools.DrawEntry("%-14s" % self.placementLabel, lineFormat) + drawEntry = uiTools.DrawEntry(" " * (width - baselineSpace - len(dst) - len(etc) + 5), lineFormat, drawEntry) + drawEntry = uiTools.DrawEntry(dst + etc, lineFormat, drawEntry) + drawEntry = uiTools.DrawEntry(bracket, curses.A_NORMAL, drawEntry, lockFormat = True) + return drawEntry + diff --git a/src/cli/connections/connEntry.py b/src/cli/connections/connEntry.py new file mode 100644 index 0000000..ac45656 --- /dev/null +++ b/src/cli/connections/connEntry.py @@ -0,0 +1,850 @@ +""" +Connection panel entries related to actual connections to or from the system +(ie, results seen by netstat, lsof, etc). +""" + +import time +import curses + +from util import connections, enum, torTools, uiTools +from cli.connections import entries + +# Connection Categories: +# Inbound Relay connection, coming to us. +# Outbound Relay connection, leaving us. +# Exit Outbound relay connection leaving the Tor network. +# Hidden Connections to a hidden service we're providing. +# Socks Socks connections for applications using Tor. +# Circuit Circuits our tor client has created. +# Directory Fetching tor consensus information. +# Control Tor controller (arm, vidalia, etc). + +Category = enum.Enum("INBOUND", "OUTBOUND", "EXIT", "HIDDEN", "SOCKS", "CIRCUIT", "DIRECTORY", "CONTROL") +CATEGORY_COLOR = {Category.INBOUND: "green", Category.OUTBOUND: "blue", + Category.EXIT: "red", Category.HIDDEN: "magenta", + Category.SOCKS: "yellow", Category.CIRCUIT: "cyan", + Category.DIRECTORY: "magenta", Category.CONTROL: "red"} + +# static data for listing format +# <src> --> <dst> <etc><padding> +LABEL_FORMAT = "%s --> %s %s%s" +LABEL_MIN_PADDING = 2 # min space between listing label and following data + +# sort value for scrubbed ip addresses +SCRUBBED_IP_VAL = 255 ** 4 + +CONFIG = {"features.connection.markInitialConnections": True, + "features.connection.showExitPort": True, + "features.connection.showColumn.fingerprint": True, + "features.connection.showColumn.nickname": True, + "features.connection.showColumn.destination": True, + "features.connection.showColumn.expandedIp": True} + +def loadConfig(config): + config.update(CONFIG) + +class Endpoint: + """ + Collection of attributes associated with a connection endpoint. This is a + thin wrapper for torUtil functions, making use of its caching for + performance. + """ + + def __init__(self, ipAddr, port): + self.ipAddr = ipAddr + self.port = port + + # if true, we treat the port as an ORPort when searching for matching + # fingerprints (otherwise the ORPort is assumed to be unknown) + self.isORPort = False + + # if set then this overwrites fingerprint lookups + self.fingerprintOverwrite = None + + def getIpAddr(self): + """ + Provides the IP address of the endpoint. + """ + + return self.ipAddr + + def getPort(self): + """ + Provides the port of the endpoint. + """ + + return self.port + + def getHostname(self, default = None): + """ + Provides the hostname associated with the relay's address. This is a + non-blocking call and returns None if the address either can't be resolved + or hasn't been resolved yet. + + Arguments: + default - return value if no hostname is available + """ + + # TODO: skipping all hostname resolution to be safe for now + #try: + # myHostname = hostnames.resolve(self.ipAddr) + #except: + # # either a ValueError or IOError depending on the source of the lookup failure + # myHostname = None + # + #if not myHostname: return default + #else: return myHostname + + return default + + def getLocale(self, default=None): + """ + Provides the two letter country code for the IP address' locale. + + Arguments: + default - return value if no locale information is available + """ + + conn = torTools.getConn() + return conn.getInfo("ip-to-country/%s" % self.ipAddr, default) + + def getFingerprint(self): + """ + Provides the fingerprint of the relay, returning "UNKNOWN" if it can't be + determined. + """ + + if self.fingerprintOverwrite: + return self.fingerprintOverwrite + + conn = torTools.getConn() + orPort = self.port if self.isORPort else None + myFingerprint = conn.getRelayFingerprint(self.ipAddr, orPort) + + if myFingerprint: return myFingerprint + else: return "UNKNOWN" + + def getNickname(self): + """ + Provides the nickname of the relay, retuning "UNKNOWN" if it can't be + determined. + """ + + myFingerprint = self.getFingerprint() + + if myFingerprint != "UNKNOWN": + conn = torTools.getConn() + myNickname = conn.getRelayNickname(myFingerprint) + + if myNickname: return myNickname + else: return "UNKNOWN" + else: return "UNKNOWN" + +class ConnectionEntry(entries.ConnectionPanelEntry): + """ + Represents a connection being made to or from this system. These only + concern real connections so it includes the inbound, outbound, directory, + application, and controller categories. + """ + + def __init__(self, lIpAddr, lPort, fIpAddr, fPort): + entries.ConnectionPanelEntry.__init__(self) + self.lines = [ConnectionLine(lIpAddr, lPort, fIpAddr, fPort)] + + def getSortValue(self, attr, listingType): + """ + Provides the value of a single attribute used for sorting purposes. + """ + + connLine = self.lines[0] + if attr == entries.SortAttr.IP_ADDRESS: + if connLine.isPrivate(): return SCRUBBED_IP_VAL # orders at the end + return connLine.sortIpAddr + elif attr == entries.SortAttr.PORT: + return connLine.sortPort + elif attr == entries.SortAttr.HOSTNAME: + if connLine.isPrivate(): return "" + return connLine.foreign.getHostname("") + elif attr == entries.SortAttr.FINGERPRINT: + return connLine.foreign.getFingerprint() + elif attr == entries.SortAttr.NICKNAME: + myNickname = connLine.foreign.getNickname() + if myNickname == "UNKNOWN": return "z" * 20 # orders at the end + else: return myNickname.lower() + elif attr == entries.SortAttr.CATEGORY: + return Category.indexOf(connLine.getType()) + elif attr == entries.SortAttr.UPTIME: + return connLine.startTime + elif attr == entries.SortAttr.COUNTRY: + if connections.isIpAddressPrivate(self.lines[0].foreign.getIpAddr()): return "" + else: return connLine.foreign.getLocale("") + else: + return entries.ConnectionPanelEntry.getSortValue(self, attr, listingType) + +class ConnectionLine(entries.ConnectionPanelLine): + """ + Display component of the ConnectionEntry. + """ + + def __init__(self, lIpAddr, lPort, fIpAddr, fPort, includePort=True, includeExpandedIpAddr=True): + entries.ConnectionPanelLine.__init__(self) + + self.local = Endpoint(lIpAddr, lPort) + self.foreign = Endpoint(fIpAddr, fPort) + self.startTime = time.time() + self.isInitialConnection = False + + # overwrite the local fingerprint with ours + conn = torTools.getConn() + self.local.fingerprintOverwrite = conn.getInfo("fingerprint") + + # True if the connection has matched the properties of a client/directory + # connection every time we've checked. The criteria we check is... + # client - first hop in an established circuit + # directory - matches an established single-hop circuit (probably a + # directory mirror) + + self._possibleClient = True + self._possibleDirectory = True + + # attributes for SOCKS, HIDDEN, and CONTROL connections + self.appName = None + self.appPid = None + self.isAppResolving = False + + myOrPort = conn.getOption("ORPort") + myDirPort = conn.getOption("DirPort") + mySocksPort = conn.getOption("SocksPort", "9050") + myCtlPort = conn.getOption("ControlPort") + myHiddenServicePorts = conn.getHiddenServicePorts() + + # the ORListenAddress can overwrite the ORPort + listenAddr = conn.getOption("ORListenAddress") + if listenAddr and ":" in listenAddr: + myOrPort = listenAddr[listenAddr.find(":") + 1:] + + if lPort in (myOrPort, myDirPort): + self.baseType = Category.INBOUND + self.local.isORPort = True + elif lPort == mySocksPort: + self.baseType = Category.SOCKS + elif fPort in myHiddenServicePorts: + self.baseType = Category.HIDDEN + elif lPort == myCtlPort: + self.baseType = Category.CONTROL + else: + self.baseType = Category.OUTBOUND + self.foreign.isORPort = True + + self.cachedType = None + + # includes the port or expanded ip address field when displaying listing + # information if true + self.includePort = includePort + self.includeExpandedIpAddr = includeExpandedIpAddr + + # cached immutable values used for sorting + self.sortIpAddr = connections.ipToInt(self.foreign.getIpAddr()) + self.sortPort = int(self.foreign.getPort()) + + def getListingEntry(self, width, currentTime, listingType): + """ + Provides the DrawEntry for this connection's listing. Lines are composed + of the following components: + <src> --> <dst> <etc> <uptime> (<type>) + + ListingType.IP_ADDRESS: + src - <internal addr:port> --> <external addr:port> + dst - <destination addr:port> + etc - <fingerprint> <nickname> + + ListingType.HOSTNAME: + src - localhost:<port> + dst - <destination hostname:port> + etc - <destination addr:port> <fingerprint> <nickname> + + ListingType.FINGERPRINT: + src - localhost + dst - <destination fingerprint> + etc - <nickname> <destination addr:port> + + ListingType.NICKNAME: + src - <source nickname> + dst - <destination nickname> + etc - <fingerprint> <destination addr:port> + + Arguments: + width - maximum length of the line + currentTime - unix timestamp for what the results should consider to be + the current time + listingType - primary attribute we're listing connections by + """ + + # fetch our (most likely cached) display entry for the listing + myListing = entries.ConnectionPanelLine.getListingEntry(self, width, currentTime, listingType) + + # fill in the current uptime and return the results + if CONFIG["features.connection.markInitialConnections"]: + timePrefix = "+" if self.isInitialConnection else " " + else: timePrefix = "" + + timeEntry = myListing.getNext() + timeEntry.text = timePrefix + "%5s" % uiTools.getTimeLabel(currentTime - self.startTime, 1) + + return myListing + + def isUnresolvedApp(self): + """ + True if our display uses application information that hasn't yet been resolved. + """ + + return self.appName == None and self.getType() in (Category.SOCKS, Category.HIDDEN, Category.CONTROL) + + def _getListingEntry(self, width, currentTime, listingType): + entryType = self.getType() + + # Lines are split into the following components in reverse: + # content - "<src> --> <dst> <etc> " + # time - "<uptime>" + # preType - " (" + # category - "<type>" + # postType - ") " + + lineFormat = uiTools.getColor(CATEGORY_COLOR[entryType]) + timeWidth = 6 if CONFIG["features.connection.markInitialConnections"] else 5 + + drawEntry = uiTools.DrawEntry(")" + " " * (9 - len(entryType)), lineFormat) + drawEntry = uiTools.DrawEntry(entryType.upper(), lineFormat | curses.A_BOLD, drawEntry) + drawEntry = uiTools.DrawEntry(" (", lineFormat, drawEntry) + drawEntry = uiTools.DrawEntry(" " * timeWidth, lineFormat, drawEntry) + drawEntry = uiTools.DrawEntry(self._getListingContent(width - (12 + timeWidth), listingType), lineFormat, drawEntry) + return drawEntry + + def _getDetails(self, width): + """ + Provides details on the connection, correlated against available consensus + data. + + Arguments: + width - available space to display in + """ + + detailFormat = curses.A_BOLD | uiTools.getColor(CATEGORY_COLOR[self.getType()]) + return [uiTools.DrawEntry(line, detailFormat) for line in self._getDetailContent(width)] + + def resetDisplay(self): + entries.ConnectionPanelLine.resetDisplay(self) + self.cachedType = None + + def isPrivate(self): + """ + Returns true if the endpoint is private, possibly belonging to a client + connection or exit traffic. + """ + + # This is used to scrub private information from the interface. Relaying + # etiquette (and wiretapping laws) say these are bad things to look at so + # DON'T CHANGE THIS UNLESS YOU HAVE A DAMN GOOD REASON! + + myType = self.getType() + + if myType == Category.INBOUND: + # if we're a guard or bridge and the connection doesn't belong to a + # known relay then it might be client traffic + + conn = torTools.getConn() + if "Guard" in conn.getMyFlags([]) or conn.getOption("BridgeRelay") == "1": + allMatches = conn.getRelayFingerprint(self.foreign.getIpAddr(), getAllMatches = True) + return allMatches == [] + elif myType == Category.EXIT: + # DNS connections exiting us aren't private (since they're hitting our + # resolvers). Everything else, however, is. + + # TODO: Ideally this would also double check that it's a UDP connection + # (since DNS is the only UDP connections Tor will relay), however this + # will take a bit more work to propagate the information up from the + # connection resolver. + return self.foreign.getPort() != "53" + + # for everything else this isn't a concern + return False + + def getType(self): + """ + Provides our best guess at the current type of the connection. This + depends on consensus results, our current client circuits, etc. Results + are cached until this entry's display is reset. + """ + + # caches both to simplify the calls and to keep the type consistent until + # we want to reflect changes + if not self.cachedType: + if self.baseType == Category.OUTBOUND: + # Currently the only non-static categories are OUTBOUND vs... + # - EXIT since this depends on the current consensus + # - CIRCUIT if this is likely to belong to our guard usage + # - DIRECTORY if this is a single-hop circuit (directory mirror?) + # + # The exitability, circuits, and fingerprints are all cached by the + # torTools util keeping this a quick lookup. + + conn = torTools.getConn() + destFingerprint = self.foreign.getFingerprint() + + if destFingerprint == "UNKNOWN": + # Not a known relay. This might be an exit connection. + + if conn.isExitingAllowed(self.foreign.getIpAddr(), self.foreign.getPort()): + self.cachedType = Category.EXIT + elif self._possibleClient or self._possibleDirectory: + # This belongs to a known relay. If we haven't eliminated ourselves as + # a possible client or directory connection then check if it still + # holds true. + + myCircuits = conn.getCircuits() + + if self._possibleClient: + # Checks that this belongs to the first hop in a circuit that's + # either unestablished or longer than a single hop (ie, anything but + # a built 1-hop connection since those are most likely a directory + # mirror). + + for _, status, _, path in myCircuits: + if path[0] == destFingerprint and (status != "BUILT" or len(path) > 1): + self.cachedType = Category.CIRCUIT # matched a probable guard connection + + # if we fell through, we can eliminate ourselves as a guard in the future + if not self.cachedType: + self._possibleClient = False + + if self._possibleDirectory: + # Checks if we match a built, single hop circuit. + + for _, status, _, path in myCircuits: + if path[0] == destFingerprint and status == "BUILT" and len(path) == 1: + self.cachedType = Category.DIRECTORY + + # if we fell through, eliminate ourselves as a directory connection + if not self.cachedType: + self._possibleDirectory = False + + if not self.cachedType: + self.cachedType = self.baseType + + return self.cachedType + + def getEtcContent(self, width, listingType): + """ + Provides the optional content for the connection. + + Arguments: + width - maximum length of the line + listingType - primary attribute we're listing connections by + """ + + # for applications show the command/pid + if self.getType() in (Category.SOCKS, Category.HIDDEN, Category.CONTROL): + displayLabel = "" + + if self.appName: + if self.appPid: displayLabel = "%s (%s)" % (self.appName, self.appPid) + else: displayLabel = self.appName + elif self.isAppResolving: + displayLabel = "resolving..." + else: displayLabel = "UNKNOWN" + + if len(displayLabel) < width: + return ("%%-%is" % width) % displayLabel + else: return "" + + # for everything else display connection/consensus information + dstAddress = self.getDestinationLabel(26, includeLocale = True) + etc, usedSpace = "", 0 + if listingType == entries.ListingType.IP_ADDRESS: + if width > usedSpace + 42 and CONFIG["features.connection.showColumn.fingerprint"]: + # show fingerprint (column width: 42 characters) + etc += "%-40s " % self.foreign.getFingerprint() + usedSpace += 42 + + if width > usedSpace + 10 and CONFIG["features.connection.showColumn.nickname"]: + # show nickname (column width: remainder) + nicknameSpace = width - usedSpace + nicknameLabel = uiTools.cropStr(self.foreign.getNickname(), nicknameSpace, 0) + etc += ("%%-%is " % nicknameSpace) % nicknameLabel + usedSpace += nicknameSpace + 2 + elif listingType == entries.ListingType.HOSTNAME: + if width > usedSpace + 28 and CONFIG["features.connection.showColumn.destination"]: + # show destination ip/port/locale (column width: 28 characters) + etc += "%-26s " % dstAddress + usedSpace += 28 + + if width > usedSpace + 42 and CONFIG["features.connection.showColumn.fingerprint"]: + # show fingerprint (column width: 42 characters) + etc += "%-40s " % self.foreign.getFingerprint() + usedSpace += 42 + + if width > usedSpace + 17 and CONFIG["features.connection.showColumn.nickname"]: + # show nickname (column width: min 17 characters, uses half of the remainder) + nicknameSpace = 15 + (width - (usedSpace + 17)) / 2 + nicknameLabel = uiTools.cropStr(self.foreign.getNickname(), nicknameSpace, 0) + etc += ("%%-%is " % nicknameSpace) % nicknameLabel + usedSpace += (nicknameSpace + 2) + elif listingType == entries.ListingType.FINGERPRINT: + if width > usedSpace + 17: + # show nickname (column width: min 17 characters, consumes any remaining space) + nicknameSpace = width - usedSpace - 2 + + # if there's room then also show a column with the destination + # ip/port/locale (column width: 28 characters) + isIpLocaleIncluded = width > usedSpace + 45 + isIpLocaleIncluded &= CONFIG["features.connection.showColumn.destination"] + if isIpLocaleIncluded: nicknameSpace -= 28 + + if CONFIG["features.connection.showColumn.nickname"]: + nicknameLabel = uiTools.cropStr(self.foreign.getNickname(), nicknameSpace, 0) + etc += ("%%-%is " % nicknameSpace) % nicknameLabel + usedSpace += nicknameSpace + 2 + + if isIpLocaleIncluded: + etc += "%-26s " % dstAddress + usedSpace += 28 + else: + if width > usedSpace + 42 and CONFIG["features.connection.showColumn.fingerprint"]: + # show fingerprint (column width: 42 characters) + etc += "%-40s " % self.foreign.getFingerprint() + usedSpace += 42 + + if width > usedSpace + 28 and CONFIG["features.connection.showColumn.destination"]: + # show destination ip/port/locale (column width: 28 characters) + etc += "%-26s " % dstAddress + usedSpace += 28 + + return ("%%-%is" % width) % etc + + def _getListingContent(self, width, listingType): + """ + Provides the source, destination, and extra info for our listing. + + Arguments: + width - maximum length of the line + listingType - primary attribute we're listing connections by + """ + + conn = torTools.getConn() + myType = self.getType() + dstAddress = self.getDestinationLabel(26, includeLocale = True) + + # The required widths are the sum of the following: + # - room for LABEL_FORMAT and LABEL_MIN_PADDING (11 characters) + # - base data for the listing + # - that extra field plus any previous + + usedSpace = len(LABEL_FORMAT % tuple([""] * 4)) + LABEL_MIN_PADDING + localPort = ":%s" % self.local.getPort() if self.includePort else "" + + src, dst, etc = "", "", "" + if listingType == entries.ListingType.IP_ADDRESS: + myExternalIpAddr = conn.getInfo("address", self.local.getIpAddr()) + addrDiffer = myExternalIpAddr != self.local.getIpAddr() + + # Expanding doesn't make sense, if the connection isn't actually + # going through Tor's external IP address. As there isn't a known + # method for checking if it is, we're checking the type instead. + # + # This isn't entirely correct. It might be a better idea to check if + # the source and destination addresses are both private, but that might + # not be perfectly reliable either. + + isExpansionType = not myType in (Category.SOCKS, Category.HIDDEN, Category.CONTROL) + + if isExpansionType: srcAddress = myExternalIpAddr + localPort + else: srcAddress = self.local.getIpAddr() + localPort + + if myType in (Category.SOCKS, Category.CONTROL): + # Like inbound connections these need their source and destination to + # be swapped. However, this only applies when listing by IP or hostname + # (their fingerprint and nickname are both for us). Reversing the + # fields here to keep the same column alignments. + + src = "%-21s" % dstAddress + dst = "%-26s" % srcAddress + else: + src = "%-21s" % srcAddress # ip:port = max of 21 characters + dst = "%-26s" % dstAddress # ip:port (xx) = max of 26 characters + + usedSpace += len(src) + len(dst) # base data requires 47 characters + + # Showing the fingerprint (which has the width of 42) has priority over + # an expanded address field. Hence check if we either have space for + # both or wouldn't be showing the fingerprint regardless. + + isExpandedAddrVisible = width > usedSpace + 28 + if isExpandedAddrVisible and CONFIG["features.connection.showColumn.fingerprint"]: + isExpandedAddrVisible = width < usedSpace + 42 or width > usedSpace + 70 + + if addrDiffer and isExpansionType and isExpandedAddrVisible and self.includeExpandedIpAddr and CONFIG["features.connection.showColumn.expandedIp"]: + # include the internal address in the src (extra 28 characters) + internalAddress = self.local.getIpAddr() + localPort + + # If this is an inbound connection then reverse ordering so it's: + # <foreign> --> <external> --> <internal> + # when the src and dst are swapped later + + if myType == Category.INBOUND: src = "%-21s --> %s" % (src, internalAddress) + else: src = "%-21s --> %s" % (internalAddress, src) + + usedSpace += 28 + + etc = self.getEtcContent(width - usedSpace, listingType) + usedSpace += len(etc) + elif listingType == entries.ListingType.HOSTNAME: + # 15 characters for source, and a min of 40 reserved for the destination + # TODO: when actually functional the src and dst need to be swapped for + # SOCKS and CONTROL connections + src = "localhost%-6s" % localPort + usedSpace += len(src) + minHostnameSpace = 40 + + etc = self.getEtcContent(width - usedSpace - minHostnameSpace, listingType) + usedSpace += len(etc) + + hostnameSpace = width - usedSpace + usedSpace = width # prevents padding at the end + if self.isPrivate(): + dst = ("%%-%is" % hostnameSpace) % "<scrubbed>" + else: + hostname = self.foreign.getHostname(self.foreign.getIpAddr()) + portLabel = ":%-5s" % self.foreign.getPort() if self.includePort else "" + + # truncates long hostnames and sets dst to <hostname>:<port> + hostname = uiTools.cropStr(hostname, hostnameSpace, 0) + dst = ("%%-%is" % hostnameSpace) % (hostname + portLabel) + elif listingType == entries.ListingType.FINGERPRINT: + src = "localhost" + if myType == Category.CONTROL: dst = "localhost" + else: dst = self.foreign.getFingerprint() + dst = "%-40s" % dst + + usedSpace += len(src) + len(dst) # base data requires 49 characters + + etc = self.getEtcContent(width - usedSpace, listingType) + usedSpace += len(etc) + else: + # base data requires 50 min characters + src = self.local.getNickname() + if myType == Category.CONTROL: dst = self.local.getNickname() + else: dst = self.foreign.getNickname() + minBaseSpace = 50 + + etc = self.getEtcContent(width - usedSpace - minBaseSpace, listingType) + usedSpace += len(etc) + + baseSpace = width - usedSpace + usedSpace = width # prevents padding at the end + + if len(src) + len(dst) > baseSpace: + src = uiTools.cropStr(src, baseSpace / 3) + dst = uiTools.cropStr(dst, baseSpace - len(src)) + + # pads dst entry to its max space + dst = ("%%-%is" % (baseSpace - len(src))) % dst + + if myType == Category.INBOUND: src, dst = dst, src + padding = " " * (width - usedSpace + LABEL_MIN_PADDING) + return LABEL_FORMAT % (src, dst, etc, padding) + + def _getDetailContent(self, width): + """ + Provides a list with detailed information for this connection. + + Arguments: + width - max length of lines + """ + + lines = [""] * 7 + lines[0] = "address: %s" % self.getDestinationLabel(width - 11) + lines[1] = "locale: %s" % ("??" if self.isPrivate() else self.foreign.getLocale("??")) + + # Remaining data concerns the consensus results, with three possible cases: + # - if there's a single match then display its details + # - if there's multiple potential relays then list all of the combinations + # of ORPorts / Fingerprints + # - if no consensus data is available then say so (probably a client or + # exit connection) + + fingerprint = self.foreign.getFingerprint() + conn = torTools.getConn() + + if fingerprint != "UNKNOWN": + # single match - display information available about it + nsEntry = conn.getConsensusEntry(fingerprint) + descEntry = conn.getDescriptorEntry(fingerprint) + + # append the fingerprint to the second line + lines[1] = "%-13sfingerprint: %s" % (lines[1], fingerprint) + + if nsEntry: + # example consensus entry: + # r murble R8sCM1ar1sS2GulQYFVmvN95xsk RJr6q+wkTFG+ng5v2bdCbVVFfA4 2011-02-21 00:25:32 195.43.157.85 443 0 + # s Exit Fast Guard Named Running Stable Valid + # w Bandwidth=2540 + # p accept 20-23,43,53,79-81,88,110,143,194,443 + + nsLines = nsEntry.split("\n") + + firstLineComp = nsLines[0].split(" ") + if len(firstLineComp) >= 9: + _, nickname, _, _, pubDate, pubTime, _, orPort, dirPort = firstLineComp[:9] + else: nickname, pubDate, pubTime, orPort, dirPort = "", "", "", "", "" + + flags = "unknown" + if len(nsLines) >= 2 and nsLines[1].startswith("s "): + flags = nsLines[1][2:] + + # The network status exit policy doesn't exist for older tor versions. + # If unavailable we'll need the full exit policy which is on the + # descriptor (if that's available). + + exitPolicy = "unknown" + if len(nsLines) >= 4 and nsLines[3].startswith("p "): + exitPolicy = nsLines[3][2:].replace(",", ", ") + elif descEntry: + # the descriptor has an individual line for each entry in the exit policy + exitPolicyEntries = [] + + for line in descEntry.split("\n"): + if line.startswith("accept") or line.startswith("reject"): + exitPolicyEntries.append(line.strip()) + + exitPolicy = ", ".join(exitPolicyEntries) + + dirPortLabel = "" if dirPort == "0" else "dirport: %s" % dirPort + lines[2] = "nickname: %-25s orport: %-10s %s" % (nickname, orPort, dirPortLabel) + lines[3] = "published: %s %s" % (pubDate, pubTime) + lines[4] = "flags: %s" % flags.replace(" ", ", ") + lines[5] = "exit policy: %s" % exitPolicy + + if descEntry: + torVersion, platform, contact = "", "", "" + + for descLine in descEntry.split("\n"): + if descLine.startswith("platform"): + # has the tor version and platform, ex: + # platform Tor 0.2.1.29 (r318f470bc5f2ad43) on Linux x86_64 + + torVersion = descLine[13:descLine.find(" ", 13)] + platform = descLine[descLine.rfind(" on ") + 4:] + elif descLine.startswith("contact"): + contact = descLine[8:] + + # clears up some highly common obscuring + for alias in (" at ", " AT "): contact = contact.replace(alias, "@") + for alias in (" dot ", " DOT "): contact = contact.replace(alias, ".") + + break # contact lines come after the platform + + lines[3] = "%-35s os: %-14s version: %s" % (lines[3], platform, torVersion) + + # contact information is an optional field + if contact: lines[6] = "contact: %s" % contact + else: + allMatches = conn.getRelayFingerprint(self.foreign.getIpAddr(), getAllMatches = True) + + if allMatches: + # multiple matches + lines[2] = "Multiple matches, possible fingerprints are:" + + for i in range(len(allMatches)): + isLastLine = i == 3 + + relayPort, relayFingerprint = allMatches[i] + lineText = "%i. or port: %-5s fingerprint: %s" % (i, relayPort, relayFingerprint) + + # if there's multiple lines remaining at the end then give a count + remainingRelays = len(allMatches) - i + if isLastLine and remainingRelays > 1: + lineText = "... %i more" % remainingRelays + + lines[3 + i] = lineText + + if isLastLine: break + else: + # no consensus entry for this ip address + lines[2] = "No consensus data found" + + # crops any lines that are too long + for i in range(len(lines)): + lines[i] = uiTools.cropStr(lines[i], width - 2) + + return lines + + def getDestinationLabel(self, maxLength, includeLocale=False, includeHostname=False): + """ + Provides a short description of the destination. This is made up of two + components, the base <ip addr>:<port> and an extra piece of information in + parentheses. The IP address is scrubbed from private connections. + + Extra information is... + - the port's purpose for exit connections + - the locale and/or hostname if set to do so, the address isn't private, + and isn't on the local network + - nothing otherwise + + Arguments: + maxLength - maximum length of the string returned + includeLocale - possibly includes the locale + includeHostname - possibly includes the hostname + """ + + # the port and port derived data can be hidden by config or without includePort + includePort = self.includePort and (CONFIG["features.connection.showExitPort"] or self.getType() != Category.EXIT) + + # destination of the connection + ipLabel = "<scrubbed>" if self.isPrivate() else self.foreign.getIpAddr() + portLabel = ":%s" % self.foreign.getPort() if includePort else "" + dstAddress = ipLabel + portLabel + + # Only append the extra info if there's at least a couple characters of + # space (this is what's needed for the country codes). + if len(dstAddress) + 5 <= maxLength: + spaceAvailable = maxLength - len(dstAddress) - 3 + + if self.getType() == Category.EXIT and includePort: + purpose = connections.getPortUsage(self.foreign.getPort()) + + if purpose: + # BitTorrent is a common protocol to truncate, so just use "Torrent" + # if there's not enough room. + if len(purpose) > spaceAvailable and purpose == "BitTorrent": + purpose = "Torrent" + + # crops with a hyphen if too long + purpose = uiTools.cropStr(purpose, spaceAvailable, endType = uiTools.Ending.HYPHEN) + + dstAddress += " (%s)" % purpose + elif not connections.isIpAddressPrivate(self.foreign.getIpAddr()): + extraInfo = [] + conn = torTools.getConn() + + if includeLocale and not conn.isGeoipUnavailable(): + foreignLocale = self.foreign.getLocale("??") + extraInfo.append(foreignLocale) + spaceAvailable -= len(foreignLocale) + 2 + + if includeHostname: + dstHostname = self.foreign.getHostname() + + if dstHostname: + # determines the full space available, taking into account the ", " + # dividers if there's multiple pieces of extra data + + maxHostnameSpace = spaceAvailable - 2 * len(extraInfo) + dstHostname = uiTools.cropStr(dstHostname, maxHostnameSpace) + extraInfo.append(dstHostname) + spaceAvailable -= len(dstHostname) + + if extraInfo: + dstAddress += " (%s)" % ", ".join(extraInfo) + + return dstAddress[:maxLength] + diff --git a/src/cli/connections/connPanel.py b/src/cli/connections/connPanel.py new file mode 100644 index 0000000..569f57c --- /dev/null +++ b/src/cli/connections/connPanel.py @@ -0,0 +1,398 @@ +""" +Listing of the currently established connections tor has made. +""" + +import time +import curses +import threading + +from cli.connections import entries, connEntry, circEntry +from util import connections, enum, panel, torTools, uiTools + +DEFAULT_CONFIG = {"features.connection.resolveApps": True, + "features.connection.listingType": 0, + "features.connection.refreshRate": 5} + +# height of the detail panel content, not counting top and bottom border +DETAILS_HEIGHT = 7 + +# listing types +Listing = enum.Enum(("IP_ADDRESS", "IP Address"), "HOSTNAME", "FINGERPRINT", "NICKNAME") + +DEFAULT_SORT_ORDER = (entries.SortAttr.CATEGORY, entries.SortAttr.LISTING, entries.SortAttr.UPTIME) + +class ConnectionPanel(panel.Panel, threading.Thread): + """ + Listing of connections tor is making, with information correlated against + the current consensus and other data sources. + """ + + def __init__(self, stdscr, config=None): + panel.Panel.__init__(self, stdscr, "conn", 0) + threading.Thread.__init__(self) + self.setDaemon(True) + + self._sortOrdering = DEFAULT_SORT_ORDER + self._config = dict(DEFAULT_CONFIG) + + if config: + config.update(self._config, { + "features.connection.listingType": (0, len(Listing.values()) - 1), + "features.connection.refreshRate": 1}) + + sortFields = entries.SortAttr.values() + customOrdering = config.getIntCSV("features.connection.order", None, 3, 0, len(sortFields)) + + if customOrdering: + self._sortOrdering = [sortFields[i] for i in customOrdering] + + self._listingType = Listing.values()[self._config["features.connection.listingType"]] + self._scroller = uiTools.Scroller(True) + self._title = "Connections:" # title line of the panel + self._entries = [] # last fetched display entries + self._entryLines = [] # individual lines rendered from the entries listing + self._showDetails = False # presents the details panel if true + + self._lastUpdate = -1 # time the content was last revised + self._isTorRunning = True # indicates if tor is currently running or not + self._isPaused = True # prevents updates if true + self._pauseTime = None # time when the panel was paused + self._halt = False # terminates thread if true + self._cond = threading.Condition() # used for pausing the thread + self.valsLock = threading.RLock() + + # Last sampling received from the ConnectionResolver, used to detect when + # it changes. + self._lastResourceFetch = -1 + + # resolver for the command/pid associated with SOCKS, HIDDEN, and CONTROL connections + self._appResolver = connections.AppResolver("arm") + + # rate limits appResolver queries to once per update + self.appResolveSinceUpdate = False + + self._update() # populates initial entries + self._resolveApps(False) # resolves initial applications + + # mark the initially exitsing connection uptimes as being estimates + for entry in self._entries: + if isinstance(entry, connEntry.ConnectionEntry): + entry.getLines()[0].isInitialConnection = True + + # listens for when tor stops so we know to stop reflecting changes + torTools.getConn().addStatusListener(self.torStateListener) + + def torStateListener(self, conn, eventType): + """ + Freezes the connection contents when Tor stops. + + Arguments: + conn - tor controller + eventType - type of event detected + """ + + self._isTorRunning = eventType == torTools.State.INIT + + if self._isPaused or not self._isTorRunning: + if not self._pauseTime: self._pauseTime = time.time() + else: self._pauseTime = None + + self.redraw(True) + + def setPaused(self, isPause): + """ + If true, prevents the panel from updating. + """ + + if not self._isPaused == isPause: + self._isPaused = isPause + + if isPause or not self._isTorRunning: + if not self._pauseTime: self._pauseTime = time.time() + else: self._pauseTime = None + + # redraws so the display reflects any changes between the last update + # and being paused + self.redraw(True) + + def setSortOrder(self, ordering = None): + """ + Sets the connection attributes we're sorting by and resorts the contents. + + Arguments: + ordering - new ordering, if undefined then this resorts with the last + set ordering + """ + + self.valsLock.acquire() + if ordering: self._sortOrdering = ordering + self._entries.sort(key=lambda i: (i.getSortValues(self._sortOrdering, self._listingType))) + + self._entryLines = [] + for entry in self._entries: + self._entryLines += entry.getLines() + self.valsLock.release() + + def setListingType(self, listingType): + """ + Sets the priority information presented by the panel. + + Arguments: + listingType - Listing instance for the primary information to be shown + """ + + self.valsLock.acquire() + self._listingType = listingType + + # if we're sorting by the listing then we need to resort + if entries.SortAttr.LISTING in self._sortOrdering: + self.setSortOrder() + + self.valsLock.release() + + def handleKey(self, key): + self.valsLock.acquire() + + if uiTools.isScrollKey(key): + pageHeight = self.getPreferredSize()[0] - 1 + if self._showDetails: pageHeight -= (DETAILS_HEIGHT + 1) + isChanged = self._scroller.handleKey(key, self._entryLines, pageHeight) + if isChanged: self.redraw(True) + elif uiTools.isSelectionKey(key): + self._showDetails = not self._showDetails + self.redraw(True) + + self.valsLock.release() + + def run(self): + """ + Keeps connections listing updated, checking for new entries at a set rate. + """ + + lastDraw = time.time() - 1 + while not self._halt: + currentTime = time.time() + + if self._isPaused or not self._isTorRunning or currentTime - lastDraw < self._config["features.connection.refreshRate"]: + self._cond.acquire() + if not self._halt: self._cond.wait(0.2) + self._cond.release() + else: + # updates content if their's new results, otherwise just redraws + self._update() + self.redraw(True) + + # we may have missed multiple updates due to being paused, showing + # another panel, etc so lastDraw might need to jump multiple ticks + drawTicks = (time.time() - lastDraw) / self._config["features.connection.refreshRate"] + lastDraw += self._config["features.connection.refreshRate"] * drawTicks + + def draw(self, width, height): + self.valsLock.acquire() + + # extra line when showing the detail panel is for the bottom border + detailPanelOffset = DETAILS_HEIGHT + 1 if self._showDetails else 0 + isScrollbarVisible = len(self._entryLines) > height - detailPanelOffset - 1 + + scrollLoc = self._scroller.getScrollLoc(self._entryLines, height - detailPanelOffset - 1) + cursorSelection = self._scroller.getCursorSelection(self._entryLines) + + # draws the detail panel if currently displaying it + if self._showDetails: + # This is a solid border unless the scrollbar is visible, in which case a + # 'T' pipe connects the border to the bar. + uiTools.drawBox(self, 0, 0, width, DETAILS_HEIGHT + 2) + if isScrollbarVisible: self.addch(DETAILS_HEIGHT + 1, 1, curses.ACS_TTEE) + + drawEntries = cursorSelection.getDetails(width) + for i in range(min(len(drawEntries), DETAILS_HEIGHT)): + drawEntries[i].render(self, 1 + i, 2) + + # title label with connection counts + title = "Connection Details:" if self._showDetails else self._title + self.addstr(0, 0, title, curses.A_STANDOUT) + + scrollOffset = 1 + if isScrollbarVisible: + scrollOffset = 3 + self.addScrollBar(scrollLoc, scrollLoc + height - detailPanelOffset - 1, len(self._entryLines), 1 + detailPanelOffset) + + currentTime = self._pauseTime if self._pauseTime else time.time() + for lineNum in range(scrollLoc, len(self._entryLines)): + entryLine = self._entryLines[lineNum] + + # if this is an unresolved SOCKS, HIDDEN, or CONTROL entry then queue up + # resolution for the applicaitions they belong to + if isinstance(entryLine, connEntry.ConnectionLine) and entryLine.isUnresolvedApp(): + self._resolveApps() + + # hilighting if this is the selected line + extraFormat = curses.A_STANDOUT if entryLine == cursorSelection else curses.A_NORMAL + + drawEntry = entryLine.getListingEntry(width - scrollOffset, currentTime, self._listingType) + drawLine = lineNum + detailPanelOffset + 1 - scrollLoc + drawEntry.render(self, drawLine, scrollOffset, extraFormat) + if drawLine >= height: break + + self.valsLock.release() + + def stop(self): + """ + Halts further resolutions and terminates the thread. + """ + + self._cond.acquire() + self._halt = True + self._cond.notifyAll() + self._cond.release() + + def _update(self): + """ + Fetches the newest resolved connections. + """ + + connResolver = connections.getResolver("tor") + currentResolutionCount = connResolver.getResolutionCount() + self.appResolveSinceUpdate = False + + if self._lastResourceFetch != currentResolutionCount: + self.valsLock.acquire() + + newEntries = [] # the new results we'll display + + # Fetches new connections and client circuits... + # newConnections [(local ip, local port, foreign ip, foreign port)...] + # newCircuits {circuitID => (status, purpose, path)...} + + newConnections = connResolver.getConnections() + newCircuits = {} + + for circuitID, status, purpose, path in torTools.getConn().getCircuits(): + # Skips established single-hop circuits (these are for directory + # fetches, not client circuits) + if not (status == "BUILT" and len(path) == 1): + newCircuits[circuitID] = (status, purpose, path) + + # Populates newEntries with any of our old entries that still exist. + # This is both for performance and to keep from resetting the uptime + # attributes. Note that CircEntries are a ConnectionEntry subclass so + # we need to check for them first. + + for oldEntry in self._entries: + if isinstance(oldEntry, circEntry.CircEntry): + newEntry = newCircuits.get(oldEntry.circuitID) + + if newEntry: + oldEntry.update(newEntry[0], newEntry[2]) + newEntries.append(oldEntry) + del newCircuits[oldEntry.circuitID] + elif isinstance(oldEntry, connEntry.ConnectionEntry): + connLine = oldEntry.getLines()[0] + connAttr = (connLine.local.getIpAddr(), connLine.local.getPort(), + connLine.foreign.getIpAddr(), connLine.foreign.getPort()) + + if connAttr in newConnections: + newEntries.append(oldEntry) + newConnections.remove(connAttr) + + # Reset any display attributes for the entries we're keeping + for entry in newEntries: entry.resetDisplay() + + # Adds any new connection and circuit entries. + for lIp, lPort, fIp, fPort in newConnections: + newConnEntry = connEntry.ConnectionEntry(lIp, lPort, fIp, fPort) + if newConnEntry.getLines()[0].getType() != connEntry.Category.CIRCUIT: + newEntries.append(newConnEntry) + + for circuitID in newCircuits: + status, purpose, path = newCircuits[circuitID] + newEntries.append(circEntry.CircEntry(circuitID, status, purpose, path)) + + # Counts the relays in each of the categories. This also flushes the + # type cache for all of the connections (in case its changed since last + # fetched). + + categoryTypes = connEntry.Category.values() + typeCounts = dict((type, 0) for type in categoryTypes) + for entry in newEntries: + if isinstance(entry, connEntry.ConnectionEntry): + typeCounts[entry.getLines()[0].getType()] += 1 + elif isinstance(entry, circEntry.CircEntry): + typeCounts[connEntry.Category.CIRCUIT] += 1 + + # makes labels for all the categories with connections (ie, + # "21 outbound", "1 control", etc) + countLabels = [] + + for category in categoryTypes: + if typeCounts[category] > 0: + countLabels.append("%i %s" % (typeCounts[category], category.lower())) + + if countLabels: self._title = "Connections (%s):" % ", ".join(countLabels) + else: self._title = "Connections:" + + self._entries = newEntries + + self._entryLines = [] + for entry in self._entries: + self._entryLines += entry.getLines() + + self.setSortOrder() + self._lastResourceFetch = currentResolutionCount + self.valsLock.release() + + def _resolveApps(self, flagQuery = True): + """ + Triggers an asynchronous query for all unresolved SOCKS, HIDDEN, and + CONTROL entries. + + Arguments: + flagQuery - sets a flag to prevent further call from being respected + until the next update if true + """ + + if self.appResolveSinceUpdate or not self._config["features.connection.resolveApps"]: return + unresolvedLines = [l for l in self._entryLines if isinstance(l, connEntry.ConnectionLine) and l.isUnresolvedApp()] + + # get the ports used for unresolved applications + appPorts = [] + + for line in unresolvedLines: + appConn = line.local if line.getType() == connEntry.Category.HIDDEN else line.foreign + appPorts.append(appConn.getPort()) + + # Queue up resolution for the unresolved ports (skips if it's still working + # on the last query). + if appPorts and not self._appResolver.isResolving: + self._appResolver.resolve(appPorts) + + # Fetches results. If the query finishes quickly then this is what we just + # asked for, otherwise these belong to an earlier resolution. + # + # The application resolver might have given up querying (for instance, if + # the lsof lookups aren't working on this platform or lacks permissions). + # The isAppResolving flag lets the unresolved entries indicate if there's + # a lookup in progress for them or not. + + appResults = self._appResolver.getResults(0.2) + + for line in unresolvedLines: + isLocal = line.getType() == connEntry.Category.HIDDEN + linePort = line.local.getPort() if isLocal else line.foreign.getPort() + + if linePort in appResults: + # sets application attributes if there's a result with this as the + # inbound port + for inboundPort, outboundPort, cmd, pid in appResults[linePort]: + appPort = outboundPort if isLocal else inboundPort + + if linePort == appPort: + line.appName = cmd + line.appPid = pid + line.isAppResolving = False + else: + line.isAppResolving = self._appResolver.isResolving + + if flagQuery: + self.appResolveSinceUpdate = True + diff --git a/src/cli/connections/entries.py b/src/cli/connections/entries.py new file mode 100644 index 0000000..6b24412 --- /dev/null +++ b/src/cli/connections/entries.py @@ -0,0 +1,164 @@ +""" +Interface for entries in the connection panel. These consist of two parts: the +entry itself (ie, Tor connection, client circuit, etc) and the lines it +consists of in the listing. +""" + +from util import enum + +# attributes we can list entries by +ListingType = enum.Enum(("IP_ADDRESS", "IP Address"), "HOSTNAME", "FINGERPRINT", "NICKNAME") + +SortAttr = enum.Enum("CATEGORY", "UPTIME", "LISTING", "IP_ADDRESS", "PORT", + "HOSTNAME", "FINGERPRINT", "NICKNAME", "COUNTRY") + +SORT_COLORS = {SortAttr.CATEGORY: "red", SortAttr.UPTIME: "yellow", + SortAttr.LISTING: "green", SortAttr.IP_ADDRESS: "blue", + SortAttr.PORT: "blue", SortAttr.HOSTNAME: "magenta", + SortAttr.FINGERPRINT: "cyan", SortAttr.NICKNAME: "cyan", + SortAttr.COUNTRY: "blue"} + +# maximum number of ports a system can have +PORT_COUNT = 65536 + +class ConnectionPanelEntry: + """ + Common parent for connection panel entries. This consists of a list of lines + in the panel listing. This caches results until the display indicates that + they should be flushed. + """ + + def __init__(self): + self.lines = [] + self.flushCache = True + + def getLines(self): + """ + Provides the individual lines in the connection listing. + """ + + if self.flushCache: + self.lines = self._getLines(self.lines) + self.flushCache = False + + return self.lines + + def _getLines(self, oldResults): + # implementation of getLines + + for line in oldResults: + line.resetDisplay() + + return oldResults + + def getSortValues(self, sortAttrs, listingType): + """ + Provides the value used in comparisons to sort based on the given + attribute. + + Arguments: + sortAttrs - list of SortAttr values for the field being sorted on + listingType - ListingType enumeration for the attribute we're listing + entries by + """ + + return [self.getSortValue(attr, listingType) for attr in sortAttrs] + + def getSortValue(self, attr, listingType): + """ + Provides the value of a single attribute used for sorting purposes. + + Arguments: + attr - list of SortAttr values for the field being sorted on + listingType - ListingType enumeration for the attribute we're listing + entries by + """ + + if attr == SortAttr.LISTING: + if listingType == ListingType.IP_ADDRESS: + # uses the IP address as the primary value, and port as secondary + sortValue = self.getSortValue(SortAttr.IP_ADDRESS, listingType) * PORT_COUNT + sortValue += self.getSortValue(SortAttr.PORT, listingType) + return sortValue + elif listingType == ListingType.HOSTNAME: + return self.getSortValue(SortAttr.HOSTNAME, listingType) + elif listingType == ListingType.FINGERPRINT: + return self.getSortValue(SortAttr.FINGERPRINT, listingType) + elif listingType == ListingType.NICKNAME: + return self.getSortValue(SortAttr.NICKNAME, listingType) + + return "" + + def resetDisplay(self): + """ + Flushes cached display results. + """ + + self.flushCache = True + +class ConnectionPanelLine: + """ + Individual line in the connection panel listing. + """ + + def __init__(self): + # cache for displayed information + self._listingCache = None + self._listingCacheArgs = (None, None) + + self._detailsCache = None + self._detailsCacheArgs = None + + self._descriptorCache = None + self._descriptorCacheArgs = None + + def getListingEntry(self, width, currentTime, listingType): + """ + Provides a DrawEntry instance for contents to be displayed in the + connection panel listing. + + Arguments: + width - available space to display in + currentTime - unix timestamp for what the results should consider to be + the current time (this may be ignored due to caching) + listingType - ListingType enumeration for the highest priority content + to be displayed + """ + + if self._listingCacheArgs != (width, listingType): + self._listingCache = self._getListingEntry(width, currentTime, listingType) + self._listingCacheArgs = (width, listingType) + + return self._listingCache + + def _getListingEntry(self, width, currentTime, listingType): + # implementation of getListingEntry + return None + + def getDetails(self, width): + """ + Provides a list of DrawEntry instances with detailed information for this + connection. + + Arguments: + width - available space to display in + """ + + if self._detailsCacheArgs != width: + self._detailsCache = self._getDetails(width) + self._detailsCacheArgs = width + + return self._detailsCache + + def _getDetails(self, width): + # implementation of getDetails + return [] + + def resetDisplay(self): + """ + Flushes cached display results. + """ + + self._listingCacheArgs = (None, None) + self._detailsCacheArgs = None + diff --git a/src/cli/controller.py b/src/cli/controller.py new file mode 100644 index 0000000..2afbf6a --- /dev/null +++ b/src/cli/controller.py @@ -0,0 +1,1584 @@ +#!/usr/bin/env python +# controller.py -- arm interface (curses monitor for relay status) +# Released under the GPL v3 (http://www.gnu.org/licenses/gpl.html) + +""" +Curses (terminal) interface for the arm relay status monitor. +""" + +import os +import re +import math +import time +import curses +import curses.textpad +import socket +from TorCtl import TorCtl + +import headerPanel +import graphing.graphPanel +import logPanel +import configPanel +import torrcPanel +import descriptorPopup + +import cli.connections.connPanel +import cli.connections.connEntry +import cli.connections.entries +from util import conf, log, connections, hostnames, panel, sysTools, torConfig, torTools, uiTools +import graphing.bandwidthStats +import graphing.connStats +import graphing.resourceStats + +CONFIRM_QUIT = True +REFRESH_RATE = 5 # seconds between redrawing screen +MAX_REGEX_FILTERS = 5 # maximum number of previous regex filters that'll be remembered + +# enums for message in control label +CTL_HELP, CTL_PAUSED = range(2) + +# panel order per page +PAGE_S = ["header", "control", "popup"] # sticky (ie, always available) page +PAGES = [ + ["graph", "log"], + ["conn"], + ["config"], + ["torrc"]] + +PAUSEABLE = ["header", "graph", "log", "conn"] + +CONFIG = {"log.torrc.readFailed": log.WARN, + "features.graph.type": 1, + "features.config.prepopulateEditValues": True, + "queries.refreshRate.rate": 5, + "log.torEventTypeUnrecognized": log.NOTICE, + "features.graph.bw.prepopulate": True, + "log.startTime": log.INFO, + "log.refreshRate": log.DEBUG, + "log.highCpuUsage": log.WARN, + "log.configEntryUndefined": log.NOTICE, + "log.torrc.validation.torStateDiffers": log.WARN, + "log.torrc.validation.unnecessaryTorrcEntries": log.NOTICE} + +class ControlPanel(panel.Panel): + """ Draws single line label for interface controls. """ + + def __init__(self, stdscr, isBlindMode): + panel.Panel.__init__(self, stdscr, "control", 0, 1) + self.msgText = CTL_HELP # message text to be displyed + self.msgAttr = curses.A_NORMAL # formatting attributes + self.page = 1 # page number currently being displayed + self.resolvingCounter = -1 # count of resolver when starting (-1 if we aren't working on a batch) + self.isBlindMode = isBlindMode + + def setMsg(self, msgText, msgAttr=curses.A_NORMAL): + """ + Sets the message and display attributes. If msgType matches CTL_HELP or + CTL_PAUSED then uses the default message for those statuses. + """ + + self.msgText = msgText + self.msgAttr = msgAttr + + def draw(self, width, height): + msgText = self.msgText + msgAttr = self.msgAttr + barTab = 2 # space between msgText and progress bar + barWidthMax = 40 # max width to progress bar + barWidth = -1 # space between "[ ]" in progress bar (not visible if -1) + barProgress = 0 # cells to fill + + if msgText == CTL_HELP: + msgAttr = curses.A_NORMAL + + if self.resolvingCounter != -1: + if hostnames.isPaused() or not hostnames.isResolving(): + # done resolving dns batch + self.resolvingCounter = -1 + curses.halfdelay(REFRESH_RATE * 10) # revert to normal refresh rate + else: + batchSize = hostnames.getRequestCount() - self.resolvingCounter + entryCount = batchSize - hostnames.getPendingCount() + if batchSize > 0: progress = 100 * entryCount / batchSize + else: progress = 0 + + additive = "or l " if self.page == 2 else "" + batchSizeDigits = int(math.log10(batchSize)) + 1 + entryCountLabel = ("%%%ii" % batchSizeDigits) % entryCount + #msgText = "Resolving hostnames (%i / %i, %i%%) - press esc %sto cancel" % (entryCount, batchSize, progress, additive) + msgText = "Resolving hostnames (press esc %sto cancel) - %s / %i, %2i%%" % (additive, entryCountLabel, batchSize, progress) + + barWidth = min(barWidthMax, width - len(msgText) - 3 - barTab) + barProgress = barWidth * entryCount / batchSize + + if self.resolvingCounter == -1: + currentPage = self.page + pageCount = len(PAGES) + + if self.isBlindMode: + if currentPage >= 2: currentPage -= 1 + pageCount -= 1 + + msgText = "page %i / %i - q: quit, p: pause, h: page help" % (currentPage, pageCount) + elif msgText == CTL_PAUSED: + msgText = "Paused" + msgAttr = curses.A_STANDOUT + + self.addstr(0, 0, msgText, msgAttr) + if barWidth > -1: + xLoc = len(msgText) + barTab + self.addstr(0, xLoc, "[", curses.A_BOLD) + self.addstr(0, xLoc + 1, " " * barProgress, curses.A_STANDOUT | uiTools.getColor("red")) + self.addstr(0, xLoc + barWidth + 1, "]", curses.A_BOLD) + +class Popup(panel.Panel): + """ + Temporarily providing old panel methods until permanent workaround for popup + can be derrived (this passive drawing method is horrible - I'll need to + provide a version using the more active repaint design later in the + revision). + """ + + def __init__(self, stdscr, height): + panel.Panel.__init__(self, stdscr, "popup", 0, height) + + # The following methods are to emulate old panel functionality (this was the + # only implementations to use these methods and will require a complete + # rewrite when refactoring gets here) + def clear(self): + if self.win: + self.isDisplaced = self.top > self.win.getparyx()[0] + if not self.isDisplaced: self.win.erase() + + def refresh(self): + if self.win and not self.isDisplaced: self.win.refresh() + + def recreate(self, stdscr, newWidth=-1, newTop=None): + self.setParent(stdscr) + self.setWidth(newWidth) + if newTop != None: self.setTop(newTop) + + newHeight, newWidth = self.getPreferredSize() + if newHeight > 0: + self.win = self.parent.subwin(newHeight, newWidth, self.top, 0) + elif self.win == None: + # don't want to leave the window as none (in very edge cases could cause + # problems) - rather, create a displaced instance + self.win = self.parent.subwin(1, newWidth, 0, 0) + + self.maxY, self.maxX = self.win.getmaxyx() + +def addstr_wrap(panel, y, x, text, formatting, startX = 0, endX = -1, maxY = -1): + """ + Writes text with word wrapping, returning the ending y/x coordinate. + y: starting write line + x: column offset from startX + text / formatting: content to be written + startX / endX: column bounds in which text may be written + """ + + # moved out of panel (trying not to polute new code!) + # TODO: unpleaseantly complex usage - replace with something else when + # rewriting confPanel and descriptorPopup (the only places this is used) + if not text: return (y, x) # nothing to write + if endX == -1: endX = panel.maxX # defaults to writing to end of panel + if maxY == -1: maxY = panel.maxY + 1 # defaults to writing to bottom of panel + lineWidth = endX - startX # room for text + while True: + if len(text) > lineWidth - x - 1: + chunkSize = text.rfind(" ", 0, lineWidth - x) + writeText = text[:chunkSize] + text = text[chunkSize:].strip() + + panel.addstr(y, x + startX, writeText, formatting) + y, x = y + 1, 0 + if y >= maxY: return (y, x) + else: + panel.addstr(y, x + startX, text, formatting) + return (y, x + len(text)) + +class sighupListener(TorCtl.PostEventListener): + """ + Listens for reload signal (hup), which is produced by: + pkill -sighup tor + causing the torrc and internal state to be reset. + """ + + def __init__(self): + TorCtl.PostEventListener.__init__(self) + self.isReset = False + + def msg_event(self, event): + self.isReset |= event.level == "NOTICE" and event.msg.startswith("Received reload signal (hup)") + +def setPauseState(panels, monitorIsPaused, currentPage, overwrite=False): + """ + Resets the isPaused state of panels. If overwrite is True then this pauses + reguardless of the monitor is paused or not. + """ + + for key in PAUSEABLE: panels[key].setPaused(overwrite or monitorIsPaused or (key not in PAGES[currentPage] and key not in PAGE_S)) + +def showMenu(stdscr, popup, title, options, initialSelection): + """ + Provides menu with options laid out in a single column. User can cancel + selection with the escape key, in which case this proives -1. Otherwise this + returns the index of the selection. If initialSelection is -1 then the first + option is used and the carrot indicating past selection is ommitted. + """ + + selection = initialSelection if initialSelection != -1 else 0 + + if popup.win: + if not panel.CURSES_LOCK.acquire(False): return -1 + try: + # TODO: should pause interface (to avoid event accumilation) + curses.cbreak() # wait indefinitely for key presses (no timeout) + + # uses smaller dimentions more fitting for small content + popup.height = len(options) + 2 + + newWidth = max([len(label) for label in options]) + 9 + popup.recreate(stdscr, newWidth) + + key = 0 + while not uiTools.isSelectionKey(key): + popup.clear() + popup.win.box() + popup.addstr(0, 0, title, curses.A_STANDOUT) + + for i in range(len(options)): + label = options[i] + format = curses.A_STANDOUT if i == selection else curses.A_NORMAL + tab = "> " if i == initialSelection else " " + popup.addstr(i + 1, 2, tab) + popup.addstr(i + 1, 4, " %s " % label, format) + + popup.refresh() + key = stdscr.getch() + if key == curses.KEY_UP: selection = max(0, selection - 1) + elif key == curses.KEY_DOWN: selection = min(len(options) - 1, selection + 1) + elif key == 27: selection, key = -1, curses.KEY_ENTER # esc - cancel + + # reverts popup dimensions and conn panel label + popup.height = 9 + popup.recreate(stdscr, 80) + + curses.halfdelay(REFRESH_RATE * 10) # reset normal pausing behavior + finally: + panel.CURSES_LOCK.release() + + return selection + +def showSortDialog(stdscr, panels, isPaused, page, titleLabel, options, oldSelection, optionColors): + """ + Displays a sorting dialog of the form: + + Current Order: <previous selection> + New Order: <selections made> + + <option 1> <option 2> <option 3> Cancel + + Options are colored when among the "Current Order" or "New Order", but not + when an option below them. If cancel is selected or the user presses escape + then this returns None. Otherwise, the new ordering is provided. + + Arguments: + stdscr, panels, isPaused, page - boiler plate arguments of the controller + (should be refactored away when rewriting) + + titleLabel - title displayed for the popup window + options - ordered listing of option labels + oldSelection - current ordering + optionColors - mappings of options to their color + + """ + + panel.CURSES_LOCK.acquire() + newSelections = [] # new ordering + + try: + setPauseState(panels, isPaused, page, True) + curses.cbreak() # wait indefinitely for key presses (no timeout) + + popup = panels["popup"] + cursorLoc = 0 # index of highlighted option + + # label for the inital ordering + formattedPrevListing = [] + for sortType in oldSelection: + colorStr = optionColors.get(sortType, "white") + formattedPrevListing.append("<%s>%s</%s>" % (colorStr, sortType, colorStr)) + prevOrderingLabel = "Current Order: %s" % ", ".join(formattedPrevListing) + + selectionOptions = list(options) + selectionOptions.append("Cancel") + + while len(newSelections) < len(oldSelection): + popup.clear() + popup.win.box() + popup.addstr(0, 0, titleLabel, curses.A_STANDOUT) + popup.addfstr(1, 2, prevOrderingLabel) + + # provides new ordering + formattedNewListing = [] + for sortType in newSelections: + colorStr = optionColors.get(sortType, "white") + formattedNewListing.append("<%s>%s</%s>" % (colorStr, sortType, colorStr)) + newOrderingLabel = "New Order: %s" % ", ".join(formattedNewListing) + popup.addfstr(2, 2, newOrderingLabel) + + # presents remaining options, each row having up to four options with + # spacing of nineteen cells + row, col = 4, 0 + for i in range(len(selectionOptions)): + popup.addstr(row, col * 19 + 2, selectionOptions[i], curses.A_STANDOUT if cursorLoc == i else curses.A_NORMAL) + col += 1 + if col == 4: row, col = row + 1, 0 + + popup.refresh() + + key = stdscr.getch() + if key == curses.KEY_LEFT: cursorLoc = max(0, cursorLoc - 1) + elif key == curses.KEY_RIGHT: cursorLoc = min(len(selectionOptions) - 1, cursorLoc + 1) + elif key == curses.KEY_UP: cursorLoc = max(0, cursorLoc - 4) + elif key == curses.KEY_DOWN: cursorLoc = min(len(selectionOptions) - 1, cursorLoc + 4) + elif uiTools.isSelectionKey(key): + # selected entry (the ord of '10' seems needed to pick up enter) + selection = selectionOptions[cursorLoc] + if selection == "Cancel": break + else: + newSelections.append(selection) + selectionOptions.remove(selection) + cursorLoc = min(cursorLoc, len(selectionOptions) - 1) + elif key == 27: break # esc - cancel + + setPauseState(panels, isPaused, page) + curses.halfdelay(REFRESH_RATE * 10) # reset normal pausing behavior + finally: + panel.CURSES_LOCK.release() + + if len(newSelections) == len(oldSelection): + return newSelections + else: return None + +def setEventListening(selectedEvents, isBlindMode): + # creates a local copy, note that a suspected python bug causes *very* + # puzzling results otherwise when trying to discard entries (silently + # returning out of this function!) + events = set(selectedEvents) + isLoggingUnknown = "UNKNOWN" in events + + # removes special types only used in arm (UNKNOWN, TORCTL, ARM_DEBUG, etc) + toDiscard = [] + for eventType in events: + if eventType not in logPanel.TOR_EVENT_TYPES.values(): toDiscard += [eventType] + + for eventType in list(toDiscard): events.discard(eventType) + + # adds events unrecognized by arm if we're listening to the 'UNKNOWN' type + if isLoggingUnknown: + events.update(set(logPanel.getMissingEventTypes())) + + setEvents = torTools.getConn().setControllerEvents(list(events)) + + # temporary hack for providing user selected events minus those that failed + # (wouldn't be a problem if I wasn't storing tor and non-tor events together...) + returnVal = list(selectedEvents.difference(torTools.FAILED_EVENTS)) + returnVal.sort() # alphabetizes + return returnVal + +def connResetListener(conn, eventType): + """ + Pauses connection resolution when tor's shut down, and resumes if started + again. + """ + + if connections.isResolverAlive("tor"): + resolver = connections.getResolver("tor") + resolver.setPaused(eventType == torTools.State.CLOSED) + +def selectiveRefresh(panels, page): + """ + This forces a redraw of content on the currently active page (should be done + after changing pages, popups, or anything else that overwrites panels). + """ + + for panelKey in PAGES[page]: + panels[panelKey].redraw(True) + +def drawTorMonitor(stdscr, startTime, loggedEvents, isBlindMode): + """ + Starts arm interface reflecting information on provided control port. + + stdscr - curses window + conn - active Tor control port connection + loggedEvents - types of events to be logged (plus an optional "UNKNOWN" for + otherwise unrecognized events) + """ + + # loads config for various interface components + config = conf.getConfig("arm") + config.update(CONFIG) + graphing.graphPanel.loadConfig(config) + cli.connections.connEntry.loadConfig(config) + + # adds events needed for arm functionality to the torTools REQ_EVENTS mapping + # (they're then included with any setControllerEvents call, and log a more + # helpful error if unavailable) + torTools.REQ_EVENTS["BW"] = "bandwidth graph won't function" + + if not isBlindMode: + torTools.REQ_EVENTS["CIRC"] = "may cause issues in identifying client connections" + + # pauses/unpauses connection resolution according to if tor's connected or not + torTools.getConn().addStatusListener(connResetListener) + + # TODO: incrementally drop this requirement until everything's using the singleton + conn = torTools.getConn().getTorCtl() + + curses.halfdelay(REFRESH_RATE * 10) # uses getch call as timer for REFRESH_RATE seconds + try: curses.use_default_colors() # allows things like semi-transparent backgrounds (call can fail with ERR) + except curses.error: pass + + # attempts to make the cursor invisible (not supported in all terminals) + try: curses.curs_set(0) + except curses.error: pass + + # attempts to determine tor's current pid (left as None if unresolveable, logging an error later) + torPid = torTools.getConn().getMyPid() + + #try: + # confLocation = conn.get_info("config-file")["config-file"] + # if confLocation[0] != "/": + # # relative path - attempt to add process pwd + # try: + # results = sysTools.call("pwdx %s" % torPid) + # if len(results) == 1 and len(results[0].split()) == 2: confLocation = "%s/%s" % (results[0].split()[1], confLocation) + # except IOError: pass # pwdx call failed + #except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed): + # confLocation = "" + + # loads the torrc and provides warnings in case of validation errors + loadedTorrc = torConfig.getTorrc() + loadedTorrc.getLock().acquire() + + try: + loadedTorrc.load() + except IOError, exc: + msg = "Unable to load torrc (%s)" % sysTools.getFileErrorMsg(exc) + log.log(CONFIG["log.torrc.readFailed"], msg) + + if loadedTorrc.isLoaded(): + corrections = loadedTorrc.getCorrections() + duplicateOptions, defaultOptions, mismatchLines, missingOptions = [], [], [], [] + + for lineNum, issue, msg in corrections: + if issue == torConfig.ValidationError.DUPLICATE: + duplicateOptions.append("%s (line %i)" % (msg, lineNum + 1)) + elif issue == torConfig.ValidationError.IS_DEFAULT: + defaultOptions.append("%s (line %i)" % (msg, lineNum + 1)) + elif issue == torConfig.ValidationError.MISMATCH: mismatchLines.append(lineNum + 1) + elif issue == torConfig.ValidationError.MISSING: missingOptions.append(msg) + + if duplicateOptions or defaultOptions: + msg = "Unneeded torrc entries found. They've been highlighted in blue on the torrc page." + + if duplicateOptions: + if len(duplicateOptions) > 1: + msg += "\n- entries ignored due to having duplicates: " + else: + msg += "\n- entry ignored due to having a duplicate: " + + duplicateOptions.sort() + msg += ", ".join(duplicateOptions) + + if defaultOptions: + if len(defaultOptions) > 1: + msg += "\n- entries match their default values: " + else: + msg += "\n- entry matches its default value: " + + defaultOptions.sort() + msg += ", ".join(defaultOptions) + + log.log(CONFIG["log.torrc.validation.unnecessaryTorrcEntries"], msg) + + if mismatchLines or missingOptions: + msg = "The torrc differ from what tor's using. You can issue a sighup to reload the torrc values by pressing x." + + if mismatchLines: + if len(mismatchLines) > 1: + msg += "\n- torrc values differ on lines: " + else: + msg += "\n- torrc value differs on line: " + + mismatchLines.sort() + msg += ", ".join([str(val + 1) for val in mismatchLines]) + + if missingOptions: + if len(missingOptions) > 1: + msg += "\n- configuration values are missing from the torrc: " + else: + msg += "\n- configuration value is missing from the torrc: " + + missingOptions.sort() + msg += ", ".join(missingOptions) + + log.log(CONFIG["log.torrc.validation.torStateDiffers"], msg) + + loadedTorrc.getLock().release() + + # minor refinements for connection resolver + if not isBlindMode: + if torPid: + # use the tor pid to help narrow connection results + torCmdName = sysTools.getProcessName(torPid, "tor") + resolver = connections.getResolver(torCmdName, torPid, "tor") + else: + resolver = connections.getResolver("tor") + + # hack to display a better (arm specific) notice if all resolvers fail + connections.RESOLVER_FINAL_FAILURE_MSG += " (connection related portions of the monitor won't function)" + + panels = { + "header": headerPanel.HeaderPanel(stdscr, startTime, config), + "popup": Popup(stdscr, 9), + "graph": graphing.graphPanel.GraphPanel(stdscr), + "log": logPanel.LogPanel(stdscr, loggedEvents, config)} + + # TODO: later it would be good to set the right 'top' values during initialization, + # but for now this is just necessary for the log panel (and a hack in the log...) + + # TODO: bug from not setting top is that the log panel might attempt to draw + # before being positioned - the following is a quick hack til rewritten + panels["log"].setPaused(True) + + panels["conn"] = cli.connections.connPanel.ConnectionPanel(stdscr, config) + + panels["control"] = ControlPanel(stdscr, isBlindMode) + panels["config"] = configPanel.ConfigPanel(stdscr, configPanel.State.TOR, config) + panels["torrc"] = torrcPanel.TorrcPanel(stdscr, torrcPanel.Config.TORRC, config) + + # provides error if pid coulnd't be determined (hopefully shouldn't happen...) + if not torPid: log.log(log.WARN, "Unable to resolve tor pid, abandoning connection listing") + + # statistical monitors for graph + panels["graph"].addStats("bandwidth", graphing.bandwidthStats.BandwidthStats(config)) + panels["graph"].addStats("system resources", graphing.resourceStats.ResourceStats()) + if not isBlindMode: panels["graph"].addStats("connections", graphing.connStats.ConnStats()) + + # sets graph based on config parameter + graphType = CONFIG["features.graph.type"] + if graphType == 0: panels["graph"].setStats(None) + elif graphType == 1: panels["graph"].setStats("bandwidth") + elif graphType == 2 and not isBlindMode: panels["graph"].setStats("connections") + elif graphType == 3: panels["graph"].setStats("system resources") + + # listeners that update bandwidth and log panels with Tor status + sighupTracker = sighupListener() + #conn.add_event_listener(panels["log"]) + conn.add_event_listener(panels["graph"].stats["bandwidth"]) + conn.add_event_listener(panels["graph"].stats["system resources"]) + if not isBlindMode: conn.add_event_listener(panels["graph"].stats["connections"]) + conn.add_event_listener(sighupTracker) + + # prepopulates bandwidth values from state file + if CONFIG["features.graph.bw.prepopulate"]: + isSuccessful = panels["graph"].stats["bandwidth"].prepopulateFromState() + if isSuccessful: panels["graph"].updateInterval = 4 + + # tells Tor to listen to the events we're interested + loggedEvents = setEventListening(loggedEvents, isBlindMode) + #panels["log"].loggedEvents = loggedEvents # strips any that couldn't be set + panels["log"].setLoggedEvents(loggedEvents) # strips any that couldn't be set + + # directs logged TorCtl events to log panel + #TorUtil.loglevel = "DEBUG" + #TorUtil.logfile = panels["log"] + #torTools.getConn().addTorCtlListener(panels["log"].tor_ctl_event) + + # provides a notice about any event types tor supports but arm doesn't + missingEventTypes = logPanel.getMissingEventTypes() + if missingEventTypes: + pluralLabel = "s" if len(missingEventTypes) > 1 else "" + log.log(CONFIG["log.torEventTypeUnrecognized"], "arm doesn't recognize the following event type%s: %s (log 'UNKNOWN' events to see them)" % (pluralLabel, ", ".join(missingEventTypes))) + + # tells revised panels to run as daemons + panels["header"].start() + panels["log"].start() + panels["conn"].start() + + # warns if tor isn't updating descriptors + #try: + # if conn.get_option("FetchUselessDescriptors")[0][1] == "0" and conn.get_option("DirPort")[0][1] == "0": + # warning = """Descriptors won't be updated (causing some connection information to be stale) unless: + #a. 'FetchUselessDescriptors 1' is set in your torrc + #b. the directory service is provided ('DirPort' defined) + #c. or tor is used as a client""" + # log.log(log.WARN, warning) + #except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed): pass + + isUnresponsive = False # true if it's been over ten seconds since the last BW event (probably due to Tor closing) + isPaused = False # if true updates are frozen + overrideKey = None # immediately runs with this input rather than waiting for the user if set + page = 0 + regexFilters = [] # previously used log regex filters + panels["popup"].redraw(True) # hack to make sure popup has a window instance (not entirely sure why...) + + # provides notice about any unused config keys + for key in config.getUnusedKeys(): + log.log(CONFIG["log.configEntryUndefined"], "Unused configuration entry: %s" % key) + + lastPerformanceLog = 0 # ensures we don't do performance logging too frequently + redrawStartTime = time.time() + + # TODO: popups need to force the panels it covers to redraw (or better, have + # a global refresh function for after changing pages, popups, etc) + + initTime = time.time() - startTime + log.log(CONFIG["log.startTime"], "arm started (initialization took %0.3f seconds)" % initTime) + + # attributes to give a WARN level event if arm's resource usage is too high + isResourceWarningGiven = False + lastResourceCheck = startTime + + lastSize = None + + # sets initial visiblity for the pages + for i in range(len(PAGES)): + isVisible = i == page + for entry in PAGES[i]: panels[entry].setVisible(isVisible) + + # TODO: come up with a nice, clean method for other threads to immediately + # terminate the draw loop and provide a stacktrace + while True: + # tried only refreshing when the screen was resized but it caused a + # noticeable lag when resizing and didn't have an appreciable effect + # on system usage + + panel.CURSES_LOCK.acquire() + try: + redrawStartTime = time.time() + + # if sighup received then reload related information + if sighupTracker.isReset: + #panels["header"]._updateParams(True) + + # other panels that use torrc data + #if not isBlindMode: panels["graph"].stats["connections"].resetOptions(conn) + #panels["graph"].stats["bandwidth"].resetOptions() + + # if bandwidth graph is being shown then height might have changed + if panels["graph"].currentDisplay == "bandwidth": + panels["graph"].setHeight(panels["graph"].stats["bandwidth"].getContentHeight()) + + # TODO: should redraw the torrcPanel + #panels["torrc"].loadConfig() + + # reload the torrc if it's previously been loaded + if loadedTorrc.isLoaded(): + try: + loadedTorrc.load() + if page == 3: panels["torrc"].redraw(True) + except IOError, exc: + msg = "Unable to load torrc (%s)" % sysTools.getFileErrorMsg(exc) + log.log(CONFIG["log.torrc.readFailed"], msg) + + sighupTracker.isReset = False + + # gives panels a chance to take advantage of the maximum bounds + # originally this checked in the bounds changed but 'recreate' is a no-op + # if panel properties are unchanged and checking every redraw is more + # resilient in case of funky changes (such as resizing during popups) + + # hack to make sure header picks layout before using the dimensions below + #panels["header"].getPreferredSize() + + startY = 0 + for panelKey in PAGE_S[:2]: + #panels[panelKey].recreate(stdscr, -1, startY) + panels[panelKey].setParent(stdscr) + panels[panelKey].setWidth(-1) + panels[panelKey].setTop(startY) + startY += panels[panelKey].getHeight() + + panels["popup"].recreate(stdscr, 80, startY) + + for panelSet in PAGES: + tmpStartY = startY + + for panelKey in panelSet: + #panels[panelKey].recreate(stdscr, -1, tmpStartY) + panels[panelKey].setParent(stdscr) + panels[panelKey].setWidth(-1) + panels[panelKey].setTop(tmpStartY) + tmpStartY += panels[panelKey].getHeight() + + # provides a notice if there's been ten seconds since the last BW event + lastHeartbeat = torTools.getConn().getHeartbeat() + if torTools.getConn().isAlive() and "BW" in torTools.getConn().getControllerEvents() and lastHeartbeat != 0: + if not isUnresponsive and (time.time() - lastHeartbeat) >= 10: + isUnresponsive = True + log.log(log.NOTICE, "Relay unresponsive (last heartbeat: %s)" % time.ctime(lastHeartbeat)) + elif isUnresponsive and (time.time() - lastHeartbeat) < 10: + # really shouldn't happen (meant Tor froze for a bit) + isUnresponsive = False + log.log(log.NOTICE, "Relay resumed") + + # TODO: part two of hack to prevent premature drawing by log panel + if page == 0 and not isPaused: panels["log"].setPaused(False) + + # I haven't the foggiest why, but doesn't work if redrawn out of order... + for panelKey in (PAGE_S + PAGES[page]): + # redrawing popup can result in display flicker when it should be hidden + if panelKey != "popup": + newSize = stdscr.getmaxyx() + isResize = lastSize != newSize + lastSize = newSize + + if panelKey in ("header", "graph", "log", "config", "torrc", "conn2"): + # revised panel (manages its own content refreshing) + panels[panelKey].redraw(isResize) + else: + panels[panelKey].redraw(True) + + stdscr.refresh() + + currentTime = time.time() + if currentTime - lastPerformanceLog >= CONFIG["queries.refreshRate.rate"]: + cpuTotal = sum(os.times()[:3]) + pythonCpuAvg = cpuTotal / (currentTime - startTime) + sysCallCpuAvg = sysTools.getSysCpuUsage() + totalCpuAvg = pythonCpuAvg + sysCallCpuAvg + + if sysCallCpuAvg > 0.00001: + log.log(CONFIG["log.refreshRate"], "refresh rate: %0.3f seconds, average cpu usage: %0.3f%% (python), %0.3f%% (system calls), %0.3f%% (total)" % (currentTime - redrawStartTime, 100 * pythonCpuAvg, 100 * sysCallCpuAvg, 100 * totalCpuAvg)) + else: + # with the proc enhancements the sysCallCpuAvg is usually zero + log.log(CONFIG["log.refreshRate"], "refresh rate: %0.3f seconds, average cpu usage: %0.3f%%" % (currentTime - redrawStartTime, 100 * totalCpuAvg)) + + lastPerformanceLog = currentTime + + # once per minute check if the sustained cpu usage is above 5%, if so + # then give a warning (and if able, some advice for lowering it) + # TODO: disabling this for now (scrolling causes cpu spikes for quick + # redraws, ie this is usually triggered by user input) + if False and not isResourceWarningGiven and currentTime > (lastResourceCheck + 60): + if totalCpuAvg >= 0.05: + msg = "Arm's cpu usage is high (averaging %0.3f%%)." % (100 * totalCpuAvg) + + if not isBlindMode: + msg += " You could lower it by dropping the connection data (running as \"arm -b\")." + + log.log(CONFIG["log.highCpuUsage"], msg) + isResourceWarningGiven = True + + lastResourceCheck = currentTime + finally: + panel.CURSES_LOCK.release() + + # wait for user keyboard input until timeout (unless an override was set) + if overrideKey: + key = overrideKey + overrideKey = None + else: + key = stdscr.getch() + + if key == ord('q') or key == ord('Q'): + quitConfirmed = not CONFIRM_QUIT + + # provides prompt to confirm that arm should exit + if CONFIRM_QUIT: + panel.CURSES_LOCK.acquire() + try: + setPauseState(panels, isPaused, page, True) + + # provides prompt + panels["control"].setMsg("Are you sure (q again to confirm)?", curses.A_BOLD) + panels["control"].redraw(True) + + curses.cbreak() + confirmationKey = stdscr.getch() + quitConfirmed = confirmationKey in (ord('q'), ord('Q')) + curses.halfdelay(REFRESH_RATE * 10) + + panels["control"].setMsg(CTL_PAUSED if isPaused else CTL_HELP) + setPauseState(panels, isPaused, page) + finally: + panel.CURSES_LOCK.release() + + if quitConfirmed: + # quits arm + # very occasionally stderr gets "close failed: [Errno 11] Resource temporarily unavailable" + # this appears to be a python bug: http://bugs.python.org/issue3014 + # (haven't seen this is quite some time... mysteriously resolved?) + + torTools.NO_SPAWN = True # prevents further worker threads from being spawned + + # stops panel daemons + panels["header"].stop() + panels["conn"].stop() + panels["log"].stop() + + panels["header"].join() + panels["conn"].join() + panels["log"].join() + + # joins on utility daemon threads - this might take a moment since + # the internal threadpools being joined might be sleeping + conn = torTools.getConn() + myPid = conn.getMyPid() + + resourceTracker = sysTools.getResourceTracker(myPid) if (myPid and sysTools.isTrackerAlive(myPid)) else None + resolver = connections.getResolver("tor") if connections.isResolverAlive("tor") else None + if resourceTracker: resourceTracker.stop() + if resolver: resolver.stop() # sets halt flag (returning immediately) + hostnames.stop() # halts and joins on hostname worker thread pool + if resourceTracker: resourceTracker.join() + if resolver: resolver.join() # joins on halted resolver + + conn.close() # joins on TorCtl event thread + break + elif key == curses.KEY_LEFT or key == curses.KEY_RIGHT: + # switch page + if key == curses.KEY_LEFT: page = (page - 1) % len(PAGES) + else: page = (page + 1) % len(PAGES) + + # skip connections listing if it's disabled + if page == 1 and isBlindMode: + if key == curses.KEY_LEFT: page = (page - 1) % len(PAGES) + else: page = (page + 1) % len(PAGES) + + # pauses panels that aren't visible to prevent events from accumilating + # (otherwise they'll wait on the curses lock which might get demanding) + setPauseState(panels, isPaused, page) + + # prevents panels on other pages from redrawing + for i in range(len(PAGES)): + isVisible = i == page + for entry in PAGES[i]: panels[entry].setVisible(isVisible) + + panels["control"].page = page + 1 + + # TODO: this redraw doesn't seem necessary (redraws anyway after this + # loop) - look into this when refactoring + panels["control"].redraw(True) + + selectiveRefresh(panels, page) + elif key == ord('p') or key == ord('P'): + # toggles update freezing + panel.CURSES_LOCK.acquire() + try: + isPaused = not isPaused + setPauseState(panels, isPaused, page) + panels["control"].setMsg(CTL_PAUSED if isPaused else CTL_HELP) + finally: + panel.CURSES_LOCK.release() + + selectiveRefresh(panels, page) + elif key == ord('x') or key == ord('X'): + # provides prompt to confirm that arm should issue a sighup + panel.CURSES_LOCK.acquire() + try: + setPauseState(panels, isPaused, page, True) + + # provides prompt + panels["control"].setMsg("This will reset Tor's internal state. Are you sure (x again to confirm)?", curses.A_BOLD) + panels["control"].redraw(True) + + curses.cbreak() + confirmationKey = stdscr.getch() + if confirmationKey in (ord('x'), ord('X')): + try: + torTools.getConn().reload() + except IOError, exc: + log.log(log.ERR, "Error detected when reloading tor: %s" % sysTools.getFileErrorMsg(exc)) + + #errorMsg = " (%s)" % str(err) if str(err) else "" + #panels["control"].setMsg("Sighup failed%s" % errorMsg, curses.A_STANDOUT) + #panels["control"].redraw(True) + #time.sleep(2) + + # reverts display settings + curses.halfdelay(REFRESH_RATE * 10) + panels["control"].setMsg(CTL_PAUSED if isPaused else CTL_HELP) + setPauseState(panels, isPaused, page) + finally: + panel.CURSES_LOCK.release() + elif key == ord('h') or key == ord('H'): + # displays popup for current page's controls + panel.CURSES_LOCK.acquire() + try: + setPauseState(panels, isPaused, page, True) + + # lists commands + popup = panels["popup"] + popup.clear() + popup.win.box() + popup.addstr(0, 0, "Page %i Commands:" % (page + 1), curses.A_STANDOUT) + + pageOverrideKeys = () + + if page == 0: + graphedStats = panels["graph"].currentDisplay + if not graphedStats: graphedStats = "none" + popup.addfstr(1, 2, "up arrow: scroll log up a line") + popup.addfstr(1, 41, "down arrow: scroll log down a line") + popup.addfstr(2, 2, "m: increase graph size") + popup.addfstr(2, 41, "n: decrease graph size") + popup.addfstr(3, 2, "s: graphed stats (%s)" % graphedStats) + popup.addfstr(3, 41, "i: graph update interval (%s)" % graphing.graphPanel.UPDATE_INTERVALS[panels["graph"].updateInterval][0]) + popup.addfstr(4, 2, "b: graph bounds (%s)" % panels["graph"].bounds.lower()) + popup.addfstr(4, 41, "a: save snapshot of the log") + popup.addfstr(5, 2, "e: change logged events") + + regexLabel = "enabled" if panels["log"].regexFilter else "disabled" + popup.addfstr(5, 41, "f: log regex filter (%s)" % regexLabel) + + hiddenEntryLabel = "visible" if panels["log"].showDuplicates else "hidden" + popup.addfstr(6, 2, "u: duplicate log entries (%s)" % hiddenEntryLabel) + popup.addfstr(6, 41, "c: clear event log") + + pageOverrideKeys = (ord('m'), ord('n'), ord('s'), ord('i'), ord('d'), ord('e'), ord('r'), ord('f'), ord('x')) + if page == 1: + popup.addfstr(1, 2, "up arrow: scroll up a line") + popup.addfstr(1, 41, "down arrow: scroll down a line") + popup.addfstr(2, 2, "page up: scroll up a page") + popup.addfstr(2, 41, "page down: scroll down a page") + + popup.addfstr(3, 2, "enter: edit configuration option") + popup.addfstr(3, 41, "d: raw consensus descriptor") + + listingType = panels["conn"]._listingType.lower() + popup.addfstr(4, 2, "l: listed identity (%s)" % listingType) + + popup.addfstr(4, 41, "s: sort ordering") + + resolverUtil = connections.getResolver("tor").overwriteResolver + if resolverUtil == None: resolverUtil = "auto" + popup.addfstr(5, 2, "u: resolving utility (%s)" % resolverUtil) + + pageOverrideKeys = (ord('d'), ord('l'), ord('s'), ord('u')) + elif page == 2: + popup.addfstr(1, 2, "up arrow: scroll up a line") + popup.addfstr(1, 41, "down arrow: scroll down a line") + popup.addfstr(2, 2, "page up: scroll up a page") + popup.addfstr(2, 41, "page down: scroll down a page") + + strippingLabel = "on" if panels["torrc"].stripComments else "off" + popup.addfstr(3, 2, "s: comment stripping (%s)" % strippingLabel) + + lineNumLabel = "on" if panels["torrc"].showLineNum else "off" + popup.addfstr(3, 41, "n: line numbering (%s)" % lineNumLabel) + + popup.addfstr(4, 2, "r: reload torrc") + popup.addfstr(4, 41, "x: reset tor (issue sighup)") + elif page == 3: + popup.addfstr(1, 2, "up arrow: scroll up a line") + popup.addfstr(1, 41, "down arrow: scroll down a line") + popup.addfstr(2, 2, "page up: scroll up a page") + popup.addfstr(2, 41, "page down: scroll down a page") + popup.addfstr(3, 2, "enter: connection details") + + popup.addstr(7, 2, "Press any key...") + popup.refresh() + + # waits for user to hit a key, if it belongs to a command then executes it + curses.cbreak() + helpExitKey = stdscr.getch() + if helpExitKey in pageOverrideKeys: overrideKey = helpExitKey + curses.halfdelay(REFRESH_RATE * 10) + + setPauseState(panels, isPaused, page) + selectiveRefresh(panels, page) + finally: + panel.CURSES_LOCK.release() + elif page == 0 and (key == ord('s') or key == ord('S')): + # provides menu to pick stats to be graphed + #options = ["None"] + [label for label in panels["graph"].stats.keys()] + options = ["None"] + + # appends stats labels with first letters of each word capitalized + initialSelection, i = -1, 1 + if not panels["graph"].currentDisplay: initialSelection = 0 + graphLabels = panels["graph"].stats.keys() + graphLabels.sort() + for label in graphLabels: + if label == panels["graph"].currentDisplay: initialSelection = i + words = label.split() + options.append(" ".join(word[0].upper() + word[1:] for word in words)) + i += 1 + + # hides top label of the graph panel and pauses panels + if panels["graph"].currentDisplay: + panels["graph"].showLabel = False + panels["graph"].redraw(True) + setPauseState(panels, isPaused, page, True) + + selection = showMenu(stdscr, panels["popup"], "Graphed Stats:", options, initialSelection) + + # reverts changes made for popup + panels["graph"].showLabel = True + setPauseState(panels, isPaused, page) + + # applies new setting + if selection != -1 and selection != initialSelection: + if selection == 0: panels["graph"].setStats(None) + else: panels["graph"].setStats(options[selection].lower()) + + selectiveRefresh(panels, page) + + # TODO: this shouldn't be necessary with the above refresh, but doesn't seem responsive otherwise... + panels["graph"].redraw(True) + elif page == 0 and (key == ord('i') or key == ord('I')): + # provides menu to pick graph panel update interval + options = [label for (label, intervalTime) in graphing.graphPanel.UPDATE_INTERVALS] + + initialSelection = panels["graph"].updateInterval + + #initialSelection = -1 + #for i in range(len(options)): + # if options[i] == panels["graph"].updateInterval: initialSelection = i + + # hides top label of the graph panel and pauses panels + if panels["graph"].currentDisplay: + panels["graph"].showLabel = False + panels["graph"].redraw(True) + setPauseState(panels, isPaused, page, True) + + selection = showMenu(stdscr, panels["popup"], "Update Interval:", options, initialSelection) + + # reverts changes made for popup + panels["graph"].showLabel = True + setPauseState(panels, isPaused, page) + + # applies new setting + if selection != -1: panels["graph"].updateInterval = selection + + selectiveRefresh(panels, page) + elif page == 0 and (key == ord('b') or key == ord('B')): + # uses the next boundary type for graph + panels["graph"].bounds = graphing.graphPanel.Bounds.next(panels["graph"].bounds) + + selectiveRefresh(panels, page) + elif page == 0 and (key == ord('a') or key == ord('A')): + # allow user to enter a path to take a snapshot - abandons if left blank + panel.CURSES_LOCK.acquire() + try: + setPauseState(panels, isPaused, page, True) + + # provides prompt + panels["control"].setMsg("Path to save log snapshot: ") + panels["control"].redraw(True) + + # gets user input (this blocks monitor updates) + pathInput = panels["control"].getstr(0, 27) + + if pathInput: + try: + panels["log"].saveSnapshot(pathInput) + panels["control"].setMsg("Saved: %s" % pathInput, curses.A_STANDOUT) + panels["control"].redraw(True) + time.sleep(2) + except IOError, exc: + panels["control"].setMsg("Unable to save snapshot: %s" % sysTools.getFileErrorMsg(exc), curses.A_STANDOUT) + panels["control"].redraw(True) + time.sleep(2) + + panels["control"].setMsg(CTL_PAUSED if isPaused else CTL_HELP) + setPauseState(panels, isPaused, page) + finally: + panel.CURSES_LOCK.release() + + panels["graph"].redraw(True) + elif page == 0 and (key == ord('e') or key == ord('E')): + # allow user to enter new types of events to log - unchanged if left blank + panel.CURSES_LOCK.acquire() + try: + setPauseState(panels, isPaused, page, True) + + # provides prompt + panels["control"].setMsg("Events to log: ") + panels["control"].redraw(True) + + # lists event types + popup = panels["popup"] + popup.height = 11 + popup.recreate(stdscr, 80) + + popup.clear() + popup.win.box() + popup.addstr(0, 0, "Event Types:", curses.A_STANDOUT) + lineNum = 1 + for line in logPanel.EVENT_LISTING.split("\n"): + line = line[6:] + popup.addstr(lineNum, 1, line) + lineNum += 1 + popup.refresh() + + # gets user input (this blocks monitor updates) + eventsInput = panels["control"].getstr(0, 15) + if eventsInput: eventsInput = eventsInput.replace(' ', '') # strips spaces + + # it would be nice to quit on esc, but looks like this might not be possible... + if eventsInput: + try: + expandedEvents = logPanel.expandEvents(eventsInput) + loggedEvents = setEventListening(expandedEvents, isBlindMode) + panels["log"].setLoggedEvents(loggedEvents) + except ValueError, exc: + panels["control"].setMsg("Invalid flags: %s" % str(exc), curses.A_STANDOUT) + panels["control"].redraw(True) + time.sleep(2) + + # reverts popup dimensions + popup.height = 9 + popup.recreate(stdscr, 80) + + panels["control"].setMsg(CTL_PAUSED if isPaused else CTL_HELP) + setPauseState(panels, isPaused, page) + finally: + panel.CURSES_LOCK.release() + + panels["graph"].redraw(True) + elif page == 0 and (key == ord('f') or key == ord('F')): + # provides menu to pick previous regular expression filters or to add a new one + # for syntax see: http://docs.python.org/library/re.html#regular-expression-syntax + options = ["None"] + regexFilters + ["New..."] + initialSelection = 0 if not panels["log"].regexFilter else 1 + + # hides top label of the graph panel and pauses panels + if panels["graph"].currentDisplay: + panels["graph"].showLabel = False + panels["graph"].redraw(True) + setPauseState(panels, isPaused, page, True) + + selection = showMenu(stdscr, panels["popup"], "Log Filter:", options, initialSelection) + + # applies new setting + if selection == 0: + panels["log"].setFilter(None) + elif selection == len(options) - 1: + # selected 'New...' option - prompt user to input regular expression + panel.CURSES_LOCK.acquire() + try: + # provides prompt + panels["control"].setMsg("Regular expression: ") + panels["control"].redraw(True) + + # gets user input (this blocks monitor updates) + regexInput = panels["control"].getstr(0, 20) + + if regexInput: + try: + panels["log"].setFilter(re.compile(regexInput)) + if regexInput in regexFilters: regexFilters.remove(regexInput) + regexFilters = [regexInput] + regexFilters + except re.error, exc: + panels["control"].setMsg("Unable to compile expression: %s" % str(exc), curses.A_STANDOUT) + panels["control"].redraw(True) + time.sleep(2) + panels["control"].setMsg(CTL_PAUSED if isPaused else CTL_HELP) + finally: + panel.CURSES_LOCK.release() + elif selection != -1: + try: + panels["log"].setFilter(re.compile(regexFilters[selection - 1])) + + # move selection to top + regexFilters = [regexFilters[selection - 1]] + regexFilters + del regexFilters[selection] + except re.error, exc: + # shouldn't happen since we've already checked validity + log.log(log.WARN, "Invalid regular expression ('%s': %s) - removing from listing" % (regexFilters[selection - 1], str(exc))) + del regexFilters[selection - 1] + + if len(regexFilters) > MAX_REGEX_FILTERS: del regexFilters[MAX_REGEX_FILTERS:] + + # reverts changes made for popup + panels["graph"].showLabel = True + setPauseState(panels, isPaused, page) + panels["graph"].redraw(True) + elif page == 0 and key in (ord('n'), ord('N'), ord('m'), ord('M')): + # Unfortunately modifier keys don't work with the up/down arrows (sending + # multiple keycodes. The only exception to this is shift + left/right, + # but for now just gonna use standard characters. + + if key in (ord('n'), ord('N')): + panels["graph"].setGraphHeight(panels["graph"].graphHeight - 1) + else: + # don't grow the graph if it's already consuming the whole display + # (plus an extra line for the graph/log gap) + maxHeight = panels["graph"].parent.getmaxyx()[0] - panels["graph"].top + currentHeight = panels["graph"].getHeight() + + if currentHeight < maxHeight + 1: + panels["graph"].setGraphHeight(panels["graph"].graphHeight + 1) + elif page == 0 and (key == ord('c') or key == ord('C')): + # provides prompt to confirm that arm should clear the log + panel.CURSES_LOCK.acquire() + try: + setPauseState(panels, isPaused, page, True) + + # provides prompt + panels["control"].setMsg("This will clear the log. Are you sure (c again to confirm)?", curses.A_BOLD) + panels["control"].redraw(True) + + curses.cbreak() + confirmationKey = stdscr.getch() + if confirmationKey in (ord('c'), ord('C')): panels["log"].clear() + + # reverts display settings + curses.halfdelay(REFRESH_RATE * 10) + panels["control"].setMsg(CTL_PAUSED if isPaused else CTL_HELP) + setPauseState(panels, isPaused, page) + finally: + panel.CURSES_LOCK.release() + elif page == 1 and (key == ord('u') or key == ord('U')): + # provides menu to pick identification resolving utility + options = ["auto"] + connections.Resolver.values() + + currentOverwrite = connections.getResolver("tor").overwriteResolver # enums correspond to indices + if currentOverwrite == None: initialSelection = 0 + else: initialSelection = options.index(currentOverwrite) + + # hides top label of conn panel and pauses panels + panelTitle = panels["conn"]._title + panels["conn"]._title = "" + panels["conn"].redraw(True) + setPauseState(panels, isPaused, page, True) + + selection = showMenu(stdscr, panels["popup"], "Resolver Util:", options, initialSelection) + selectedOption = options[selection] if selection != "auto" else None + + # reverts changes made for popup + panels["conn"]._title = panelTitle + setPauseState(panels, isPaused, page) + + # applies new setting + if selection != -1 and selectedOption != connections.getResolver("tor").overwriteResolver: + connections.getResolver("tor").overwriteResolver = selectedOption + elif page == 1 and key in (ord('d'), ord('D')): + # presents popup for raw consensus data + panel.CURSES_LOCK.acquire() + try: + setPauseState(panels, isPaused, page, True) + curses.cbreak() # wait indefinitely for key presses (no timeout) + panelTitle = panels["conn"]._title + panels["conn"]._title = "" + panels["conn"].redraw(True) + + descriptorPopup.showDescriptorPopup(panels["popup"], stdscr, panels["conn"]) + + panels["conn"]._title = panelTitle + setPauseState(panels, isPaused, page) + curses.halfdelay(REFRESH_RATE * 10) # reset normal pausing behavior + finally: + panel.CURSES_LOCK.release() + elif page == 1 and (key == ord('l') or key == ord('L')): + # provides a menu to pick the primary information we list connections by + options = cli.connections.entries.ListingType.values() + + # dropping the HOSTNAME listing type until we support displaying that content + options.remove(cli.connections.entries.ListingType.HOSTNAME) + + initialSelection = options.index(panels["conn"]._listingType) + + # hides top label of connection panel and pauses the display + panelTitle = panels["conn"]._title + panels["conn"]._title = "" + panels["conn"].redraw(True) + setPauseState(panels, isPaused, page, True) + + selection = showMenu(stdscr, panels["popup"], "List By:", options, initialSelection) + + # reverts changes made for popup + panels["conn"]._title = panelTitle + setPauseState(panels, isPaused, page) + + # applies new setting + if selection != -1 and options[selection] != panels["conn"]._listingType: + panels["conn"].setListingType(options[selection]) + panels["conn"].redraw(True) + elif page == 1 and (key == ord('s') or key == ord('S')): + # set ordering for connection options + titleLabel = "Connection Ordering:" + options = cli.connections.entries.SortAttr.values() + oldSelection = panels["conn"]._sortOrdering + optionColors = dict([(attr, cli.connections.entries.SORT_COLORS[attr]) for attr in options]) + results = showSortDialog(stdscr, panels, isPaused, page, titleLabel, options, oldSelection, optionColors) + + if results: + panels["conn"].setSortOrder(results) + + panels["conn"].redraw(True) + elif page == 2 and (key == ord('c') or key == ord('C')) and False: + # TODO: disabled for now (probably gonna be going with separate pages + # rather than popup menu) + # provides menu to pick config being displayed + #options = [confPanel.CONFIG_LABELS[confType] for confType in range(4)] + options = [] + initialSelection = panels["torrc"].configType + + # hides top label of the graph panel and pauses panels + panels["torrc"].showLabel = False + panels["torrc"].redraw(True) + setPauseState(panels, isPaused, page, True) + + selection = showMenu(stdscr, panels["popup"], "Configuration:", options, initialSelection) + + # reverts changes made for popup + panels["torrc"].showLabel = True + setPauseState(panels, isPaused, page) + + # applies new setting + if selection != -1: panels["torrc"].setConfigType(selection) + + selectiveRefresh(panels, page) + elif page == 2 and (key == ord('w') or key == ord('W')): + # display a popup for saving the current configuration + panel.CURSES_LOCK.acquire() + try: + configLines = torConfig.getCustomOptions(True) + + # lists event types + popup = panels["popup"] + popup.height = len(configLines) + 3 + popup.recreate(stdscr) + displayHeight, displayWidth = panels["popup"].getPreferredSize() + + # displayed options (truncating the labels if there's limited room) + if displayWidth >= 30: selectionOptions = ("Save", "Save As...", "Cancel") + else: selectionOptions = ("Save", "Save As", "X") + + # checks if we can show options beside the last line of visible content + lastIndex = min(displayHeight - 3, len(configLines) - 1) + isOptionLineSeparate = displayWidth < (30 + len(configLines[lastIndex])) + + # if we're showing all the content and have room to display selection + # options besides the text then shrink the popup by a row + if not isOptionLineSeparate and displayHeight == len(configLines) + 3: + popup.height -= 1 + popup.recreate(stdscr) + + key, selection = 0, 2 + while not uiTools.isSelectionKey(key): + # if the popup has been resized then recreate it (needed for the + # proper border height) + newHeight, newWidth = panels["popup"].getPreferredSize() + if (displayHeight, displayWidth) != (newHeight, newWidth): + displayHeight, displayWidth = newHeight, newWidth + popup.recreate(stdscr) + + # if there isn't room to display the popup then cancel it + if displayHeight <= 2: + selection = 2 + break + + popup.clear() + popup.win.box() + popup.addstr(0, 0, "Configuration being saved:", curses.A_STANDOUT) + + visibleConfigLines = displayHeight - 3 if isOptionLineSeparate else displayHeight - 2 + for i in range(visibleConfigLines): + line = uiTools.cropStr(configLines[i], displayWidth - 2) + + if " " in line: + option, arg = line.split(" ", 1) + popup.addstr(i + 1, 1, option, curses.A_BOLD | uiTools.getColor("green")) + popup.addstr(i + 1, len(option) + 2, arg, curses.A_BOLD | uiTools.getColor("cyan")) + else: + popup.addstr(i + 1, 1, line, curses.A_BOLD | uiTools.getColor("green")) + + # draws 'T' between the lower left and the covered panel's scroll bar + if displayWidth > 1: popup.win.addch(displayHeight - 1, 1, curses.ACS_TTEE) + + # draws selection options (drawn right to left) + drawX = displayWidth - 1 + for i in range(len(selectionOptions) - 1, -1, -1): + optionLabel = selectionOptions[i] + drawX -= (len(optionLabel) + 2) + + # if we've run out of room then drop the option (this will only + # occure on tiny displays) + if drawX < 1: break + + selectionFormat = curses.A_STANDOUT if i == selection else curses.A_NORMAL + popup.addstr(displayHeight - 2, drawX, "[") + popup.addstr(displayHeight - 2, drawX + 1, optionLabel, selectionFormat | curses.A_BOLD) + popup.addstr(displayHeight - 2, drawX + len(optionLabel) + 1, "]") + + drawX -= 1 # space gap between the options + + popup.refresh() + + key = stdscr.getch() + if key == curses.KEY_LEFT: selection = max(0, selection - 1) + elif key == curses.KEY_RIGHT: selection = min(len(selectionOptions) - 1, selection + 1) + + if selection in (0, 1): + loadedTorrc = torConfig.getTorrc() + try: configLocation = loadedTorrc.getConfigLocation() + except IOError: configLocation = "" + + if selection == 1: + # prompts user for a configuration location + promptMsg = "Save to (esc to cancel): " + panels["control"].setMsg(promptMsg) + panels["control"].redraw(True) + configLocation = panels["control"].getstr(0, len(promptMsg), configLocation) + if configLocation: configLocation = os.path.abspath(configLocation) + + if configLocation: + try: + # make dir if the path doesn't already exist + baseDir = os.path.dirname(configLocation) + if not os.path.exists(baseDir): os.makedirs(baseDir) + + # saves the configuration to the file + configFile = open(configLocation, "w") + configFile.write("\n".join(configLines)) + configFile.close() + + # reloads the cached torrc if overwriting it + if configLocation == loadedTorrc.getConfigLocation(): + try: + loadedTorrc.load() + panels["torrc"]._lastContentHeightArgs = None + except IOError: pass + + msg = "Saved configuration to %s" % configLocation + except (IOError, OSError), exc: + msg = "Unable to save configuration (%s)" % sysTools.getFileErrorMsg(exc) + + panels["control"].setMsg(msg, curses.A_STANDOUT) + panels["control"].redraw(True) + time.sleep(2) + + panels["control"].setMsg(CTL_PAUSED if isPaused else CTL_HELP) + + # reverts popup dimensions + popup.height = 9 + popup.recreate(stdscr, 80) + finally: + panel.CURSES_LOCK.release() + + panels["config"].redraw(True) + elif page == 2 and (key == ord('s') or key == ord('S')): + # set ordering for config options + titleLabel = "Config Option Ordering:" + options = [configPanel.FIELD_ATTR[field][0] for field in configPanel.Field.values()] + oldSelection = [configPanel.FIELD_ATTR[field][0] for field in panels["config"].sortOrdering] + optionColors = dict([configPanel.FIELD_ATTR[field] for field in configPanel.Field.values()]) + results = showSortDialog(stdscr, panels, isPaused, page, titleLabel, options, oldSelection, optionColors) + + if results: + # converts labels back to enums + resultEnums = [] + + for label in results: + for entryEnum in configPanel.FIELD_ATTR: + if label == configPanel.FIELD_ATTR[entryEnum][0]: + resultEnums.append(entryEnum) + break + + panels["config"].setSortOrder(resultEnums) + + panels["config"].redraw(True) + elif page == 2 and uiTools.isSelectionKey(key): + # let the user edit the configuration value, unchanged if left blank + panel.CURSES_LOCK.acquire() + try: + setPauseState(panels, isPaused, page, True) + + # provides prompt + selection = panels["config"].getSelection() + configOption = selection.get(configPanel.Field.OPTION) + titleMsg = "%s Value (esc to cancel): " % configOption + panels["control"].setMsg(titleMsg) + panels["control"].redraw(True) + + displayWidth = panels["control"].getPreferredSize()[1] + initialValue = selection.get(configPanel.Field.VALUE) + + # initial input for the text field + initialText = "" + if CONFIG["features.config.prepopulateEditValues"] and initialValue != "<none>": + initialText = initialValue + + newConfigValue = panels["control"].getstr(0, len(titleMsg), initialText) + + # it would be nice to quit on esc, but looks like this might not be possible... + if newConfigValue != None and newConfigValue != initialValue: + conn = torTools.getConn() + + # if the value's a boolean then allow for 'true' and 'false' inputs + if selection.get(configPanel.Field.TYPE) == "Boolean": + if newConfigValue.lower() == "true": newConfigValue = "1" + elif newConfigValue.lower() == "false": newConfigValue = "0" + + try: + if selection.get(configPanel.Field.TYPE) == "LineList": + newConfigValue = newConfigValue.split(",") + + conn.setOption(configOption, newConfigValue) + + # resets the isDefault flag + customOptions = torConfig.getCustomOptions() + selection.fields[configPanel.Field.IS_DEFAULT] = not configOption in customOptions + + panels["config"].redraw(True) + except Exception, exc: + errorMsg = "%s (press any key)" % exc + panels["control"].setMsg(uiTools.cropStr(errorMsg, displayWidth), curses.A_STANDOUT) + panels["control"].redraw(True) + + curses.cbreak() # wait indefinitely for key presses (no timeout) + stdscr.getch() + curses.halfdelay(REFRESH_RATE * 10) + + panels["control"].setMsg(CTL_PAUSED if isPaused else CTL_HELP) + setPauseState(panels, isPaused, page) + finally: + panel.CURSES_LOCK.release() + elif page == 3 and key == ord('r') or key == ord('R'): + # reloads torrc, providing a notice if successful or not + loadedTorrc = torConfig.getTorrc() + loadedTorrc.getLock().acquire() + + try: + loadedTorrc.load() + isSuccessful = True + except IOError: + isSuccessful = False + + loadedTorrc.getLock().release() + + #isSuccessful = panels["torrc"].loadConfig(logErrors = False) + #confTypeLabel = confPanel.CONFIG_LABELS[panels["torrc"].configType] + resetMsg = "torrc reloaded" if isSuccessful else "failed to reload torrc" + if isSuccessful: + panels["torrc"]._lastContentHeightArgs = None + panels["torrc"].redraw(True) + + panels["control"].setMsg(resetMsg, curses.A_STANDOUT) + panels["control"].redraw(True) + time.sleep(1) + + panels["control"].setMsg(CTL_PAUSED if isPaused else CTL_HELP) + elif page == 0: + panels["log"].handleKey(key) + elif page == 1: + panels["conn"].handleKey(key) + elif page == 2: + panels["config"].handleKey(key) + elif page == 3: + panels["torrc"].handleKey(key) + +def startTorMonitor(startTime, loggedEvents, isBlindMode): + try: + curses.wrapper(drawTorMonitor, startTime, loggedEvents, isBlindMode) + except KeyboardInterrupt: + pass # skip printing stack trace in case of keyboard interrupt + diff --git a/src/cli/descriptorPopup.py b/src/cli/descriptorPopup.py new file mode 100644 index 0000000..cdc959d --- /dev/null +++ b/src/cli/descriptorPopup.py @@ -0,0 +1,181 @@ +#!/usr/bin/env python +# descriptorPopup.py -- popup panel used to show raw consensus data +# Released under the GPL v3 (http://www.gnu.org/licenses/gpl.html) + +import math +import socket +import curses +from TorCtl import TorCtl + +import controller +import connections.connEntry +from util import panel, torTools, uiTools + +# field keywords used to identify areas for coloring +LINE_NUM_COLOR = "yellow" +HEADER_COLOR = "cyan" +HEADER_PREFIX = ["ns/id/", "desc/id/"] + +SIG_COLOR = "red" +SIG_START_KEYS = ["-----BEGIN RSA PUBLIC KEY-----", "-----BEGIN SIGNATURE-----"] +SIG_END_KEYS = ["-----END RSA PUBLIC KEY-----", "-----END SIGNATURE-----"] + +UNRESOLVED_MSG = "No consensus data available" +ERROR_MSG = "Unable to retrieve data" + +class PopupProperties: + """ + State attributes of popup window for consensus descriptions. + """ + + def __init__(self): + self.fingerprint = "" + self.entryColor = "white" + self.text = [] + self.scroll = 0 + self.showLineNum = True + + def reset(self, fingerprint, entryColor): + self.fingerprint = fingerprint + self.entryColor = entryColor + self.text = [] + self.scroll = 0 + + if fingerprint == "UNKNOWN": + self.fingerprint = None + self.showLineNum = False + self.text.append(UNRESOLVED_MSG) + else: + conn = torTools.getConn() + + try: + self.showLineNum = True + self.text.append("ns/id/%s" % fingerprint) + self.text += conn.getConsensusEntry(fingerprint).split("\n") + except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed): + self.text = self.text + [ERROR_MSG, ""] + + try: + descCommand = "desc/id/%s" % fingerprint + self.text.append("desc/id/%s" % fingerprint) + self.text += conn.getDescriptorEntry(fingerprint).split("\n") + except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed): + self.text = self.text + [ERROR_MSG] + + def handleKey(self, key, height): + if key == curses.KEY_UP: self.scroll = max(self.scroll - 1, 0) + elif key == curses.KEY_DOWN: self.scroll = max(0, min(self.scroll + 1, len(self.text) - height)) + elif key == curses.KEY_PPAGE: self.scroll = max(self.scroll - height, 0) + elif key == curses.KEY_NPAGE: self.scroll = max(0, min(self.scroll + height, len(self.text) - height)) + +def showDescriptorPopup(popup, stdscr, connectionPanel): + """ + Presents consensus descriptor in popup window with the following controls: + Up, Down, Page Up, Page Down - scroll descriptor + Right, Left - next / previous connection + Enter, Space, d, D - close popup + """ + + properties = PopupProperties() + isVisible = True + + if not panel.CURSES_LOCK.acquire(False): return + try: + while isVisible: + selection = connectionPanel._scroller.getCursorSelection(connectionPanel._entryLines) + if not selection: break + fingerprint = selection.foreign.getFingerprint() + entryColor = connections.connEntry.CATEGORY_COLOR[selection.getType()] + properties.reset(fingerprint, entryColor) + + # constrains popup size to match text + width, height = 0, 0 + for line in properties.text: + # width includes content, line number field, and border + lineWidth = len(line) + 5 + if properties.showLineNum: lineWidth += int(math.log10(len(properties.text))) + 1 + width = max(width, lineWidth) + + # tracks number of extra lines that will be taken due to text wrap + height += (lineWidth - 2) / connectionPanel.maxX + + popup.setHeight(min(len(properties.text) + height + 2, connectionPanel.maxY)) + popup.recreate(stdscr, width) + + while isVisible: + draw(popup, properties) + key = stdscr.getch() + + if uiTools.isSelectionKey(key) or key in (ord('d'), ord('D')): + # closes popup + isVisible = False + elif key in (curses.KEY_LEFT, curses.KEY_RIGHT): + # navigation - pass on to connPanel and recreate popup + connectionPanel.handleKey(curses.KEY_UP if key == curses.KEY_LEFT else curses.KEY_DOWN) + break + else: properties.handleKey(key, popup.height - 2) + + popup.setHeight(9) + popup.recreate(stdscr, 80) + finally: + panel.CURSES_LOCK.release() + +def draw(popup, properties): + popup.clear() + popup.win.box() + xOffset = 2 + + if properties.text: + if properties.fingerprint: popup.addstr(0, 0, "Consensus Descriptor (%s):" % properties.fingerprint, curses.A_STANDOUT) + else: popup.addstr(0, 0, "Consensus Descriptor:", curses.A_STANDOUT) + + isEncryption = False # true if line is part of an encryption block + + # checks if first line is in an encryption block + for i in range(0, properties.scroll): + lineText = properties.text[i].strip() + if lineText in SIG_START_KEYS: isEncryption = True + elif lineText in SIG_END_KEYS: isEncryption = False + + pageHeight = popup.maxY - 2 + numFieldWidth = int(math.log10(len(properties.text))) + 1 + lineNum = 1 + for i in range(properties.scroll, min(len(properties.text), properties.scroll + pageHeight)): + lineText = properties.text[i].strip() + + numOffset = 0 # offset for line numbering + if properties.showLineNum: + popup.addstr(lineNum, xOffset, ("%%%ii" % numFieldWidth) % (i + 1), curses.A_BOLD | uiTools.getColor(LINE_NUM_COLOR)) + numOffset = numFieldWidth + 1 + + if lineText: + keyword = lineText.split()[0] # first word of line + remainder = lineText[len(keyword):] + keywordFormat = curses.A_BOLD | uiTools.getColor(properties.entryColor) + remainderFormat = uiTools.getColor(properties.entryColor) + + if lineText.startswith(HEADER_PREFIX[0]) or lineText.startswith(HEADER_PREFIX[1]): + keyword, remainder = lineText, "" + keywordFormat = curses.A_BOLD | uiTools.getColor(HEADER_COLOR) + if lineText == UNRESOLVED_MSG or lineText == ERROR_MSG: + keyword, remainder = lineText, "" + if lineText in SIG_START_KEYS: + keyword, remainder = lineText, "" + isEncryption = True + keywordFormat = curses.A_BOLD | uiTools.getColor(SIG_COLOR) + elif lineText in SIG_END_KEYS: + keyword, remainder = lineText, "" + isEncryption = False + keywordFormat = curses.A_BOLD | uiTools.getColor(SIG_COLOR) + elif isEncryption: + keyword, remainder = lineText, "" + keywordFormat = uiTools.getColor(SIG_COLOR) + + lineNum, xLoc = controller.addstr_wrap(popup, lineNum, 0, keyword, keywordFormat, xOffset + numOffset, popup.maxX - 1, popup.maxY - 1) + lineNum, xLoc = controller.addstr_wrap(popup, lineNum, xLoc, remainder, remainderFormat, xOffset + numOffset, popup.maxX - 1, popup.maxY - 1) + + lineNum += 1 + if lineNum > pageHeight: break + + popup.refresh() + diff --git a/src/cli/graphing/__init__.py b/src/cli/graphing/__init__.py new file mode 100644 index 0000000..2dddaa3 --- /dev/null +++ b/src/cli/graphing/__init__.py @@ -0,0 +1,6 @@ +""" +Graphing panel resources. +""" + +__all__ = ["graphPanel", "bandwidthStats", "connStats", "resourceStats"] + diff --git a/src/cli/graphing/bandwidthStats.py b/src/cli/graphing/bandwidthStats.py new file mode 100644 index 0000000..2864dd8 --- /dev/null +++ b/src/cli/graphing/bandwidthStats.py @@ -0,0 +1,398 @@ +""" +Tracks bandwidth usage of the tor process, expanding to include accounting +stats if they're set. +""" + +import time + +from cli.graphing import graphPanel +from util import log, sysTools, torTools, uiTools + +DL_COLOR, UL_COLOR = "green", "cyan" + +# width at which panel abandons placing optional stats (avg and total) with +# header in favor of replacing the x-axis label +COLLAPSE_WIDTH = 135 + +# valid keys for the accountingInfo mapping +ACCOUNTING_ARGS = ("status", "resetTime", "read", "written", "readLimit", "writtenLimit") + +PREPOPULATE_SUCCESS_MSG = "Read the last day of bandwidth history from the state file" +PREPOPULATE_FAILURE_MSG = "Unable to prepopulate bandwidth information (%s)" + +DEFAULT_CONFIG = {"features.graph.bw.transferInBytes": False, + "features.graph.bw.accounting.show": True, + "features.graph.bw.accounting.rate": 10, + "features.graph.bw.accounting.isTimeLong": False, + "log.graph.bw.prepopulateSuccess": log.NOTICE, + "log.graph.bw.prepopulateFailure": log.NOTICE} + +class BandwidthStats(graphPanel.GraphStats): + """ + Uses tor BW events to generate bandwidth usage graph. + """ + + def __init__(self, config=None): + graphPanel.GraphStats.__init__(self) + + self._config = dict(DEFAULT_CONFIG) + if config: + config.update(self._config, {"features.graph.bw.accounting.rate": 1}) + + # stats prepopulated from tor's state file + self.prepopulatePrimaryTotal = 0 + self.prepopulateSecondaryTotal = 0 + self.prepopulateTicks = 0 + + # accounting data (set by _updateAccountingInfo method) + self.accountingLastUpdated = 0 + self.accountingInfo = dict([(arg, "") for arg in ACCOUNTING_ARGS]) + + # listens for tor reload (sighup) events which can reset the bandwidth + # rate/burst and if tor's using accounting + conn = torTools.getConn() + self._titleStats, self.isAccounting = [], False + self.resetListener(conn, torTools.State.INIT) # initializes values + conn.addStatusListener(self.resetListener) + + # Initialized the bandwidth totals to the values reported by Tor. This + # uses a controller options introduced in ticket 2345: + # https://trac.torproject.org/projects/tor/ticket/2345 + # + # further updates are still handled via BW events to avoid unnecessary + # GETINFO requests. + + self.initialPrimaryTotal = 0 + self.initialSecondaryTotal = 0 + + readTotal = conn.getInfo("traffic/read") + if readTotal and readTotal.isdigit(): + self.initialPrimaryTotal = int(readTotal) / 1024 # Bytes -> KB + + writeTotal = conn.getInfo("traffic/written") + if writeTotal and writeTotal.isdigit(): + self.initialSecondaryTotal = int(writeTotal) / 1024 # Bytes -> KB + + def resetListener(self, conn, eventType): + # updates title parameters and accounting status if they changed + self._titleStats = [] # force reset of title + self.new_desc_event(None) # updates title params + + if eventType == torTools.State.INIT and self._config["features.graph.bw.accounting.show"]: + self.isAccounting = conn.getInfo('accounting/enabled') == '1' + + def prepopulateFromState(self): + """ + Attempts to use tor's state file to prepopulate values for the 15 minute + interval via the BWHistoryReadValues/BWHistoryWriteValues values. This + returns True if successful and False otherwise. + """ + + # checks that this is a relay (if ORPort is unset, then skip) + conn = torTools.getConn() + orPort = conn.getOption("ORPort") + if orPort == "0": return + + # gets the uptime (using the same parameters as the header panel to take + # advantage of caching + uptime = None + queryPid = conn.getMyPid() + if queryPid: + queryParam = ["%cpu", "rss", "%mem", "etime"] + queryCmd = "ps -p %s -o %s" % (queryPid, ",".join(queryParam)) + psCall = sysTools.call(queryCmd, 3600, True) + + if psCall and len(psCall) == 2: + stats = psCall[1].strip().split() + if len(stats) == 4: uptime = stats[3] + + # checks if tor has been running for at least a day, the reason being that + # the state tracks a day's worth of data and this should only prepopulate + # results associated with this tor instance + if not uptime or not "-" in uptime: + msg = PREPOPULATE_FAILURE_MSG % "insufficient uptime" + log.log(self._config["log.graph.bw.prepopulateFailure"], msg) + return False + + # get the user's data directory (usually '~/.tor') + dataDir = conn.getOption("DataDirectory") + if not dataDir: + msg = PREPOPULATE_FAILURE_MSG % "data directory not found" + log.log(self._config["log.graph.bw.prepopulateFailure"], msg) + return False + + # attempt to open the state file + try: stateFile = open("%s%s/state" % (conn.getPathPrefix(), dataDir), "r") + except IOError: + msg = PREPOPULATE_FAILURE_MSG % "unable to read the state file" + log.log(self._config["log.graph.bw.prepopulateFailure"], msg) + return False + + # get the BWHistory entries (ordered oldest to newest) and number of + # intervals since last recorded + bwReadEntries, bwWriteEntries = None, None + missingReadEntries, missingWriteEntries = None, None + + # converts from gmt to local with respect to DST + tz_offset = time.altzone if time.localtime()[8] else time.timezone + + for line in stateFile: + line = line.strip() + + # According to the rep_hist_update_state() function the BWHistory*Ends + # correspond to the start of the following sampling period. Also, the + # most recent values of BWHistory*Values appear to be an incremental + # counter for the current sampling period. Hence, offsets are added to + # account for both. + + if line.startswith("BWHistoryReadValues"): + bwReadEntries = line[20:].split(",") + bwReadEntries = [int(entry) / 1024.0 / 900 for entry in bwReadEntries] + bwReadEntries.pop() + elif line.startswith("BWHistoryWriteValues"): + bwWriteEntries = line[21:].split(",") + bwWriteEntries = [int(entry) / 1024.0 / 900 for entry in bwWriteEntries] + bwWriteEntries.pop() + elif line.startswith("BWHistoryReadEnds"): + lastReadTime = time.mktime(time.strptime(line[18:], "%Y-%m-%d %H:%M:%S")) - tz_offset + lastReadTime -= 900 + missingReadEntries = int((time.time() - lastReadTime) / 900) + elif line.startswith("BWHistoryWriteEnds"): + lastWriteTime = time.mktime(time.strptime(line[19:], "%Y-%m-%d %H:%M:%S")) - tz_offset + lastWriteTime -= 900 + missingWriteEntries = int((time.time() - lastWriteTime) / 900) + + if not bwReadEntries or not bwWriteEntries or not lastReadTime or not lastWriteTime: + msg = PREPOPULATE_FAILURE_MSG % "bandwidth stats missing from state file" + log.log(self._config["log.graph.bw.prepopulateFailure"], msg) + return False + + # fills missing entries with the last value + bwReadEntries += [bwReadEntries[-1]] * missingReadEntries + bwWriteEntries += [bwWriteEntries[-1]] * missingWriteEntries + + # crops starting entries so they're the same size + entryCount = min(len(bwReadEntries), len(bwWriteEntries), self.maxCol) + bwReadEntries = bwReadEntries[len(bwReadEntries) - entryCount:] + bwWriteEntries = bwWriteEntries[len(bwWriteEntries) - entryCount:] + + # gets index for 15-minute interval + intervalIndex = 0 + for indexEntry in graphPanel.UPDATE_INTERVALS: + if indexEntry[1] == 900: break + else: intervalIndex += 1 + + # fills the graphing parameters with state information + for i in range(entryCount): + readVal, writeVal = bwReadEntries[i], bwWriteEntries[i] + + self.lastPrimary, self.lastSecondary = readVal, writeVal + + self.prepopulatePrimaryTotal += readVal * 900 + self.prepopulateSecondaryTotal += writeVal * 900 + self.prepopulateTicks += 900 + + self.primaryCounts[intervalIndex].insert(0, readVal) + self.secondaryCounts[intervalIndex].insert(0, writeVal) + + self.maxPrimary[intervalIndex] = max(self.primaryCounts) + self.maxSecondary[intervalIndex] = max(self.secondaryCounts) + del self.primaryCounts[intervalIndex][self.maxCol + 1:] + del self.secondaryCounts[intervalIndex][self.maxCol + 1:] + + msg = PREPOPULATE_SUCCESS_MSG + missingSec = time.time() - min(lastReadTime, lastWriteTime) + if missingSec: msg += " (%s is missing)" % uiTools.getTimeLabel(missingSec, 0, True) + log.log(self._config["log.graph.bw.prepopulateSuccess"], msg) + + return True + + def bandwidth_event(self, event): + if self.isAccounting and self.isNextTickRedraw(): + if time.time() - self.accountingLastUpdated >= self._config["features.graph.bw.accounting.rate"]: + self._updateAccountingInfo() + + # scales units from B to KB for graphing + self._processEvent(event.read / 1024.0, event.written / 1024.0) + + def draw(self, panel, width, height): + # line of the graph's x-axis labeling + labelingLine = graphPanel.GraphStats.getContentHeight(self) + panel.graphHeight - 2 + + # if display is narrow, overwrites x-axis labels with avg / total stats + if width <= COLLAPSE_WIDTH: + # clears line + panel.addstr(labelingLine, 0, " " * width) + graphCol = min((width - 10) / 2, self.maxCol) + + primaryFooter = "%s, %s" % (self._getAvgLabel(True), self._getTotalLabel(True)) + secondaryFooter = "%s, %s" % (self._getAvgLabel(False), self._getTotalLabel(False)) + + panel.addstr(labelingLine, 1, primaryFooter, uiTools.getColor(self.getColor(True))) + panel.addstr(labelingLine, graphCol + 6, secondaryFooter, uiTools.getColor(self.getColor(False))) + + # provides accounting stats if enabled + if self.isAccounting: + if torTools.getConn().isAlive(): + status = self.accountingInfo["status"] + + hibernateColor = "green" + if status == "soft": hibernateColor = "yellow" + elif status == "hard": hibernateColor = "red" + elif status == "": + # failed to be queried + status, hibernateColor = "unknown", "red" + + panel.addfstr(labelingLine + 2, 0, "Accounting (<%s>%s</%s>)" % (hibernateColor, status, hibernateColor)) + + resetTime = self.accountingInfo["resetTime"] + if not resetTime: resetTime = "unknown" + panel.addstr(labelingLine + 2, 35, "Time to reset: %s" % resetTime) + + used, total = self.accountingInfo["read"], self.accountingInfo["readLimit"] + if used and total: + panel.addstr(labelingLine + 3, 2, "%s / %s" % (used, total), uiTools.getColor(self.getColor(True))) + + used, total = self.accountingInfo["written"], self.accountingInfo["writtenLimit"] + if used and total: + panel.addstr(labelingLine + 3, 37, "%s / %s" % (used, total), uiTools.getColor(self.getColor(False))) + else: + panel.addfstr(labelingLine + 2, 0, "Accounting: Connection Closed...") + + def getTitle(self, width): + stats = list(self._titleStats) + + while True: + if not stats: return "Bandwidth:" + else: + label = "Bandwidth (%s):" % ", ".join(stats) + + if len(label) > width: del stats[-1] + else: return label + + def getHeaderLabel(self, width, isPrimary): + graphType = "Download" if isPrimary else "Upload" + stats = [""] + + # if wide then avg and total are part of the header, otherwise they're on + # the x-axis + if width * 2 > COLLAPSE_WIDTH: + stats = [""] * 3 + stats[1] = "- %s" % self._getAvgLabel(isPrimary) + stats[2] = ", %s" % self._getTotalLabel(isPrimary) + + stats[0] = "%-14s" % ("%s/sec" % uiTools.getSizeLabel((self.lastPrimary if isPrimary else self.lastSecondary) * 1024, 1, False, self._config["features.graph.bw.transferInBytes"])) + + # drops label's components if there's not enough space + labeling = graphType + " (" + "".join(stats).strip() + "):" + while len(labeling) >= width: + if len(stats) > 1: + del stats[-1] + labeling = graphType + " (" + "".join(stats).strip() + "):" + else: + labeling = graphType + ":" + break + + return labeling + + def getColor(self, isPrimary): + return DL_COLOR if isPrimary else UL_COLOR + + def getContentHeight(self): + baseHeight = graphPanel.GraphStats.getContentHeight(self) + return baseHeight + 3 if self.isAccounting else baseHeight + + def new_desc_event(self, event): + # updates self._titleStats with updated values + conn = torTools.getConn() + if not conn.isAlive(): return # keep old values + + myFingerprint = conn.getInfo("fingerprint") + if not self._titleStats or not myFingerprint or (event and myFingerprint in event.idlist): + stats = [] + bwRate = conn.getMyBandwidthRate() + bwBurst = conn.getMyBandwidthBurst() + bwObserved = conn.getMyBandwidthObserved() + bwMeasured = conn.getMyBandwidthMeasured() + labelInBytes = self._config["features.graph.bw.transferInBytes"] + + if bwRate and bwBurst: + bwRateLabel = uiTools.getSizeLabel(bwRate, 1, False, labelInBytes) + bwBurstLabel = uiTools.getSizeLabel(bwBurst, 1, False, labelInBytes) + + # if both are using rounded values then strip off the ".0" decimal + if ".0" in bwRateLabel and ".0" in bwBurstLabel: + bwRateLabel = bwRateLabel.replace(".0", "") + bwBurstLabel = bwBurstLabel.replace(".0", "") + + stats.append("limit: %s/s" % bwRateLabel) + stats.append("burst: %s/s" % bwBurstLabel) + + # Provide the observed bandwidth either if the measured bandwidth isn't + # available or if the measured bandwidth is the observed (this happens + # if there isn't yet enough bandwidth measurements). + if bwObserved and (not bwMeasured or bwMeasured == bwObserved): + stats.append("observed: %s/s" % uiTools.getSizeLabel(bwObserved, 1, False, labelInBytes)) + elif bwMeasured: + stats.append("measured: %s/s" % uiTools.getSizeLabel(bwMeasured, 1, False, labelInBytes)) + + self._titleStats = stats + + def _getAvgLabel(self, isPrimary): + total = self.primaryTotal if isPrimary else self.secondaryTotal + total += self.prepopulatePrimaryTotal if isPrimary else self.prepopulateSecondaryTotal + return "avg: %s/sec" % uiTools.getSizeLabel((total / max(1, self.tick + self.prepopulateTicks)) * 1024, 1, False, self._config["features.graph.bw.transferInBytes"]) + + def _getTotalLabel(self, isPrimary): + total = self.primaryTotal if isPrimary else self.secondaryTotal + total += self.initialPrimaryTotal if isPrimary else self.initialSecondaryTotal + return "total: %s" % uiTools.getSizeLabel(total * 1024, 1) + + def _updateAccountingInfo(self): + """ + Updates mapping used for accounting info. This includes the following keys: + status, resetTime, read, written, readLimit, writtenLimit + + Any failed lookups result in a mapping to an empty string. + """ + + conn = torTools.getConn() + queried = dict([(arg, "") for arg in ACCOUNTING_ARGS]) + queried["status"] = conn.getInfo("accounting/hibernating") + + # provides a nicely formatted reset time + endInterval = conn.getInfo("accounting/interval-end") + if endInterval: + # converts from gmt to local with respect to DST + if time.localtime()[8]: tz_offset = time.altzone + else: tz_offset = time.timezone + + sec = time.mktime(time.strptime(endInterval, "%Y-%m-%d %H:%M:%S")) - time.time() - tz_offset + if self._config["features.graph.bw.accounting.isTimeLong"]: + queried["resetTime"] = ", ".join(uiTools.getTimeLabels(sec, True)) + else: + days = sec / 86400 + sec %= 86400 + hours = sec / 3600 + sec %= 3600 + minutes = sec / 60 + sec %= 60 + queried["resetTime"] = "%i:%02i:%02i:%02i" % (days, hours, minutes, sec) + + # number of bytes used and in total for the accounting period + used = conn.getInfo("accounting/bytes") + left = conn.getInfo("accounting/bytes-left") + + if used and left: + usedComp, leftComp = used.split(" "), left.split(" ") + read, written = int(usedComp[0]), int(usedComp[1]) + readLeft, writtenLeft = int(leftComp[0]), int(leftComp[1]) + + queried["read"] = uiTools.getSizeLabel(read) + queried["written"] = uiTools.getSizeLabel(written) + queried["readLimit"] = uiTools.getSizeLabel(read + readLeft) + queried["writtenLimit"] = uiTools.getSizeLabel(written + writtenLeft) + + self.accountingInfo = queried + self.accountingLastUpdated = time.time() + diff --git a/src/cli/graphing/connStats.py b/src/cli/graphing/connStats.py new file mode 100644 index 0000000..51227b7 --- /dev/null +++ b/src/cli/graphing/connStats.py @@ -0,0 +1,54 @@ +""" +Tracks stats concerning tor's current connections. +""" + +from cli.graphing import graphPanel +from util import connections, torTools + +class ConnStats(graphPanel.GraphStats): + """ + Tracks number of connections, counting client and directory connections as + outbound. Control connections are excluded from counts. + """ + + def __init__(self): + graphPanel.GraphStats.__init__(self) + + # listens for tor reload (sighup) events which can reset the ports tor uses + conn = torTools.getConn() + self.orPort, self.dirPort, self.controlPort = "0", "0", "0" + self.resetListener(conn, torTools.State.INIT) # initialize port values + conn.addStatusListener(self.resetListener) + + def resetListener(self, conn, eventType): + if eventType == torTools.State.INIT: + self.orPort = conn.getOption("ORPort", "0") + self.dirPort = conn.getOption("DirPort", "0") + self.controlPort = conn.getOption("ControlPort", "0") + + def eventTick(self): + """ + Fetches connection stats from cached information. + """ + + inboundCount, outboundCount = 0, 0 + + for entry in connections.getResolver("tor").getConnections(): + localPort = entry[1] + if localPort in (self.orPort, self.dirPort): inboundCount += 1 + elif localPort == self.controlPort: pass # control connection + else: outboundCount += 1 + + self._processEvent(inboundCount, outboundCount) + + def getTitle(self, width): + return "Connection Count:" + + def getHeaderLabel(self, width, isPrimary): + avg = (self.primaryTotal if isPrimary else self.secondaryTotal) / max(1, self.tick) + if isPrimary: return "Inbound (%s, avg: %s):" % (self.lastPrimary, avg) + else: return "Outbound (%s, avg: %s):" % (self.lastSecondary, avg) + + def getRefreshRate(self): + return 5 + diff --git a/src/cli/graphing/graphPanel.py b/src/cli/graphing/graphPanel.py new file mode 100644 index 0000000..e4b493d --- /dev/null +++ b/src/cli/graphing/graphPanel.py @@ -0,0 +1,407 @@ +""" +Flexible panel for presenting bar graphs for a variety of stats. This panel is +just concerned with the rendering of information, which is actually collected +and stored by implementations of the GraphStats interface. Panels are made up +of a title, followed by headers and graphs for two sets of stats. For +instance... + +Bandwidth (cap: 5 MB, burst: 10 MB): +Downloaded (0.0 B/sec): Uploaded (0.0 B/sec): + 34 30 + * * + ** * * * ** + * * * ** ** ** *** ** ** ** ** + ********* ****** ****** ********* ****** ****** + 0 ************ **************** 0 ************ **************** + 25s 50 1m 1.6 2.0 25s 50 1m 1.6 2.0 +""" + +import copy +import curses +from TorCtl import TorCtl + +from util import enum, panel, uiTools + +# time intervals at which graphs can be updated +UPDATE_INTERVALS = [("each second", 1), ("5 seconds", 5), ("30 seconds", 30), + ("minutely", 60), ("15 minute", 900), ("30 minute", 1800), + ("hourly", 3600), ("daily", 86400)] + +DEFAULT_CONTENT_HEIGHT = 4 # space needed for labeling above and below the graph +DEFAULT_COLOR_PRIMARY, DEFAULT_COLOR_SECONDARY = "green", "cyan" +MIN_GRAPH_HEIGHT = 1 + +# enums for graph bounds: +# Bounds.GLOBAL_MAX - global maximum (highest value ever seen) +# Bounds.LOCAL_MAX - local maximum (highest value currently on the graph) +# Bounds.TIGHT - local maximum and minimum +Bounds = enum.Enum("GLOBAL_MAX", "LOCAL_MAX", "TIGHT") + +WIDE_LABELING_GRAPH_COL = 50 # minimum graph columns to use wide spacing for x-axis labels + +# used for setting defaults when initializing GraphStats and GraphPanel instances +CONFIG = {"features.graph.height": 7, + "features.graph.interval": 0, + "features.graph.bound": 1, + "features.graph.maxWidth": 150, + "features.graph.showIntermediateBounds": True} + +def loadConfig(config): + config.update(CONFIG, { + "features.graph.height": MIN_GRAPH_HEIGHT, + "features.graph.maxWidth": 1, + "features.graph.interval": (0, len(UPDATE_INTERVALS) - 1), + "features.graph.bound": (0, 2)}) + +class GraphStats(TorCtl.PostEventListener): + """ + Module that's expected to update dynamically and provide attributes to be + graphed. Up to two graphs (a 'primary' and 'secondary') can be displayed at a + time and timescale parameters use the labels defined in UPDATE_INTERVALS. + """ + + def __init__(self, isPauseBuffer=False): + """ + Initializes parameters needed to present a graph. + """ + + TorCtl.PostEventListener.__init__(self) + + # panel to be redrawn when updated (set when added to GraphPanel) + self._graphPanel = None + + # mirror instance used to track updates when paused + self.isPaused, self.isPauseBuffer = False, isPauseBuffer + if isPauseBuffer: self._pauseBuffer = None + else: self._pauseBuffer = GraphStats(True) + + # tracked stats + self.tick = 0 # number of processed events + self.lastPrimary, self.lastSecondary = 0, 0 # most recent registered stats + self.primaryTotal, self.secondaryTotal = 0, 0 # sum of all stats seen + + # timescale dependent stats + self.maxCol = CONFIG["features.graph.maxWidth"] + self.maxPrimary, self.maxSecondary = {}, {} + self.primaryCounts, self.secondaryCounts = {}, {} + + for i in range(len(UPDATE_INTERVALS)): + # recent rates for graph + self.maxPrimary[i] = 0 + self.maxSecondary[i] = 0 + + # historic stats for graph, first is accumulator + # iterative insert needed to avoid making shallow copies (nasty, nasty gotcha) + self.primaryCounts[i] = (self.maxCol + 1) * [0] + self.secondaryCounts[i] = (self.maxCol + 1) * [0] + + def eventTick(self): + """ + Called when it's time to process another event. All graphs use tor BW + events to keep in sync with each other (this happens once a second). + """ + + pass + + def isNextTickRedraw(self): + """ + Provides true if the following tick (call to _processEvent) will result in + being redrawn. + """ + + if self._graphPanel and not self.isPauseBuffer and not self.isPaused: + # use the minimum of the current refresh rate and the panel's + updateRate = UPDATE_INTERVALS[self._graphPanel.updateInterval][1] + return (self.tick + 1) % min(updateRate, self.getRefreshRate()) == 0 + else: return False + + def getTitle(self, width): + """ + Provides top label. + """ + + return "" + + def getHeaderLabel(self, width, isPrimary): + """ + Provides labeling presented at the top of the graph. + """ + + return "" + + def getColor(self, isPrimary): + """ + Provides the color to be used for the graph and stats. + """ + + return DEFAULT_COLOR_PRIMARY if isPrimary else DEFAULT_COLOR_SECONDARY + + def getContentHeight(self): + """ + Provides the height content should take up (not including the graph). + """ + + return DEFAULT_CONTENT_HEIGHT + + def getRefreshRate(self): + """ + Provides the number of ticks between when the stats have new values to be + redrawn. + """ + + return 1 + + def isVisible(self): + """ + True if the stat has content to present, false if it should be hidden. + """ + + return True + + def draw(self, panel, width, height): + """ + Allows for any custom drawing monitor wishes to append. + """ + + pass + + def setPaused(self, isPause): + """ + If true, prevents bandwidth updates from being presented. This is a no-op + if a pause buffer. + """ + + if isPause == self.isPaused or self.isPauseBuffer: return + self.isPaused = isPause + + if self.isPaused: active, inactive = self._pauseBuffer, self + else: active, inactive = self, self._pauseBuffer + self._parameterSwap(active, inactive) + + def bandwidth_event(self, event): + self.eventTick() + + def _parameterSwap(self, active, inactive): + """ + Either overwrites parameters of pauseBuffer or with the current values or + vice versa. This is a helper method for setPaused and should be overwritten + to append with additional parameters that need to be preserved when paused. + """ + + # The pause buffer is constructed as a GraphStats instance which will + # become problematic if this is overridden by any implementations (which + # currently isn't the case). If this happens then the pause buffer will + # need to be of the requester's type (not quite sure how to do this + # gracefully...). + + active.tick = inactive.tick + active.lastPrimary = inactive.lastPrimary + active.lastSecondary = inactive.lastSecondary + active.primaryTotal = inactive.primaryTotal + active.secondaryTotal = inactive.secondaryTotal + active.maxPrimary = dict(inactive.maxPrimary) + active.maxSecondary = dict(inactive.maxSecondary) + active.primaryCounts = copy.deepcopy(inactive.primaryCounts) + active.secondaryCounts = copy.deepcopy(inactive.secondaryCounts) + + def _processEvent(self, primary, secondary): + """ + Includes new stats in graphs and notifies associated GraphPanel of changes. + """ + + if self.isPaused: self._pauseBuffer._processEvent(primary, secondary) + else: + isRedraw = self.isNextTickRedraw() + + self.lastPrimary, self.lastSecondary = primary, secondary + self.primaryTotal += primary + self.secondaryTotal += secondary + + # updates for all time intervals + self.tick += 1 + for i in range(len(UPDATE_INTERVALS)): + lable, timescale = UPDATE_INTERVALS[i] + + self.primaryCounts[i][0] += primary + self.secondaryCounts[i][0] += secondary + + if self.tick % timescale == 0: + self.maxPrimary[i] = max(self.maxPrimary[i], self.primaryCounts[i][0] / timescale) + self.primaryCounts[i][0] /= timescale + self.primaryCounts[i].insert(0, 0) + del self.primaryCounts[i][self.maxCol + 1:] + + self.maxSecondary[i] = max(self.maxSecondary[i], self.secondaryCounts[i][0] / timescale) + self.secondaryCounts[i][0] /= timescale + self.secondaryCounts[i].insert(0, 0) + del self.secondaryCounts[i][self.maxCol + 1:] + + if isRedraw: self._graphPanel.redraw(True) + +class GraphPanel(panel.Panel): + """ + Panel displaying a graph, drawing statistics from custom GraphStats + implementations. + """ + + def __init__(self, stdscr): + panel.Panel.__init__(self, stdscr, "graph", 0) + self.updateInterval = CONFIG["features.graph.interval"] + self.bounds = Bounds.values()[CONFIG["features.graph.bound"]] + self.graphHeight = CONFIG["features.graph.height"] + self.currentDisplay = None # label of the stats currently being displayed + self.stats = {} # available stats (mappings of label -> instance) + self.showLabel = True # shows top label if true, hides otherwise + self.isPaused = False + + def getHeight(self): + """ + Provides the height requested by the currently displayed GraphStats (zero + if hidden). + """ + + if self.currentDisplay and self.stats[self.currentDisplay].isVisible(): + return self.stats[self.currentDisplay].getContentHeight() + self.graphHeight + else: return 0 + + def setGraphHeight(self, newGraphHeight): + """ + Sets the preferred height used for the graph (restricted to the + MIN_GRAPH_HEIGHT minimum). + + Arguments: + newGraphHeight - new height for the graph + """ + + self.graphHeight = max(MIN_GRAPH_HEIGHT, newGraphHeight) + + def draw(self, width, height): + """ Redraws graph panel """ + + if self.currentDisplay: + param = self.stats[self.currentDisplay] + graphCol = min((width - 10) / 2, param.maxCol) + + primaryColor = uiTools.getColor(param.getColor(True)) + secondaryColor = uiTools.getColor(param.getColor(False)) + + if self.showLabel: self.addstr(0, 0, param.getTitle(width), curses.A_STANDOUT) + + # top labels + left, right = param.getHeaderLabel(width / 2, True), param.getHeaderLabel(width / 2, False) + if left: self.addstr(1, 0, left, curses.A_BOLD | primaryColor) + if right: self.addstr(1, graphCol + 5, right, curses.A_BOLD | secondaryColor) + + # determines max/min value on the graph + if self.bounds == Bounds.GLOBAL_MAX: + primaryMaxBound = int(param.maxPrimary[self.updateInterval]) + secondaryMaxBound = int(param.maxSecondary[self.updateInterval]) + else: + # both Bounds.LOCAL_MAX and Bounds.TIGHT use local maxima + if graphCol < 2: + # nothing being displayed + primaryMaxBound, secondaryMaxBound = 0, 0 + else: + primaryMaxBound = int(max(param.primaryCounts[self.updateInterval][1:graphCol + 1])) + secondaryMaxBound = int(max(param.secondaryCounts[self.updateInterval][1:graphCol + 1])) + + primaryMinBound = secondaryMinBound = 0 + if self.bounds == Bounds.TIGHT: + primaryMinBound = int(min(param.primaryCounts[self.updateInterval][1:graphCol + 1])) + secondaryMinBound = int(min(param.secondaryCounts[self.updateInterval][1:graphCol + 1])) + + # if the max = min (ie, all values are the same) then use zero lower + # bound so a graph is still displayed + if primaryMinBound == primaryMaxBound: primaryMinBound = 0 + if secondaryMinBound == secondaryMaxBound: secondaryMinBound = 0 + + # displays upper and lower bounds + self.addstr(2, 0, "%4i" % primaryMaxBound, primaryColor) + self.addstr(self.graphHeight + 1, 0, "%4i" % primaryMinBound, primaryColor) + + self.addstr(2, graphCol + 5, "%4i" % secondaryMaxBound, secondaryColor) + self.addstr(self.graphHeight + 1, graphCol + 5, "%4i" % secondaryMinBound, secondaryColor) + + # displays intermediate bounds on every other row + if CONFIG["features.graph.showIntermediateBounds"]: + ticks = (self.graphHeight - 3) / 2 + for i in range(ticks): + row = self.graphHeight - (2 * i) - 3 + if self.graphHeight % 2 == 0 and i >= (ticks / 2): row -= 1 + + if primaryMinBound != primaryMaxBound: + primaryVal = (primaryMaxBound - primaryMinBound) / (self.graphHeight - 1) * (self.graphHeight - row - 1) + if not primaryVal in (primaryMinBound, primaryMaxBound): self.addstr(row + 2, 0, "%4i" % primaryVal, primaryColor) + + if secondaryMinBound != secondaryMaxBound: + secondaryVal = (secondaryMaxBound - secondaryMinBound) / (self.graphHeight - 1) * (self.graphHeight - row - 1) + if not secondaryVal in (secondaryMinBound, secondaryMaxBound): self.addstr(row + 2, graphCol + 5, "%4i" % secondaryVal, secondaryColor) + + # creates bar graph (both primary and secondary) + for col in range(graphCol): + colCount = int(param.primaryCounts[self.updateInterval][col + 1]) - primaryMinBound + colHeight = min(self.graphHeight, self.graphHeight * colCount / (max(1, primaryMaxBound) - primaryMinBound)) + for row in range(colHeight): self.addstr(self.graphHeight + 1 - row, col + 5, " ", curses.A_STANDOUT | primaryColor) + + colCount = int(param.secondaryCounts[self.updateInterval][col + 1]) - secondaryMinBound + colHeight = min(self.graphHeight, self.graphHeight * colCount / (max(1, secondaryMaxBound) - secondaryMinBound)) + for row in range(colHeight): self.addstr(self.graphHeight + 1 - row, col + graphCol + 10, " ", curses.A_STANDOUT | secondaryColor) + + # bottom labeling of x-axis + intervalSec = 1 # seconds per labeling + for i in range(len(UPDATE_INTERVALS)): + if i == self.updateInterval: intervalSec = UPDATE_INTERVALS[i][1] + + intervalSpacing = 10 if graphCol >= WIDE_LABELING_GRAPH_COL else 5 + unitsLabel, decimalPrecision = None, 0 + for i in range((graphCol - 4) / intervalSpacing): + loc = (i + 1) * intervalSpacing + timeLabel = uiTools.getTimeLabel(loc * intervalSec, decimalPrecision) + + if not unitsLabel: unitsLabel = timeLabel[-1] + elif unitsLabel != timeLabel[-1]: + # upped scale so also up precision of future measurements + unitsLabel = timeLabel[-1] + decimalPrecision += 1 + else: + # if constrained on space then strips labeling since already provided + timeLabel = timeLabel[:-1] + + self.addstr(self.graphHeight + 2, 4 + loc, timeLabel, primaryColor) + self.addstr(self.graphHeight + 2, graphCol + 10 + loc, timeLabel, secondaryColor) + + param.draw(self, width, height) # allows current stats to modify the display + + def addStats(self, label, stats): + """ + Makes GraphStats instance available in the panel. + """ + + stats._graphPanel = self + stats.isPaused = True + self.stats[label] = stats + + def setStats(self, label): + """ + Sets the currently displayed stats instance, hiding panel if None. + """ + + if label != self.currentDisplay: + if self.currentDisplay: self.stats[self.currentDisplay].setPaused(True) + + if not label: + self.currentDisplay = None + elif label in self.stats.keys(): + self.currentDisplay = label + self.stats[label].setPaused(self.isPaused) + else: raise ValueError("Unrecognized stats label: %s" % label) + + def setPaused(self, isPause): + """ + If true, prevents bandwidth updates from being presented. + """ + + if isPause == self.isPaused: return + self.isPaused = isPause + if self.currentDisplay: self.stats[self.currentDisplay].setPaused(self.isPaused) + diff --git a/src/cli/graphing/resourceStats.py b/src/cli/graphing/resourceStats.py new file mode 100644 index 0000000..f26d5c1 --- /dev/null +++ b/src/cli/graphing/resourceStats.py @@ -0,0 +1,47 @@ +""" +Tracks the system resource usage (cpu and memory) of the tor process. +""" + +from cli.graphing import graphPanel +from util import sysTools, torTools, uiTools + +class ResourceStats(graphPanel.GraphStats): + """ + System resource usage tracker. + """ + + def __init__(self): + graphPanel.GraphStats.__init__(self) + self.queryPid = torTools.getConn().getMyPid() + + def getTitle(self, width): + return "System Resources:" + + def getHeaderLabel(self, width, isPrimary): + avg = (self.primaryTotal if isPrimary else self.secondaryTotal) / max(1, self.tick) + lastAmount = self.lastPrimary if isPrimary else self.lastSecondary + + if isPrimary: + return "CPU (%0.1f%%, avg: %0.1f%%):" % (lastAmount, avg) + else: + # memory sizes are converted from MB to B before generating labels + usageLabel = uiTools.getSizeLabel(lastAmount * 1048576, 1) + avgLabel = uiTools.getSizeLabel(avg * 1048576, 1) + return "Memory (%s, avg: %s):" % (usageLabel, avgLabel) + + def eventTick(self): + """ + Fetch the cached measurement of resource usage from the ResourceTracker. + """ + + primary, secondary = 0, 0 + if self.queryPid: + resourceTracker = sysTools.getResourceTracker(self.queryPid) + + if not resourceTracker.lastQueryFailed(): + primary, _, secondary, _ = resourceTracker.getResourceUsage() + primary *= 100 # decimal percentage to whole numbers + secondary /= 1048576 # translate size to MB so axis labels are short + + self._processEvent(primary, secondary) + diff --git a/src/cli/headerPanel.py b/src/cli/headerPanel.py new file mode 100644 index 0000000..f653299 --- /dev/null +++ b/src/cli/headerPanel.py @@ -0,0 +1,474 @@ +""" +Top panel for every page, containing basic system and tor related information. +If there's room available then this expands to present its information in two +columns, otherwise it's laid out as follows: + arm - <hostname> (<os> <sys/version>) Tor <tor/version> (<new, old, recommended, etc>) + <nickname> - <address>:<orPort>, [Dir Port: <dirPort>, ]Control Port (<open, password, cookie>): <controlPort> + cpu: <cpu%> mem: <mem> (<mem%>) uid: <uid> uptime: <upmin>:<upsec> + fingerprint: <fingerprint> + +Example: + arm - odin (Linux 2.6.24-24-generic) Tor 0.2.1.19 (recommended) + odin - 76.104.132.98:9001, Dir Port: 9030, Control Port (cookie): 9051 + cpu: 14.6% mem: 42 MB (4.2%) pid: 20060 uptime: 48:27 + fingerprint: BDAD31F6F318E0413833E8EBDA956F76E4D66788 +""" + +import os +import time +import curses +import threading + +from util import log, panel, sysTools, torTools, uiTools + +# minimum width for which panel attempts to double up contents (two columns to +# better use screen real estate) +MIN_DUAL_COL_WIDTH = 141 + +FLAG_COLORS = {"Authority": "white", "BadExit": "red", "BadDirectory": "red", "Exit": "cyan", + "Fast": "yellow", "Guard": "green", "HSDir": "magenta", "Named": "blue", + "Stable": "blue", "Running": "yellow", "Unnamed": "magenta", "Valid": "green", + "V2Dir": "cyan", "V3Dir": "white"} + +VERSION_STATUS_COLORS = {"new": "blue", "new in series": "blue", "obsolete": "red", "recommended": "green", + "old": "red", "unrecommended": "red", "unknown": "cyan"} + +DEFAULT_CONFIG = {"features.showFdUsage": False, + "log.fdUsageSixtyPercent": log.NOTICE, + "log.fdUsageNinetyPercent": log.WARN} + +class HeaderPanel(panel.Panel, threading.Thread): + """ + Top area contenting tor settings and system information. Stats are stored in + the vals mapping, keys including: + tor/ version, versionStatus, nickname, orPort, dirPort, controlPort, + exitPolicy, isAuthPassword (bool), isAuthCookie (bool), + orListenAddr, *address, *fingerprint, *flags, pid, startTime, + *fdUsed, fdLimit, isFdLimitEstimate + sys/ hostname, os, version + stat/ *%torCpu, *%armCpu, *rss, *%mem + + * volatile parameter that'll be reset on each update + """ + + def __init__(self, stdscr, startTime, config = None): + panel.Panel.__init__(self, stdscr, "header", 0) + threading.Thread.__init__(self) + self.setDaemon(True) + + self._config = dict(DEFAULT_CONFIG) + if config: config.update(self._config) + + self._isTorConnected = True + self._lastUpdate = -1 # time the content was last revised + self._isPaused = False # prevents updates if true + self._halt = False # terminates thread if true + self._cond = threading.Condition() # used for pausing the thread + + # Time when the panel was paused or tor was stopped. This is used to + # freeze the uptime statistic (uptime increments normally when None). + self._haltTime = None + + # The last arm cpu usage sampling taken. This is a tuple of the form: + # (total arm cpu time, sampling timestamp) + # + # The initial cpu total should be zero. However, at startup the cpu time + # in practice is often greater than the real time causing the initially + # reported cpu usage to be over 100% (which shouldn't be possible on + # single core systems). + # + # Setting the initial cpu total to the value at this panel's init tends to + # give smoother results (staying in the same ballpark as the second + # sampling) so fudging the numbers this way for now. + + self._armCpuSampling = (sum(os.times()[:3]), startTime) + + # Last sampling received from the ResourceTracker, used to detect when it + # changes. + self._lastResourceFetch = -1 + + # flag to indicate if we've already given file descriptor warnings + self._isFdSixtyPercentWarned = False + self._isFdNinetyPercentWarned = False + + self.vals = {} + self.valsLock = threading.RLock() + self._update(True) + + # listens for tor reload (sighup) events + torTools.getConn().addStatusListener(self.resetListener) + + def getHeight(self): + """ + Provides the height of the content, which is dynamically determined by the + panel's maximum width. + """ + + isWide = self.getParent().getmaxyx()[1] >= MIN_DUAL_COL_WIDTH + if self.vals["tor/orPort"]: return 4 if isWide else 6 + else: return 3 if isWide else 4 + + def draw(self, width, height): + self.valsLock.acquire() + isWide = width + 1 >= MIN_DUAL_COL_WIDTH + + # space available for content + if isWide: + leftWidth = max(width / 2, 77) + rightWidth = width - leftWidth + else: leftWidth = rightWidth = width + + # Line 1 / Line 1 Left (system and tor version information) + sysNameLabel = "arm - %s" % self.vals["sys/hostname"] + contentSpace = min(leftWidth, 40) + + if len(sysNameLabel) + 10 <= contentSpace: + sysTypeLabel = "%s %s" % (self.vals["sys/os"], self.vals["sys/version"]) + sysTypeLabel = uiTools.cropStr(sysTypeLabel, contentSpace - len(sysNameLabel) - 3, 4) + self.addstr(0, 0, "%s (%s)" % (sysNameLabel, sysTypeLabel)) + else: + self.addstr(0, 0, uiTools.cropStr(sysNameLabel, contentSpace)) + + contentSpace = leftWidth - 43 + if 7 + len(self.vals["tor/version"]) + len(self.vals["tor/versionStatus"]) <= contentSpace: + versionColor = VERSION_STATUS_COLORS[self.vals["tor/versionStatus"]] if \ + self.vals["tor/versionStatus"] in VERSION_STATUS_COLORS else "white" + versionStatusMsg = "<%s>%s</%s>" % (versionColor, self.vals["tor/versionStatus"], versionColor) + self.addfstr(0, 43, "Tor %s (%s)" % (self.vals["tor/version"], versionStatusMsg)) + elif 11 <= contentSpace: + self.addstr(0, 43, uiTools.cropStr("Tor %s" % self.vals["tor/version"], contentSpace, 4)) + + # Line 2 / Line 2 Left (tor ip/port information) + if self.vals["tor/orPort"]: + myAddress = "Unknown" + if self.vals["tor/orListenAddr"]: myAddress = self.vals["tor/orListenAddr"] + elif self.vals["tor/address"]: myAddress = self.vals["tor/address"] + + # acting as a relay (we can assume certain parameters are set + entry = "" + dirPortLabel = ", Dir Port: %s" % self.vals["tor/dirPort"] if self.vals["tor/dirPort"] != "0" else "" + for label in (self.vals["tor/nickname"], " - " + myAddress, ":" + self.vals["tor/orPort"], dirPortLabel): + if len(entry) + len(label) <= leftWidth: entry += label + else: break + else: + # non-relay (client only) + # TODO: not sure what sort of stats to provide... + entry = "<red>Relaying Disabled</red>" + + if self.vals["tor/isAuthPassword"]: authType = "password" + elif self.vals["tor/isAuthCookie"]: authType = "cookie" + else: authType = "open" + + if len(entry) + 19 + len(self.vals["tor/controlPort"]) + len(authType) <= leftWidth: + authColor = "red" if authType == "open" else "green" + authLabel = "<%s>%s</%s>" % (authColor, authType, authColor) + self.addfstr(1, 0, "%s, Control Port (%s): %s" % (entry, authLabel, self.vals["tor/controlPort"])) + elif len(entry) + 16 + len(self.vals["tor/controlPort"]) <= leftWidth: + self.addstr(1, 0, "%s, Control Port: %s" % (entry, self.vals["tor/controlPort"])) + else: self.addstr(1, 0, entry) + + # Line 3 / Line 1 Right (system usage info) + y, x = (0, leftWidth) if isWide else (2, 0) + if self.vals["stat/rss"] != "0": memoryLabel = uiTools.getSizeLabel(int(self.vals["stat/rss"])) + else: memoryLabel = "0" + + uptimeLabel = "" + if self.vals["tor/startTime"]: + if self._haltTime: + # freeze the uptime when paused or the tor process is stopped + uptimeLabel = uiTools.getShortTimeLabel(self._haltTime - self.vals["tor/startTime"]) + else: + uptimeLabel = uiTools.getShortTimeLabel(time.time() - self.vals["tor/startTime"]) + + sysFields = ((0, "cpu: %s%% tor, %s%% arm" % (self.vals["stat/%torCpu"], self.vals["stat/%armCpu"])), + (27, "mem: %s (%s%%)" % (memoryLabel, self.vals["stat/%mem"])), + (47, "pid: %s" % (self.vals["tor/pid"] if self._isTorConnected else "")), + (59, "uptime: %s" % uptimeLabel)) + + for (start, label) in sysFields: + if start + len(label) <= rightWidth: self.addstr(y, x + start, label) + else: break + + if self.vals["tor/orPort"]: + # Line 4 / Line 2 Right (fingerprint, and possibly file descriptor usage) + y, x = (1, leftWidth) if isWide else (3, 0) + + fingerprintLabel = uiTools.cropStr("fingerprint: %s" % self.vals["tor/fingerprint"], width) + self.addstr(y, x, fingerprintLabel) + + # if there's room and we're able to retrieve both the file descriptor + # usage and limit then it might be presented + if width - x - 59 >= 20 and self.vals["tor/fdUsed"] and self.vals["tor/fdLimit"]: + # display file descriptor usage if we're either configured to do so or + # running out + + fdPercent = 100 * self.vals["tor/fdUsed"] / self.vals["tor/fdLimit"] + + if fdPercent >= 60 or self._config["features.showFdUsage"]: + fdPercentLabel, fdPercentFormat = "%i%%" % fdPercent, curses.A_NORMAL + if fdPercent >= 95: + fdPercentFormat = curses.A_BOLD | uiTools.getColor("red") + elif fdPercent >= 90: + fdPercentFormat = uiTools.getColor("red") + elif fdPercent >= 60: + fdPercentFormat = uiTools.getColor("yellow") + + estimateChar = "?" if self.vals["tor/isFdLimitEstimate"] else "" + baseLabel = "file desc: %i / %i%s (" % (self.vals["tor/fdUsed"], self.vals["tor/fdLimit"], estimateChar) + + self.addstr(y, x + 59, baseLabel) + self.addstr(y, x + 59 + len(baseLabel), fdPercentLabel, fdPercentFormat) + self.addstr(y, x + 59 + len(baseLabel) + len(fdPercentLabel), ")") + + # Line 5 / Line 3 Left (flags) + if self._isTorConnected: + flagLine = "flags: " + for flag in self.vals["tor/flags"]: + flagColor = FLAG_COLORS[flag] if flag in FLAG_COLORS.keys() else "white" + flagLine += "<%s>%s</%s>, " % (flagColor, flag, flagColor) + + if len(self.vals["tor/flags"]) > 0: flagLine = flagLine[:-2] + else: flagLine += "<cyan>none</cyan>" + + self.addfstr(2 if isWide else 4, 0, flagLine) + else: + statusTime = torTools.getConn().getStatus()[1] + statusTimeLabel = time.strftime("%H:%M %m/%d/%Y", time.localtime(statusTime)) + self.addfstr(2 if isWide else 4, 0, "<red>Tor Disconnected</red> (%s)" % statusTimeLabel) + + # Undisplayed / Line 3 Right (exit policy) + if isWide: + exitPolicy = self.vals["tor/exitPolicy"] + + # adds note when default exit policy is appended + if exitPolicy == "": exitPolicy = "<default>" + elif not exitPolicy.endswith((" *:*", " *")): exitPolicy += ", <default>" + + # color codes accepts to be green, rejects to be red, and default marker to be cyan + isSimple = len(exitPolicy) > rightWidth - 13 + policies = exitPolicy.split(", ") + for i in range(len(policies)): + policy = policies[i].strip() + displayedPolicy = policy.replace("accept", "").replace("reject", "").strip() if isSimple else policy + if policy.startswith("accept"): policy = "<green>%s</green>" % displayedPolicy + elif policy.startswith("reject"): policy = "<red>%s</red>" % displayedPolicy + elif policy.startswith("<default>"): policy = "<cyan>%s</cyan>" % displayedPolicy + policies[i] = policy + + self.addfstr(2, leftWidth, "exit policy: %s" % ", ".join(policies)) + else: + # Client only + # TODO: not sure what information to provide here... + pass + + self.valsLock.release() + + def setPaused(self, isPause): + """ + If true, prevents updates from being presented. + """ + + if not self._isPaused == isPause: + self._isPaused = isPause + if self._isTorConnected: + if isPause: self._haltTime = time.time() + else: self._haltTime = None + + # Redraw now so we'll be displaying the state right when paused + # (otherwise the uptime might be off by a second, and change when + # the panel's redrawn for other reasons). + self.redraw(True) + + def run(self): + """ + Keeps stats updated, checking for new information at a set rate. + """ + + lastDraw = time.time() - 1 + while not self._halt: + currentTime = time.time() + + if self._isPaused or currentTime - lastDraw < 1 or not self._isTorConnected: + self._cond.acquire() + if not self._halt: self._cond.wait(0.2) + self._cond.release() + else: + # Update the volatile attributes (cpu, memory, flags, etc) if we have + # a new resource usage sampling (the most dynamic stat) or its been + # twenty seconds since last fetched (so we still refresh occasionally + # when resource fetches fail). + # + # Otherwise, just redraw the panel to change the uptime field. + + isChanged = False + if self.vals["tor/pid"]: + resourceTracker = sysTools.getResourceTracker(self.vals["tor/pid"]) + isChanged = self._lastResourceFetch != resourceTracker.getRunCount() + + if isChanged or currentTime - self._lastUpdate >= 20: + self._update() + + self.redraw(True) + lastDraw += 1 + + def stop(self): + """ + Halts further resolutions and terminates the thread. + """ + + self._cond.acquire() + self._halt = True + self._cond.notifyAll() + self._cond.release() + + def resetListener(self, conn, eventType): + """ + Updates static parameters on tor reload (sighup) events. + + Arguments: + conn - tor controller + eventType - type of event detected + """ + + if eventType == torTools.State.INIT: + self._isTorConnected = True + if self._isPaused: self._haltTime = time.time() + else: self._haltTime = None + + self._update(True) + self.redraw(True) + elif eventType == torTools.State.CLOSED: + self._isTorConnected = False + self._haltTime = time.time() + self._update() + self.redraw(True) + + def _update(self, setStatic=False): + """ + Updates stats in the vals mapping. By default this just revises volatile + attributes. + + Arguments: + setStatic - resets all parameters, including relatively static values + """ + + self.valsLock.acquire() + conn = torTools.getConn() + + if setStatic: + # version is truncated to first part, for instance: + # 0.2.2.13-alpha (git-feb8c1b5f67f2c6f) -> 0.2.2.13-alpha + self.vals["tor/version"] = conn.getInfo("version", "Unknown").split()[0] + self.vals["tor/versionStatus"] = conn.getInfo("status/version/current", "Unknown") + self.vals["tor/nickname"] = conn.getOption("Nickname", "") + self.vals["tor/orPort"] = conn.getOption("ORPort", "0") + self.vals["tor/dirPort"] = conn.getOption("DirPort", "0") + self.vals["tor/controlPort"] = conn.getOption("ControlPort", "") + self.vals["tor/isAuthPassword"] = conn.getOption("HashedControlPassword") != None + self.vals["tor/isAuthCookie"] = conn.getOption("CookieAuthentication") == "1" + + # orport is reported as zero if unset + if self.vals["tor/orPort"] == "0": self.vals["tor/orPort"] = "" + + # overwrite address if ORListenAddress is set (and possibly orPort too) + self.vals["tor/orListenAddr"] = "" + listenAddr = conn.getOption("ORListenAddress") + if listenAddr: + if ":" in listenAddr: + # both ip and port overwritten + self.vals["tor/orListenAddr"] = listenAddr[:listenAddr.find(":")] + self.vals["tor/orPort"] = listenAddr[listenAddr.find(":") + 1:] + else: + self.vals["tor/orListenAddr"] = listenAddr + + # fetch exit policy (might span over multiple lines) + policyEntries = [] + for exitPolicy in conn.getOption("ExitPolicy", [], True): + policyEntries += [policy.strip() for policy in exitPolicy.split(",")] + self.vals["tor/exitPolicy"] = ", ".join(policyEntries) + + # file descriptor limit for the process, if this can't be determined + # then the limit is None + fdLimit, fdIsEstimate = conn.getMyFileDescriptorLimit() + self.vals["tor/fdLimit"] = fdLimit + self.vals["tor/isFdLimitEstimate"] = fdIsEstimate + + # system information + unameVals = os.uname() + self.vals["sys/hostname"] = unameVals[1] + self.vals["sys/os"] = unameVals[0] + self.vals["sys/version"] = unameVals[2] + + pid = conn.getMyPid() + self.vals["tor/pid"] = pid if pid else "" + + startTime = conn.getStartTime() + self.vals["tor/startTime"] = startTime if startTime else "" + + # reverts volatile parameters to defaults + self.vals["tor/fingerprint"] = "Unknown" + self.vals["tor/flags"] = [] + self.vals["tor/fdUsed"] = 0 + self.vals["stat/%torCpu"] = "0" + self.vals["stat/%armCpu"] = "0" + self.vals["stat/rss"] = "0" + self.vals["stat/%mem"] = "0" + + # sets volatile parameters + # TODO: This can change, being reported by STATUS_SERVER -> EXTERNAL_ADDRESS + # events. Introduce caching via torTools? + self.vals["tor/address"] = conn.getInfo("address", "") + + self.vals["tor/fingerprint"] = conn.getInfo("fingerprint", self.vals["tor/fingerprint"]) + self.vals["tor/flags"] = conn.getMyFlags(self.vals["tor/flags"]) + + # Updates file descriptor usage and logs if the usage is high. If we don't + # have a known limit or it's obviously faulty (being lower than our + # current usage) then omit file descriptor functionality. + if self.vals["tor/fdLimit"]: + fdUsed = conn.getMyFileDescriptorUsage() + if fdUsed and fdUsed <= self.vals["tor/fdLimit"]: self.vals["tor/fdUsed"] = fdUsed + else: self.vals["tor/fdUsed"] = 0 + + if self.vals["tor/fdUsed"] and self.vals["tor/fdLimit"]: + fdPercent = 100 * self.vals["tor/fdUsed"] / self.vals["tor/fdLimit"] + estimatedLabel = " estimated" if self.vals["tor/isFdLimitEstimate"] else "" + msg = "Tor's%s file descriptor usage is at %i%%." % (estimatedLabel, fdPercent) + + if fdPercent >= 90 and not self._isFdNinetyPercentWarned: + self._isFdSixtyPercentWarned, self._isFdNinetyPercentWarned = True, True + msg += " If you run out Tor will be unable to continue functioning." + log.log(self._config["log.fdUsageNinetyPercent"], msg) + elif fdPercent >= 60 and not self._isFdSixtyPercentWarned: + self._isFdSixtyPercentWarned = True + log.log(self._config["log.fdUsageSixtyPercent"], msg) + + # ps or proc derived resource usage stats + if self.vals["tor/pid"]: + resourceTracker = sysTools.getResourceTracker(self.vals["tor/pid"]) + + if resourceTracker.lastQueryFailed(): + self.vals["stat/%torCpu"] = "0" + self.vals["stat/rss"] = "0" + self.vals["stat/%mem"] = "0" + else: + cpuUsage, _, memUsage, memUsagePercent = resourceTracker.getResourceUsage() + self._lastResourceFetch = resourceTracker.getRunCount() + self.vals["stat/%torCpu"] = "%0.1f" % (100 * cpuUsage) + self.vals["stat/rss"] = str(memUsage) + self.vals["stat/%mem"] = "%0.1f" % (100 * memUsagePercent) + + # determines the cpu time for the arm process (including user and system + # time of both the primary and child processes) + + totalArmCpuTime, currentTime = sum(os.times()[:3]), time.time() + armCpuDelta = totalArmCpuTime - self._armCpuSampling[0] + armTimeDelta = currentTime - self._armCpuSampling[1] + pythonCpuTime = armCpuDelta / armTimeDelta + sysCallCpuTime = sysTools.getSysCpuUsage() + self.vals["stat/%armCpu"] = "%0.1f" % (100 * (pythonCpuTime + sysCallCpuTime)) + self._armCpuSampling = (totalArmCpuTime, currentTime) + + self._lastUpdate = currentTime + self.valsLock.release() + diff --git a/src/cli/logPanel.py b/src/cli/logPanel.py new file mode 100644 index 0000000..86e680f --- /dev/null +++ b/src/cli/logPanel.py @@ -0,0 +1,1100 @@ +""" +Panel providing a chronological log of events its been configured to listen +for. This provides prepopulation from the log file and supports filtering by +regular expressions. +""" + +import time +import os +import curses +import threading + +from TorCtl import TorCtl + +from version import VERSION +from util import conf, log, panel, sysTools, torTools, uiTools + +TOR_EVENT_TYPES = { + "d": "DEBUG", "a": "ADDRMAP", "k": "DESCCHANGED", "s": "STREAM", + "i": "INFO", "f": "AUTHDIR_NEWDESCS", "g": "GUARD", "r": "STREAM_BW", + "n": "NOTICE", "h": "BUILDTIMEOUT_SET", "l": "NEWCONSENSUS", "t": "STATUS_CLIENT", + "w": "WARN", "b": "BW", "m": "NEWDESC", "u": "STATUS_GENERAL", + "e": "ERR", "c": "CIRC", "p": "NS", "v": "STATUS_SERVER", + "j": "CLIENTS_SEEN", "q": "ORCONN"} + +EVENT_LISTING = """ d DEBUG a ADDRMAP k DESCCHANGED s STREAM + i INFO f AUTHDIR_NEWDESCS g GUARD r STREAM_BW + n NOTICE h BUILDTIMEOUT_SET l NEWCONSENSUS t STATUS_CLIENT + w WARN b BW m NEWDESC u STATUS_GENERAL + e ERR c CIRC p NS v STATUS_SERVER + j CLIENTS_SEEN q ORCONN + DINWE tor runlevel+ A All Events + 12345 arm runlevel+ X No Events + 67890 torctl runlevel+ U Unknown Events""" + +RUNLEVEL_EVENT_COLOR = {log.DEBUG: "magenta", log.INFO: "blue", log.NOTICE: "green", + log.WARN: "yellow", log.ERR: "red"} +DAYBREAK_EVENT = "DAYBREAK" # special event for marking when the date changes +TIMEZONE_OFFSET = time.altzone if time.localtime()[8] else time.timezone + +ENTRY_INDENT = 2 # spaces an entry's message is indented after the first line +DEFAULT_CONFIG = {"features.logFile": "", + "features.log.showDateDividers": True, + "features.log.showDuplicateEntries": False, + "features.log.entryDuration": 7, + "features.log.maxLinesPerEntry": 4, + "features.log.prepopulate": True, + "features.log.prepopulateReadLimit": 5000, + "features.log.maxRefreshRate": 300, + "cache.logPanel.size": 1000, + "log.logPanel.prepopulateSuccess": log.INFO, + "log.logPanel.prepopulateFailed": log.WARN, + "log.logPanel.logFileOpened": log.NOTICE, + "log.logPanel.logFileWriteFailed": log.ERR, + "log.logPanel.forceDoubleRedraw": log.DEBUG} + +DUPLICATE_MSG = " [%i duplicate%s hidden]" + +# The height of the drawn content is estimated based on the last time we redrew +# the panel. It's chiefly used for scrolling and the bar indicating its +# position. Letting the estimate be too inaccurate results in a display bug, so +# redraws the display if it's off by this threshold. +CONTENT_HEIGHT_REDRAW_THRESHOLD = 3 + +# static starting portion of common log entries, fetched from the config when +# needed if None +COMMON_LOG_MESSAGES = None + +# cached values and the arguments that generated it for the getDaybreaks and +# getDuplicates functions +CACHED_DAYBREAKS_ARGUMENTS = (None, None) # events, current day +CACHED_DAYBREAKS_RESULT = None +CACHED_DUPLICATES_ARGUMENTS = None # events +CACHED_DUPLICATES_RESULT = None + +# duration we'll wait for the deduplication function before giving up (in ms) +DEDUPLICATION_TIMEOUT = 100 + +def daysSince(timestamp=None): + """ + Provides the number of days since the epoch converted to local time (rounded + down). + + Arguments: + timestamp - unix timestamp to convert, current time if undefined + """ + + if timestamp == None: timestamp = time.time() + return int((timestamp - TIMEZONE_OFFSET) / 86400) + +def expandEvents(eventAbbr): + """ + Expands event abbreviations to their full names. Beside mappings provided in + TOR_EVENT_TYPES this recognizes the following special events and aliases: + U - UKNOWN events + A - all events + X - no events + DINWE - runlevel and higher + 12345 - arm runlevel and higher (ARM_DEBUG - ARM_ERR) + 67890 - torctl runlevel and higher (TORCTL_DEBUG - TORCTL_ERR) + Raises ValueError with invalid input if any part isn't recognized. + + Examples: + "inUt" -> ["INFO", "NOTICE", "UNKNOWN", "STREAM_BW"] + "N4" -> ["NOTICE", "WARN", "ERR", "ARM_WARN", "ARM_ERR"] + "cfX" -> [] + + Arguments: + eventAbbr - flags to be parsed to event types + """ + + expandedEvents, invalidFlags = set(), "" + + for flag in eventAbbr: + if flag == "A": + armRunlevels = ["ARM_" + runlevel for runlevel in log.Runlevel.values()] + torctlRunlevels = ["TORCTL_" + runlevel for runlevel in log.Runlevel.values()] + expandedEvents = set(TOR_EVENT_TYPES.values() + armRunlevels + torctlRunlevels + ["UNKNOWN"]) + break + elif flag == "X": + expandedEvents = set() + break + elif flag in "DINWE1234567890": + # all events for a runlevel and higher + if flag in "DINWE": typePrefix = "" + elif flag in "12345": typePrefix = "ARM_" + elif flag in "67890": typePrefix = "TORCTL_" + + if flag in "D16": runlevelIndex = 0 + elif flag in "I27": runlevelIndex = 1 + elif flag in "N38": runlevelIndex = 2 + elif flag in "W49": runlevelIndex = 3 + elif flag in "E50": runlevelIndex = 4 + + runlevelSet = [typePrefix + runlevel for runlevel in log.Runlevel.values()[runlevelIndex:]] + expandedEvents = expandedEvents.union(set(runlevelSet)) + elif flag == "U": + expandedEvents.add("UNKNOWN") + elif flag in TOR_EVENT_TYPES: + expandedEvents.add(TOR_EVENT_TYPES[flag]) + else: + invalidFlags += flag + + if invalidFlags: raise ValueError(invalidFlags) + else: return expandedEvents + +def getMissingEventTypes(): + """ + Provides the event types the current torctl connection supports but arm + doesn't. This provides an empty list if no event types are missing, and None + if the GETINFO query fails. + """ + + torEventTypes = torTools.getConn().getInfo("events/names") + + if torEventTypes: + torEventTypes = torEventTypes.split(" ") + armEventTypes = TOR_EVENT_TYPES.values() + return [event for event in torEventTypes if not event in armEventTypes] + else: return None # GETINFO call failed + +def loadLogMessages(): + """ + Fetches a mapping of common log messages to their runlevels from the config. + """ + + global COMMON_LOG_MESSAGES + armConf = conf.getConfig("arm") + + COMMON_LOG_MESSAGES = {} + for confKey in armConf.getKeys(): + if confKey.startswith("msg."): + eventType = confKey[4:].upper() + messages = armConf.get(confKey, []) + COMMON_LOG_MESSAGES[eventType] = messages + +def getLogFileEntries(runlevels, readLimit = None, addLimit = None, config = None): + """ + Parses tor's log file for past events matching the given runlevels, providing + a list of log entries (ordered newest to oldest). Limiting the number of read + entries is suggested to avoid parsing everything from logs in the GB and TB + range. + + Arguments: + runlevels - event types (DEBUG - ERR) to be returned + readLimit - max lines of the log file that'll be read (unlimited if None) + addLimit - maximum entries to provide back (unlimited if None) + config - configuration parameters related to this panel, uses defaults + if left as None + """ + + startTime = time.time() + if not runlevels: return [] + + if not config: config = DEFAULT_CONFIG + + # checks tor's configuration for the log file's location (if any exists) + loggingTypes, loggingLocation = None, None + for loggingEntry in torTools.getConn().getOption("Log", [], True): + # looks for an entry like: notice file /var/log/tor/notices.log + entryComp = loggingEntry.split() + + if entryComp[1] == "file": + loggingTypes, loggingLocation = entryComp[0], entryComp[2] + break + + if not loggingLocation: return [] + + # includes the prefix for tor paths + loggingLocation = torTools.getConn().getPathPrefix() + loggingLocation + + # if the runlevels argument is a superset of the log file then we can + # limit the read contents to the addLimit + runlevels = log.Runlevel.values() + loggingTypes = loggingTypes.upper() + if addLimit and (not readLimit or readLimit > addLimit): + if "-" in loggingTypes: + divIndex = loggingTypes.find("-") + sIndex = runlevels.index(loggingTypes[:divIndex]) + eIndex = runlevels.index(loggingTypes[divIndex+1:]) + logFileRunlevels = runlevels[sIndex:eIndex+1] + else: + sIndex = runlevels.index(loggingTypes) + logFileRunlevels = runlevels[sIndex:] + + # checks if runlevels we're reporting are a superset of the file's contents + isFileSubset = True + for runlevelType in logFileRunlevels: + if runlevelType not in runlevels: + isFileSubset = False + break + + if isFileSubset: readLimit = addLimit + + # tries opening the log file, cropping results to avoid choking on huge logs + lines = [] + try: + if readLimit: + lines = sysTools.call("tail -n %i %s" % (readLimit, loggingLocation)) + if not lines: raise IOError() + else: + logFile = open(loggingLocation, "r") + lines = logFile.readlines() + logFile.close() + except IOError: + msg = "Unable to read tor's log file: %s" % loggingLocation + log.log(config["log.logPanel.prepopulateFailed"], msg) + + if not lines: return [] + + loggedEvents = [] + currentUnixTime, currentLocalTime = time.time(), time.localtime() + for i in range(len(lines) - 1, -1, -1): + line = lines[i] + + # entries look like: + # Jul 15 18:29:48.806 [notice] Parsing GEOIP file. + lineComp = line.split() + eventType = lineComp[3][1:-1].upper() + + if eventType in runlevels: + # converts timestamp to unix time + timestamp = " ".join(lineComp[:3]) + + # strips the decimal seconds + if "." in timestamp: timestamp = timestamp[:timestamp.find(".")] + + # overwrites missing time parameters with the local time (ignoring wday + # and yday since they aren't used) + eventTimeComp = list(time.strptime(timestamp, "%b %d %H:%M:%S")) + eventTimeComp[0] = currentLocalTime.tm_year + eventTimeComp[8] = currentLocalTime.tm_isdst + eventTime = time.mktime(eventTimeComp) # converts local to unix time + + # The above is gonna be wrong if the logs are for the previous year. If + # the event's in the future then correct for this. + if eventTime > currentUnixTime + 60: + eventTimeComp[0] -= 1 + eventTime = time.mktime(eventTimeComp) + + eventMsg = " ".join(lineComp[4:]) + loggedEvents.append(LogEntry(eventTime, eventType, eventMsg, RUNLEVEL_EVENT_COLOR[eventType])) + + if "opening log file" in line: + break # this entry marks the start of this tor instance + + if addLimit: loggedEvents = loggedEvents[:addLimit] + msg = "Read %i entries from tor's log file: %s (read limit: %i, runtime: %0.3f)" % (len(loggedEvents), loggingLocation, readLimit, time.time() - startTime) + log.log(config["log.logPanel.prepopulateSuccess"], msg) + return loggedEvents + +def getDaybreaks(events, ignoreTimeForCache = False): + """ + Provides the input events back with special 'DAYBREAK_EVENT' markers inserted + whenever the date changed between log entries (or since the most recent + event). The timestamp matches the beginning of the day for the following + entry. + + Arguments: + events - chronologically ordered listing of events + ignoreTimeForCache - skips taking the day into consideration for providing + cached results if true + """ + + global CACHED_DAYBREAKS_ARGUMENTS, CACHED_DAYBREAKS_RESULT + if not events: return [] + + newListing = [] + currentDay = daysSince() + lastDay = currentDay + + if CACHED_DAYBREAKS_ARGUMENTS[0] == events and \ + (ignoreTimeForCache or CACHED_DAYBREAKS_ARGUMENTS[1] == currentDay): + return list(CACHED_DAYBREAKS_RESULT) + + for entry in events: + eventDay = daysSince(entry.timestamp) + if eventDay != lastDay: + markerTimestamp = (eventDay * 86400) + TIMEZONE_OFFSET + newListing.append(LogEntry(markerTimestamp, DAYBREAK_EVENT, "", "white")) + + newListing.append(entry) + lastDay = eventDay + + CACHED_DAYBREAKS_ARGUMENTS = (list(events), currentDay) + CACHED_DAYBREAKS_RESULT = list(newListing) + + return newListing + +def getDuplicates(events): + """ + Deduplicates a list of log entries, providing back a tuple listing with the + log entry and count of duplicates following it. Entries in different days are + not considered to be duplicates. This times out, returning None if it takes + longer than DEDUPLICATION_TIMEOUT. + + Arguments: + events - chronologically ordered listing of events + """ + + global CACHED_DUPLICATES_ARGUMENTS, CACHED_DUPLICATES_RESULT + if CACHED_DUPLICATES_ARGUMENTS == events: + return list(CACHED_DUPLICATES_RESULT) + + # loads common log entries from the config if they haven't been + if COMMON_LOG_MESSAGES == None: loadLogMessages() + + startTime = time.time() + eventsRemaining = list(events) + returnEvents = [] + + while eventsRemaining: + entry = eventsRemaining.pop(0) + duplicateIndices = isDuplicate(entry, eventsRemaining, True) + + # checks if the call timeout has been reached + if (time.time() - startTime) > DEDUPLICATION_TIMEOUT / 1000.0: + return None + + # drops duplicate entries + duplicateIndices.reverse() + for i in duplicateIndices: del eventsRemaining[i] + + returnEvents.append((entry, len(duplicateIndices))) + + CACHED_DUPLICATES_ARGUMENTS = list(events) + CACHED_DUPLICATES_RESULT = list(returnEvents) + + return returnEvents + +def isDuplicate(event, eventSet, getDuplicates = False): + """ + True if the event is a duplicate for something in the eventSet, false + otherwise. If the getDuplicates flag is set this provides the indices of + the duplicates instead. + + Arguments: + event - event to search for duplicates of + eventSet - set to look for the event in + getDuplicates - instead of providing back a boolean this gives a list of + the duplicate indices in the eventSet + """ + + duplicateIndices = [] + for i in range(len(eventSet)): + forwardEntry = eventSet[i] + + # if showing dates then do duplicate detection for each day, rather + # than globally + if forwardEntry.type == DAYBREAK_EVENT: break + + if event.type == forwardEntry.type: + isDuplicate = False + if event.msg == forwardEntry.msg: isDuplicate = True + elif event.type in COMMON_LOG_MESSAGES: + for commonMsg in COMMON_LOG_MESSAGES[event.type]: + # if it starts with an asterisk then check the whole message rather + # than just the start + if commonMsg[0] == "*": + isDuplicate = commonMsg[1:] in event.msg and commonMsg[1:] in forwardEntry.msg + else: + isDuplicate = event.msg.startswith(commonMsg) and forwardEntry.msg.startswith(commonMsg) + + if isDuplicate: break + + if isDuplicate: + if getDuplicates: duplicateIndices.append(i) + else: return True + + if getDuplicates: return duplicateIndices + else: return False + +class LogEntry(): + """ + Individual log file entry, having the following attributes: + timestamp - unix timestamp for when the event occurred + eventType - event type that occurred ("INFO", "BW", "ARM_WARN", etc) + msg - message that was logged + color - color of the log entry + """ + + def __init__(self, timestamp, eventType, msg, color): + self.timestamp = timestamp + self.type = eventType + self.msg = msg + self.color = color + self._displayMessage = None + + def getDisplayMessage(self, includeDate = False): + """ + Provides the entry's message for the log. + + Arguments: + includeDate - appends the event's date to the start of the message + """ + + if includeDate: + # not the common case so skip caching + entryTime = time.localtime(self.timestamp) + timeLabel = "%i/%i/%i %02i:%02i:%02i" % (entryTime[1], entryTime[2], entryTime[0], entryTime[3], entryTime[4], entryTime[5]) + return "%s [%s] %s" % (timeLabel, self.type, self.msg) + + if not self._displayMessage: + entryTime = time.localtime(self.timestamp) + self._displayMessage = "%02i:%02i:%02i [%s] %s" % (entryTime[3], entryTime[4], entryTime[5], self.type, self.msg) + + return self._displayMessage + +class TorEventObserver(TorCtl.PostEventListener): + """ + Listens for all types of events provided by TorCtl, providing an LogEntry + instance to the given callback function. + """ + + def __init__(self, callback): + """ + Tor event listener with the purpose of translating events to nicely + formatted calls of a callback function. + + Arguments: + callback - function accepting a LogEntry, called when an event of these + types occur + """ + + TorCtl.PostEventListener.__init__(self) + self.callback = callback + + def circ_status_event(self, event): + msg = "ID: %-3s STATUS: %-10s PATH: %s" % (event.circ_id, event.status, ", ".join(event.path)) + if event.purpose: msg += " PURPOSE: %s" % event.purpose + if event.reason: msg += " REASON: %s" % event.reason + if event.remote_reason: msg += " REMOTE_REASON: %s" % event.remote_reason + self._notify(event, msg, "yellow") + + def buildtimeout_set_event(self, event): + self._notify(event, "SET_TYPE: %s, TOTAL_TIMES: %s, TIMEOUT_MS: %s, XM: %s, ALPHA: %s, CUTOFF_QUANTILE: %s" % (event.set_type, event.total_times, event.timeout_ms, event.xm, event.alpha, event.cutoff_quantile)) + + def stream_status_event(self, event): + self._notify(event, "ID: %s STATUS: %s CIRC_ID: %s TARGET: %s:%s REASON: %s REMOTE_REASON: %s SOURCE: %s SOURCE_ADDR: %s PURPOSE: %s" % (event.strm_id, event.status, event.circ_id, event.target_host, event.target_port, event.reason, event.remote_reason, event.source, event.source_addr, event.purpose)) + + def or_conn_status_event(self, event): + msg = "STATUS: %-10s ENDPOINT: %-20s" % (event.status, event.endpoint) + if event.age: msg += " AGE: %-3s" % event.age + if event.read_bytes: msg += " READ: %-4i" % event.read_bytes + if event.wrote_bytes: msg += " WRITTEN: %-4i" % event.wrote_bytes + if event.reason: msg += " REASON: %-6s" % event.reason + if event.ncircs: msg += " NCIRCS: %i" % event.ncircs + self._notify(event, msg) + + def stream_bw_event(self, event): + self._notify(event, "ID: %s READ: %s WRITTEN: %s" % (event.strm_id, event.bytes_read, event.bytes_written)) + + def bandwidth_event(self, event): + self._notify(event, "READ: %i, WRITTEN: %i" % (event.read, event.written), "cyan") + + def msg_event(self, event): + self._notify(event, event.msg, RUNLEVEL_EVENT_COLOR[event.level]) + + def new_desc_event(self, event): + idlistStr = [str(item) for item in event.idlist] + self._notify(event, ", ".join(idlistStr)) + + def address_mapped_event(self, event): + self._notify(event, "%s, %s -> %s" % (event.when, event.from_addr, event.to_addr)) + + def ns_event(self, event): + # NetworkStatus params: nickname, idhash, orhash, ip, orport (int), + # dirport (int), flags, idhex, bandwidth, updated (datetime) + msg = ", ".join(["%s (%s)" % (ns.idhex, ns.nickname) for ns in event.nslist]) + self._notify(event, "Listed (%i): %s" % (len(event.nslist), msg), "blue") + + def new_consensus_event(self, event): + msg = ", ".join(["%s (%s)" % (ns.idhex, ns.nickname) for ns in event.nslist]) + self._notify(event, "Listed (%i): %s" % (len(event.nslist), msg), "magenta") + + def unknown_event(self, event): + msg = "(%s) %s" % (event.event_name, event.event_string) + self.callback(LogEntry(event.arrived_at, "UNKNOWN", msg, "red")) + + def _notify(self, event, msg, color="white"): + self.callback(LogEntry(event.arrived_at, event.event_name, msg, color)) + +class LogPanel(panel.Panel, threading.Thread): + """ + Listens for and displays tor, arm, and torctl events. This can prepopulate + from tor's log file if it exists. + """ + + def __init__(self, stdscr, loggedEvents, config=None): + panel.Panel.__init__(self, stdscr, "log", 0) + threading.Thread.__init__(self) + self.setDaemon(True) + + self._config = dict(DEFAULT_CONFIG) + + if config: + config.update(self._config, { + "features.log.maxLinesPerEntry": 1, + "features.log.prepopulateReadLimit": 0, + "features.log.maxRefreshRate": 10, + "cache.logPanel.size": 1000}) + + # collapses duplicate log entries if false, showing only the most recent + self.showDuplicates = self._config["features.log.showDuplicateEntries"] + + self.msgLog = [] # log entries, sorted by the timestamp + self.loggedEvents = loggedEvents # events we're listening to + self.regexFilter = None # filter for presented log events (no filtering if None) + self.lastContentHeight = 0 # height of the rendered content when last drawn + self.logFile = None # file log messages are saved to (skipped if None) + self.scroll = 0 + self._isPaused = False + self._pauseBuffer = [] # location where messages are buffered if paused + + self._lastUpdate = -1 # time the content was last revised + self._halt = False # terminates thread if true + self._cond = threading.Condition() # used for pausing/resuming the thread + + # restricts concurrent write access to attributes used to draw the display + # and pausing: + # msgLog, loggedEvents, regexFilter, scroll, _pauseBuffer + self.valsLock = threading.RLock() + + # cached parameters (invalidated if arguments for them change) + # last set of events we've drawn with + self._lastLoggedEvents = [] + + # _getTitle (args: loggedEvents, regexFilter pattern, width) + self._titleCache = None + self._titleArgs = (None, None, None) + + # fetches past tor events from log file, if available + torEventBacklog = [] + if self._config["features.log.prepopulate"]: + setRunlevels = list(set.intersection(set(self.loggedEvents), set(log.Runlevel.values()))) + readLimit = self._config["features.log.prepopulateReadLimit"] + addLimit = self._config["cache.logPanel.size"] + torEventBacklog = getLogFileEntries(setRunlevels, readLimit, addLimit, self._config) + + # adds arm listener and fetches past events + log.LOG_LOCK.acquire() + try: + armRunlevels = [log.DEBUG, log.INFO, log.NOTICE, log.WARN, log.ERR] + log.addListeners(armRunlevels, self._registerArmEvent) + + # gets the set of arm events we're logging + setRunlevels = [] + for i in range(len(armRunlevels)): + if "ARM_" + log.Runlevel.values()[i] in self.loggedEvents: + setRunlevels.append(armRunlevels[i]) + + armEventBacklog = [] + for level, msg, eventTime in log._getEntries(setRunlevels): + armEventEntry = LogEntry(eventTime, "ARM_" + level, msg, RUNLEVEL_EVENT_COLOR[level]) + armEventBacklog.insert(0, armEventEntry) + + # joins armEventBacklog and torEventBacklog chronologically into msgLog + while armEventBacklog or torEventBacklog: + if not armEventBacklog: + self.msgLog.append(torEventBacklog.pop(0)) + elif not torEventBacklog: + self.msgLog.append(armEventBacklog.pop(0)) + elif armEventBacklog[0].timestamp < torEventBacklog[0].timestamp: + self.msgLog.append(torEventBacklog.pop(0)) + else: + self.msgLog.append(armEventBacklog.pop(0)) + finally: + log.LOG_LOCK.release() + + # crops events that are either too old, or more numerous than the caching size + self._trimEvents(self.msgLog) + + # leaving lastContentHeight as being too low causes initialization problems + self.lastContentHeight = len(self.msgLog) + + # adds listeners for tor and torctl events + conn = torTools.getConn() + conn.addEventListener(TorEventObserver(self.registerEvent)) + conn.addTorCtlListener(self._registerTorCtlEvent) + + # opens log file if we'll be saving entries + if self._config["features.logFile"]: + logPath = self._config["features.logFile"] + + try: + # make dir if the path doesn't already exist + baseDir = os.path.dirname(logPath) + if not os.path.exists(baseDir): os.makedirs(baseDir) + + self.logFile = open(logPath, "a") + log.log(self._config["log.logPanel.logFileOpened"], "arm %s opening log file (%s)" % (VERSION, logPath)) + except (IOError, OSError), exc: + log.log(self._config["log.logPanel.logFileWriteFailed"], "Unable to write to log file: %s" % sysTools.getFileErrorMsg(exc)) + self.logFile = None + + def registerEvent(self, event): + """ + Notes event and redraws log. If paused it's held in a temporary buffer. + + Arguments: + event - LogEntry for the event that occurred + """ + + if not event.type in self.loggedEvents: return + + # strips control characters to avoid screwing up the terminal + event.msg = uiTools.getPrintable(event.msg) + + # note event in the log file if we're saving them + if self.logFile: + try: + self.logFile.write(event.getDisplayMessage(True) + "\n") + self.logFile.flush() + except IOError, exc: + log.log(self._config["log.logPanel.logFileWriteFailed"], "Unable to write to log file: %s" % sysTools.getFileErrorMsg(exc)) + self.logFile = None + + if self._isPaused: + self.valsLock.acquire() + self._pauseBuffer.insert(0, event) + self._trimEvents(self._pauseBuffer) + self.valsLock.release() + else: + self.valsLock.acquire() + self.msgLog.insert(0, event) + self._trimEvents(self.msgLog) + + # notifies the display that it has new content + if not self.regexFilter or self.regexFilter.search(event.getDisplayMessage()): + self._cond.acquire() + self._cond.notifyAll() + self._cond.release() + + self.valsLock.release() + + def _registerArmEvent(self, level, msg, eventTime): + eventColor = RUNLEVEL_EVENT_COLOR[level] + self.registerEvent(LogEntry(eventTime, "ARM_%s" % level, msg, eventColor)) + + def _registerTorCtlEvent(self, level, msg): + eventColor = RUNLEVEL_EVENT_COLOR[level] + self.registerEvent(LogEntry(time.time(), "TORCTL_%s" % level, msg, eventColor)) + + def setLoggedEvents(self, eventTypes): + """ + Sets the event types recognized by the panel. + + Arguments: + eventTypes - event types to be logged + """ + + if eventTypes == self.loggedEvents: return + + self.valsLock.acquire() + self.loggedEvents = eventTypes + self.redraw(True) + self.valsLock.release() + + def setFilter(self, logFilter): + """ + Filters log entries according to the given regular expression. + + Arguments: + logFilter - regular expression used to determine which messages are + shown, None if no filter should be applied + """ + + if logFilter == self.regexFilter: return + + self.valsLock.acquire() + self.regexFilter = logFilter + self.redraw(True) + self.valsLock.release() + + def clear(self): + """ + Clears the contents of the event log. + """ + + self.valsLock.acquire() + self.msgLog = [] + self.redraw(True) + self.valsLock.release() + + def saveSnapshot(self, path): + """ + Saves the log events currently being displayed to the given path. This + takes filers into account. This overwrites the file if it already exists, + and raises an IOError if there's a problem. + + Arguments: + path - path where to save the log snapshot + """ + + # make dir if the path doesn't already exist + baseDir = os.path.dirname(path) + if not os.path.exists(baseDir): os.makedirs(baseDir) + + snapshotFile = open(path, "w") + self.valsLock.acquire() + try: + for entry in self.msgLog: + isVisible = not self.regexFilter or self.regexFilter.search(entry.getDisplayMessage()) + if isVisible: snapshotFile.write(entry.getDisplayMessage(True) + "\n") + + self.valsLock.release() + except Exception, exc: + self.valsLock.release() + raise exc + + def handleKey(self, key): + if uiTools.isScrollKey(key): + pageHeight = self.getPreferredSize()[0] - 1 + newScroll = uiTools.getScrollPosition(key, self.scroll, pageHeight, self.lastContentHeight) + + if self.scroll != newScroll: + self.valsLock.acquire() + self.scroll = newScroll + self.redraw(True) + self.valsLock.release() + elif key in (ord('u'), ord('U')): + self.valsLock.acquire() + self.showDuplicates = not self.showDuplicates + self.redraw(True) + self.valsLock.release() + + def setPaused(self, isPause): + """ + If true, prevents message log from being updated with new events. + """ + + if isPause == self._isPaused: return + + self._isPaused = isPause + if self._isPaused: self._pauseBuffer = [] + else: + self.valsLock.acquire() + self.msgLog = (self._pauseBuffer + self.msgLog)[:self._config["cache.logPanel.size"]] + self.redraw(True) + self.valsLock.release() + + def draw(self, width, height): + """ + Redraws message log. Entries stretch to use available space and may + contain up to two lines. Starts with newest entries. + """ + + self.valsLock.acquire() + self._lastLoggedEvents, self._lastUpdate = list(self.msgLog), time.time() + + # draws the top label + self.addstr(0, 0, self._getTitle(width), curses.A_STANDOUT) + + # restricts scroll location to valid bounds + self.scroll = max(0, min(self.scroll, self.lastContentHeight - height + 1)) + + # draws left-hand scroll bar if content's longer than the height + msgIndent, dividerIndent = 1, 0 # offsets for scroll bar + isScrollBarVisible = self.lastContentHeight > height - 1 + if isScrollBarVisible: + msgIndent, dividerIndent = 3, 2 + self.addScrollBar(self.scroll, self.scroll + height - 1, self.lastContentHeight, 1) + + # draws log entries + lineCount = 1 - self.scroll + seenFirstDateDivider = False + dividerAttr, duplicateAttr = curses.A_BOLD | uiTools.getColor("yellow"), curses.A_BOLD | uiTools.getColor("green") + + isDatesShown = self.regexFilter == None and self._config["features.log.showDateDividers"] + eventLog = getDaybreaks(self.msgLog, self._isPaused) if isDatesShown else list(self.msgLog) + if not self.showDuplicates: + deduplicatedLog = getDuplicates(eventLog) + + if deduplicatedLog == None: + msg = "Deduplication took too long. Its current implementation has difficulty handling large logs so disabling it to keep the interface responsive." + log.log(log.WARN, msg) + self.showDuplicates = True + deduplicatedLog = [(entry, 0) for entry in eventLog] + else: deduplicatedLog = [(entry, 0) for entry in eventLog] + + # determines if we have the minimum width to show date dividers + showDaybreaks = width - dividerIndent >= 3 + + while deduplicatedLog: + entry, duplicateCount = deduplicatedLog.pop(0) + + if self.regexFilter and not self.regexFilter.search(entry.getDisplayMessage()): + continue # filter doesn't match log message - skip + + # checks if we should be showing a divider with the date + if entry.type == DAYBREAK_EVENT: + # bottom of the divider + if seenFirstDateDivider: + if lineCount >= 1 and lineCount < height and showDaybreaks: + self.addch(lineCount, dividerIndent, curses.ACS_LLCORNER, dividerAttr) + self.hline(lineCount, dividerIndent + 1, width - dividerIndent - 1, dividerAttr) + self.addch(lineCount, width, curses.ACS_LRCORNER, dividerAttr) + + lineCount += 1 + + # top of the divider + if lineCount >= 1 and lineCount < height and showDaybreaks: + timeLabel = time.strftime(" %B %d, %Y ", time.localtime(entry.timestamp)) + self.addch(lineCount, dividerIndent, curses.ACS_ULCORNER, dividerAttr) + self.addch(lineCount, dividerIndent + 1, curses.ACS_HLINE, dividerAttr) + self.addstr(lineCount, dividerIndent + 2, timeLabel, curses.A_BOLD | dividerAttr) + + lineLength = width - dividerIndent - len(timeLabel) - 2 + self.hline(lineCount, dividerIndent + len(timeLabel) + 2, lineLength, dividerAttr) + self.addch(lineCount, dividerIndent + len(timeLabel) + 2 + lineLength, curses.ACS_URCORNER, dividerAttr) + + seenFirstDateDivider = True + lineCount += 1 + else: + # entry contents to be displayed, tuples of the form: + # (msg, formatting, includeLinebreak) + displayQueue = [] + + msgComp = entry.getDisplayMessage().split("\n") + for i in range(len(msgComp)): + font = curses.A_BOLD if "ERR" in entry.type else curses.A_NORMAL # emphasizes ERR messages + displayQueue.append((msgComp[i].strip(), font | uiTools.getColor(entry.color), i != len(msgComp) - 1)) + + if duplicateCount: + pluralLabel = "s" if duplicateCount > 1 else "" + duplicateMsg = DUPLICATE_MSG % (duplicateCount, pluralLabel) + displayQueue.append((duplicateMsg, duplicateAttr, False)) + + cursorLoc, lineOffset = msgIndent, 0 + maxEntriesPerLine = self._config["features.log.maxLinesPerEntry"] + while displayQueue: + msg, format, includeBreak = displayQueue.pop(0) + drawLine = lineCount + lineOffset + if lineOffset == maxEntriesPerLine: break + + maxMsgSize = width - cursorLoc + if len(msg) > maxMsgSize: + # message is too long - break it up + if lineOffset == maxEntriesPerLine - 1: + msg = uiTools.cropStr(msg, maxMsgSize) + else: + msg, remainder = uiTools.cropStr(msg, maxMsgSize, 4, 4, uiTools.Ending.HYPHEN, True) + displayQueue.insert(0, (remainder.strip(), format, includeBreak)) + + includeBreak = True + + if drawLine < height and drawLine >= 1: + if seenFirstDateDivider and width - dividerIndent >= 3 and showDaybreaks: + self.addch(drawLine, dividerIndent, curses.ACS_VLINE, dividerAttr) + self.addch(drawLine, width, curses.ACS_VLINE, dividerAttr) + + self.addstr(drawLine, cursorLoc, msg, format) + + cursorLoc += len(msg) + + if includeBreak or not displayQueue: + lineOffset += 1 + cursorLoc = msgIndent + ENTRY_INDENT + + lineCount += lineOffset + + # if this is the last line and there's room, then draw the bottom of the divider + if not deduplicatedLog and seenFirstDateDivider: + if lineCount < height and showDaybreaks: + self.addch(lineCount, dividerIndent, curses.ACS_LLCORNER, dividerAttr) + self.hline(lineCount, dividerIndent + 1, width - dividerIndent - 1, dividerAttr) + self.addch(lineCount, width, curses.ACS_LRCORNER, dividerAttr) + + lineCount += 1 + + # redraw the display if... + # - lastContentHeight was off by too much + # - we're off the bottom of the page + newContentHeight = lineCount + self.scroll - 1 + contentHeightDelta = abs(self.lastContentHeight - newContentHeight) + forceRedraw, forceRedrawReason = True, "" + + if contentHeightDelta >= CONTENT_HEIGHT_REDRAW_THRESHOLD: + forceRedrawReason = "estimate was off by %i" % contentHeightDelta + elif newContentHeight > height and self.scroll + height - 1 > newContentHeight: + forceRedrawReason = "scrolled off the bottom of the page" + elif not isScrollBarVisible and newContentHeight > height - 1: + forceRedrawReason = "scroll bar wasn't previously visible" + elif isScrollBarVisible and newContentHeight <= height - 1: + forceRedrawReason = "scroll bar shouldn't be visible" + else: forceRedraw = False + + self.lastContentHeight = newContentHeight + if forceRedraw: + forceRedrawReason = "redrawing the log panel with the corrected content height (%s)" % forceRedrawReason + log.log(self._config["log.logPanel.forceDoubleRedraw"], forceRedrawReason) + self.redraw(True) + + self.valsLock.release() + + def redraw(self, forceRedraw=False, block=False): + # determines if the content needs to be redrawn or not + panel.Panel.redraw(self, forceRedraw, block) + + def run(self): + """ + Redraws the display, coalescing updates if events are rapidly logged (for + instance running at the DEBUG runlevel) while also being immediately + responsive if additions are less frequent. + """ + + lastDay = daysSince() # used to determine if the date has changed + while not self._halt: + currentDay = daysSince() + timeSinceReset = time.time() - self._lastUpdate + maxLogUpdateRate = self._config["features.log.maxRefreshRate"] / 1000.0 + + sleepTime = 0 + if (self.msgLog == self._lastLoggedEvents and lastDay == currentDay) or self._isPaused: + sleepTime = 5 + elif timeSinceReset < maxLogUpdateRate: + sleepTime = max(0.05, maxLogUpdateRate - timeSinceReset) + + if sleepTime: + self._cond.acquire() + if not self._halt: self._cond.wait(sleepTime) + self._cond.release() + else: + lastDay = currentDay + self.redraw(True) + + def stop(self): + """ + Halts further resolutions and terminates the thread. + """ + + self._cond.acquire() + self._halt = True + self._cond.notifyAll() + self._cond.release() + + def _getTitle(self, width): + """ + Provides the label used for the panel, looking like: + Events (ARM NOTICE - ERR, BW - filter: prepopulate): + + This truncates the attributes (with an ellipse) if too long, and condenses + runlevel ranges if there's three or more in a row (for instance ARM_INFO, + ARM_NOTICE, and ARM_WARN becomes "ARM_INFO - WARN"). + + Arguments: + width - width constraint the label needs to fix in + """ + + # usually the attributes used to make the label are decently static, so + # provide cached results if they're unchanged + self.valsLock.acquire() + currentPattern = self.regexFilter.pattern if self.regexFilter else None + isUnchanged = self._titleArgs[0] == self.loggedEvents + isUnchanged &= self._titleArgs[1] == currentPattern + isUnchanged &= self._titleArgs[2] == width + if isUnchanged: + self.valsLock.release() + return self._titleCache + + eventsList = list(self.loggedEvents) + if not eventsList: + if not currentPattern: + panelLabel = "Events:" + else: + labelPattern = uiTools.cropStr(currentPattern, width - 18) + panelLabel = "Events (filter: %s):" % labelPattern + else: + # does the following with all runlevel types (tor, arm, and torctl): + # - pulls to the start of the list + # - condenses range if there's three or more in a row (ex. "ARM_INFO - WARN") + # - condense further if there's identical runlevel ranges for multiple + # types (ex. "NOTICE - ERR, ARM_NOTICE - ERR" becomes "TOR/ARM NOTICE - ERR") + tmpRunlevels = [] # runlevels pulled from the list (just the runlevel part) + runlevelRanges = [] # tuple of type, startLevel, endLevel for ranges to be consensed + + # reverses runlevels and types so they're appended in the right order + reversedRunlevels = log.Runlevel.values() + reversedRunlevels.reverse() + for prefix in ("TORCTL_", "ARM_", ""): + # blank ending runlevel forces the break condition to be reached at the end + for runlevel in reversedRunlevels + [""]: + eventType = prefix + runlevel + if runlevel and eventType in eventsList: + # runlevel event found, move to the tmp list + eventsList.remove(eventType) + tmpRunlevels.append(runlevel) + elif tmpRunlevels: + # adds all tmp list entries to the start of eventsList + if len(tmpRunlevels) >= 3: + # save condense sequential runlevels to be added later + runlevelRanges.append((prefix, tmpRunlevels[-1], tmpRunlevels[0])) + else: + # adds runlevels individaully + for tmpRunlevel in tmpRunlevels: + eventsList.insert(0, prefix + tmpRunlevel) + + tmpRunlevels = [] + + # adds runlevel ranges, condensing if there's identical ranges + for i in range(len(runlevelRanges)): + if runlevelRanges[i]: + prefix, startLevel, endLevel = runlevelRanges[i] + + # check for matching ranges + matches = [] + for j in range(i + 1, len(runlevelRanges)): + if runlevelRanges[j] and runlevelRanges[j][1] == startLevel and runlevelRanges[j][2] == endLevel: + matches.append(runlevelRanges[j]) + runlevelRanges[j] = None + + if matches: + # strips underscores and replaces empty entries with "TOR" + prefixes = [entry[0] for entry in matches] + [prefix] + for k in range(len(prefixes)): + if prefixes[k] == "": prefixes[k] = "TOR" + else: prefixes[k] = prefixes[k].replace("_", "") + + eventsList.insert(0, "%s %s - %s" % ("/".join(prefixes), startLevel, endLevel)) + else: + eventsList.insert(0, "%s%s - %s" % (prefix, startLevel, endLevel)) + + # truncates to use an ellipsis if too long, for instance: + attrLabel = ", ".join(eventsList) + if currentPattern: attrLabel += " - filter: %s" % currentPattern + attrLabel = uiTools.cropStr(attrLabel, width - 10, 1) + if attrLabel: attrLabel = " (%s)" % attrLabel + panelLabel = "Events%s:" % attrLabel + + # cache results and return + self._titleCache = panelLabel + self._titleArgs = (list(self.loggedEvents), currentPattern, width) + self.valsLock.release() + return panelLabel + + def _trimEvents(self, eventListing): + """ + Crops events that have either: + - grown beyond the cache limit + - outlived the configured log duration + + Argument: + eventListing - listing of log entries + """ + + cacheSize = self._config["cache.logPanel.size"] + if len(eventListing) > cacheSize: del eventListing[cacheSize:] + + logTTL = self._config["features.log.entryDuration"] + if logTTL > 0: + currentDay = daysSince() + + breakpoint = None # index at which to crop from + for i in range(len(eventListing) - 1, -1, -1): + daysSinceEvent = currentDay - daysSince(eventListing[i].timestamp) + if daysSinceEvent > logTTL: breakpoint = i # older than the ttl + else: break + + # removes entries older than the ttl + if breakpoint != None: del eventListing[breakpoint:] + diff --git a/src/cli/torrcPanel.py b/src/cli/torrcPanel.py new file mode 100644 index 0000000..b7cad86 --- /dev/null +++ b/src/cli/torrcPanel.py @@ -0,0 +1,221 @@ +""" +Panel displaying the torrc or armrc with the validation done against it. +""" + +import math +import curses +import threading + +from util import conf, enum, panel, torConfig, uiTools + +DEFAULT_CONFIG = {"features.config.file.showScrollbars": True, + "features.config.file.maxLinesPerEntry": 8} + +# TODO: The armrc use case is incomplete. There should be equivilant reloading +# and validation capabilities to the torrc. +Config = enum.Enum("TORRC", "ARMRC") # configuration file types that can be displayed + +class TorrcPanel(panel.Panel): + """ + Renders the current torrc or armrc with syntax highlighting in a scrollable + area. + """ + + def __init__(self, stdscr, configType, config=None): + panel.Panel.__init__(self, stdscr, "configFile", 0) + + self._config = dict(DEFAULT_CONFIG) + if config: + config.update(self._config, {"features.config.file.maxLinesPerEntry": 1}) + + self.valsLock = threading.RLock() + self.configType = configType + self.scroll = 0 + self.showLabel = True # shows top label (hides otherwise) + self.showLineNum = True # shows left aligned line numbers + self.stripComments = False # drops comments and extra whitespace + + # height of the content when last rendered (the cached value is invalid if + # _lastContentHeightArgs is None or differs from the current dimensions) + self._lastContentHeight = 1 + self._lastContentHeightArgs = None + + def handleKey(self, key): + self.valsLock.acquire() + if uiTools.isScrollKey(key): + pageHeight = self.getPreferredSize()[0] - 1 + newScroll = uiTools.getScrollPosition(key, self.scroll, pageHeight, self._lastContentHeight) + + if self.scroll != newScroll: + self.scroll = newScroll + self.redraw(True) + elif key == ord('n') or key == ord('N'): + self.showLineNum = not self.showLineNum + self._lastContentHeightArgs = None + self.redraw(True) + elif key == ord('s') or key == ord('S'): + self.stripComments = not self.stripComments + self._lastContentHeightArgs = None + self.redraw(True) + + self.valsLock.release() + + def draw(self, width, height): + self.valsLock.acquire() + + # If true, we assume that the cached value in self._lastContentHeight is + # still accurate, and stop drawing when there's nothing more to display. + # Otherwise the self._lastContentHeight is suspect, and we'll process all + # the content to check if it's right (and redraw again with the corrected + # height if not). + trustLastContentHeight = self._lastContentHeightArgs == (width, height) + + # restricts scroll location to valid bounds + self.scroll = max(0, min(self.scroll, self._lastContentHeight - height + 1)) + + renderedContents, corrections, confLocation = None, {}, None + if self.configType == Config.TORRC: + loadedTorrc = torConfig.getTorrc() + loadedTorrc.getLock().acquire() + confLocation = loadedTorrc.getConfigLocation() + + if not loadedTorrc.isLoaded(): + renderedContents = ["### Unable to load the torrc ###"] + else: + renderedContents = loadedTorrc.getDisplayContents(self.stripComments) + + # constructs a mapping of line numbers to the issue on it + corrections = dict((lineNum, (issue, msg)) for lineNum, issue, msg in loadedTorrc.getCorrections()) + + loadedTorrc.getLock().release() + else: + loadedArmrc = conf.getConfig("arm") + confLocation = loadedArmrc.path + renderedContents = list(loadedArmrc.rawContents) + + # offset to make room for the line numbers + lineNumOffset = 0 + if self.showLineNum: + if len(renderedContents) == 0: lineNumOffset = 2 + else: lineNumOffset = int(math.log10(len(renderedContents))) + 2 + + # draws left-hand scroll bar if content's longer than the height + scrollOffset = 0 + if self._config["features.config.file.showScrollbars"] and self._lastContentHeight > height - 1: + scrollOffset = 3 + self.addScrollBar(self.scroll, self.scroll + height - 1, self._lastContentHeight, 1) + + displayLine = -self.scroll + 1 # line we're drawing on + + # draws the top label + if self.showLabel: + sourceLabel = "Tor" if self.configType == Config.TORRC else "Arm" + locationLabel = " (%s)" % confLocation if confLocation else "" + self.addstr(0, 0, "%s Configuration File%s:" % (sourceLabel, locationLabel), curses.A_STANDOUT) + + isMultiline = False # true if we're in the middle of a multiline torrc entry + for lineNumber in range(0, len(renderedContents)): + lineText = renderedContents[lineNumber] + lineText = lineText.rstrip() # remove ending whitespace + + # blank lines are hidden when stripping comments + if self.stripComments and not lineText: continue + + # splits the line into its component (msg, format) tuples + lineComp = {"option": ["", curses.A_BOLD | uiTools.getColor("green")], + "argument": ["", curses.A_BOLD | uiTools.getColor("cyan")], + "correction": ["", curses.A_BOLD | uiTools.getColor("cyan")], + "comment": ["", uiTools.getColor("white")]} + + # parses the comment + commentIndex = lineText.find("#") + if commentIndex != -1: + lineComp["comment"][0] = lineText[commentIndex:] + lineText = lineText[:commentIndex] + + # splits the option and argument, preserving any whitespace around them + strippedLine = lineText.strip() + optionIndex = strippedLine.find(" ") + if isMultiline: + # part of a multiline entry started on a previous line so everything + # is part of the argument + lineComp["argument"][0] = lineText + elif optionIndex == -1: + # no argument provided + lineComp["option"][0] = lineText + else: + optionText = strippedLine[:optionIndex] + optionEnd = lineText.find(optionText) + len(optionText) + lineComp["option"][0] = lineText[:optionEnd] + lineComp["argument"][0] = lineText[optionEnd:] + + # flags following lines as belonging to this multiline entry if it ends + # with a slash + if strippedLine: isMultiline = strippedLine.endswith("\\") + + # gets the correction + if lineNumber in corrections: + lineIssue, lineIssueMsg = corrections[lineNumber] + + if lineIssue in (torConfig.ValidationError.DUPLICATE, torConfig.ValidationError.IS_DEFAULT): + lineComp["option"][1] = curses.A_BOLD | uiTools.getColor("blue") + lineComp["argument"][1] = curses.A_BOLD | uiTools.getColor("blue") + elif lineIssue == torConfig.ValidationError.MISMATCH: + lineComp["argument"][1] = curses.A_BOLD | uiTools.getColor("red") + lineComp["correction"][0] = " (%s)" % lineIssueMsg + else: + # For some types of configs the correction field is simply used to + # provide extra data (for instance, the type for tor state fields). + lineComp["correction"][0] = " (%s)" % lineIssueMsg + lineComp["correction"][1] = curses.A_BOLD | uiTools.getColor("magenta") + + # draws the line number + if self.showLineNum and displayLine < height and displayLine >= 1: + lineNumStr = ("%%%ii" % (lineNumOffset - 1)) % (lineNumber + 1) + self.addstr(displayLine, scrollOffset, lineNumStr, curses.A_BOLD | uiTools.getColor("yellow")) + + # draws the rest of the components with line wrap + cursorLoc, lineOffset = lineNumOffset + scrollOffset, 0 + maxLinesPerEntry = self._config["features.config.file.maxLinesPerEntry"] + displayQueue = [lineComp[entry] for entry in ("option", "argument", "correction", "comment")] + + while displayQueue: + msg, format = displayQueue.pop(0) + + maxMsgSize, includeBreak = width - cursorLoc, False + if len(msg) >= maxMsgSize: + # message is too long - break it up + if lineOffset == maxLinesPerEntry - 1: + msg = uiTools.cropStr(msg, maxMsgSize) + else: + includeBreak = True + msg, remainder = uiTools.cropStr(msg, maxMsgSize, 4, 4, uiTools.Ending.HYPHEN, True) + displayQueue.insert(0, (remainder.strip(), format)) + + drawLine = displayLine + lineOffset + if msg and drawLine < height and drawLine >= 1: + self.addstr(drawLine, cursorLoc, msg, format) + + # If we're done, and have added content to this line, then start + # further content on the next line. + cursorLoc += len(msg) + includeBreak |= not displayQueue and cursorLoc != lineNumOffset + scrollOffset + + if includeBreak: + lineOffset += 1 + cursorLoc = lineNumOffset + scrollOffset + + displayLine += max(lineOffset, 1) + + if trustLastContentHeight and displayLine >= height: break + + if not trustLastContentHeight: + self._lastContentHeightArgs = (width, height) + newContentHeight = displayLine + self.scroll - 1 + + if self._lastContentHeight != newContentHeight: + self._lastContentHeight = newContentHeight + self.redraw(True) + + self.valsLock.release() + diff --git a/src/interface/__init__.py b/src/interface/__init__.py deleted file mode 100644 index 0f11fc1..0000000 --- a/src/interface/__init__.py +++ /dev/null @@ -1,6 +0,0 @@ -""" -Panels, popups, and handlers comprising the arm user interface. -""" - -__all__ = ["configPanel", "connPanel", "controller", "descriptorPopup", "headerPanel", "logPanel", "torrcPanel"] - diff --git a/src/interface/configPanel.py b/src/interface/configPanel.py deleted file mode 100644 index fd6fb54..0000000 --- a/src/interface/configPanel.py +++ /dev/null @@ -1,364 +0,0 @@ -""" -Panel presenting the configuration state for tor or arm. Options can be edited -and the resulting configuration files saved. -""" - -import curses -import threading - -from util import conf, enum, panel, torTools, torConfig, uiTools - -DEFAULT_CONFIG = {"features.config.selectionDetails.height": 6, - "features.config.state.showPrivateOptions": False, - "features.config.state.showVirtualOptions": False, - "features.config.state.colWidth.option": 25, - "features.config.state.colWidth.value": 15} - -# TODO: The arm use cases are incomplete since they currently can't be -# modified, have their descriptions fetched, or even get a complete listing -# of what's available. -State = enum.Enum("TOR", "ARM") # state to be presented - -# mappings of option categories to the color for their entries -CATEGORY_COLOR = {torConfig.Category.GENERAL: "green", - torConfig.Category.CLIENT: "blue", - torConfig.Category.RELAY: "yellow", - torConfig.Category.DIRECTORY: "magenta", - torConfig.Category.AUTHORITY: "red", - torConfig.Category.HIDDEN_SERVICE: "cyan", - torConfig.Category.TESTING: "white", - torConfig.Category.UNKNOWN: "white"} - -# attributes of a ConfigEntry -Field = enum.Enum("CATEGORY", "OPTION", "VALUE", "TYPE", "ARG_USAGE", - "SUMMARY", "DESCRIPTION", "MAN_ENTRY", "IS_DEFAULT") -DEFAULT_SORT_ORDER = (Field.MAN_ENTRY, Field.OPTION, Field.IS_DEFAULT) -FIELD_ATTR = {Field.CATEGORY: ("Category", "red"), - Field.OPTION: ("Option Name", "blue"), - Field.VALUE: ("Value", "cyan"), - Field.TYPE: ("Arg Type", "green"), - Field.ARG_USAGE: ("Arg Usage", "yellow"), - Field.SUMMARY: ("Summary", "green"), - Field.DESCRIPTION: ("Description", "white"), - Field.MAN_ENTRY: ("Man Page Entry", "blue"), - Field.IS_DEFAULT: ("Is Default", "magenta")} - -class ConfigEntry(): - """ - Configuration option in the panel. - """ - - def __init__(self, option, type, isDefault): - self.fields = {} - self.fields[Field.OPTION] = option - self.fields[Field.TYPE] = type - self.fields[Field.IS_DEFAULT] = isDefault - - # Fetches extra infromation from external sources (the arm config and tor - # man page). These are None if unavailable for this config option. - summary = torConfig.getConfigSummary(option) - manEntry = torConfig.getConfigDescription(option) - - if manEntry: - self.fields[Field.MAN_ENTRY] = manEntry.index - self.fields[Field.CATEGORY] = manEntry.category - self.fields[Field.ARG_USAGE] = manEntry.argUsage - self.fields[Field.DESCRIPTION] = manEntry.description - else: - self.fields[Field.MAN_ENTRY] = 99999 # sorts non-man entries last - self.fields[Field.CATEGORY] = torConfig.Category.UNKNOWN - self.fields[Field.ARG_USAGE] = "" - self.fields[Field.DESCRIPTION] = "" - - # uses the full man page description if a summary is unavailable - self.fields[Field.SUMMARY] = summary if summary != None else self.fields[Field.DESCRIPTION] - - # cache of what's displayed for this configuration option - self.labelCache = None - self.labelCacheArgs = None - - def get(self, field): - """ - Provides back the value in the given field. - - Arguments: - field - enum for the field to be provided back - """ - - if field == Field.VALUE: return self._getValue() - else: return self.fields[field] - - def getAll(self, fields): - """ - Provides back a list with the given field values. - - Arguments: - field - enums for the fields to be provided back - """ - - return [self.get(field) for field in fields] - - def getLabel(self, optionWidth, valueWidth, summaryWidth): - """ - Provides display string of the configuration entry with the given - constraints on the width of the contents. - - Arguments: - optionWidth - width of the option column - valueWidth - width of the value column - summaryWidth - width of the summary column - """ - - # Fetching the display entries is very common so this caches the values. - # Doing this substantially drops cpu usage when scrolling (by around 40%). - - argSet = (optionWidth, valueWidth, summaryWidth) - if not self.labelCache or self.labelCacheArgs != argSet: - optionLabel = uiTools.cropStr(self.get(Field.OPTION), optionWidth) - valueLabel = uiTools.cropStr(self.get(Field.VALUE), valueWidth) - summaryLabel = uiTools.cropStr(self.get(Field.SUMMARY), summaryWidth, None) - lineTextLayout = "%%-%is %%-%is %%-%is" % (optionWidth, valueWidth, summaryWidth) - self.labelCache = lineTextLayout % (optionLabel, valueLabel, summaryLabel) - self.labelCacheArgs = argSet - - return self.labelCache - - def _getValue(self): - """ - Provides the current value of the configuration entry, taking advantage of - the torTools caching to effectively query the accurate value. This uses the - value's type to provide a user friendly representation if able. - """ - - confValue = ", ".join(torTools.getConn().getOption(self.get(Field.OPTION), [], True)) - - # provides nicer values for recognized types - if not confValue: confValue = "<none>" - elif self.get(Field.TYPE) == "Boolean" and confValue in ("0", "1"): - confValue = "False" if confValue == "0" else "True" - elif self.get(Field.TYPE) == "DataSize" and confValue.isdigit(): - confValue = uiTools.getSizeLabel(int(confValue)) - elif self.get(Field.TYPE) == "TimeInterval" and confValue.isdigit(): - confValue = uiTools.getTimeLabel(int(confValue), isLong = True) - - return confValue - -class ConfigPanel(panel.Panel): - """ - Renders a listing of the tor or arm configuration state, allowing options to - be selected and edited. - """ - - def __init__(self, stdscr, configType, config=None): - panel.Panel.__init__(self, stdscr, "configState", 0) - - self.sortOrdering = DEFAULT_SORT_ORDER - self._config = dict(DEFAULT_CONFIG) - if config: - config.update(self._config, { - "features.config.selectionDetails.height": 0, - "features.config.state.colWidth.option": 5, - "features.config.state.colWidth.value": 5}) - - sortFields = Field.values() - customOrdering = config.getIntCSV("features.config.order", None, 3, 0, len(sortFields)) - - if customOrdering: - self.sortOrdering = [sortFields[i] for i in customOrdering] - - self.configType = configType - self.confContents = [] - self.scroller = uiTools.Scroller(True) - self.valsLock = threading.RLock() - - # shows all configuration options if true, otherwise only the ones with - # the 'important' flag are shown - self.showAll = False - - if self.configType == State.TOR: - conn = torTools.getConn() - customOptions = torConfig.getCustomOptions() - configOptionLines = conn.getInfo("config/names", "").strip().split("\n") - - for line in configOptionLines: - # lines are of the form "<option> <type>", like: - # UseEntryGuards Boolean - confOption, confType = line.strip().split(" ", 1) - - # skips private and virtual entries if not configured to show them - if not self._config["features.config.state.showPrivateOptions"] and confOption.startswith("__"): - continue - elif not self._config["features.config.state.showVirtualOptions"] and confType == "Virtual": - continue - - self.confContents.append(ConfigEntry(confOption, confType, not confOption in customOptions)) - elif self.configType == State.ARM: - # loaded via the conf utility - armConf = conf.getConfig("arm") - for key in armConf.getKeys(): - pass # TODO: implement - - # mirror listing with only the important configuration options - self.confImportantContents = [] - for entry in self.confContents: - if torConfig.isImportant(entry.get(Field.OPTION)): - self.confImportantContents.append(entry) - - # if there aren't any important options then show everything - if not self.confImportantContents: - self.confImportantContents = self.confContents - - self.setSortOrder() # initial sorting of the contents - - def getSelection(self): - """ - Provides the currently selected entry. - """ - - return self.scroller.getCursorSelection(self._getConfigOptions()) - - def setSortOrder(self, ordering = None): - """ - Sets the configuration attributes we're sorting by and resorts the - contents. - - Arguments: - ordering - new ordering, if undefined then this resorts with the last - set ordering - """ - - self.valsLock.acquire() - if ordering: self.sortOrdering = ordering - self.confContents.sort(key=lambda i: (i.getAll(self.sortOrdering))) - self.confImportantContents.sort(key=lambda i: (i.getAll(self.sortOrdering))) - self.valsLock.release() - - def handleKey(self, key): - self.valsLock.acquire() - if uiTools.isScrollKey(key): - pageHeight = self.getPreferredSize()[0] - 1 - detailPanelHeight = self._config["features.config.selectionDetails.height"] - if detailPanelHeight > 0 and detailPanelHeight + 2 <= pageHeight: - pageHeight -= (detailPanelHeight + 1) - - isChanged = self.scroller.handleKey(key, self._getConfigOptions(), pageHeight) - if isChanged: self.redraw(True) - elif key == ord('a') or key == ord('A'): - self.showAll = not self.showAll - self.redraw(True) - self.valsLock.release() - - def draw(self, width, height): - self.valsLock.acquire() - - # draws the top label - configType = "Tor" if self.configType == State.TOR else "Arm" - hiddenMsg = "press 'a' to hide most options" if self.showAll else "press 'a' to show all options" - - # panel with details for the current selection - detailPanelHeight = self._config["features.config.selectionDetails.height"] - isScrollbarVisible = False - if detailPanelHeight == 0 or detailPanelHeight + 2 >= height: - # no detail panel - detailPanelHeight = 0 - scrollLoc = self.scroller.getScrollLoc(self._getConfigOptions(), height - 1) - cursorSelection = self.getSelection() - isScrollbarVisible = len(self._getConfigOptions()) > height - 1 - else: - # Shrink detail panel if there isn't sufficient room for the whole - # thing. The extra line is for the bottom border. - detailPanelHeight = min(height - 1, detailPanelHeight + 1) - scrollLoc = self.scroller.getScrollLoc(self._getConfigOptions(), height - 1 - detailPanelHeight) - cursorSelection = self.getSelection() - isScrollbarVisible = len(self._getConfigOptions()) > height - detailPanelHeight - 1 - - self._drawSelectionPanel(cursorSelection, width, detailPanelHeight, isScrollbarVisible) - - titleLabel = "%s Configuration (%s):" % (configType, hiddenMsg) - self.addstr(0, 0, titleLabel, curses.A_STANDOUT) - - # draws left-hand scroll bar if content's longer than the height - scrollOffset = 1 - if isScrollbarVisible: - scrollOffset = 3 - self.addScrollBar(scrollLoc, scrollLoc + height - detailPanelHeight - 1, len(self._getConfigOptions()), 1 + detailPanelHeight) - - optionWidth = self._config["features.config.state.colWidth.option"] - valueWidth = self._config["features.config.state.colWidth.value"] - descriptionWidth = max(0, width - scrollOffset - optionWidth - valueWidth - 2) - - for lineNum in range(scrollLoc, len(self._getConfigOptions())): - entry = self._getConfigOptions()[lineNum] - drawLine = lineNum + detailPanelHeight + 1 - scrollLoc - - lineFormat = curses.A_NORMAL if entry.get(Field.IS_DEFAULT) else curses.A_BOLD - if entry.get(Field.CATEGORY): lineFormat |= uiTools.getColor(CATEGORY_COLOR[entry.get(Field.CATEGORY)]) - if entry == cursorSelection: lineFormat |= curses.A_STANDOUT - - lineText = entry.getLabel(optionWidth, valueWidth, descriptionWidth) - self.addstr(drawLine, scrollOffset, lineText, lineFormat) - - if drawLine >= height: break - - self.valsLock.release() - - def _getConfigOptions(self): - return self.confContents if self.showAll else self.confImportantContents - - def _drawSelectionPanel(self, selection, width, detailPanelHeight, isScrollbarVisible): - """ - Renders a panel for the selected configuration option. - """ - - # This is a solid border unless the scrollbar is visible, in which case a - # 'T' pipe connects the border to the bar. - uiTools.drawBox(self, 0, 0, width, detailPanelHeight + 1) - if isScrollbarVisible: self.addch(detailPanelHeight, 1, curses.ACS_TTEE) - - selectionFormat = curses.A_BOLD | uiTools.getColor(CATEGORY_COLOR[selection.get(Field.CATEGORY)]) - - # first entry: - # <option> (<category> Option) - optionLabel =" (%s Option)" % selection.get(Field.CATEGORY) - self.addstr(1, 2, selection.get(Field.OPTION) + optionLabel, selectionFormat) - - # second entry: - # Value: <value> ([default|custom], <type>, usage: <argument usage>) - if detailPanelHeight >= 3: - valueAttr = [] - valueAttr.append("default" if selection.get(Field.IS_DEFAULT) else "custom") - valueAttr.append(selection.get(Field.TYPE)) - valueAttr.append("usage: %s" % (selection.get(Field.ARG_USAGE))) - valueAttrLabel = ", ".join(valueAttr) - - valueLabelWidth = width - 12 - len(valueAttrLabel) - valueLabel = uiTools.cropStr(selection.get(Field.VALUE), valueLabelWidth) - - self.addstr(2, 2, "Value: %s (%s)" % (valueLabel, valueAttrLabel), selectionFormat) - - # remainder is filled with the man page description - descriptionHeight = max(0, detailPanelHeight - 3) - descriptionContent = "Description: " + selection.get(Field.DESCRIPTION) - - for i in range(descriptionHeight): - # checks if we're done writing the description - if not descriptionContent: break - - # there's a leading indent after the first line - if i > 0: descriptionContent = " " + descriptionContent - - # we only want to work with content up until the next newline - if "\n" in descriptionContent: - lineContent, descriptionContent = descriptionContent.split("\n", 1) - else: lineContent, descriptionContent = descriptionContent, "" - - if i != descriptionHeight - 1: - # there's more lines to display - msg, remainder = uiTools.cropStr(lineContent, width - 2, 4, 4, uiTools.Ending.HYPHEN, True) - descriptionContent = remainder.strip() + descriptionContent - else: - # this is the last line, end it with an ellipse - msg = uiTools.cropStr(lineContent, width - 2, 4, 4) - - self.addstr(3 + i, 2, msg, selectionFormat) - diff --git a/src/interface/connections/__init__.py b/src/interface/connections/__init__.py deleted file mode 100644 index 7a32d77..0000000 --- a/src/interface/connections/__init__.py +++ /dev/null @@ -1,6 +0,0 @@ -""" -Panels, popups, and handlers comprising the arm user interface. -""" - -__all__ = ["circEntry", "connEntry", "connPanel", "entries"] - diff --git a/src/interface/connections/circEntry.py b/src/interface/connections/circEntry.py deleted file mode 100644 index 80c3f81..0000000 --- a/src/interface/connections/circEntry.py +++ /dev/null @@ -1,216 +0,0 @@ -""" -Connection panel entries for client circuits. This includes a header entry -followed by an entry for each hop in the circuit. For instance: - -89.188.20.246:42667 --> 217.172.182.26 (de) General / Built 8.6m (CIRCUIT) -| 85.8.28.4 (se) 98FBC3B2B93897A78CDD797EF549E6B62C9A8523 1 / Guard -| 91.121.204.76 (fr) 546387D93F8D40CFF8842BB9D3A8EC477CEDA984 2 / Middle -+- 217.172.182.26 (de) 5CFA9EA136C0EA0AC096E5CEA7EB674F1207CF86 3 / Exit -""" - -import curses - -from interface.connections import entries, connEntry -from util import torTools, uiTools - -# cached fingerprint -> (IP Address, ORPort) results -RELAY_INFO = {} - -def getRelayInfo(fingerprint): - """ - Provides the (IP Address, ORPort) tuple for the given relay. If the lookup - fails then this returns ("192.168.0.1", "0"). - - Arguments: - fingerprint - relay to look up - """ - - if not fingerprint in RELAY_INFO: - conn = torTools.getConn() - failureResult = ("192.168.0.1", "0") - - nsEntry = conn.getConsensusEntry(fingerprint) - if not nsEntry: return failureResult - - nsLineComp = nsEntry.split("\n")[0].split(" ") - if len(nsLineComp) < 8: return failureResult - - RELAY_INFO[fingerprint] = (nsLineComp[6], nsLineComp[7]) - - return RELAY_INFO[fingerprint] - -class CircEntry(connEntry.ConnectionEntry): - def __init__(self, circuitID, status, purpose, path): - connEntry.ConnectionEntry.__init__(self, "127.0.0.1", "0", "127.0.0.1", "0") - - self.circuitID = circuitID - self.status = status - - # drops to lowercase except the first letter - if len(purpose) >= 2: - purpose = purpose[0].upper() + purpose[1:].lower() - - self.lines = [CircHeaderLine(self.circuitID, purpose)] - - # Overwrites attributes of the initial line to make it more fitting as the - # header for our listing. - - self.lines[0].baseType = connEntry.Category.CIRCUIT - - self.update(status, path) - - def update(self, status, path): - """ - Our status and path can change over time if the circuit is still in the - process of being built. Updates these attributes of our relay. - - Arguments: - status - new status of the circuit - path - list of fingerprints for the series of relays involved in the - circuit - """ - - self.status = status - self.lines = [self.lines[0]] - - if status == "BUILT" and not self.lines[0].isBuilt: - exitIp, exitORPort = getRelayInfo(path[-1]) - self.lines[0].setExit(exitIp, exitORPort, path[-1]) - - for i in range(len(path)): - relayFingerprint = path[i] - relayIp, relayOrPort = getRelayInfo(relayFingerprint) - - if i == len(path) - 1: - if status == "BUILT": placementType = "Exit" - else: placementType = "Extending" - elif i == 0: placementType = "Guard" - else: placementType = "Middle" - - placementLabel = "%i / %s" % (i + 1, placementType) - - self.lines.append(CircLine(relayIp, relayOrPort, relayFingerprint, placementLabel)) - - self.lines[-1].isLast = True - -class CircHeaderLine(connEntry.ConnectionLine): - """ - Initial line of a client entry. This has the same basic format as connection - lines except that its etc field has circuit attributes. - """ - - def __init__(self, circuitID, purpose): - connEntry.ConnectionLine.__init__(self, "127.0.0.1", "0", "0.0.0.0", "0", False, False) - self.circuitID = circuitID - self.purpose = purpose - self.isBuilt = False - - def setExit(self, exitIpAddr, exitPort, exitFingerprint): - connEntry.ConnectionLine.__init__(self, "127.0.0.1", "0", exitIpAddr, exitPort, False, False) - self.isBuilt = True - self.foreign.fingerprintOverwrite = exitFingerprint - - def getType(self): - return connEntry.Category.CIRCUIT - - def getDestinationLabel(self, maxLength, includeLocale=False, includeHostname=False): - if not self.isBuilt: return "Building..." - return connEntry.ConnectionLine.getDestinationLabel(self, maxLength, includeLocale, includeHostname) - - def getEtcContent(self, width, listingType): - """ - Attempts to provide all circuit related stats. Anything that can't be - shown completely (not enough room) is dropped. - """ - - etcAttr = ["Purpose: %s" % self.purpose, "Circuit ID: %i" % self.circuitID] - - for i in range(len(etcAttr), -1, -1): - etcLabel = ", ".join(etcAttr[:i]) - if len(etcLabel) <= width: - return ("%%-%is" % width) % etcLabel - - return "" - - def getDetails(self, width): - if not self.isBuilt: - detailFormat = curses.A_BOLD | uiTools.getColor(connEntry.CATEGORY_COLOR[self.getType()]) - return [uiTools.DrawEntry("Building Circuit...", detailFormat)] - else: return connEntry.ConnectionLine.getDetails(self, width) - -class CircLine(connEntry.ConnectionLine): - """ - An individual hop in a circuit. This overwrites the displayed listing, but - otherwise makes use of the ConnectionLine attributes (for the detail display, - caching, etc). - """ - - def __init__(self, fIpAddr, fPort, fFingerprint, placementLabel): - connEntry.ConnectionLine.__init__(self, "127.0.0.1", "0", fIpAddr, fPort) - self.foreign.fingerprintOverwrite = fFingerprint - self.placementLabel = placementLabel - self.includePort = False - - # determines the sort of left hand bracketing we use - self.isLast = False - - def getType(self): - return connEntry.Category.CIRCUIT - - def getListingEntry(self, width, currentTime, listingType): - """ - Provides the DrawEntry for this relay in the circuilt listing. Lines are - composed of the following components: - <bracket> <dst> <etc> <placement label> - - The dst and etc entries largely match their ConnectionEntry counterparts. - - Arguments: - width - maximum length of the line - currentTime - the current unix time (ignored) - listingType - primary attribute we're listing connections by - """ - - return entries.ConnectionPanelLine.getListingEntry(self, width, currentTime, listingType) - - def _getListingEntry(self, width, currentTime, listingType): - lineFormat = uiTools.getColor(connEntry.CATEGORY_COLOR[self.getType()]) - - # The required widths are the sum of the following: - # bracketing (3 characters) - # placementLabel (14 characters) - # gap between etc and placement label (5 characters) - - if self.isLast: bracket = (curses.ACS_LLCORNER, curses.ACS_HLINE, ord(' ')) - else: bracket = (curses.ACS_VLINE, ord(' '), ord(' ')) - baselineSpace = len(bracket) + 14 + 5 - - dst, etc = "", "" - if listingType == entries.ListingType.IP_ADDRESS: - # TODO: include hostname when that's available - # dst width is derived as: - # src (21) + dst (26) + divider (7) + right gap (2) - bracket (3) = 53 char - dst = "%-53s" % self.getDestinationLabel(53, includeLocale = True) - etc = self.getEtcContent(width - baselineSpace - len(dst), listingType) - elif listingType == entries.ListingType.HOSTNAME: - # min space for the hostname is 40 characters - etc = self.getEtcContent(width - baselineSpace - 40, listingType) - dstLayout = "%%-%is" % (width - baselineSpace - len(etc)) - dst = dstLayout % self.foreign.getHostname(self.foreign.getIpAddr()) - elif listingType == entries.ListingType.FINGERPRINT: - # dst width is derived as: - # src (9) + dst (40) + divider (7) + right gap (2) - bracket (3) = 55 char - dst = "%-55s" % self.foreign.getFingerprint() - etc = self.getEtcContent(width - baselineSpace - len(dst), listingType) - else: - # min space for the nickname is 56 characters - etc = self.getEtcContent(width - baselineSpace - 56, listingType) - dstLayout = "%%-%is" % (width - baselineSpace - len(etc)) - dst = dstLayout % self.foreign.getNickname() - - drawEntry = uiTools.DrawEntry("%-14s" % self.placementLabel, lineFormat) - drawEntry = uiTools.DrawEntry(" " * (width - baselineSpace - len(dst) - len(etc) + 5), lineFormat, drawEntry) - drawEntry = uiTools.DrawEntry(dst + etc, lineFormat, drawEntry) - drawEntry = uiTools.DrawEntry(bracket, curses.A_NORMAL, drawEntry, lockFormat = True) - return drawEntry - diff --git a/src/interface/connections/connEntry.py b/src/interface/connections/connEntry.py deleted file mode 100644 index e6c0d92..0000000 --- a/src/interface/connections/connEntry.py +++ /dev/null @@ -1,850 +0,0 @@ -""" -Connection panel entries related to actual connections to or from the system -(ie, results seen by netstat, lsof, etc). -""" - -import time -import curses - -from util import connections, enum, torTools, uiTools -from interface.connections import entries - -# Connection Categories: -# Inbound Relay connection, coming to us. -# Outbound Relay connection, leaving us. -# Exit Outbound relay connection leaving the Tor network. -# Hidden Connections to a hidden service we're providing. -# Socks Socks connections for applications using Tor. -# Circuit Circuits our tor client has created. -# Directory Fetching tor consensus information. -# Control Tor controller (arm, vidalia, etc). - -Category = enum.Enum("INBOUND", "OUTBOUND", "EXIT", "HIDDEN", "SOCKS", "CIRCUIT", "DIRECTORY", "CONTROL") -CATEGORY_COLOR = {Category.INBOUND: "green", Category.OUTBOUND: "blue", - Category.EXIT: "red", Category.HIDDEN: "magenta", - Category.SOCKS: "yellow", Category.CIRCUIT: "cyan", - Category.DIRECTORY: "magenta", Category.CONTROL: "red"} - -# static data for listing format -# <src> --> <dst> <etc><padding> -LABEL_FORMAT = "%s --> %s %s%s" -LABEL_MIN_PADDING = 2 # min space between listing label and following data - -# sort value for scrubbed ip addresses -SCRUBBED_IP_VAL = 255 ** 4 - -CONFIG = {"features.connection.markInitialConnections": True, - "features.connection.showExitPort": True, - "features.connection.showColumn.fingerprint": True, - "features.connection.showColumn.nickname": True, - "features.connection.showColumn.destination": True, - "features.connection.showColumn.expandedIp": True} - -def loadConfig(config): - config.update(CONFIG) - -class Endpoint: - """ - Collection of attributes associated with a connection endpoint. This is a - thin wrapper for torUtil functions, making use of its caching for - performance. - """ - - def __init__(self, ipAddr, port): - self.ipAddr = ipAddr - self.port = port - - # if true, we treat the port as an ORPort when searching for matching - # fingerprints (otherwise the ORPort is assumed to be unknown) - self.isORPort = False - - # if set then this overwrites fingerprint lookups - self.fingerprintOverwrite = None - - def getIpAddr(self): - """ - Provides the IP address of the endpoint. - """ - - return self.ipAddr - - def getPort(self): - """ - Provides the port of the endpoint. - """ - - return self.port - - def getHostname(self, default = None): - """ - Provides the hostname associated with the relay's address. This is a - non-blocking call and returns None if the address either can't be resolved - or hasn't been resolved yet. - - Arguments: - default - return value if no hostname is available - """ - - # TODO: skipping all hostname resolution to be safe for now - #try: - # myHostname = hostnames.resolve(self.ipAddr) - #except: - # # either a ValueError or IOError depending on the source of the lookup failure - # myHostname = None - # - #if not myHostname: return default - #else: return myHostname - - return default - - def getLocale(self, default=None): - """ - Provides the two letter country code for the IP address' locale. - - Arguments: - default - return value if no locale information is available - """ - - conn = torTools.getConn() - return conn.getInfo("ip-to-country/%s" % self.ipAddr, default) - - def getFingerprint(self): - """ - Provides the fingerprint of the relay, returning "UNKNOWN" if it can't be - determined. - """ - - if self.fingerprintOverwrite: - return self.fingerprintOverwrite - - conn = torTools.getConn() - orPort = self.port if self.isORPort else None - myFingerprint = conn.getRelayFingerprint(self.ipAddr, orPort) - - if myFingerprint: return myFingerprint - else: return "UNKNOWN" - - def getNickname(self): - """ - Provides the nickname of the relay, retuning "UNKNOWN" if it can't be - determined. - """ - - myFingerprint = self.getFingerprint() - - if myFingerprint != "UNKNOWN": - conn = torTools.getConn() - myNickname = conn.getRelayNickname(myFingerprint) - - if myNickname: return myNickname - else: return "UNKNOWN" - else: return "UNKNOWN" - -class ConnectionEntry(entries.ConnectionPanelEntry): - """ - Represents a connection being made to or from this system. These only - concern real connections so it includes the inbound, outbound, directory, - application, and controller categories. - """ - - def __init__(self, lIpAddr, lPort, fIpAddr, fPort): - entries.ConnectionPanelEntry.__init__(self) - self.lines = [ConnectionLine(lIpAddr, lPort, fIpAddr, fPort)] - - def getSortValue(self, attr, listingType): - """ - Provides the value of a single attribute used for sorting purposes. - """ - - connLine = self.lines[0] - if attr == entries.SortAttr.IP_ADDRESS: - if connLine.isPrivate(): return SCRUBBED_IP_VAL # orders at the end - return connLine.sortIpAddr - elif attr == entries.SortAttr.PORT: - return connLine.sortPort - elif attr == entries.SortAttr.HOSTNAME: - if connLine.isPrivate(): return "" - return connLine.foreign.getHostname("") - elif attr == entries.SortAttr.FINGERPRINT: - return connLine.foreign.getFingerprint() - elif attr == entries.SortAttr.NICKNAME: - myNickname = connLine.foreign.getNickname() - if myNickname == "UNKNOWN": return "z" * 20 # orders at the end - else: return myNickname.lower() - elif attr == entries.SortAttr.CATEGORY: - return Category.indexOf(connLine.getType()) - elif attr == entries.SortAttr.UPTIME: - return connLine.startTime - elif attr == entries.SortAttr.COUNTRY: - if connections.isIpAddressPrivate(self.lines[0].foreign.getIpAddr()): return "" - else: return connLine.foreign.getLocale("") - else: - return entries.ConnectionPanelEntry.getSortValue(self, attr, listingType) - -class ConnectionLine(entries.ConnectionPanelLine): - """ - Display component of the ConnectionEntry. - """ - - def __init__(self, lIpAddr, lPort, fIpAddr, fPort, includePort=True, includeExpandedIpAddr=True): - entries.ConnectionPanelLine.__init__(self) - - self.local = Endpoint(lIpAddr, lPort) - self.foreign = Endpoint(fIpAddr, fPort) - self.startTime = time.time() - self.isInitialConnection = False - - # overwrite the local fingerprint with ours - conn = torTools.getConn() - self.local.fingerprintOverwrite = conn.getInfo("fingerprint") - - # True if the connection has matched the properties of a client/directory - # connection every time we've checked. The criteria we check is... - # client - first hop in an established circuit - # directory - matches an established single-hop circuit (probably a - # directory mirror) - - self._possibleClient = True - self._possibleDirectory = True - - # attributes for SOCKS, HIDDEN, and CONTROL connections - self.appName = None - self.appPid = None - self.isAppResolving = False - - myOrPort = conn.getOption("ORPort") - myDirPort = conn.getOption("DirPort") - mySocksPort = conn.getOption("SocksPort", "9050") - myCtlPort = conn.getOption("ControlPort") - myHiddenServicePorts = conn.getHiddenServicePorts() - - # the ORListenAddress can overwrite the ORPort - listenAddr = conn.getOption("ORListenAddress") - if listenAddr and ":" in listenAddr: - myOrPort = listenAddr[listenAddr.find(":") + 1:] - - if lPort in (myOrPort, myDirPort): - self.baseType = Category.INBOUND - self.local.isORPort = True - elif lPort == mySocksPort: - self.baseType = Category.SOCKS - elif fPort in myHiddenServicePorts: - self.baseType = Category.HIDDEN - elif lPort == myCtlPort: - self.baseType = Category.CONTROL - else: - self.baseType = Category.OUTBOUND - self.foreign.isORPort = True - - self.cachedType = None - - # includes the port or expanded ip address field when displaying listing - # information if true - self.includePort = includePort - self.includeExpandedIpAddr = includeExpandedIpAddr - - # cached immutable values used for sorting - self.sortIpAddr = connections.ipToInt(self.foreign.getIpAddr()) - self.sortPort = int(self.foreign.getPort()) - - def getListingEntry(self, width, currentTime, listingType): - """ - Provides the DrawEntry for this connection's listing. Lines are composed - of the following components: - <src> --> <dst> <etc> <uptime> (<type>) - - ListingType.IP_ADDRESS: - src - <internal addr:port> --> <external addr:port> - dst - <destination addr:port> - etc - <fingerprint> <nickname> - - ListingType.HOSTNAME: - src - localhost:<port> - dst - <destination hostname:port> - etc - <destination addr:port> <fingerprint> <nickname> - - ListingType.FINGERPRINT: - src - localhost - dst - <destination fingerprint> - etc - <nickname> <destination addr:port> - - ListingType.NICKNAME: - src - <source nickname> - dst - <destination nickname> - etc - <fingerprint> <destination addr:port> - - Arguments: - width - maximum length of the line - currentTime - unix timestamp for what the results should consider to be - the current time - listingType - primary attribute we're listing connections by - """ - - # fetch our (most likely cached) display entry for the listing - myListing = entries.ConnectionPanelLine.getListingEntry(self, width, currentTime, listingType) - - # fill in the current uptime and return the results - if CONFIG["features.connection.markInitialConnections"]: - timePrefix = "+" if self.isInitialConnection else " " - else: timePrefix = "" - - timeEntry = myListing.getNext() - timeEntry.text = timePrefix + "%5s" % uiTools.getTimeLabel(currentTime - self.startTime, 1) - - return myListing - - def isUnresolvedApp(self): - """ - True if our display uses application information that hasn't yet been resolved. - """ - - return self.appName == None and self.getType() in (Category.SOCKS, Category.HIDDEN, Category.CONTROL) - - def _getListingEntry(self, width, currentTime, listingType): - entryType = self.getType() - - # Lines are split into the following components in reverse: - # content - "<src> --> <dst> <etc> " - # time - "<uptime>" - # preType - " (" - # category - "<type>" - # postType - ") " - - lineFormat = uiTools.getColor(CATEGORY_COLOR[entryType]) - timeWidth = 6 if CONFIG["features.connection.markInitialConnections"] else 5 - - drawEntry = uiTools.DrawEntry(")" + " " * (9 - len(entryType)), lineFormat) - drawEntry = uiTools.DrawEntry(entryType.upper(), lineFormat | curses.A_BOLD, drawEntry) - drawEntry = uiTools.DrawEntry(" (", lineFormat, drawEntry) - drawEntry = uiTools.DrawEntry(" " * timeWidth, lineFormat, drawEntry) - drawEntry = uiTools.DrawEntry(self._getListingContent(width - (12 + timeWidth), listingType), lineFormat, drawEntry) - return drawEntry - - def _getDetails(self, width): - """ - Provides details on the connection, correlated against available consensus - data. - - Arguments: - width - available space to display in - """ - - detailFormat = curses.A_BOLD | uiTools.getColor(CATEGORY_COLOR[self.getType()]) - return [uiTools.DrawEntry(line, detailFormat) for line in self._getDetailContent(width)] - - def resetDisplay(self): - entries.ConnectionPanelLine.resetDisplay(self) - self.cachedType = None - - def isPrivate(self): - """ - Returns true if the endpoint is private, possibly belonging to a client - connection or exit traffic. - """ - - # This is used to scrub private information from the interface. Relaying - # etiquette (and wiretapping laws) say these are bad things to look at so - # DON'T CHANGE THIS UNLESS YOU HAVE A DAMN GOOD REASON! - - myType = self.getType() - - if myType == Category.INBOUND: - # if we're a guard or bridge and the connection doesn't belong to a - # known relay then it might be client traffic - - conn = torTools.getConn() - if "Guard" in conn.getMyFlags([]) or conn.getOption("BridgeRelay") == "1": - allMatches = conn.getRelayFingerprint(self.foreign.getIpAddr(), getAllMatches = True) - return allMatches == [] - elif myType == Category.EXIT: - # DNS connections exiting us aren't private (since they're hitting our - # resolvers). Everything else, however, is. - - # TODO: Ideally this would also double check that it's a UDP connection - # (since DNS is the only UDP connections Tor will relay), however this - # will take a bit more work to propagate the information up from the - # connection resolver. - return self.foreign.getPort() != "53" - - # for everything else this isn't a concern - return False - - def getType(self): - """ - Provides our best guess at the current type of the connection. This - depends on consensus results, our current client circuits, etc. Results - are cached until this entry's display is reset. - """ - - # caches both to simplify the calls and to keep the type consistent until - # we want to reflect changes - if not self.cachedType: - if self.baseType == Category.OUTBOUND: - # Currently the only non-static categories are OUTBOUND vs... - # - EXIT since this depends on the current consensus - # - CIRCUIT if this is likely to belong to our guard usage - # - DIRECTORY if this is a single-hop circuit (directory mirror?) - # - # The exitability, circuits, and fingerprints are all cached by the - # torTools util keeping this a quick lookup. - - conn = torTools.getConn() - destFingerprint = self.foreign.getFingerprint() - - if destFingerprint == "UNKNOWN": - # Not a known relay. This might be an exit connection. - - if conn.isExitingAllowed(self.foreign.getIpAddr(), self.foreign.getPort()): - self.cachedType = Category.EXIT - elif self._possibleClient or self._possibleDirectory: - # This belongs to a known relay. If we haven't eliminated ourselves as - # a possible client or directory connection then check if it still - # holds true. - - myCircuits = conn.getCircuits() - - if self._possibleClient: - # Checks that this belongs to the first hop in a circuit that's - # either unestablished or longer than a single hop (ie, anything but - # a built 1-hop connection since those are most likely a directory - # mirror). - - for _, status, _, path in myCircuits: - if path[0] == destFingerprint and (status != "BUILT" or len(path) > 1): - self.cachedType = Category.CIRCUIT # matched a probable guard connection - - # if we fell through, we can eliminate ourselves as a guard in the future - if not self.cachedType: - self._possibleClient = False - - if self._possibleDirectory: - # Checks if we match a built, single hop circuit. - - for _, status, _, path in myCircuits: - if path[0] == destFingerprint and status == "BUILT" and len(path) == 1: - self.cachedType = Category.DIRECTORY - - # if we fell through, eliminate ourselves as a directory connection - if not self.cachedType: - self._possibleDirectory = False - - if not self.cachedType: - self.cachedType = self.baseType - - return self.cachedType - - def getEtcContent(self, width, listingType): - """ - Provides the optional content for the connection. - - Arguments: - width - maximum length of the line - listingType - primary attribute we're listing connections by - """ - - # for applications show the command/pid - if self.getType() in (Category.SOCKS, Category.HIDDEN, Category.CONTROL): - displayLabel = "" - - if self.appName: - if self.appPid: displayLabel = "%s (%s)" % (self.appName, self.appPid) - else: displayLabel = self.appName - elif self.isAppResolving: - displayLabel = "resolving..." - else: displayLabel = "UNKNOWN" - - if len(displayLabel) < width: - return ("%%-%is" % width) % displayLabel - else: return "" - - # for everything else display connection/consensus information - dstAddress = self.getDestinationLabel(26, includeLocale = True) - etc, usedSpace = "", 0 - if listingType == entries.ListingType.IP_ADDRESS: - if width > usedSpace + 42 and CONFIG["features.connection.showColumn.fingerprint"]: - # show fingerprint (column width: 42 characters) - etc += "%-40s " % self.foreign.getFingerprint() - usedSpace += 42 - - if width > usedSpace + 10 and CONFIG["features.connection.showColumn.nickname"]: - # show nickname (column width: remainder) - nicknameSpace = width - usedSpace - nicknameLabel = uiTools.cropStr(self.foreign.getNickname(), nicknameSpace, 0) - etc += ("%%-%is " % nicknameSpace) % nicknameLabel - usedSpace += nicknameSpace + 2 - elif listingType == entries.ListingType.HOSTNAME: - if width > usedSpace + 28 and CONFIG["features.connection.showColumn.destination"]: - # show destination ip/port/locale (column width: 28 characters) - etc += "%-26s " % dstAddress - usedSpace += 28 - - if width > usedSpace + 42 and CONFIG["features.connection.showColumn.fingerprint"]: - # show fingerprint (column width: 42 characters) - etc += "%-40s " % self.foreign.getFingerprint() - usedSpace += 42 - - if width > usedSpace + 17 and CONFIG["features.connection.showColumn.nickname"]: - # show nickname (column width: min 17 characters, uses half of the remainder) - nicknameSpace = 15 + (width - (usedSpace + 17)) / 2 - nicknameLabel = uiTools.cropStr(self.foreign.getNickname(), nicknameSpace, 0) - etc += ("%%-%is " % nicknameSpace) % nicknameLabel - usedSpace += (nicknameSpace + 2) - elif listingType == entries.ListingType.FINGERPRINT: - if width > usedSpace + 17: - # show nickname (column width: min 17 characters, consumes any remaining space) - nicknameSpace = width - usedSpace - 2 - - # if there's room then also show a column with the destination - # ip/port/locale (column width: 28 characters) - isIpLocaleIncluded = width > usedSpace + 45 - isIpLocaleIncluded &= CONFIG["features.connection.showColumn.destination"] - if isIpLocaleIncluded: nicknameSpace -= 28 - - if CONFIG["features.connection.showColumn.nickname"]: - nicknameLabel = uiTools.cropStr(self.foreign.getNickname(), nicknameSpace, 0) - etc += ("%%-%is " % nicknameSpace) % nicknameLabel - usedSpace += nicknameSpace + 2 - - if isIpLocaleIncluded: - etc += "%-26s " % dstAddress - usedSpace += 28 - else: - if width > usedSpace + 42 and CONFIG["features.connection.showColumn.fingerprint"]: - # show fingerprint (column width: 42 characters) - etc += "%-40s " % self.foreign.getFingerprint() - usedSpace += 42 - - if width > usedSpace + 28 and CONFIG["features.connection.showColumn.destination"]: - # show destination ip/port/locale (column width: 28 characters) - etc += "%-26s " % dstAddress - usedSpace += 28 - - return ("%%-%is" % width) % etc - - def _getListingContent(self, width, listingType): - """ - Provides the source, destination, and extra info for our listing. - - Arguments: - width - maximum length of the line - listingType - primary attribute we're listing connections by - """ - - conn = torTools.getConn() - myType = self.getType() - dstAddress = self.getDestinationLabel(26, includeLocale = True) - - # The required widths are the sum of the following: - # - room for LABEL_FORMAT and LABEL_MIN_PADDING (11 characters) - # - base data for the listing - # - that extra field plus any previous - - usedSpace = len(LABEL_FORMAT % tuple([""] * 4)) + LABEL_MIN_PADDING - localPort = ":%s" % self.local.getPort() if self.includePort else "" - - src, dst, etc = "", "", "" - if listingType == entries.ListingType.IP_ADDRESS: - myExternalIpAddr = conn.getInfo("address", self.local.getIpAddr()) - addrDiffer = myExternalIpAddr != self.local.getIpAddr() - - # Expanding doesn't make sense, if the connection isn't actually - # going through Tor's external IP address. As there isn't a known - # method for checking if it is, we're checking the type instead. - # - # This isn't entirely correct. It might be a better idea to check if - # the source and destination addresses are both private, but that might - # not be perfectly reliable either. - - isExpansionType = not myType in (Category.SOCKS, Category.HIDDEN, Category.CONTROL) - - if isExpansionType: srcAddress = myExternalIpAddr + localPort - else: srcAddress = self.local.getIpAddr() + localPort - - if myType in (Category.SOCKS, Category.CONTROL): - # Like inbound connections these need their source and destination to - # be swapped. However, this only applies when listing by IP or hostname - # (their fingerprint and nickname are both for us). Reversing the - # fields here to keep the same column alignments. - - src = "%-21s" % dstAddress - dst = "%-26s" % srcAddress - else: - src = "%-21s" % srcAddress # ip:port = max of 21 characters - dst = "%-26s" % dstAddress # ip:port (xx) = max of 26 characters - - usedSpace += len(src) + len(dst) # base data requires 47 characters - - # Showing the fingerprint (which has the width of 42) has priority over - # an expanded address field. Hence check if we either have space for - # both or wouldn't be showing the fingerprint regardless. - - isExpandedAddrVisible = width > usedSpace + 28 - if isExpandedAddrVisible and CONFIG["features.connection.showColumn.fingerprint"]: - isExpandedAddrVisible = width < usedSpace + 42 or width > usedSpace + 70 - - if addrDiffer and isExpansionType and isExpandedAddrVisible and self.includeExpandedIpAddr and CONFIG["features.connection.showColumn.expandedIp"]: - # include the internal address in the src (extra 28 characters) - internalAddress = self.local.getIpAddr() + localPort - - # If this is an inbound connection then reverse ordering so it's: - # <foreign> --> <external> --> <internal> - # when the src and dst are swapped later - - if myType == Category.INBOUND: src = "%-21s --> %s" % (src, internalAddress) - else: src = "%-21s --> %s" % (internalAddress, src) - - usedSpace += 28 - - etc = self.getEtcContent(width - usedSpace, listingType) - usedSpace += len(etc) - elif listingType == entries.ListingType.HOSTNAME: - # 15 characters for source, and a min of 40 reserved for the destination - # TODO: when actually functional the src and dst need to be swapped for - # SOCKS and CONTROL connections - src = "localhost%-6s" % localPort - usedSpace += len(src) - minHostnameSpace = 40 - - etc = self.getEtcContent(width - usedSpace - minHostnameSpace, listingType) - usedSpace += len(etc) - - hostnameSpace = width - usedSpace - usedSpace = width # prevents padding at the end - if self.isPrivate(): - dst = ("%%-%is" % hostnameSpace) % "<scrubbed>" - else: - hostname = self.foreign.getHostname(self.foreign.getIpAddr()) - portLabel = ":%-5s" % self.foreign.getPort() if self.includePort else "" - - # truncates long hostnames and sets dst to <hostname>:<port> - hostname = uiTools.cropStr(hostname, hostnameSpace, 0) - dst = ("%%-%is" % hostnameSpace) % (hostname + portLabel) - elif listingType == entries.ListingType.FINGERPRINT: - src = "localhost" - if myType == Category.CONTROL: dst = "localhost" - else: dst = self.foreign.getFingerprint() - dst = "%-40s" % dst - - usedSpace += len(src) + len(dst) # base data requires 49 characters - - etc = self.getEtcContent(width - usedSpace, listingType) - usedSpace += len(etc) - else: - # base data requires 50 min characters - src = self.local.getNickname() - if myType == Category.CONTROL: dst = self.local.getNickname() - else: dst = self.foreign.getNickname() - minBaseSpace = 50 - - etc = self.getEtcContent(width - usedSpace - minBaseSpace, listingType) - usedSpace += len(etc) - - baseSpace = width - usedSpace - usedSpace = width # prevents padding at the end - - if len(src) + len(dst) > baseSpace: - src = uiTools.cropStr(src, baseSpace / 3) - dst = uiTools.cropStr(dst, baseSpace - len(src)) - - # pads dst entry to its max space - dst = ("%%-%is" % (baseSpace - len(src))) % dst - - if myType == Category.INBOUND: src, dst = dst, src - padding = " " * (width - usedSpace + LABEL_MIN_PADDING) - return LABEL_FORMAT % (src, dst, etc, padding) - - def _getDetailContent(self, width): - """ - Provides a list with detailed information for this connection. - - Arguments: - width - max length of lines - """ - - lines = [""] * 7 - lines[0] = "address: %s" % self.getDestinationLabel(width - 11) - lines[1] = "locale: %s" % ("??" if self.isPrivate() else self.foreign.getLocale("??")) - - # Remaining data concerns the consensus results, with three possible cases: - # - if there's a single match then display its details - # - if there's multiple potential relays then list all of the combinations - # of ORPorts / Fingerprints - # - if no consensus data is available then say so (probably a client or - # exit connection) - - fingerprint = self.foreign.getFingerprint() - conn = torTools.getConn() - - if fingerprint != "UNKNOWN": - # single match - display information available about it - nsEntry = conn.getConsensusEntry(fingerprint) - descEntry = conn.getDescriptorEntry(fingerprint) - - # append the fingerprint to the second line - lines[1] = "%-13sfingerprint: %s" % (lines[1], fingerprint) - - if nsEntry: - # example consensus entry: - # r murble R8sCM1ar1sS2GulQYFVmvN95xsk RJr6q+wkTFG+ng5v2bdCbVVFfA4 2011-02-21 00:25:32 195.43.157.85 443 0 - # s Exit Fast Guard Named Running Stable Valid - # w Bandwidth=2540 - # p accept 20-23,43,53,79-81,88,110,143,194,443 - - nsLines = nsEntry.split("\n") - - firstLineComp = nsLines[0].split(" ") - if len(firstLineComp) >= 9: - _, nickname, _, _, pubDate, pubTime, _, orPort, dirPort = firstLineComp[:9] - else: nickname, pubDate, pubTime, orPort, dirPort = "", "", "", "", "" - - flags = "unknown" - if len(nsLines) >= 2 and nsLines[1].startswith("s "): - flags = nsLines[1][2:] - - # The network status exit policy doesn't exist for older tor versions. - # If unavailable we'll need the full exit policy which is on the - # descriptor (if that's available). - - exitPolicy = "unknown" - if len(nsLines) >= 4 and nsLines[3].startswith("p "): - exitPolicy = nsLines[3][2:].replace(",", ", ") - elif descEntry: - # the descriptor has an individual line for each entry in the exit policy - exitPolicyEntries = [] - - for line in descEntry.split("\n"): - if line.startswith("accept") or line.startswith("reject"): - exitPolicyEntries.append(line.strip()) - - exitPolicy = ", ".join(exitPolicyEntries) - - dirPortLabel = "" if dirPort == "0" else "dirport: %s" % dirPort - lines[2] = "nickname: %-25s orport: %-10s %s" % (nickname, orPort, dirPortLabel) - lines[3] = "published: %s %s" % (pubDate, pubTime) - lines[4] = "flags: %s" % flags.replace(" ", ", ") - lines[5] = "exit policy: %s" % exitPolicy - - if descEntry: - torVersion, platform, contact = "", "", "" - - for descLine in descEntry.split("\n"): - if descLine.startswith("platform"): - # has the tor version and platform, ex: - # platform Tor 0.2.1.29 (r318f470bc5f2ad43) on Linux x86_64 - - torVersion = descLine[13:descLine.find(" ", 13)] - platform = descLine[descLine.rfind(" on ") + 4:] - elif descLine.startswith("contact"): - contact = descLine[8:] - - # clears up some highly common obscuring - for alias in (" at ", " AT "): contact = contact.replace(alias, "@") - for alias in (" dot ", " DOT "): contact = contact.replace(alias, ".") - - break # contact lines come after the platform - - lines[3] = "%-35s os: %-14s version: %s" % (lines[3], platform, torVersion) - - # contact information is an optional field - if contact: lines[6] = "contact: %s" % contact - else: - allMatches = conn.getRelayFingerprint(self.foreign.getIpAddr(), getAllMatches = True) - - if allMatches: - # multiple matches - lines[2] = "Multiple matches, possible fingerprints are:" - - for i in range(len(allMatches)): - isLastLine = i == 3 - - relayPort, relayFingerprint = allMatches[i] - lineText = "%i. or port: %-5s fingerprint: %s" % (i, relayPort, relayFingerprint) - - # if there's multiple lines remaining at the end then give a count - remainingRelays = len(allMatches) - i - if isLastLine and remainingRelays > 1: - lineText = "... %i more" % remainingRelays - - lines[3 + i] = lineText - - if isLastLine: break - else: - # no consensus entry for this ip address - lines[2] = "No consensus data found" - - # crops any lines that are too long - for i in range(len(lines)): - lines[i] = uiTools.cropStr(lines[i], width - 2) - - return lines - - def getDestinationLabel(self, maxLength, includeLocale=False, includeHostname=False): - """ - Provides a short description of the destination. This is made up of two - components, the base <ip addr>:<port> and an extra piece of information in - parentheses. The IP address is scrubbed from private connections. - - Extra information is... - - the port's purpose for exit connections - - the locale and/or hostname if set to do so, the address isn't private, - and isn't on the local network - - nothing otherwise - - Arguments: - maxLength - maximum length of the string returned - includeLocale - possibly includes the locale - includeHostname - possibly includes the hostname - """ - - # the port and port derived data can be hidden by config or without includePort - includePort = self.includePort and (CONFIG["features.connection.showExitPort"] or self.getType() != Category.EXIT) - - # destination of the connection - ipLabel = "<scrubbed>" if self.isPrivate() else self.foreign.getIpAddr() - portLabel = ":%s" % self.foreign.getPort() if includePort else "" - dstAddress = ipLabel + portLabel - - # Only append the extra info if there's at least a couple characters of - # space (this is what's needed for the country codes). - if len(dstAddress) + 5 <= maxLength: - spaceAvailable = maxLength - len(dstAddress) - 3 - - if self.getType() == Category.EXIT and includePort: - purpose = connections.getPortUsage(self.foreign.getPort()) - - if purpose: - # BitTorrent is a common protocol to truncate, so just use "Torrent" - # if there's not enough room. - if len(purpose) > spaceAvailable and purpose == "BitTorrent": - purpose = "Torrent" - - # crops with a hyphen if too long - purpose = uiTools.cropStr(purpose, spaceAvailable, endType = uiTools.Ending.HYPHEN) - - dstAddress += " (%s)" % purpose - elif not connections.isIpAddressPrivate(self.foreign.getIpAddr()): - extraInfo = [] - conn = torTools.getConn() - - if includeLocale and not conn.isGeoipUnavailable(): - foreignLocale = self.foreign.getLocale("??") - extraInfo.append(foreignLocale) - spaceAvailable -= len(foreignLocale) + 2 - - if includeHostname: - dstHostname = self.foreign.getHostname() - - if dstHostname: - # determines the full space available, taking into account the ", " - # dividers if there's multiple pieces of extra data - - maxHostnameSpace = spaceAvailable - 2 * len(extraInfo) - dstHostname = uiTools.cropStr(dstHostname, maxHostnameSpace) - extraInfo.append(dstHostname) - spaceAvailable -= len(dstHostname) - - if extraInfo: - dstAddress += " (%s)" % ", ".join(extraInfo) - - return dstAddress[:maxLength] - diff --git a/src/interface/connections/connPanel.py b/src/interface/connections/connPanel.py deleted file mode 100644 index 79fe9df..0000000 --- a/src/interface/connections/connPanel.py +++ /dev/null @@ -1,398 +0,0 @@ -""" -Listing of the currently established connections tor has made. -""" - -import time -import curses -import threading - -from interface.connections import entries, connEntry, circEntry -from util import connections, enum, panel, torTools, uiTools - -DEFAULT_CONFIG = {"features.connection.resolveApps": True, - "features.connection.listingType": 0, - "features.connection.refreshRate": 5} - -# height of the detail panel content, not counting top and bottom border -DETAILS_HEIGHT = 7 - -# listing types -Listing = enum.Enum(("IP_ADDRESS", "IP Address"), "HOSTNAME", "FINGERPRINT", "NICKNAME") - -DEFAULT_SORT_ORDER = (entries.SortAttr.CATEGORY, entries.SortAttr.LISTING, entries.SortAttr.UPTIME) - -class ConnectionPanel(panel.Panel, threading.Thread): - """ - Listing of connections tor is making, with information correlated against - the current consensus and other data sources. - """ - - def __init__(self, stdscr, config=None): - panel.Panel.__init__(self, stdscr, "conn", 0) - threading.Thread.__init__(self) - self.setDaemon(True) - - self._sortOrdering = DEFAULT_SORT_ORDER - self._config = dict(DEFAULT_CONFIG) - - if config: - config.update(self._config, { - "features.connection.listingType": (0, len(Listing.values()) - 1), - "features.connection.refreshRate": 1}) - - sortFields = entries.SortAttr.values() - customOrdering = config.getIntCSV("features.connection.order", None, 3, 0, len(sortFields)) - - if customOrdering: - self._sortOrdering = [sortFields[i] for i in customOrdering] - - self._listingType = Listing.values()[self._config["features.connection.listingType"]] - self._scroller = uiTools.Scroller(True) - self._title = "Connections:" # title line of the panel - self._entries = [] # last fetched display entries - self._entryLines = [] # individual lines rendered from the entries listing - self._showDetails = False # presents the details panel if true - - self._lastUpdate = -1 # time the content was last revised - self._isTorRunning = True # indicates if tor is currently running or not - self._isPaused = True # prevents updates if true - self._pauseTime = None # time when the panel was paused - self._halt = False # terminates thread if true - self._cond = threading.Condition() # used for pausing the thread - self.valsLock = threading.RLock() - - # Last sampling received from the ConnectionResolver, used to detect when - # it changes. - self._lastResourceFetch = -1 - - # resolver for the command/pid associated with SOCKS, HIDDEN, and CONTROL connections - self._appResolver = connections.AppResolver("arm") - - # rate limits appResolver queries to once per update - self.appResolveSinceUpdate = False - - self._update() # populates initial entries - self._resolveApps(False) # resolves initial applications - - # mark the initially exitsing connection uptimes as being estimates - for entry in self._entries: - if isinstance(entry, connEntry.ConnectionEntry): - entry.getLines()[0].isInitialConnection = True - - # listens for when tor stops so we know to stop reflecting changes - torTools.getConn().addStatusListener(self.torStateListener) - - def torStateListener(self, conn, eventType): - """ - Freezes the connection contents when Tor stops. - - Arguments: - conn - tor controller - eventType - type of event detected - """ - - self._isTorRunning = eventType == torTools.State.INIT - - if self._isPaused or not self._isTorRunning: - if not self._pauseTime: self._pauseTime = time.time() - else: self._pauseTime = None - - self.redraw(True) - - def setPaused(self, isPause): - """ - If true, prevents the panel from updating. - """ - - if not self._isPaused == isPause: - self._isPaused = isPause - - if isPause or not self._isTorRunning: - if not self._pauseTime: self._pauseTime = time.time() - else: self._pauseTime = None - - # redraws so the display reflects any changes between the last update - # and being paused - self.redraw(True) - - def setSortOrder(self, ordering = None): - """ - Sets the connection attributes we're sorting by and resorts the contents. - - Arguments: - ordering - new ordering, if undefined then this resorts with the last - set ordering - """ - - self.valsLock.acquire() - if ordering: self._sortOrdering = ordering - self._entries.sort(key=lambda i: (i.getSortValues(self._sortOrdering, self._listingType))) - - self._entryLines = [] - for entry in self._entries: - self._entryLines += entry.getLines() - self.valsLock.release() - - def setListingType(self, listingType): - """ - Sets the priority information presented by the panel. - - Arguments: - listingType - Listing instance for the primary information to be shown - """ - - self.valsLock.acquire() - self._listingType = listingType - - # if we're sorting by the listing then we need to resort - if entries.SortAttr.LISTING in self._sortOrdering: - self.setSortOrder() - - self.valsLock.release() - - def handleKey(self, key): - self.valsLock.acquire() - - if uiTools.isScrollKey(key): - pageHeight = self.getPreferredSize()[0] - 1 - if self._showDetails: pageHeight -= (DETAILS_HEIGHT + 1) - isChanged = self._scroller.handleKey(key, self._entryLines, pageHeight) - if isChanged: self.redraw(True) - elif uiTools.isSelectionKey(key): - self._showDetails = not self._showDetails - self.redraw(True) - - self.valsLock.release() - - def run(self): - """ - Keeps connections listing updated, checking for new entries at a set rate. - """ - - lastDraw = time.time() - 1 - while not self._halt: - currentTime = time.time() - - if self._isPaused or not self._isTorRunning or currentTime - lastDraw < self._config["features.connection.refreshRate"]: - self._cond.acquire() - if not self._halt: self._cond.wait(0.2) - self._cond.release() - else: - # updates content if their's new results, otherwise just redraws - self._update() - self.redraw(True) - - # we may have missed multiple updates due to being paused, showing - # another panel, etc so lastDraw might need to jump multiple ticks - drawTicks = (time.time() - lastDraw) / self._config["features.connection.refreshRate"] - lastDraw += self._config["features.connection.refreshRate"] * drawTicks - - def draw(self, width, height): - self.valsLock.acquire() - - # extra line when showing the detail panel is for the bottom border - detailPanelOffset = DETAILS_HEIGHT + 1 if self._showDetails else 0 - isScrollbarVisible = len(self._entryLines) > height - detailPanelOffset - 1 - - scrollLoc = self._scroller.getScrollLoc(self._entryLines, height - detailPanelOffset - 1) - cursorSelection = self._scroller.getCursorSelection(self._entryLines) - - # draws the detail panel if currently displaying it - if self._showDetails: - # This is a solid border unless the scrollbar is visible, in which case a - # 'T' pipe connects the border to the bar. - uiTools.drawBox(self, 0, 0, width, DETAILS_HEIGHT + 2) - if isScrollbarVisible: self.addch(DETAILS_HEIGHT + 1, 1, curses.ACS_TTEE) - - drawEntries = cursorSelection.getDetails(width) - for i in range(min(len(drawEntries), DETAILS_HEIGHT)): - drawEntries[i].render(self, 1 + i, 2) - - # title label with connection counts - title = "Connection Details:" if self._showDetails else self._title - self.addstr(0, 0, title, curses.A_STANDOUT) - - scrollOffset = 1 - if isScrollbarVisible: - scrollOffset = 3 - self.addScrollBar(scrollLoc, scrollLoc + height - detailPanelOffset - 1, len(self._entryLines), 1 + detailPanelOffset) - - currentTime = self._pauseTime if self._pauseTime else time.time() - for lineNum in range(scrollLoc, len(self._entryLines)): - entryLine = self._entryLines[lineNum] - - # if this is an unresolved SOCKS, HIDDEN, or CONTROL entry then queue up - # resolution for the applicaitions they belong to - if isinstance(entryLine, connEntry.ConnectionLine) and entryLine.isUnresolvedApp(): - self._resolveApps() - - # hilighting if this is the selected line - extraFormat = curses.A_STANDOUT if entryLine == cursorSelection else curses.A_NORMAL - - drawEntry = entryLine.getListingEntry(width - scrollOffset, currentTime, self._listingType) - drawLine = lineNum + detailPanelOffset + 1 - scrollLoc - drawEntry.render(self, drawLine, scrollOffset, extraFormat) - if drawLine >= height: break - - self.valsLock.release() - - def stop(self): - """ - Halts further resolutions and terminates the thread. - """ - - self._cond.acquire() - self._halt = True - self._cond.notifyAll() - self._cond.release() - - def _update(self): - """ - Fetches the newest resolved connections. - """ - - connResolver = connections.getResolver("tor") - currentResolutionCount = connResolver.getResolutionCount() - self.appResolveSinceUpdate = False - - if self._lastResourceFetch != currentResolutionCount: - self.valsLock.acquire() - - newEntries = [] # the new results we'll display - - # Fetches new connections and client circuits... - # newConnections [(local ip, local port, foreign ip, foreign port)...] - # newCircuits {circuitID => (status, purpose, path)...} - - newConnections = connResolver.getConnections() - newCircuits = {} - - for circuitID, status, purpose, path in torTools.getConn().getCircuits(): - # Skips established single-hop circuits (these are for directory - # fetches, not client circuits) - if not (status == "BUILT" and len(path) == 1): - newCircuits[circuitID] = (status, purpose, path) - - # Populates newEntries with any of our old entries that still exist. - # This is both for performance and to keep from resetting the uptime - # attributes. Note that CircEntries are a ConnectionEntry subclass so - # we need to check for them first. - - for oldEntry in self._entries: - if isinstance(oldEntry, circEntry.CircEntry): - newEntry = newCircuits.get(oldEntry.circuitID) - - if newEntry: - oldEntry.update(newEntry[0], newEntry[2]) - newEntries.append(oldEntry) - del newCircuits[oldEntry.circuitID] - elif isinstance(oldEntry, connEntry.ConnectionEntry): - connLine = oldEntry.getLines()[0] - connAttr = (connLine.local.getIpAddr(), connLine.local.getPort(), - connLine.foreign.getIpAddr(), connLine.foreign.getPort()) - - if connAttr in newConnections: - newEntries.append(oldEntry) - newConnections.remove(connAttr) - - # Reset any display attributes for the entries we're keeping - for entry in newEntries: entry.resetDisplay() - - # Adds any new connection and circuit entries. - for lIp, lPort, fIp, fPort in newConnections: - newConnEntry = connEntry.ConnectionEntry(lIp, lPort, fIp, fPort) - if newConnEntry.getLines()[0].getType() != connEntry.Category.CIRCUIT: - newEntries.append(newConnEntry) - - for circuitID in newCircuits: - status, purpose, path = newCircuits[circuitID] - newEntries.append(circEntry.CircEntry(circuitID, status, purpose, path)) - - # Counts the relays in each of the categories. This also flushes the - # type cache for all of the connections (in case its changed since last - # fetched). - - categoryTypes = connEntry.Category.values() - typeCounts = dict((type, 0) for type in categoryTypes) - for entry in newEntries: - if isinstance(entry, connEntry.ConnectionEntry): - typeCounts[entry.getLines()[0].getType()] += 1 - elif isinstance(entry, circEntry.CircEntry): - typeCounts[connEntry.Category.CIRCUIT] += 1 - - # makes labels for all the categories with connections (ie, - # "21 outbound", "1 control", etc) - countLabels = [] - - for category in categoryTypes: - if typeCounts[category] > 0: - countLabels.append("%i %s" % (typeCounts[category], category.lower())) - - if countLabels: self._title = "Connections (%s):" % ", ".join(countLabels) - else: self._title = "Connections:" - - self._entries = newEntries - - self._entryLines = [] - for entry in self._entries: - self._entryLines += entry.getLines() - - self.setSortOrder() - self._lastResourceFetch = currentResolutionCount - self.valsLock.release() - - def _resolveApps(self, flagQuery = True): - """ - Triggers an asynchronous query for all unresolved SOCKS, HIDDEN, and - CONTROL entries. - - Arguments: - flagQuery - sets a flag to prevent further call from being respected - until the next update if true - """ - - if self.appResolveSinceUpdate or not self._config["features.connection.resolveApps"]: return - unresolvedLines = [l for l in self._entryLines if isinstance(l, connEntry.ConnectionLine) and l.isUnresolvedApp()] - - # get the ports used for unresolved applications - appPorts = [] - - for line in unresolvedLines: - appConn = line.local if line.getType() == connEntry.Category.HIDDEN else line.foreign - appPorts.append(appConn.getPort()) - - # Queue up resolution for the unresolved ports (skips if it's still working - # on the last query). - if appPorts and not self._appResolver.isResolving: - self._appResolver.resolve(appPorts) - - # Fetches results. If the query finishes quickly then this is what we just - # asked for, otherwise these belong to an earlier resolution. - # - # The application resolver might have given up querying (for instance, if - # the lsof lookups aren't working on this platform or lacks permissions). - # The isAppResolving flag lets the unresolved entries indicate if there's - # a lookup in progress for them or not. - - appResults = self._appResolver.getResults(0.2) - - for line in unresolvedLines: - isLocal = line.getType() == connEntry.Category.HIDDEN - linePort = line.local.getPort() if isLocal else line.foreign.getPort() - - if linePort in appResults: - # sets application attributes if there's a result with this as the - # inbound port - for inboundPort, outboundPort, cmd, pid in appResults[linePort]: - appPort = outboundPort if isLocal else inboundPort - - if linePort == appPort: - line.appName = cmd - line.appPid = pid - line.isAppResolving = False - else: - line.isAppResolving = self._appResolver.isResolving - - if flagQuery: - self.appResolveSinceUpdate = True - diff --git a/src/interface/connections/entries.py b/src/interface/connections/entries.py deleted file mode 100644 index 6b24412..0000000 --- a/src/interface/connections/entries.py +++ /dev/null @@ -1,164 +0,0 @@ -""" -Interface for entries in the connection panel. These consist of two parts: the -entry itself (ie, Tor connection, client circuit, etc) and the lines it -consists of in the listing. -""" - -from util import enum - -# attributes we can list entries by -ListingType = enum.Enum(("IP_ADDRESS", "IP Address"), "HOSTNAME", "FINGERPRINT", "NICKNAME") - -SortAttr = enum.Enum("CATEGORY", "UPTIME", "LISTING", "IP_ADDRESS", "PORT", - "HOSTNAME", "FINGERPRINT", "NICKNAME", "COUNTRY") - -SORT_COLORS = {SortAttr.CATEGORY: "red", SortAttr.UPTIME: "yellow", - SortAttr.LISTING: "green", SortAttr.IP_ADDRESS: "blue", - SortAttr.PORT: "blue", SortAttr.HOSTNAME: "magenta", - SortAttr.FINGERPRINT: "cyan", SortAttr.NICKNAME: "cyan", - SortAttr.COUNTRY: "blue"} - -# maximum number of ports a system can have -PORT_COUNT = 65536 - -class ConnectionPanelEntry: - """ - Common parent for connection panel entries. This consists of a list of lines - in the panel listing. This caches results until the display indicates that - they should be flushed. - """ - - def __init__(self): - self.lines = [] - self.flushCache = True - - def getLines(self): - """ - Provides the individual lines in the connection listing. - """ - - if self.flushCache: - self.lines = self._getLines(self.lines) - self.flushCache = False - - return self.lines - - def _getLines(self, oldResults): - # implementation of getLines - - for line in oldResults: - line.resetDisplay() - - return oldResults - - def getSortValues(self, sortAttrs, listingType): - """ - Provides the value used in comparisons to sort based on the given - attribute. - - Arguments: - sortAttrs - list of SortAttr values for the field being sorted on - listingType - ListingType enumeration for the attribute we're listing - entries by - """ - - return [self.getSortValue(attr, listingType) for attr in sortAttrs] - - def getSortValue(self, attr, listingType): - """ - Provides the value of a single attribute used for sorting purposes. - - Arguments: - attr - list of SortAttr values for the field being sorted on - listingType - ListingType enumeration for the attribute we're listing - entries by - """ - - if attr == SortAttr.LISTING: - if listingType == ListingType.IP_ADDRESS: - # uses the IP address as the primary value, and port as secondary - sortValue = self.getSortValue(SortAttr.IP_ADDRESS, listingType) * PORT_COUNT - sortValue += self.getSortValue(SortAttr.PORT, listingType) - return sortValue - elif listingType == ListingType.HOSTNAME: - return self.getSortValue(SortAttr.HOSTNAME, listingType) - elif listingType == ListingType.FINGERPRINT: - return self.getSortValue(SortAttr.FINGERPRINT, listingType) - elif listingType == ListingType.NICKNAME: - return self.getSortValue(SortAttr.NICKNAME, listingType) - - return "" - - def resetDisplay(self): - """ - Flushes cached display results. - """ - - self.flushCache = True - -class ConnectionPanelLine: - """ - Individual line in the connection panel listing. - """ - - def __init__(self): - # cache for displayed information - self._listingCache = None - self._listingCacheArgs = (None, None) - - self._detailsCache = None - self._detailsCacheArgs = None - - self._descriptorCache = None - self._descriptorCacheArgs = None - - def getListingEntry(self, width, currentTime, listingType): - """ - Provides a DrawEntry instance for contents to be displayed in the - connection panel listing. - - Arguments: - width - available space to display in - currentTime - unix timestamp for what the results should consider to be - the current time (this may be ignored due to caching) - listingType - ListingType enumeration for the highest priority content - to be displayed - """ - - if self._listingCacheArgs != (width, listingType): - self._listingCache = self._getListingEntry(width, currentTime, listingType) - self._listingCacheArgs = (width, listingType) - - return self._listingCache - - def _getListingEntry(self, width, currentTime, listingType): - # implementation of getListingEntry - return None - - def getDetails(self, width): - """ - Provides a list of DrawEntry instances with detailed information for this - connection. - - Arguments: - width - available space to display in - """ - - if self._detailsCacheArgs != width: - self._detailsCache = self._getDetails(width) - self._detailsCacheArgs = width - - return self._detailsCache - - def _getDetails(self, width): - # implementation of getDetails - return [] - - def resetDisplay(self): - """ - Flushes cached display results. - """ - - self._listingCacheArgs = (None, None) - self._detailsCacheArgs = None - diff --git a/src/interface/controller.py b/src/interface/controller.py deleted file mode 100644 index 5060188..0000000 --- a/src/interface/controller.py +++ /dev/null @@ -1,1584 +0,0 @@ -#!/usr/bin/env python -# controller.py -- arm interface (curses monitor for relay status) -# Released under the GPL v3 (http://www.gnu.org/licenses/gpl.html) - -""" -Curses (terminal) interface for the arm relay status monitor. -""" - -import os -import re -import math -import time -import curses -import curses.textpad -import socket -from TorCtl import TorCtl - -import headerPanel -import graphing.graphPanel -import logPanel -import configPanel -import torrcPanel -import descriptorPopup - -import interface.connections.connPanel -import interface.connections.connEntry -import interface.connections.entries -from util import conf, log, connections, hostnames, panel, sysTools, torConfig, torTools, uiTools -import graphing.bandwidthStats -import graphing.connStats -import graphing.resourceStats - -CONFIRM_QUIT = True -REFRESH_RATE = 5 # seconds between redrawing screen -MAX_REGEX_FILTERS = 5 # maximum number of previous regex filters that'll be remembered - -# enums for message in control label -CTL_HELP, CTL_PAUSED = range(2) - -# panel order per page -PAGE_S = ["header", "control", "popup"] # sticky (ie, always available) page -PAGES = [ - ["graph", "log"], - ["conn"], - ["config"], - ["torrc"]] - -PAUSEABLE = ["header", "graph", "log", "conn"] - -CONFIG = {"log.torrc.readFailed": log.WARN, - "features.graph.type": 1, - "features.config.prepopulateEditValues": True, - "queries.refreshRate.rate": 5, - "log.torEventTypeUnrecognized": log.NOTICE, - "features.graph.bw.prepopulate": True, - "log.startTime": log.INFO, - "log.refreshRate": log.DEBUG, - "log.highCpuUsage": log.WARN, - "log.configEntryUndefined": log.NOTICE, - "log.torrc.validation.torStateDiffers": log.WARN, - "log.torrc.validation.unnecessaryTorrcEntries": log.NOTICE} - -class ControlPanel(panel.Panel): - """ Draws single line label for interface controls. """ - - def __init__(self, stdscr, isBlindMode): - panel.Panel.__init__(self, stdscr, "control", 0, 1) - self.msgText = CTL_HELP # message text to be displyed - self.msgAttr = curses.A_NORMAL # formatting attributes - self.page = 1 # page number currently being displayed - self.resolvingCounter = -1 # count of resolver when starting (-1 if we aren't working on a batch) - self.isBlindMode = isBlindMode - - def setMsg(self, msgText, msgAttr=curses.A_NORMAL): - """ - Sets the message and display attributes. If msgType matches CTL_HELP or - CTL_PAUSED then uses the default message for those statuses. - """ - - self.msgText = msgText - self.msgAttr = msgAttr - - def draw(self, width, height): - msgText = self.msgText - msgAttr = self.msgAttr - barTab = 2 # space between msgText and progress bar - barWidthMax = 40 # max width to progress bar - barWidth = -1 # space between "[ ]" in progress bar (not visible if -1) - barProgress = 0 # cells to fill - - if msgText == CTL_HELP: - msgAttr = curses.A_NORMAL - - if self.resolvingCounter != -1: - if hostnames.isPaused() or not hostnames.isResolving(): - # done resolving dns batch - self.resolvingCounter = -1 - curses.halfdelay(REFRESH_RATE * 10) # revert to normal refresh rate - else: - batchSize = hostnames.getRequestCount() - self.resolvingCounter - entryCount = batchSize - hostnames.getPendingCount() - if batchSize > 0: progress = 100 * entryCount / batchSize - else: progress = 0 - - additive = "or l " if self.page == 2 else "" - batchSizeDigits = int(math.log10(batchSize)) + 1 - entryCountLabel = ("%%%ii" % batchSizeDigits) % entryCount - #msgText = "Resolving hostnames (%i / %i, %i%%) - press esc %sto cancel" % (entryCount, batchSize, progress, additive) - msgText = "Resolving hostnames (press esc %sto cancel) - %s / %i, %2i%%" % (additive, entryCountLabel, batchSize, progress) - - barWidth = min(barWidthMax, width - len(msgText) - 3 - barTab) - barProgress = barWidth * entryCount / batchSize - - if self.resolvingCounter == -1: - currentPage = self.page - pageCount = len(PAGES) - - if self.isBlindMode: - if currentPage >= 2: currentPage -= 1 - pageCount -= 1 - - msgText = "page %i / %i - q: quit, p: pause, h: page help" % (currentPage, pageCount) - elif msgText == CTL_PAUSED: - msgText = "Paused" - msgAttr = curses.A_STANDOUT - - self.addstr(0, 0, msgText, msgAttr) - if barWidth > -1: - xLoc = len(msgText) + barTab - self.addstr(0, xLoc, "[", curses.A_BOLD) - self.addstr(0, xLoc + 1, " " * barProgress, curses.A_STANDOUT | uiTools.getColor("red")) - self.addstr(0, xLoc + barWidth + 1, "]", curses.A_BOLD) - -class Popup(panel.Panel): - """ - Temporarily providing old panel methods until permanent workaround for popup - can be derrived (this passive drawing method is horrible - I'll need to - provide a version using the more active repaint design later in the - revision). - """ - - def __init__(self, stdscr, height): - panel.Panel.__init__(self, stdscr, "popup", 0, height) - - # The following methods are to emulate old panel functionality (this was the - # only implementations to use these methods and will require a complete - # rewrite when refactoring gets here) - def clear(self): - if self.win: - self.isDisplaced = self.top > self.win.getparyx()[0] - if not self.isDisplaced: self.win.erase() - - def refresh(self): - if self.win and not self.isDisplaced: self.win.refresh() - - def recreate(self, stdscr, newWidth=-1, newTop=None): - self.setParent(stdscr) - self.setWidth(newWidth) - if newTop != None: self.setTop(newTop) - - newHeight, newWidth = self.getPreferredSize() - if newHeight > 0: - self.win = self.parent.subwin(newHeight, newWidth, self.top, 0) - elif self.win == None: - # don't want to leave the window as none (in very edge cases could cause - # problems) - rather, create a displaced instance - self.win = self.parent.subwin(1, newWidth, 0, 0) - - self.maxY, self.maxX = self.win.getmaxyx() - -def addstr_wrap(panel, y, x, text, formatting, startX = 0, endX = -1, maxY = -1): - """ - Writes text with word wrapping, returning the ending y/x coordinate. - y: starting write line - x: column offset from startX - text / formatting: content to be written - startX / endX: column bounds in which text may be written - """ - - # moved out of panel (trying not to polute new code!) - # TODO: unpleaseantly complex usage - replace with something else when - # rewriting confPanel and descriptorPopup (the only places this is used) - if not text: return (y, x) # nothing to write - if endX == -1: endX = panel.maxX # defaults to writing to end of panel - if maxY == -1: maxY = panel.maxY + 1 # defaults to writing to bottom of panel - lineWidth = endX - startX # room for text - while True: - if len(text) > lineWidth - x - 1: - chunkSize = text.rfind(" ", 0, lineWidth - x) - writeText = text[:chunkSize] - text = text[chunkSize:].strip() - - panel.addstr(y, x + startX, writeText, formatting) - y, x = y + 1, 0 - if y >= maxY: return (y, x) - else: - panel.addstr(y, x + startX, text, formatting) - return (y, x + len(text)) - -class sighupListener(TorCtl.PostEventListener): - """ - Listens for reload signal (hup), which is produced by: - pkill -sighup tor - causing the torrc and internal state to be reset. - """ - - def __init__(self): - TorCtl.PostEventListener.__init__(self) - self.isReset = False - - def msg_event(self, event): - self.isReset |= event.level == "NOTICE" and event.msg.startswith("Received reload signal (hup)") - -def setPauseState(panels, monitorIsPaused, currentPage, overwrite=False): - """ - Resets the isPaused state of panels. If overwrite is True then this pauses - reguardless of the monitor is paused or not. - """ - - for key in PAUSEABLE: panels[key].setPaused(overwrite or monitorIsPaused or (key not in PAGES[currentPage] and key not in PAGE_S)) - -def showMenu(stdscr, popup, title, options, initialSelection): - """ - Provides menu with options laid out in a single column. User can cancel - selection with the escape key, in which case this proives -1. Otherwise this - returns the index of the selection. If initialSelection is -1 then the first - option is used and the carrot indicating past selection is ommitted. - """ - - selection = initialSelection if initialSelection != -1 else 0 - - if popup.win: - if not panel.CURSES_LOCK.acquire(False): return -1 - try: - # TODO: should pause interface (to avoid event accumilation) - curses.cbreak() # wait indefinitely for key presses (no timeout) - - # uses smaller dimentions more fitting for small content - popup.height = len(options) + 2 - - newWidth = max([len(label) for label in options]) + 9 - popup.recreate(stdscr, newWidth) - - key = 0 - while not uiTools.isSelectionKey(key): - popup.clear() - popup.win.box() - popup.addstr(0, 0, title, curses.A_STANDOUT) - - for i in range(len(options)): - label = options[i] - format = curses.A_STANDOUT if i == selection else curses.A_NORMAL - tab = "> " if i == initialSelection else " " - popup.addstr(i + 1, 2, tab) - popup.addstr(i + 1, 4, " %s " % label, format) - - popup.refresh() - key = stdscr.getch() - if key == curses.KEY_UP: selection = max(0, selection - 1) - elif key == curses.KEY_DOWN: selection = min(len(options) - 1, selection + 1) - elif key == 27: selection, key = -1, curses.KEY_ENTER # esc - cancel - - # reverts popup dimensions and conn panel label - popup.height = 9 - popup.recreate(stdscr, 80) - - curses.halfdelay(REFRESH_RATE * 10) # reset normal pausing behavior - finally: - panel.CURSES_LOCK.release() - - return selection - -def showSortDialog(stdscr, panels, isPaused, page, titleLabel, options, oldSelection, optionColors): - """ - Displays a sorting dialog of the form: - - Current Order: <previous selection> - New Order: <selections made> - - <option 1> <option 2> <option 3> Cancel - - Options are colored when among the "Current Order" or "New Order", but not - when an option below them. If cancel is selected or the user presses escape - then this returns None. Otherwise, the new ordering is provided. - - Arguments: - stdscr, panels, isPaused, page - boiler plate arguments of the controller - (should be refactored away when rewriting) - - titleLabel - title displayed for the popup window - options - ordered listing of option labels - oldSelection - current ordering - optionColors - mappings of options to their color - - """ - - panel.CURSES_LOCK.acquire() - newSelections = [] # new ordering - - try: - setPauseState(panels, isPaused, page, True) - curses.cbreak() # wait indefinitely for key presses (no timeout) - - popup = panels["popup"] - cursorLoc = 0 # index of highlighted option - - # label for the inital ordering - formattedPrevListing = [] - for sortType in oldSelection: - colorStr = optionColors.get(sortType, "white") - formattedPrevListing.append("<%s>%s</%s>" % (colorStr, sortType, colorStr)) - prevOrderingLabel = "Current Order: %s" % ", ".join(formattedPrevListing) - - selectionOptions = list(options) - selectionOptions.append("Cancel") - - while len(newSelections) < len(oldSelection): - popup.clear() - popup.win.box() - popup.addstr(0, 0, titleLabel, curses.A_STANDOUT) - popup.addfstr(1, 2, prevOrderingLabel) - - # provides new ordering - formattedNewListing = [] - for sortType in newSelections: - colorStr = optionColors.get(sortType, "white") - formattedNewListing.append("<%s>%s</%s>" % (colorStr, sortType, colorStr)) - newOrderingLabel = "New Order: %s" % ", ".join(formattedNewListing) - popup.addfstr(2, 2, newOrderingLabel) - - # presents remaining options, each row having up to four options with - # spacing of nineteen cells - row, col = 4, 0 - for i in range(len(selectionOptions)): - popup.addstr(row, col * 19 + 2, selectionOptions[i], curses.A_STANDOUT if cursorLoc == i else curses.A_NORMAL) - col += 1 - if col == 4: row, col = row + 1, 0 - - popup.refresh() - - key = stdscr.getch() - if key == curses.KEY_LEFT: cursorLoc = max(0, cursorLoc - 1) - elif key == curses.KEY_RIGHT: cursorLoc = min(len(selectionOptions) - 1, cursorLoc + 1) - elif key == curses.KEY_UP: cursorLoc = max(0, cursorLoc - 4) - elif key == curses.KEY_DOWN: cursorLoc = min(len(selectionOptions) - 1, cursorLoc + 4) - elif uiTools.isSelectionKey(key): - # selected entry (the ord of '10' seems needed to pick up enter) - selection = selectionOptions[cursorLoc] - if selection == "Cancel": break - else: - newSelections.append(selection) - selectionOptions.remove(selection) - cursorLoc = min(cursorLoc, len(selectionOptions) - 1) - elif key == 27: break # esc - cancel - - setPauseState(panels, isPaused, page) - curses.halfdelay(REFRESH_RATE * 10) # reset normal pausing behavior - finally: - panel.CURSES_LOCK.release() - - if len(newSelections) == len(oldSelection): - return newSelections - else: return None - -def setEventListening(selectedEvents, isBlindMode): - # creates a local copy, note that a suspected python bug causes *very* - # puzzling results otherwise when trying to discard entries (silently - # returning out of this function!) - events = set(selectedEvents) - isLoggingUnknown = "UNKNOWN" in events - - # removes special types only used in arm (UNKNOWN, TORCTL, ARM_DEBUG, etc) - toDiscard = [] - for eventType in events: - if eventType not in logPanel.TOR_EVENT_TYPES.values(): toDiscard += [eventType] - - for eventType in list(toDiscard): events.discard(eventType) - - # adds events unrecognized by arm if we're listening to the 'UNKNOWN' type - if isLoggingUnknown: - events.update(set(logPanel.getMissingEventTypes())) - - setEvents = torTools.getConn().setControllerEvents(list(events)) - - # temporary hack for providing user selected events minus those that failed - # (wouldn't be a problem if I wasn't storing tor and non-tor events together...) - returnVal = list(selectedEvents.difference(torTools.FAILED_EVENTS)) - returnVal.sort() # alphabetizes - return returnVal - -def connResetListener(conn, eventType): - """ - Pauses connection resolution when tor's shut down, and resumes if started - again. - """ - - if connections.isResolverAlive("tor"): - resolver = connections.getResolver("tor") - resolver.setPaused(eventType == torTools.State.CLOSED) - -def selectiveRefresh(panels, page): - """ - This forces a redraw of content on the currently active page (should be done - after changing pages, popups, or anything else that overwrites panels). - """ - - for panelKey in PAGES[page]: - panels[panelKey].redraw(True) - -def drawTorMonitor(stdscr, startTime, loggedEvents, isBlindMode): - """ - Starts arm interface reflecting information on provided control port. - - stdscr - curses window - conn - active Tor control port connection - loggedEvents - types of events to be logged (plus an optional "UNKNOWN" for - otherwise unrecognized events) - """ - - # loads config for various interface components - config = conf.getConfig("arm") - config.update(CONFIG) - graphing.graphPanel.loadConfig(config) - interface.connections.connEntry.loadConfig(config) - - # adds events needed for arm functionality to the torTools REQ_EVENTS mapping - # (they're then included with any setControllerEvents call, and log a more - # helpful error if unavailable) - torTools.REQ_EVENTS["BW"] = "bandwidth graph won't function" - - if not isBlindMode: - torTools.REQ_EVENTS["CIRC"] = "may cause issues in identifying client connections" - - # pauses/unpauses connection resolution according to if tor's connected or not - torTools.getConn().addStatusListener(connResetListener) - - # TODO: incrementally drop this requirement until everything's using the singleton - conn = torTools.getConn().getTorCtl() - - curses.halfdelay(REFRESH_RATE * 10) # uses getch call as timer for REFRESH_RATE seconds - try: curses.use_default_colors() # allows things like semi-transparent backgrounds (call can fail with ERR) - except curses.error: pass - - # attempts to make the cursor invisible (not supported in all terminals) - try: curses.curs_set(0) - except curses.error: pass - - # attempts to determine tor's current pid (left as None if unresolveable, logging an error later) - torPid = torTools.getConn().getMyPid() - - #try: - # confLocation = conn.get_info("config-file")["config-file"] - # if confLocation[0] != "/": - # # relative path - attempt to add process pwd - # try: - # results = sysTools.call("pwdx %s" % torPid) - # if len(results) == 1 and len(results[0].split()) == 2: confLocation = "%s/%s" % (results[0].split()[1], confLocation) - # except IOError: pass # pwdx call failed - #except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed): - # confLocation = "" - - # loads the torrc and provides warnings in case of validation errors - loadedTorrc = torConfig.getTorrc() - loadedTorrc.getLock().acquire() - - try: - loadedTorrc.load() - except IOError, exc: - msg = "Unable to load torrc (%s)" % sysTools.getFileErrorMsg(exc) - log.log(CONFIG["log.torrc.readFailed"], msg) - - if loadedTorrc.isLoaded(): - corrections = loadedTorrc.getCorrections() - duplicateOptions, defaultOptions, mismatchLines, missingOptions = [], [], [], [] - - for lineNum, issue, msg in corrections: - if issue == torConfig.ValidationError.DUPLICATE: - duplicateOptions.append("%s (line %i)" % (msg, lineNum + 1)) - elif issue == torConfig.ValidationError.IS_DEFAULT: - defaultOptions.append("%s (line %i)" % (msg, lineNum + 1)) - elif issue == torConfig.ValidationError.MISMATCH: mismatchLines.append(lineNum + 1) - elif issue == torConfig.ValidationError.MISSING: missingOptions.append(msg) - - if duplicateOptions or defaultOptions: - msg = "Unneeded torrc entries found. They've been highlighted in blue on the torrc page." - - if duplicateOptions: - if len(duplicateOptions) > 1: - msg += "\n- entries ignored due to having duplicates: " - else: - msg += "\n- entry ignored due to having a duplicate: " - - duplicateOptions.sort() - msg += ", ".join(duplicateOptions) - - if defaultOptions: - if len(defaultOptions) > 1: - msg += "\n- entries match their default values: " - else: - msg += "\n- entry matches its default value: " - - defaultOptions.sort() - msg += ", ".join(defaultOptions) - - log.log(CONFIG["log.torrc.validation.unnecessaryTorrcEntries"], msg) - - if mismatchLines or missingOptions: - msg = "The torrc differ from what tor's using. You can issue a sighup to reload the torrc values by pressing x." - - if mismatchLines: - if len(mismatchLines) > 1: - msg += "\n- torrc values differ on lines: " - else: - msg += "\n- torrc value differs on line: " - - mismatchLines.sort() - msg += ", ".join([str(val + 1) for val in mismatchLines]) - - if missingOptions: - if len(missingOptions) > 1: - msg += "\n- configuration values are missing from the torrc: " - else: - msg += "\n- configuration value is missing from the torrc: " - - missingOptions.sort() - msg += ", ".join(missingOptions) - - log.log(CONFIG["log.torrc.validation.torStateDiffers"], msg) - - loadedTorrc.getLock().release() - - # minor refinements for connection resolver - if not isBlindMode: - if torPid: - # use the tor pid to help narrow connection results - torCmdName = sysTools.getProcessName(torPid, "tor") - resolver = connections.getResolver(torCmdName, torPid, "tor") - else: - resolver = connections.getResolver("tor") - - # hack to display a better (arm specific) notice if all resolvers fail - connections.RESOLVER_FINAL_FAILURE_MSG += " (connection related portions of the monitor won't function)" - - panels = { - "header": headerPanel.HeaderPanel(stdscr, startTime, config), - "popup": Popup(stdscr, 9), - "graph": graphing.graphPanel.GraphPanel(stdscr), - "log": logPanel.LogPanel(stdscr, loggedEvents, config)} - - # TODO: later it would be good to set the right 'top' values during initialization, - # but for now this is just necessary for the log panel (and a hack in the log...) - - # TODO: bug from not setting top is that the log panel might attempt to draw - # before being positioned - the following is a quick hack til rewritten - panels["log"].setPaused(True) - - panels["conn"] = interface.connections.connPanel.ConnectionPanel(stdscr, config) - - panels["control"] = ControlPanel(stdscr, isBlindMode) - panels["config"] = configPanel.ConfigPanel(stdscr, configPanel.State.TOR, config) - panels["torrc"] = torrcPanel.TorrcPanel(stdscr, torrcPanel.Config.TORRC, config) - - # provides error if pid coulnd't be determined (hopefully shouldn't happen...) - if not torPid: log.log(log.WARN, "Unable to resolve tor pid, abandoning connection listing") - - # statistical monitors for graph - panels["graph"].addStats("bandwidth", graphing.bandwidthStats.BandwidthStats(config)) - panels["graph"].addStats("system resources", graphing.resourceStats.ResourceStats()) - if not isBlindMode: panels["graph"].addStats("connections", graphing.connStats.ConnStats()) - - # sets graph based on config parameter - graphType = CONFIG["features.graph.type"] - if graphType == 0: panels["graph"].setStats(None) - elif graphType == 1: panels["graph"].setStats("bandwidth") - elif graphType == 2 and not isBlindMode: panels["graph"].setStats("connections") - elif graphType == 3: panels["graph"].setStats("system resources") - - # listeners that update bandwidth and log panels with Tor status - sighupTracker = sighupListener() - #conn.add_event_listener(panels["log"]) - conn.add_event_listener(panels["graph"].stats["bandwidth"]) - conn.add_event_listener(panels["graph"].stats["system resources"]) - if not isBlindMode: conn.add_event_listener(panels["graph"].stats["connections"]) - conn.add_event_listener(sighupTracker) - - # prepopulates bandwidth values from state file - if CONFIG["features.graph.bw.prepopulate"]: - isSuccessful = panels["graph"].stats["bandwidth"].prepopulateFromState() - if isSuccessful: panels["graph"].updateInterval = 4 - - # tells Tor to listen to the events we're interested - loggedEvents = setEventListening(loggedEvents, isBlindMode) - #panels["log"].loggedEvents = loggedEvents # strips any that couldn't be set - panels["log"].setLoggedEvents(loggedEvents) # strips any that couldn't be set - - # directs logged TorCtl events to log panel - #TorUtil.loglevel = "DEBUG" - #TorUtil.logfile = panels["log"] - #torTools.getConn().addTorCtlListener(panels["log"].tor_ctl_event) - - # provides a notice about any event types tor supports but arm doesn't - missingEventTypes = logPanel.getMissingEventTypes() - if missingEventTypes: - pluralLabel = "s" if len(missingEventTypes) > 1 else "" - log.log(CONFIG["log.torEventTypeUnrecognized"], "arm doesn't recognize the following event type%s: %s (log 'UNKNOWN' events to see them)" % (pluralLabel, ", ".join(missingEventTypes))) - - # tells revised panels to run as daemons - panels["header"].start() - panels["log"].start() - panels["conn"].start() - - # warns if tor isn't updating descriptors - #try: - # if conn.get_option("FetchUselessDescriptors")[0][1] == "0" and conn.get_option("DirPort")[0][1] == "0": - # warning = """Descriptors won't be updated (causing some connection information to be stale) unless: - #a. 'FetchUselessDescriptors 1' is set in your torrc - #b. the directory service is provided ('DirPort' defined) - #c. or tor is used as a client""" - # log.log(log.WARN, warning) - #except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed): pass - - isUnresponsive = False # true if it's been over ten seconds since the last BW event (probably due to Tor closing) - isPaused = False # if true updates are frozen - overrideKey = None # immediately runs with this input rather than waiting for the user if set - page = 0 - regexFilters = [] # previously used log regex filters - panels["popup"].redraw(True) # hack to make sure popup has a window instance (not entirely sure why...) - - # provides notice about any unused config keys - for key in config.getUnusedKeys(): - log.log(CONFIG["log.configEntryUndefined"], "Unused configuration entry: %s" % key) - - lastPerformanceLog = 0 # ensures we don't do performance logging too frequently - redrawStartTime = time.time() - - # TODO: popups need to force the panels it covers to redraw (or better, have - # a global refresh function for after changing pages, popups, etc) - - initTime = time.time() - startTime - log.log(CONFIG["log.startTime"], "arm started (initialization took %0.3f seconds)" % initTime) - - # attributes to give a WARN level event if arm's resource usage is too high - isResourceWarningGiven = False - lastResourceCheck = startTime - - lastSize = None - - # sets initial visiblity for the pages - for i in range(len(PAGES)): - isVisible = i == page - for entry in PAGES[i]: panels[entry].setVisible(isVisible) - - # TODO: come up with a nice, clean method for other threads to immediately - # terminate the draw loop and provide a stacktrace - while True: - # tried only refreshing when the screen was resized but it caused a - # noticeable lag when resizing and didn't have an appreciable effect - # on system usage - - panel.CURSES_LOCK.acquire() - try: - redrawStartTime = time.time() - - # if sighup received then reload related information - if sighupTracker.isReset: - #panels["header"]._updateParams(True) - - # other panels that use torrc data - #if not isBlindMode: panels["graph"].stats["connections"].resetOptions(conn) - #panels["graph"].stats["bandwidth"].resetOptions() - - # if bandwidth graph is being shown then height might have changed - if panels["graph"].currentDisplay == "bandwidth": - panels["graph"].setHeight(panels["graph"].stats["bandwidth"].getContentHeight()) - - # TODO: should redraw the torrcPanel - #panels["torrc"].loadConfig() - - # reload the torrc if it's previously been loaded - if loadedTorrc.isLoaded(): - try: - loadedTorrc.load() - if page == 3: panels["torrc"].redraw(True) - except IOError, exc: - msg = "Unable to load torrc (%s)" % sysTools.getFileErrorMsg(exc) - log.log(CONFIG["log.torrc.readFailed"], msg) - - sighupTracker.isReset = False - - # gives panels a chance to take advantage of the maximum bounds - # originally this checked in the bounds changed but 'recreate' is a no-op - # if panel properties are unchanged and checking every redraw is more - # resilient in case of funky changes (such as resizing during popups) - - # hack to make sure header picks layout before using the dimensions below - #panels["header"].getPreferredSize() - - startY = 0 - for panelKey in PAGE_S[:2]: - #panels[panelKey].recreate(stdscr, -1, startY) - panels[panelKey].setParent(stdscr) - panels[panelKey].setWidth(-1) - panels[panelKey].setTop(startY) - startY += panels[panelKey].getHeight() - - panels["popup"].recreate(stdscr, 80, startY) - - for panelSet in PAGES: - tmpStartY = startY - - for panelKey in panelSet: - #panels[panelKey].recreate(stdscr, -1, tmpStartY) - panels[panelKey].setParent(stdscr) - panels[panelKey].setWidth(-1) - panels[panelKey].setTop(tmpStartY) - tmpStartY += panels[panelKey].getHeight() - - # provides a notice if there's been ten seconds since the last BW event - lastHeartbeat = torTools.getConn().getHeartbeat() - if torTools.getConn().isAlive() and "BW" in torTools.getConn().getControllerEvents() and lastHeartbeat != 0: - if not isUnresponsive and (time.time() - lastHeartbeat) >= 10: - isUnresponsive = True - log.log(log.NOTICE, "Relay unresponsive (last heartbeat: %s)" % time.ctime(lastHeartbeat)) - elif isUnresponsive and (time.time() - lastHeartbeat) < 10: - # really shouldn't happen (meant Tor froze for a bit) - isUnresponsive = False - log.log(log.NOTICE, "Relay resumed") - - # TODO: part two of hack to prevent premature drawing by log panel - if page == 0 and not isPaused: panels["log"].setPaused(False) - - # I haven't the foggiest why, but doesn't work if redrawn out of order... - for panelKey in (PAGE_S + PAGES[page]): - # redrawing popup can result in display flicker when it should be hidden - if panelKey != "popup": - newSize = stdscr.getmaxyx() - isResize = lastSize != newSize - lastSize = newSize - - if panelKey in ("header", "graph", "log", "config", "torrc", "conn2"): - # revised panel (manages its own content refreshing) - panels[panelKey].redraw(isResize) - else: - panels[panelKey].redraw(True) - - stdscr.refresh() - - currentTime = time.time() - if currentTime - lastPerformanceLog >= CONFIG["queries.refreshRate.rate"]: - cpuTotal = sum(os.times()[:3]) - pythonCpuAvg = cpuTotal / (currentTime - startTime) - sysCallCpuAvg = sysTools.getSysCpuUsage() - totalCpuAvg = pythonCpuAvg + sysCallCpuAvg - - if sysCallCpuAvg > 0.00001: - log.log(CONFIG["log.refreshRate"], "refresh rate: %0.3f seconds, average cpu usage: %0.3f%% (python), %0.3f%% (system calls), %0.3f%% (total)" % (currentTime - redrawStartTime, 100 * pythonCpuAvg, 100 * sysCallCpuAvg, 100 * totalCpuAvg)) - else: - # with the proc enhancements the sysCallCpuAvg is usually zero - log.log(CONFIG["log.refreshRate"], "refresh rate: %0.3f seconds, average cpu usage: %0.3f%%" % (currentTime - redrawStartTime, 100 * totalCpuAvg)) - - lastPerformanceLog = currentTime - - # once per minute check if the sustained cpu usage is above 5%, if so - # then give a warning (and if able, some advice for lowering it) - # TODO: disabling this for now (scrolling causes cpu spikes for quick - # redraws, ie this is usually triggered by user input) - if False and not isResourceWarningGiven and currentTime > (lastResourceCheck + 60): - if totalCpuAvg >= 0.05: - msg = "Arm's cpu usage is high (averaging %0.3f%%)." % (100 * totalCpuAvg) - - if not isBlindMode: - msg += " You could lower it by dropping the connection data (running as \"arm -b\")." - - log.log(CONFIG["log.highCpuUsage"], msg) - isResourceWarningGiven = True - - lastResourceCheck = currentTime - finally: - panel.CURSES_LOCK.release() - - # wait for user keyboard input until timeout (unless an override was set) - if overrideKey: - key = overrideKey - overrideKey = None - else: - key = stdscr.getch() - - if key == ord('q') or key == ord('Q'): - quitConfirmed = not CONFIRM_QUIT - - # provides prompt to confirm that arm should exit - if CONFIRM_QUIT: - panel.CURSES_LOCK.acquire() - try: - setPauseState(panels, isPaused, page, True) - - # provides prompt - panels["control"].setMsg("Are you sure (q again to confirm)?", curses.A_BOLD) - panels["control"].redraw(True) - - curses.cbreak() - confirmationKey = stdscr.getch() - quitConfirmed = confirmationKey in (ord('q'), ord('Q')) - curses.halfdelay(REFRESH_RATE * 10) - - panels["control"].setMsg(CTL_PAUSED if isPaused else CTL_HELP) - setPauseState(panels, isPaused, page) - finally: - panel.CURSES_LOCK.release() - - if quitConfirmed: - # quits arm - # very occasionally stderr gets "close failed: [Errno 11] Resource temporarily unavailable" - # this appears to be a python bug: http://bugs.python.org/issue3014 - # (haven't seen this is quite some time... mysteriously resolved?) - - torTools.NO_SPAWN = True # prevents further worker threads from being spawned - - # stops panel daemons - panels["header"].stop() - panels["conn"].stop() - panels["log"].stop() - - panels["header"].join() - panels["conn"].join() - panels["log"].join() - - # joins on utility daemon threads - this might take a moment since - # the internal threadpools being joined might be sleeping - conn = torTools.getConn() - myPid = conn.getMyPid() - - resourceTracker = sysTools.getResourceTracker(myPid) if (myPid and sysTools.isTrackerAlive(myPid)) else None - resolver = connections.getResolver("tor") if connections.isResolverAlive("tor") else None - if resourceTracker: resourceTracker.stop() - if resolver: resolver.stop() # sets halt flag (returning immediately) - hostnames.stop() # halts and joins on hostname worker thread pool - if resourceTracker: resourceTracker.join() - if resolver: resolver.join() # joins on halted resolver - - conn.close() # joins on TorCtl event thread - break - elif key == curses.KEY_LEFT or key == curses.KEY_RIGHT: - # switch page - if key == curses.KEY_LEFT: page = (page - 1) % len(PAGES) - else: page = (page + 1) % len(PAGES) - - # skip connections listing if it's disabled - if page == 1 and isBlindMode: - if key == curses.KEY_LEFT: page = (page - 1) % len(PAGES) - else: page = (page + 1) % len(PAGES) - - # pauses panels that aren't visible to prevent events from accumilating - # (otherwise they'll wait on the curses lock which might get demanding) - setPauseState(panels, isPaused, page) - - # prevents panels on other pages from redrawing - for i in range(len(PAGES)): - isVisible = i == page - for entry in PAGES[i]: panels[entry].setVisible(isVisible) - - panels["control"].page = page + 1 - - # TODO: this redraw doesn't seem necessary (redraws anyway after this - # loop) - look into this when refactoring - panels["control"].redraw(True) - - selectiveRefresh(panels, page) - elif key == ord('p') or key == ord('P'): - # toggles update freezing - panel.CURSES_LOCK.acquire() - try: - isPaused = not isPaused - setPauseState(panels, isPaused, page) - panels["control"].setMsg(CTL_PAUSED if isPaused else CTL_HELP) - finally: - panel.CURSES_LOCK.release() - - selectiveRefresh(panels, page) - elif key == ord('x') or key == ord('X'): - # provides prompt to confirm that arm should issue a sighup - panel.CURSES_LOCK.acquire() - try: - setPauseState(panels, isPaused, page, True) - - # provides prompt - panels["control"].setMsg("This will reset Tor's internal state. Are you sure (x again to confirm)?", curses.A_BOLD) - panels["control"].redraw(True) - - curses.cbreak() - confirmationKey = stdscr.getch() - if confirmationKey in (ord('x'), ord('X')): - try: - torTools.getConn().reload() - except IOError, exc: - log.log(log.ERR, "Error detected when reloading tor: %s" % sysTools.getFileErrorMsg(exc)) - - #errorMsg = " (%s)" % str(err) if str(err) else "" - #panels["control"].setMsg("Sighup failed%s" % errorMsg, curses.A_STANDOUT) - #panels["control"].redraw(True) - #time.sleep(2) - - # reverts display settings - curses.halfdelay(REFRESH_RATE * 10) - panels["control"].setMsg(CTL_PAUSED if isPaused else CTL_HELP) - setPauseState(panels, isPaused, page) - finally: - panel.CURSES_LOCK.release() - elif key == ord('h') or key == ord('H'): - # displays popup for current page's controls - panel.CURSES_LOCK.acquire() - try: - setPauseState(panels, isPaused, page, True) - - # lists commands - popup = panels["popup"] - popup.clear() - popup.win.box() - popup.addstr(0, 0, "Page %i Commands:" % (page + 1), curses.A_STANDOUT) - - pageOverrideKeys = () - - if page == 0: - graphedStats = panels["graph"].currentDisplay - if not graphedStats: graphedStats = "none" - popup.addfstr(1, 2, "up arrow: scroll log up a line") - popup.addfstr(1, 41, "down arrow: scroll log down a line") - popup.addfstr(2, 2, "m: increase graph size") - popup.addfstr(2, 41, "n: decrease graph size") - popup.addfstr(3, 2, "s: graphed stats (%s)" % graphedStats) - popup.addfstr(3, 41, "i: graph update interval (%s)" % graphing.graphPanel.UPDATE_INTERVALS[panels["graph"].updateInterval][0]) - popup.addfstr(4, 2, "b: graph bounds (%s)" % panels["graph"].bounds.lower()) - popup.addfstr(4, 41, "a: save snapshot of the log") - popup.addfstr(5, 2, "e: change logged events") - - regexLabel = "enabled" if panels["log"].regexFilter else "disabled" - popup.addfstr(5, 41, "f: log regex filter (%s)" % regexLabel) - - hiddenEntryLabel = "visible" if panels["log"].showDuplicates else "hidden" - popup.addfstr(6, 2, "u: duplicate log entries (%s)" % hiddenEntryLabel) - popup.addfstr(6, 41, "c: clear event log") - - pageOverrideKeys = (ord('m'), ord('n'), ord('s'), ord('i'), ord('d'), ord('e'), ord('r'), ord('f'), ord('x')) - if page == 1: - popup.addfstr(1, 2, "up arrow: scroll up a line") - popup.addfstr(1, 41, "down arrow: scroll down a line") - popup.addfstr(2, 2, "page up: scroll up a page") - popup.addfstr(2, 41, "page down: scroll down a page") - - popup.addfstr(3, 2, "enter: edit configuration option") - popup.addfstr(3, 41, "d: raw consensus descriptor") - - listingType = panels["conn"]._listingType.lower() - popup.addfstr(4, 2, "l: listed identity (%s)" % listingType) - - popup.addfstr(4, 41, "s: sort ordering") - - resolverUtil = connections.getResolver("tor").overwriteResolver - if resolverUtil == None: resolverUtil = "auto" - popup.addfstr(5, 2, "u: resolving utility (%s)" % resolverUtil) - - pageOverrideKeys = (ord('d'), ord('l'), ord('s'), ord('u')) - elif page == 2: - popup.addfstr(1, 2, "up arrow: scroll up a line") - popup.addfstr(1, 41, "down arrow: scroll down a line") - popup.addfstr(2, 2, "page up: scroll up a page") - popup.addfstr(2, 41, "page down: scroll down a page") - - strippingLabel = "on" if panels["torrc"].stripComments else "off" - popup.addfstr(3, 2, "s: comment stripping (%s)" % strippingLabel) - - lineNumLabel = "on" if panels["torrc"].showLineNum else "off" - popup.addfstr(3, 41, "n: line numbering (%s)" % lineNumLabel) - - popup.addfstr(4, 2, "r: reload torrc") - popup.addfstr(4, 41, "x: reset tor (issue sighup)") - elif page == 3: - popup.addfstr(1, 2, "up arrow: scroll up a line") - popup.addfstr(1, 41, "down arrow: scroll down a line") - popup.addfstr(2, 2, "page up: scroll up a page") - popup.addfstr(2, 41, "page down: scroll down a page") - popup.addfstr(3, 2, "enter: connection details") - - popup.addstr(7, 2, "Press any key...") - popup.refresh() - - # waits for user to hit a key, if it belongs to a command then executes it - curses.cbreak() - helpExitKey = stdscr.getch() - if helpExitKey in pageOverrideKeys: overrideKey = helpExitKey - curses.halfdelay(REFRESH_RATE * 10) - - setPauseState(panels, isPaused, page) - selectiveRefresh(panels, page) - finally: - panel.CURSES_LOCK.release() - elif page == 0 and (key == ord('s') or key == ord('S')): - # provides menu to pick stats to be graphed - #options = ["None"] + [label for label in panels["graph"].stats.keys()] - options = ["None"] - - # appends stats labels with first letters of each word capitalized - initialSelection, i = -1, 1 - if not panels["graph"].currentDisplay: initialSelection = 0 - graphLabels = panels["graph"].stats.keys() - graphLabels.sort() - for label in graphLabels: - if label == panels["graph"].currentDisplay: initialSelection = i - words = label.split() - options.append(" ".join(word[0].upper() + word[1:] for word in words)) - i += 1 - - # hides top label of the graph panel and pauses panels - if panels["graph"].currentDisplay: - panels["graph"].showLabel = False - panels["graph"].redraw(True) - setPauseState(panels, isPaused, page, True) - - selection = showMenu(stdscr, panels["popup"], "Graphed Stats:", options, initialSelection) - - # reverts changes made for popup - panels["graph"].showLabel = True - setPauseState(panels, isPaused, page) - - # applies new setting - if selection != -1 and selection != initialSelection: - if selection == 0: panels["graph"].setStats(None) - else: panels["graph"].setStats(options[selection].lower()) - - selectiveRefresh(panels, page) - - # TODO: this shouldn't be necessary with the above refresh, but doesn't seem responsive otherwise... - panels["graph"].redraw(True) - elif page == 0 and (key == ord('i') or key == ord('I')): - # provides menu to pick graph panel update interval - options = [label for (label, intervalTime) in graphing.graphPanel.UPDATE_INTERVALS] - - initialSelection = panels["graph"].updateInterval - - #initialSelection = -1 - #for i in range(len(options)): - # if options[i] == panels["graph"].updateInterval: initialSelection = i - - # hides top label of the graph panel and pauses panels - if panels["graph"].currentDisplay: - panels["graph"].showLabel = False - panels["graph"].redraw(True) - setPauseState(panels, isPaused, page, True) - - selection = showMenu(stdscr, panels["popup"], "Update Interval:", options, initialSelection) - - # reverts changes made for popup - panels["graph"].showLabel = True - setPauseState(panels, isPaused, page) - - # applies new setting - if selection != -1: panels["graph"].updateInterval = selection - - selectiveRefresh(panels, page) - elif page == 0 and (key == ord('b') or key == ord('B')): - # uses the next boundary type for graph - panels["graph"].bounds = graphing.graphPanel.Bounds.next(panels["graph"].bounds) - - selectiveRefresh(panels, page) - elif page == 0 and (key == ord('a') or key == ord('A')): - # allow user to enter a path to take a snapshot - abandons if left blank - panel.CURSES_LOCK.acquire() - try: - setPauseState(panels, isPaused, page, True) - - # provides prompt - panels["control"].setMsg("Path to save log snapshot: ") - panels["control"].redraw(True) - - # gets user input (this blocks monitor updates) - pathInput = panels["control"].getstr(0, 27) - - if pathInput: - try: - panels["log"].saveSnapshot(pathInput) - panels["control"].setMsg("Saved: %s" % pathInput, curses.A_STANDOUT) - panels["control"].redraw(True) - time.sleep(2) - except IOError, exc: - panels["control"].setMsg("Unable to save snapshot: %s" % sysTools.getFileErrorMsg(exc), curses.A_STANDOUT) - panels["control"].redraw(True) - time.sleep(2) - - panels["control"].setMsg(CTL_PAUSED if isPaused else CTL_HELP) - setPauseState(panels, isPaused, page) - finally: - panel.CURSES_LOCK.release() - - panels["graph"].redraw(True) - elif page == 0 and (key == ord('e') or key == ord('E')): - # allow user to enter new types of events to log - unchanged if left blank - panel.CURSES_LOCK.acquire() - try: - setPauseState(panels, isPaused, page, True) - - # provides prompt - panels["control"].setMsg("Events to log: ") - panels["control"].redraw(True) - - # lists event types - popup = panels["popup"] - popup.height = 11 - popup.recreate(stdscr, 80) - - popup.clear() - popup.win.box() - popup.addstr(0, 0, "Event Types:", curses.A_STANDOUT) - lineNum = 1 - for line in logPanel.EVENT_LISTING.split("\n"): - line = line[6:] - popup.addstr(lineNum, 1, line) - lineNum += 1 - popup.refresh() - - # gets user input (this blocks monitor updates) - eventsInput = panels["control"].getstr(0, 15) - if eventsInput: eventsInput = eventsInput.replace(' ', '') # strips spaces - - # it would be nice to quit on esc, but looks like this might not be possible... - if eventsInput: - try: - expandedEvents = logPanel.expandEvents(eventsInput) - loggedEvents = setEventListening(expandedEvents, isBlindMode) - panels["log"].setLoggedEvents(loggedEvents) - except ValueError, exc: - panels["control"].setMsg("Invalid flags: %s" % str(exc), curses.A_STANDOUT) - panels["control"].redraw(True) - time.sleep(2) - - # reverts popup dimensions - popup.height = 9 - popup.recreate(stdscr, 80) - - panels["control"].setMsg(CTL_PAUSED if isPaused else CTL_HELP) - setPauseState(panels, isPaused, page) - finally: - panel.CURSES_LOCK.release() - - panels["graph"].redraw(True) - elif page == 0 and (key == ord('f') or key == ord('F')): - # provides menu to pick previous regular expression filters or to add a new one - # for syntax see: http://docs.python.org/library/re.html#regular-expression-syntax - options = ["None"] + regexFilters + ["New..."] - initialSelection = 0 if not panels["log"].regexFilter else 1 - - # hides top label of the graph panel and pauses panels - if panels["graph"].currentDisplay: - panels["graph"].showLabel = False - panels["graph"].redraw(True) - setPauseState(panels, isPaused, page, True) - - selection = showMenu(stdscr, panels["popup"], "Log Filter:", options, initialSelection) - - # applies new setting - if selection == 0: - panels["log"].setFilter(None) - elif selection == len(options) - 1: - # selected 'New...' option - prompt user to input regular expression - panel.CURSES_LOCK.acquire() - try: - # provides prompt - panels["control"].setMsg("Regular expression: ") - panels["control"].redraw(True) - - # gets user input (this blocks monitor updates) - regexInput = panels["control"].getstr(0, 20) - - if regexInput: - try: - panels["log"].setFilter(re.compile(regexInput)) - if regexInput in regexFilters: regexFilters.remove(regexInput) - regexFilters = [regexInput] + regexFilters - except re.error, exc: - panels["control"].setMsg("Unable to compile expression: %s" % str(exc), curses.A_STANDOUT) - panels["control"].redraw(True) - time.sleep(2) - panels["control"].setMsg(CTL_PAUSED if isPaused else CTL_HELP) - finally: - panel.CURSES_LOCK.release() - elif selection != -1: - try: - panels["log"].setFilter(re.compile(regexFilters[selection - 1])) - - # move selection to top - regexFilters = [regexFilters[selection - 1]] + regexFilters - del regexFilters[selection] - except re.error, exc: - # shouldn't happen since we've already checked validity - log.log(log.WARN, "Invalid regular expression ('%s': %s) - removing from listing" % (regexFilters[selection - 1], str(exc))) - del regexFilters[selection - 1] - - if len(regexFilters) > MAX_REGEX_FILTERS: del regexFilters[MAX_REGEX_FILTERS:] - - # reverts changes made for popup - panels["graph"].showLabel = True - setPauseState(panels, isPaused, page) - panels["graph"].redraw(True) - elif page == 0 and key in (ord('n'), ord('N'), ord('m'), ord('M')): - # Unfortunately modifier keys don't work with the up/down arrows (sending - # multiple keycodes. The only exception to this is shift + left/right, - # but for now just gonna use standard characters. - - if key in (ord('n'), ord('N')): - panels["graph"].setGraphHeight(panels["graph"].graphHeight - 1) - else: - # don't grow the graph if it's already consuming the whole display - # (plus an extra line for the graph/log gap) - maxHeight = panels["graph"].parent.getmaxyx()[0] - panels["graph"].top - currentHeight = panels["graph"].getHeight() - - if currentHeight < maxHeight + 1: - panels["graph"].setGraphHeight(panels["graph"].graphHeight + 1) - elif page == 0 and (key == ord('c') or key == ord('C')): - # provides prompt to confirm that arm should clear the log - panel.CURSES_LOCK.acquire() - try: - setPauseState(panels, isPaused, page, True) - - # provides prompt - panels["control"].setMsg("This will clear the log. Are you sure (c again to confirm)?", curses.A_BOLD) - panels["control"].redraw(True) - - curses.cbreak() - confirmationKey = stdscr.getch() - if confirmationKey in (ord('c'), ord('C')): panels["log"].clear() - - # reverts display settings - curses.halfdelay(REFRESH_RATE * 10) - panels["control"].setMsg(CTL_PAUSED if isPaused else CTL_HELP) - setPauseState(panels, isPaused, page) - finally: - panel.CURSES_LOCK.release() - elif page == 1 and (key == ord('u') or key == ord('U')): - # provides menu to pick identification resolving utility - options = ["auto"] + connections.Resolver.values() - - currentOverwrite = connections.getResolver("tor").overwriteResolver # enums correspond to indices - if currentOverwrite == None: initialSelection = 0 - else: initialSelection = options.index(currentOverwrite) - - # hides top label of conn panel and pauses panels - panelTitle = panels["conn"]._title - panels["conn"]._title = "" - panels["conn"].redraw(True) - setPauseState(panels, isPaused, page, True) - - selection = showMenu(stdscr, panels["popup"], "Resolver Util:", options, initialSelection) - selectedOption = options[selection] if selection != "auto" else None - - # reverts changes made for popup - panels["conn"]._title = panelTitle - setPauseState(panels, isPaused, page) - - # applies new setting - if selection != -1 and selectedOption != connections.getResolver("tor").overwriteResolver: - connections.getResolver("tor").overwriteResolver = selectedOption - elif page == 1 and key in (ord('d'), ord('D')): - # presents popup for raw consensus data - panel.CURSES_LOCK.acquire() - try: - setPauseState(panels, isPaused, page, True) - curses.cbreak() # wait indefinitely for key presses (no timeout) - panelTitle = panels["conn"]._title - panels["conn"]._title = "" - panels["conn"].redraw(True) - - descriptorPopup.showDescriptorPopup(panels["popup"], stdscr, panels["conn"]) - - panels["conn"]._title = panelTitle - setPauseState(panels, isPaused, page) - curses.halfdelay(REFRESH_RATE * 10) # reset normal pausing behavior - finally: - panel.CURSES_LOCK.release() - elif page == 1 and (key == ord('l') or key == ord('L')): - # provides a menu to pick the primary information we list connections by - options = interface.connections.entries.ListingType.values() - - # dropping the HOSTNAME listing type until we support displaying that content - options.remove(interface.connections.entries.ListingType.HOSTNAME) - - initialSelection = options.index(panels["conn"]._listingType) - - # hides top label of connection panel and pauses the display - panelTitle = panels["conn"]._title - panels["conn"]._title = "" - panels["conn"].redraw(True) - setPauseState(panels, isPaused, page, True) - - selection = showMenu(stdscr, panels["popup"], "List By:", options, initialSelection) - - # reverts changes made for popup - panels["conn"]._title = panelTitle - setPauseState(panels, isPaused, page) - - # applies new setting - if selection != -1 and options[selection] != panels["conn"]._listingType: - panels["conn"].setListingType(options[selection]) - panels["conn"].redraw(True) - elif page == 1 and (key == ord('s') or key == ord('S')): - # set ordering for connection options - titleLabel = "Connection Ordering:" - options = interface.connections.entries.SortAttr.values() - oldSelection = panels["conn"]._sortOrdering - optionColors = dict([(attr, interface.connections.entries.SORT_COLORS[attr]) for attr in options]) - results = showSortDialog(stdscr, panels, isPaused, page, titleLabel, options, oldSelection, optionColors) - - if results: - panels["conn"].setSortOrder(results) - - panels["conn"].redraw(True) - elif page == 2 and (key == ord('c') or key == ord('C')) and False: - # TODO: disabled for now (probably gonna be going with separate pages - # rather than popup menu) - # provides menu to pick config being displayed - #options = [confPanel.CONFIG_LABELS[confType] for confType in range(4)] - options = [] - initialSelection = panels["torrc"].configType - - # hides top label of the graph panel and pauses panels - panels["torrc"].showLabel = False - panels["torrc"].redraw(True) - setPauseState(panels, isPaused, page, True) - - selection = showMenu(stdscr, panels["popup"], "Configuration:", options, initialSelection) - - # reverts changes made for popup - panels["torrc"].showLabel = True - setPauseState(panels, isPaused, page) - - # applies new setting - if selection != -1: panels["torrc"].setConfigType(selection) - - selectiveRefresh(panels, page) - elif page == 2 and (key == ord('w') or key == ord('W')): - # display a popup for saving the current configuration - panel.CURSES_LOCK.acquire() - try: - configLines = torConfig.getCustomOptions(True) - - # lists event types - popup = panels["popup"] - popup.height = len(configLines) + 3 - popup.recreate(stdscr) - displayHeight, displayWidth = panels["popup"].getPreferredSize() - - # displayed options (truncating the labels if there's limited room) - if displayWidth >= 30: selectionOptions = ("Save", "Save As...", "Cancel") - else: selectionOptions = ("Save", "Save As", "X") - - # checks if we can show options beside the last line of visible content - lastIndex = min(displayHeight - 3, len(configLines) - 1) - isOptionLineSeparate = displayWidth < (30 + len(configLines[lastIndex])) - - # if we're showing all the content and have room to display selection - # options besides the text then shrink the popup by a row - if not isOptionLineSeparate and displayHeight == len(configLines) + 3: - popup.height -= 1 - popup.recreate(stdscr) - - key, selection = 0, 2 - while not uiTools.isSelectionKey(key): - # if the popup has been resized then recreate it (needed for the - # proper border height) - newHeight, newWidth = panels["popup"].getPreferredSize() - if (displayHeight, displayWidth) != (newHeight, newWidth): - displayHeight, displayWidth = newHeight, newWidth - popup.recreate(stdscr) - - # if there isn't room to display the popup then cancel it - if displayHeight <= 2: - selection = 2 - break - - popup.clear() - popup.win.box() - popup.addstr(0, 0, "Configuration being saved:", curses.A_STANDOUT) - - visibleConfigLines = displayHeight - 3 if isOptionLineSeparate else displayHeight - 2 - for i in range(visibleConfigLines): - line = uiTools.cropStr(configLines[i], displayWidth - 2) - - if " " in line: - option, arg = line.split(" ", 1) - popup.addstr(i + 1, 1, option, curses.A_BOLD | uiTools.getColor("green")) - popup.addstr(i + 1, len(option) + 2, arg, curses.A_BOLD | uiTools.getColor("cyan")) - else: - popup.addstr(i + 1, 1, line, curses.A_BOLD | uiTools.getColor("green")) - - # draws 'T' between the lower left and the covered panel's scroll bar - if displayWidth > 1: popup.win.addch(displayHeight - 1, 1, curses.ACS_TTEE) - - # draws selection options (drawn right to left) - drawX = displayWidth - 1 - for i in range(len(selectionOptions) - 1, -1, -1): - optionLabel = selectionOptions[i] - drawX -= (len(optionLabel) + 2) - - # if we've run out of room then drop the option (this will only - # occure on tiny displays) - if drawX < 1: break - - selectionFormat = curses.A_STANDOUT if i == selection else curses.A_NORMAL - popup.addstr(displayHeight - 2, drawX, "[") - popup.addstr(displayHeight - 2, drawX + 1, optionLabel, selectionFormat | curses.A_BOLD) - popup.addstr(displayHeight - 2, drawX + len(optionLabel) + 1, "]") - - drawX -= 1 # space gap between the options - - popup.refresh() - - key = stdscr.getch() - if key == curses.KEY_LEFT: selection = max(0, selection - 1) - elif key == curses.KEY_RIGHT: selection = min(len(selectionOptions) - 1, selection + 1) - - if selection in (0, 1): - loadedTorrc = torConfig.getTorrc() - try: configLocation = loadedTorrc.getConfigLocation() - except IOError: configLocation = "" - - if selection == 1: - # prompts user for a configuration location - promptMsg = "Save to (esc to cancel): " - panels["control"].setMsg(promptMsg) - panels["control"].redraw(True) - configLocation = panels["control"].getstr(0, len(promptMsg), configLocation) - if configLocation: configLocation = os.path.abspath(configLocation) - - if configLocation: - try: - # make dir if the path doesn't already exist - baseDir = os.path.dirname(configLocation) - if not os.path.exists(baseDir): os.makedirs(baseDir) - - # saves the configuration to the file - configFile = open(configLocation, "w") - configFile.write("\n".join(configLines)) - configFile.close() - - # reloads the cached torrc if overwriting it - if configLocation == loadedTorrc.getConfigLocation(): - try: - loadedTorrc.load() - panels["torrc"]._lastContentHeightArgs = None - except IOError: pass - - msg = "Saved configuration to %s" % configLocation - except (IOError, OSError), exc: - msg = "Unable to save configuration (%s)" % sysTools.getFileErrorMsg(exc) - - panels["control"].setMsg(msg, curses.A_STANDOUT) - panels["control"].redraw(True) - time.sleep(2) - - panels["control"].setMsg(CTL_PAUSED if isPaused else CTL_HELP) - - # reverts popup dimensions - popup.height = 9 - popup.recreate(stdscr, 80) - finally: - panel.CURSES_LOCK.release() - - panels["config"].redraw(True) - elif page == 2 and (key == ord('s') or key == ord('S')): - # set ordering for config options - titleLabel = "Config Option Ordering:" - options = [configPanel.FIELD_ATTR[field][0] for field in configPanel.Field.values()] - oldSelection = [configPanel.FIELD_ATTR[field][0] for field in panels["config"].sortOrdering] - optionColors = dict([configPanel.FIELD_ATTR[field] for field in configPanel.Field.values()]) - results = showSortDialog(stdscr, panels, isPaused, page, titleLabel, options, oldSelection, optionColors) - - if results: - # converts labels back to enums - resultEnums = [] - - for label in results: - for entryEnum in configPanel.FIELD_ATTR: - if label == configPanel.FIELD_ATTR[entryEnum][0]: - resultEnums.append(entryEnum) - break - - panels["config"].setSortOrder(resultEnums) - - panels["config"].redraw(True) - elif page == 2 and uiTools.isSelectionKey(key): - # let the user edit the configuration value, unchanged if left blank - panel.CURSES_LOCK.acquire() - try: - setPauseState(panels, isPaused, page, True) - - # provides prompt - selection = panels["config"].getSelection() - configOption = selection.get(configPanel.Field.OPTION) - titleMsg = "%s Value (esc to cancel): " % configOption - panels["control"].setMsg(titleMsg) - panels["control"].redraw(True) - - displayWidth = panels["control"].getPreferredSize()[1] - initialValue = selection.get(configPanel.Field.VALUE) - - # initial input for the text field - initialText = "" - if CONFIG["features.config.prepopulateEditValues"] and initialValue != "<none>": - initialText = initialValue - - newConfigValue = panels["control"].getstr(0, len(titleMsg), initialText) - - # it would be nice to quit on esc, but looks like this might not be possible... - if newConfigValue != None and newConfigValue != initialValue: - conn = torTools.getConn() - - # if the value's a boolean then allow for 'true' and 'false' inputs - if selection.get(configPanel.Field.TYPE) == "Boolean": - if newConfigValue.lower() == "true": newConfigValue = "1" - elif newConfigValue.lower() == "false": newConfigValue = "0" - - try: - if selection.get(configPanel.Field.TYPE) == "LineList": - newConfigValue = newConfigValue.split(",") - - conn.setOption(configOption, newConfigValue) - - # resets the isDefault flag - customOptions = torConfig.getCustomOptions() - selection.fields[configPanel.Field.IS_DEFAULT] = not configOption in customOptions - - panels["config"].redraw(True) - except Exception, exc: - errorMsg = "%s (press any key)" % exc - panels["control"].setMsg(uiTools.cropStr(errorMsg, displayWidth), curses.A_STANDOUT) - panels["control"].redraw(True) - - curses.cbreak() # wait indefinitely for key presses (no timeout) - stdscr.getch() - curses.halfdelay(REFRESH_RATE * 10) - - panels["control"].setMsg(CTL_PAUSED if isPaused else CTL_HELP) - setPauseState(panels, isPaused, page) - finally: - panel.CURSES_LOCK.release() - elif page == 3 and key == ord('r') or key == ord('R'): - # reloads torrc, providing a notice if successful or not - loadedTorrc = torConfig.getTorrc() - loadedTorrc.getLock().acquire() - - try: - loadedTorrc.load() - isSuccessful = True - except IOError: - isSuccessful = False - - loadedTorrc.getLock().release() - - #isSuccessful = panels["torrc"].loadConfig(logErrors = False) - #confTypeLabel = confPanel.CONFIG_LABELS[panels["torrc"].configType] - resetMsg = "torrc reloaded" if isSuccessful else "failed to reload torrc" - if isSuccessful: - panels["torrc"]._lastContentHeightArgs = None - panels["torrc"].redraw(True) - - panels["control"].setMsg(resetMsg, curses.A_STANDOUT) - panels["control"].redraw(True) - time.sleep(1) - - panels["control"].setMsg(CTL_PAUSED if isPaused else CTL_HELP) - elif page == 0: - panels["log"].handleKey(key) - elif page == 1: - panels["conn"].handleKey(key) - elif page == 2: - panels["config"].handleKey(key) - elif page == 3: - panels["torrc"].handleKey(key) - -def startTorMonitor(startTime, loggedEvents, isBlindMode): - try: - curses.wrapper(drawTorMonitor, startTime, loggedEvents, isBlindMode) - except KeyboardInterrupt: - pass # skip printing stack trace in case of keyboard interrupt - diff --git a/src/interface/descriptorPopup.py b/src/interface/descriptorPopup.py deleted file mode 100644 index cdc959d..0000000 --- a/src/interface/descriptorPopup.py +++ /dev/null @@ -1,181 +0,0 @@ -#!/usr/bin/env python -# descriptorPopup.py -- popup panel used to show raw consensus data -# Released under the GPL v3 (http://www.gnu.org/licenses/gpl.html) - -import math -import socket -import curses -from TorCtl import TorCtl - -import controller -import connections.connEntry -from util import panel, torTools, uiTools - -# field keywords used to identify areas for coloring -LINE_NUM_COLOR = "yellow" -HEADER_COLOR = "cyan" -HEADER_PREFIX = ["ns/id/", "desc/id/"] - -SIG_COLOR = "red" -SIG_START_KEYS = ["-----BEGIN RSA PUBLIC KEY-----", "-----BEGIN SIGNATURE-----"] -SIG_END_KEYS = ["-----END RSA PUBLIC KEY-----", "-----END SIGNATURE-----"] - -UNRESOLVED_MSG = "No consensus data available" -ERROR_MSG = "Unable to retrieve data" - -class PopupProperties: - """ - State attributes of popup window for consensus descriptions. - """ - - def __init__(self): - self.fingerprint = "" - self.entryColor = "white" - self.text = [] - self.scroll = 0 - self.showLineNum = True - - def reset(self, fingerprint, entryColor): - self.fingerprint = fingerprint - self.entryColor = entryColor - self.text = [] - self.scroll = 0 - - if fingerprint == "UNKNOWN": - self.fingerprint = None - self.showLineNum = False - self.text.append(UNRESOLVED_MSG) - else: - conn = torTools.getConn() - - try: - self.showLineNum = True - self.text.append("ns/id/%s" % fingerprint) - self.text += conn.getConsensusEntry(fingerprint).split("\n") - except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed): - self.text = self.text + [ERROR_MSG, ""] - - try: - descCommand = "desc/id/%s" % fingerprint - self.text.append("desc/id/%s" % fingerprint) - self.text += conn.getDescriptorEntry(fingerprint).split("\n") - except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed): - self.text = self.text + [ERROR_MSG] - - def handleKey(self, key, height): - if key == curses.KEY_UP: self.scroll = max(self.scroll - 1, 0) - elif key == curses.KEY_DOWN: self.scroll = max(0, min(self.scroll + 1, len(self.text) - height)) - elif key == curses.KEY_PPAGE: self.scroll = max(self.scroll - height, 0) - elif key == curses.KEY_NPAGE: self.scroll = max(0, min(self.scroll + height, len(self.text) - height)) - -def showDescriptorPopup(popup, stdscr, connectionPanel): - """ - Presents consensus descriptor in popup window with the following controls: - Up, Down, Page Up, Page Down - scroll descriptor - Right, Left - next / previous connection - Enter, Space, d, D - close popup - """ - - properties = PopupProperties() - isVisible = True - - if not panel.CURSES_LOCK.acquire(False): return - try: - while isVisible: - selection = connectionPanel._scroller.getCursorSelection(connectionPanel._entryLines) - if not selection: break - fingerprint = selection.foreign.getFingerprint() - entryColor = connections.connEntry.CATEGORY_COLOR[selection.getType()] - properties.reset(fingerprint, entryColor) - - # constrains popup size to match text - width, height = 0, 0 - for line in properties.text: - # width includes content, line number field, and border - lineWidth = len(line) + 5 - if properties.showLineNum: lineWidth += int(math.log10(len(properties.text))) + 1 - width = max(width, lineWidth) - - # tracks number of extra lines that will be taken due to text wrap - height += (lineWidth - 2) / connectionPanel.maxX - - popup.setHeight(min(len(properties.text) + height + 2, connectionPanel.maxY)) - popup.recreate(stdscr, width) - - while isVisible: - draw(popup, properties) - key = stdscr.getch() - - if uiTools.isSelectionKey(key) or key in (ord('d'), ord('D')): - # closes popup - isVisible = False - elif key in (curses.KEY_LEFT, curses.KEY_RIGHT): - # navigation - pass on to connPanel and recreate popup - connectionPanel.handleKey(curses.KEY_UP if key == curses.KEY_LEFT else curses.KEY_DOWN) - break - else: properties.handleKey(key, popup.height - 2) - - popup.setHeight(9) - popup.recreate(stdscr, 80) - finally: - panel.CURSES_LOCK.release() - -def draw(popup, properties): - popup.clear() - popup.win.box() - xOffset = 2 - - if properties.text: - if properties.fingerprint: popup.addstr(0, 0, "Consensus Descriptor (%s):" % properties.fingerprint, curses.A_STANDOUT) - else: popup.addstr(0, 0, "Consensus Descriptor:", curses.A_STANDOUT) - - isEncryption = False # true if line is part of an encryption block - - # checks if first line is in an encryption block - for i in range(0, properties.scroll): - lineText = properties.text[i].strip() - if lineText in SIG_START_KEYS: isEncryption = True - elif lineText in SIG_END_KEYS: isEncryption = False - - pageHeight = popup.maxY - 2 - numFieldWidth = int(math.log10(len(properties.text))) + 1 - lineNum = 1 - for i in range(properties.scroll, min(len(properties.text), properties.scroll + pageHeight)): - lineText = properties.text[i].strip() - - numOffset = 0 # offset for line numbering - if properties.showLineNum: - popup.addstr(lineNum, xOffset, ("%%%ii" % numFieldWidth) % (i + 1), curses.A_BOLD | uiTools.getColor(LINE_NUM_COLOR)) - numOffset = numFieldWidth + 1 - - if lineText: - keyword = lineText.split()[0] # first word of line - remainder = lineText[len(keyword):] - keywordFormat = curses.A_BOLD | uiTools.getColor(properties.entryColor) - remainderFormat = uiTools.getColor(properties.entryColor) - - if lineText.startswith(HEADER_PREFIX[0]) or lineText.startswith(HEADER_PREFIX[1]): - keyword, remainder = lineText, "" - keywordFormat = curses.A_BOLD | uiTools.getColor(HEADER_COLOR) - if lineText == UNRESOLVED_MSG or lineText == ERROR_MSG: - keyword, remainder = lineText, "" - if lineText in SIG_START_KEYS: - keyword, remainder = lineText, "" - isEncryption = True - keywordFormat = curses.A_BOLD | uiTools.getColor(SIG_COLOR) - elif lineText in SIG_END_KEYS: - keyword, remainder = lineText, "" - isEncryption = False - keywordFormat = curses.A_BOLD | uiTools.getColor(SIG_COLOR) - elif isEncryption: - keyword, remainder = lineText, "" - keywordFormat = uiTools.getColor(SIG_COLOR) - - lineNum, xLoc = controller.addstr_wrap(popup, lineNum, 0, keyword, keywordFormat, xOffset + numOffset, popup.maxX - 1, popup.maxY - 1) - lineNum, xLoc = controller.addstr_wrap(popup, lineNum, xLoc, remainder, remainderFormat, xOffset + numOffset, popup.maxX - 1, popup.maxY - 1) - - lineNum += 1 - if lineNum > pageHeight: break - - popup.refresh() - diff --git a/src/interface/graphing/__init__.py b/src/interface/graphing/__init__.py deleted file mode 100644 index 9a81dbd..0000000 --- a/src/interface/graphing/__init__.py +++ /dev/null @@ -1,6 +0,0 @@ -""" -Panels, popups, and handlers comprising the arm user interface. -""" - -__all__ = ["graphPanel", "bandwidthStats", "connStats", "resourceStats"] - diff --git a/src/interface/graphing/bandwidthStats.py b/src/interface/graphing/bandwidthStats.py deleted file mode 100644 index f8e3020..0000000 --- a/src/interface/graphing/bandwidthStats.py +++ /dev/null @@ -1,398 +0,0 @@ -""" -Tracks bandwidth usage of the tor process, expanding to include accounting -stats if they're set. -""" - -import time - -from interface.graphing import graphPanel -from util import log, sysTools, torTools, uiTools - -DL_COLOR, UL_COLOR = "green", "cyan" - -# width at which panel abandons placing optional stats (avg and total) with -# header in favor of replacing the x-axis label -COLLAPSE_WIDTH = 135 - -# valid keys for the accountingInfo mapping -ACCOUNTING_ARGS = ("status", "resetTime", "read", "written", "readLimit", "writtenLimit") - -PREPOPULATE_SUCCESS_MSG = "Read the last day of bandwidth history from the state file" -PREPOPULATE_FAILURE_MSG = "Unable to prepopulate bandwidth information (%s)" - -DEFAULT_CONFIG = {"features.graph.bw.transferInBytes": False, - "features.graph.bw.accounting.show": True, - "features.graph.bw.accounting.rate": 10, - "features.graph.bw.accounting.isTimeLong": False, - "log.graph.bw.prepopulateSuccess": log.NOTICE, - "log.graph.bw.prepopulateFailure": log.NOTICE} - -class BandwidthStats(graphPanel.GraphStats): - """ - Uses tor BW events to generate bandwidth usage graph. - """ - - def __init__(self, config=None): - graphPanel.GraphStats.__init__(self) - - self._config = dict(DEFAULT_CONFIG) - if config: - config.update(self._config, {"features.graph.bw.accounting.rate": 1}) - - # stats prepopulated from tor's state file - self.prepopulatePrimaryTotal = 0 - self.prepopulateSecondaryTotal = 0 - self.prepopulateTicks = 0 - - # accounting data (set by _updateAccountingInfo method) - self.accountingLastUpdated = 0 - self.accountingInfo = dict([(arg, "") for arg in ACCOUNTING_ARGS]) - - # listens for tor reload (sighup) events which can reset the bandwidth - # rate/burst and if tor's using accounting - conn = torTools.getConn() - self._titleStats, self.isAccounting = [], False - self.resetListener(conn, torTools.State.INIT) # initializes values - conn.addStatusListener(self.resetListener) - - # Initialized the bandwidth totals to the values reported by Tor. This - # uses a controller options introduced in ticket 2345: - # https://trac.torproject.org/projects/tor/ticket/2345 - # - # further updates are still handled via BW events to avoid unnecessary - # GETINFO requests. - - self.initialPrimaryTotal = 0 - self.initialSecondaryTotal = 0 - - readTotal = conn.getInfo("traffic/read") - if readTotal and readTotal.isdigit(): - self.initialPrimaryTotal = int(readTotal) / 1024 # Bytes -> KB - - writeTotal = conn.getInfo("traffic/written") - if writeTotal and writeTotal.isdigit(): - self.initialSecondaryTotal = int(writeTotal) / 1024 # Bytes -> KB - - def resetListener(self, conn, eventType): - # updates title parameters and accounting status if they changed - self._titleStats = [] # force reset of title - self.new_desc_event(None) # updates title params - - if eventType == torTools.State.INIT and self._config["features.graph.bw.accounting.show"]: - self.isAccounting = conn.getInfo('accounting/enabled') == '1' - - def prepopulateFromState(self): - """ - Attempts to use tor's state file to prepopulate values for the 15 minute - interval via the BWHistoryReadValues/BWHistoryWriteValues values. This - returns True if successful and False otherwise. - """ - - # checks that this is a relay (if ORPort is unset, then skip) - conn = torTools.getConn() - orPort = conn.getOption("ORPort") - if orPort == "0": return - - # gets the uptime (using the same parameters as the header panel to take - # advantage of caching - uptime = None - queryPid = conn.getMyPid() - if queryPid: - queryParam = ["%cpu", "rss", "%mem", "etime"] - queryCmd = "ps -p %s -o %s" % (queryPid, ",".join(queryParam)) - psCall = sysTools.call(queryCmd, 3600, True) - - if psCall and len(psCall) == 2: - stats = psCall[1].strip().split() - if len(stats) == 4: uptime = stats[3] - - # checks if tor has been running for at least a day, the reason being that - # the state tracks a day's worth of data and this should only prepopulate - # results associated with this tor instance - if not uptime or not "-" in uptime: - msg = PREPOPULATE_FAILURE_MSG % "insufficient uptime" - log.log(self._config["log.graph.bw.prepopulateFailure"], msg) - return False - - # get the user's data directory (usually '~/.tor') - dataDir = conn.getOption("DataDirectory") - if not dataDir: - msg = PREPOPULATE_FAILURE_MSG % "data directory not found" - log.log(self._config["log.graph.bw.prepopulateFailure"], msg) - return False - - # attempt to open the state file - try: stateFile = open("%s%s/state" % (conn.getPathPrefix(), dataDir), "r") - except IOError: - msg = PREPOPULATE_FAILURE_MSG % "unable to read the state file" - log.log(self._config["log.graph.bw.prepopulateFailure"], msg) - return False - - # get the BWHistory entries (ordered oldest to newest) and number of - # intervals since last recorded - bwReadEntries, bwWriteEntries = None, None - missingReadEntries, missingWriteEntries = None, None - - # converts from gmt to local with respect to DST - tz_offset = time.altzone if time.localtime()[8] else time.timezone - - for line in stateFile: - line = line.strip() - - # According to the rep_hist_update_state() function the BWHistory*Ends - # correspond to the start of the following sampling period. Also, the - # most recent values of BWHistory*Values appear to be an incremental - # counter for the current sampling period. Hence, offsets are added to - # account for both. - - if line.startswith("BWHistoryReadValues"): - bwReadEntries = line[20:].split(",") - bwReadEntries = [int(entry) / 1024.0 / 900 for entry in bwReadEntries] - bwReadEntries.pop() - elif line.startswith("BWHistoryWriteValues"): - bwWriteEntries = line[21:].split(",") - bwWriteEntries = [int(entry) / 1024.0 / 900 for entry in bwWriteEntries] - bwWriteEntries.pop() - elif line.startswith("BWHistoryReadEnds"): - lastReadTime = time.mktime(time.strptime(line[18:], "%Y-%m-%d %H:%M:%S")) - tz_offset - lastReadTime -= 900 - missingReadEntries = int((time.time() - lastReadTime) / 900) - elif line.startswith("BWHistoryWriteEnds"): - lastWriteTime = time.mktime(time.strptime(line[19:], "%Y-%m-%d %H:%M:%S")) - tz_offset - lastWriteTime -= 900 - missingWriteEntries = int((time.time() - lastWriteTime) / 900) - - if not bwReadEntries or not bwWriteEntries or not lastReadTime or not lastWriteTime: - msg = PREPOPULATE_FAILURE_MSG % "bandwidth stats missing from state file" - log.log(self._config["log.graph.bw.prepopulateFailure"], msg) - return False - - # fills missing entries with the last value - bwReadEntries += [bwReadEntries[-1]] * missingReadEntries - bwWriteEntries += [bwWriteEntries[-1]] * missingWriteEntries - - # crops starting entries so they're the same size - entryCount = min(len(bwReadEntries), len(bwWriteEntries), self.maxCol) - bwReadEntries = bwReadEntries[len(bwReadEntries) - entryCount:] - bwWriteEntries = bwWriteEntries[len(bwWriteEntries) - entryCount:] - - # gets index for 15-minute interval - intervalIndex = 0 - for indexEntry in graphPanel.UPDATE_INTERVALS: - if indexEntry[1] == 900: break - else: intervalIndex += 1 - - # fills the graphing parameters with state information - for i in range(entryCount): - readVal, writeVal = bwReadEntries[i], bwWriteEntries[i] - - self.lastPrimary, self.lastSecondary = readVal, writeVal - - self.prepopulatePrimaryTotal += readVal * 900 - self.prepopulateSecondaryTotal += writeVal * 900 - self.prepopulateTicks += 900 - - self.primaryCounts[intervalIndex].insert(0, readVal) - self.secondaryCounts[intervalIndex].insert(0, writeVal) - - self.maxPrimary[intervalIndex] = max(self.primaryCounts) - self.maxSecondary[intervalIndex] = max(self.secondaryCounts) - del self.primaryCounts[intervalIndex][self.maxCol + 1:] - del self.secondaryCounts[intervalIndex][self.maxCol + 1:] - - msg = PREPOPULATE_SUCCESS_MSG - missingSec = time.time() - min(lastReadTime, lastWriteTime) - if missingSec: msg += " (%s is missing)" % uiTools.getTimeLabel(missingSec, 0, True) - log.log(self._config["log.graph.bw.prepopulateSuccess"], msg) - - return True - - def bandwidth_event(self, event): - if self.isAccounting and self.isNextTickRedraw(): - if time.time() - self.accountingLastUpdated >= self._config["features.graph.bw.accounting.rate"]: - self._updateAccountingInfo() - - # scales units from B to KB for graphing - self._processEvent(event.read / 1024.0, event.written / 1024.0) - - def draw(self, panel, width, height): - # line of the graph's x-axis labeling - labelingLine = graphPanel.GraphStats.getContentHeight(self) + panel.graphHeight - 2 - - # if display is narrow, overwrites x-axis labels with avg / total stats - if width <= COLLAPSE_WIDTH: - # clears line - panel.addstr(labelingLine, 0, " " * width) - graphCol = min((width - 10) / 2, self.maxCol) - - primaryFooter = "%s, %s" % (self._getAvgLabel(True), self._getTotalLabel(True)) - secondaryFooter = "%s, %s" % (self._getAvgLabel(False), self._getTotalLabel(False)) - - panel.addstr(labelingLine, 1, primaryFooter, uiTools.getColor(self.getColor(True))) - panel.addstr(labelingLine, graphCol + 6, secondaryFooter, uiTools.getColor(self.getColor(False))) - - # provides accounting stats if enabled - if self.isAccounting: - if torTools.getConn().isAlive(): - status = self.accountingInfo["status"] - - hibernateColor = "green" - if status == "soft": hibernateColor = "yellow" - elif status == "hard": hibernateColor = "red" - elif status == "": - # failed to be queried - status, hibernateColor = "unknown", "red" - - panel.addfstr(labelingLine + 2, 0, "Accounting (<%s>%s</%s>)" % (hibernateColor, status, hibernateColor)) - - resetTime = self.accountingInfo["resetTime"] - if not resetTime: resetTime = "unknown" - panel.addstr(labelingLine + 2, 35, "Time to reset: %s" % resetTime) - - used, total = self.accountingInfo["read"], self.accountingInfo["readLimit"] - if used and total: - panel.addstr(labelingLine + 3, 2, "%s / %s" % (used, total), uiTools.getColor(self.getColor(True))) - - used, total = self.accountingInfo["written"], self.accountingInfo["writtenLimit"] - if used and total: - panel.addstr(labelingLine + 3, 37, "%s / %s" % (used, total), uiTools.getColor(self.getColor(False))) - else: - panel.addfstr(labelingLine + 2, 0, "Accounting: Connection Closed...") - - def getTitle(self, width): - stats = list(self._titleStats) - - while True: - if not stats: return "Bandwidth:" - else: - label = "Bandwidth (%s):" % ", ".join(stats) - - if len(label) > width: del stats[-1] - else: return label - - def getHeaderLabel(self, width, isPrimary): - graphType = "Download" if isPrimary else "Upload" - stats = [""] - - # if wide then avg and total are part of the header, otherwise they're on - # the x-axis - if width * 2 > COLLAPSE_WIDTH: - stats = [""] * 3 - stats[1] = "- %s" % self._getAvgLabel(isPrimary) - stats[2] = ", %s" % self._getTotalLabel(isPrimary) - - stats[0] = "%-14s" % ("%s/sec" % uiTools.getSizeLabel((self.lastPrimary if isPrimary else self.lastSecondary) * 1024, 1, False, self._config["features.graph.bw.transferInBytes"])) - - # drops label's components if there's not enough space - labeling = graphType + " (" + "".join(stats).strip() + "):" - while len(labeling) >= width: - if len(stats) > 1: - del stats[-1] - labeling = graphType + " (" + "".join(stats).strip() + "):" - else: - labeling = graphType + ":" - break - - return labeling - - def getColor(self, isPrimary): - return DL_COLOR if isPrimary else UL_COLOR - - def getContentHeight(self): - baseHeight = graphPanel.GraphStats.getContentHeight(self) - return baseHeight + 3 if self.isAccounting else baseHeight - - def new_desc_event(self, event): - # updates self._titleStats with updated values - conn = torTools.getConn() - if not conn.isAlive(): return # keep old values - - myFingerprint = conn.getInfo("fingerprint") - if not self._titleStats or not myFingerprint or (event and myFingerprint in event.idlist): - stats = [] - bwRate = conn.getMyBandwidthRate() - bwBurst = conn.getMyBandwidthBurst() - bwObserved = conn.getMyBandwidthObserved() - bwMeasured = conn.getMyBandwidthMeasured() - labelInBytes = self._config["features.graph.bw.transferInBytes"] - - if bwRate and bwBurst: - bwRateLabel = uiTools.getSizeLabel(bwRate, 1, False, labelInBytes) - bwBurstLabel = uiTools.getSizeLabel(bwBurst, 1, False, labelInBytes) - - # if both are using rounded values then strip off the ".0" decimal - if ".0" in bwRateLabel and ".0" in bwBurstLabel: - bwRateLabel = bwRateLabel.replace(".0", "") - bwBurstLabel = bwBurstLabel.replace(".0", "") - - stats.append("limit: %s/s" % bwRateLabel) - stats.append("burst: %s/s" % bwBurstLabel) - - # Provide the observed bandwidth either if the measured bandwidth isn't - # available or if the measured bandwidth is the observed (this happens - # if there isn't yet enough bandwidth measurements). - if bwObserved and (not bwMeasured or bwMeasured == bwObserved): - stats.append("observed: %s/s" % uiTools.getSizeLabel(bwObserved, 1, False, labelInBytes)) - elif bwMeasured: - stats.append("measured: %s/s" % uiTools.getSizeLabel(bwMeasured, 1, False, labelInBytes)) - - self._titleStats = stats - - def _getAvgLabel(self, isPrimary): - total = self.primaryTotal if isPrimary else self.secondaryTotal - total += self.prepopulatePrimaryTotal if isPrimary else self.prepopulateSecondaryTotal - return "avg: %s/sec" % uiTools.getSizeLabel((total / max(1, self.tick + self.prepopulateTicks)) * 1024, 1, False, self._config["features.graph.bw.transferInBytes"]) - - def _getTotalLabel(self, isPrimary): - total = self.primaryTotal if isPrimary else self.secondaryTotal - total += self.initialPrimaryTotal if isPrimary else self.initialSecondaryTotal - return "total: %s" % uiTools.getSizeLabel(total * 1024, 1) - - def _updateAccountingInfo(self): - """ - Updates mapping used for accounting info. This includes the following keys: - status, resetTime, read, written, readLimit, writtenLimit - - Any failed lookups result in a mapping to an empty string. - """ - - conn = torTools.getConn() - queried = dict([(arg, "") for arg in ACCOUNTING_ARGS]) - queried["status"] = conn.getInfo("accounting/hibernating") - - # provides a nicely formatted reset time - endInterval = conn.getInfo("accounting/interval-end") - if endInterval: - # converts from gmt to local with respect to DST - if time.localtime()[8]: tz_offset = time.altzone - else: tz_offset = time.timezone - - sec = time.mktime(time.strptime(endInterval, "%Y-%m-%d %H:%M:%S")) - time.time() - tz_offset - if self._config["features.graph.bw.accounting.isTimeLong"]: - queried["resetTime"] = ", ".join(uiTools.getTimeLabels(sec, True)) - else: - days = sec / 86400 - sec %= 86400 - hours = sec / 3600 - sec %= 3600 - minutes = sec / 60 - sec %= 60 - queried["resetTime"] = "%i:%02i:%02i:%02i" % (days, hours, minutes, sec) - - # number of bytes used and in total for the accounting period - used = conn.getInfo("accounting/bytes") - left = conn.getInfo("accounting/bytes-left") - - if used and left: - usedComp, leftComp = used.split(" "), left.split(" ") - read, written = int(usedComp[0]), int(usedComp[1]) - readLeft, writtenLeft = int(leftComp[0]), int(leftComp[1]) - - queried["read"] = uiTools.getSizeLabel(read) - queried["written"] = uiTools.getSizeLabel(written) - queried["readLimit"] = uiTools.getSizeLabel(read + readLeft) - queried["writtenLimit"] = uiTools.getSizeLabel(written + writtenLeft) - - self.accountingInfo = queried - self.accountingLastUpdated = time.time() - diff --git a/src/interface/graphing/connStats.py b/src/interface/graphing/connStats.py deleted file mode 100644 index 511490c..0000000 --- a/src/interface/graphing/connStats.py +++ /dev/null @@ -1,54 +0,0 @@ -""" -Tracks stats concerning tor's current connections. -""" - -from interface.graphing import graphPanel -from util import connections, torTools - -class ConnStats(graphPanel.GraphStats): - """ - Tracks number of connections, counting client and directory connections as - outbound. Control connections are excluded from counts. - """ - - def __init__(self): - graphPanel.GraphStats.__init__(self) - - # listens for tor reload (sighup) events which can reset the ports tor uses - conn = torTools.getConn() - self.orPort, self.dirPort, self.controlPort = "0", "0", "0" - self.resetListener(conn, torTools.State.INIT) # initialize port values - conn.addStatusListener(self.resetListener) - - def resetListener(self, conn, eventType): - if eventType == torTools.State.INIT: - self.orPort = conn.getOption("ORPort", "0") - self.dirPort = conn.getOption("DirPort", "0") - self.controlPort = conn.getOption("ControlPort", "0") - - def eventTick(self): - """ - Fetches connection stats from cached information. - """ - - inboundCount, outboundCount = 0, 0 - - for entry in connections.getResolver("tor").getConnections(): - localPort = entry[1] - if localPort in (self.orPort, self.dirPort): inboundCount += 1 - elif localPort == self.controlPort: pass # control connection - else: outboundCount += 1 - - self._processEvent(inboundCount, outboundCount) - - def getTitle(self, width): - return "Connection Count:" - - def getHeaderLabel(self, width, isPrimary): - avg = (self.primaryTotal if isPrimary else self.secondaryTotal) / max(1, self.tick) - if isPrimary: return "Inbound (%s, avg: %s):" % (self.lastPrimary, avg) - else: return "Outbound (%s, avg: %s):" % (self.lastSecondary, avg) - - def getRefreshRate(self): - return 5 - diff --git a/src/interface/graphing/graphPanel.py b/src/interface/graphing/graphPanel.py deleted file mode 100644 index e4b493d..0000000 --- a/src/interface/graphing/graphPanel.py +++ /dev/null @@ -1,407 +0,0 @@ -""" -Flexible panel for presenting bar graphs for a variety of stats. This panel is -just concerned with the rendering of information, which is actually collected -and stored by implementations of the GraphStats interface. Panels are made up -of a title, followed by headers and graphs for two sets of stats. For -instance... - -Bandwidth (cap: 5 MB, burst: 10 MB): -Downloaded (0.0 B/sec): Uploaded (0.0 B/sec): - 34 30 - * * - ** * * * ** - * * * ** ** ** *** ** ** ** ** - ********* ****** ****** ********* ****** ****** - 0 ************ **************** 0 ************ **************** - 25s 50 1m 1.6 2.0 25s 50 1m 1.6 2.0 -""" - -import copy -import curses -from TorCtl import TorCtl - -from util import enum, panel, uiTools - -# time intervals at which graphs can be updated -UPDATE_INTERVALS = [("each second", 1), ("5 seconds", 5), ("30 seconds", 30), - ("minutely", 60), ("15 minute", 900), ("30 minute", 1800), - ("hourly", 3600), ("daily", 86400)] - -DEFAULT_CONTENT_HEIGHT = 4 # space needed for labeling above and below the graph -DEFAULT_COLOR_PRIMARY, DEFAULT_COLOR_SECONDARY = "green", "cyan" -MIN_GRAPH_HEIGHT = 1 - -# enums for graph bounds: -# Bounds.GLOBAL_MAX - global maximum (highest value ever seen) -# Bounds.LOCAL_MAX - local maximum (highest value currently on the graph) -# Bounds.TIGHT - local maximum and minimum -Bounds = enum.Enum("GLOBAL_MAX", "LOCAL_MAX", "TIGHT") - -WIDE_LABELING_GRAPH_COL = 50 # minimum graph columns to use wide spacing for x-axis labels - -# used for setting defaults when initializing GraphStats and GraphPanel instances -CONFIG = {"features.graph.height": 7, - "features.graph.interval": 0, - "features.graph.bound": 1, - "features.graph.maxWidth": 150, - "features.graph.showIntermediateBounds": True} - -def loadConfig(config): - config.update(CONFIG, { - "features.graph.height": MIN_GRAPH_HEIGHT, - "features.graph.maxWidth": 1, - "features.graph.interval": (0, len(UPDATE_INTERVALS) - 1), - "features.graph.bound": (0, 2)}) - -class GraphStats(TorCtl.PostEventListener): - """ - Module that's expected to update dynamically and provide attributes to be - graphed. Up to two graphs (a 'primary' and 'secondary') can be displayed at a - time and timescale parameters use the labels defined in UPDATE_INTERVALS. - """ - - def __init__(self, isPauseBuffer=False): - """ - Initializes parameters needed to present a graph. - """ - - TorCtl.PostEventListener.__init__(self) - - # panel to be redrawn when updated (set when added to GraphPanel) - self._graphPanel = None - - # mirror instance used to track updates when paused - self.isPaused, self.isPauseBuffer = False, isPauseBuffer - if isPauseBuffer: self._pauseBuffer = None - else: self._pauseBuffer = GraphStats(True) - - # tracked stats - self.tick = 0 # number of processed events - self.lastPrimary, self.lastSecondary = 0, 0 # most recent registered stats - self.primaryTotal, self.secondaryTotal = 0, 0 # sum of all stats seen - - # timescale dependent stats - self.maxCol = CONFIG["features.graph.maxWidth"] - self.maxPrimary, self.maxSecondary = {}, {} - self.primaryCounts, self.secondaryCounts = {}, {} - - for i in range(len(UPDATE_INTERVALS)): - # recent rates for graph - self.maxPrimary[i] = 0 - self.maxSecondary[i] = 0 - - # historic stats for graph, first is accumulator - # iterative insert needed to avoid making shallow copies (nasty, nasty gotcha) - self.primaryCounts[i] = (self.maxCol + 1) * [0] - self.secondaryCounts[i] = (self.maxCol + 1) * [0] - - def eventTick(self): - """ - Called when it's time to process another event. All graphs use tor BW - events to keep in sync with each other (this happens once a second). - """ - - pass - - def isNextTickRedraw(self): - """ - Provides true if the following tick (call to _processEvent) will result in - being redrawn. - """ - - if self._graphPanel and not self.isPauseBuffer and not self.isPaused: - # use the minimum of the current refresh rate and the panel's - updateRate = UPDATE_INTERVALS[self._graphPanel.updateInterval][1] - return (self.tick + 1) % min(updateRate, self.getRefreshRate()) == 0 - else: return False - - def getTitle(self, width): - """ - Provides top label. - """ - - return "" - - def getHeaderLabel(self, width, isPrimary): - """ - Provides labeling presented at the top of the graph. - """ - - return "" - - def getColor(self, isPrimary): - """ - Provides the color to be used for the graph and stats. - """ - - return DEFAULT_COLOR_PRIMARY if isPrimary else DEFAULT_COLOR_SECONDARY - - def getContentHeight(self): - """ - Provides the height content should take up (not including the graph). - """ - - return DEFAULT_CONTENT_HEIGHT - - def getRefreshRate(self): - """ - Provides the number of ticks between when the stats have new values to be - redrawn. - """ - - return 1 - - def isVisible(self): - """ - True if the stat has content to present, false if it should be hidden. - """ - - return True - - def draw(self, panel, width, height): - """ - Allows for any custom drawing monitor wishes to append. - """ - - pass - - def setPaused(self, isPause): - """ - If true, prevents bandwidth updates from being presented. This is a no-op - if a pause buffer. - """ - - if isPause == self.isPaused or self.isPauseBuffer: return - self.isPaused = isPause - - if self.isPaused: active, inactive = self._pauseBuffer, self - else: active, inactive = self, self._pauseBuffer - self._parameterSwap(active, inactive) - - def bandwidth_event(self, event): - self.eventTick() - - def _parameterSwap(self, active, inactive): - """ - Either overwrites parameters of pauseBuffer or with the current values or - vice versa. This is a helper method for setPaused and should be overwritten - to append with additional parameters that need to be preserved when paused. - """ - - # The pause buffer is constructed as a GraphStats instance which will - # become problematic if this is overridden by any implementations (which - # currently isn't the case). If this happens then the pause buffer will - # need to be of the requester's type (not quite sure how to do this - # gracefully...). - - active.tick = inactive.tick - active.lastPrimary = inactive.lastPrimary - active.lastSecondary = inactive.lastSecondary - active.primaryTotal = inactive.primaryTotal - active.secondaryTotal = inactive.secondaryTotal - active.maxPrimary = dict(inactive.maxPrimary) - active.maxSecondary = dict(inactive.maxSecondary) - active.primaryCounts = copy.deepcopy(inactive.primaryCounts) - active.secondaryCounts = copy.deepcopy(inactive.secondaryCounts) - - def _processEvent(self, primary, secondary): - """ - Includes new stats in graphs and notifies associated GraphPanel of changes. - """ - - if self.isPaused: self._pauseBuffer._processEvent(primary, secondary) - else: - isRedraw = self.isNextTickRedraw() - - self.lastPrimary, self.lastSecondary = primary, secondary - self.primaryTotal += primary - self.secondaryTotal += secondary - - # updates for all time intervals - self.tick += 1 - for i in range(len(UPDATE_INTERVALS)): - lable, timescale = UPDATE_INTERVALS[i] - - self.primaryCounts[i][0] += primary - self.secondaryCounts[i][0] += secondary - - if self.tick % timescale == 0: - self.maxPrimary[i] = max(self.maxPrimary[i], self.primaryCounts[i][0] / timescale) - self.primaryCounts[i][0] /= timescale - self.primaryCounts[i].insert(0, 0) - del self.primaryCounts[i][self.maxCol + 1:] - - self.maxSecondary[i] = max(self.maxSecondary[i], self.secondaryCounts[i][0] / timescale) - self.secondaryCounts[i][0] /= timescale - self.secondaryCounts[i].insert(0, 0) - del self.secondaryCounts[i][self.maxCol + 1:] - - if isRedraw: self._graphPanel.redraw(True) - -class GraphPanel(panel.Panel): - """ - Panel displaying a graph, drawing statistics from custom GraphStats - implementations. - """ - - def __init__(self, stdscr): - panel.Panel.__init__(self, stdscr, "graph", 0) - self.updateInterval = CONFIG["features.graph.interval"] - self.bounds = Bounds.values()[CONFIG["features.graph.bound"]] - self.graphHeight = CONFIG["features.graph.height"] - self.currentDisplay = None # label of the stats currently being displayed - self.stats = {} # available stats (mappings of label -> instance) - self.showLabel = True # shows top label if true, hides otherwise - self.isPaused = False - - def getHeight(self): - """ - Provides the height requested by the currently displayed GraphStats (zero - if hidden). - """ - - if self.currentDisplay and self.stats[self.currentDisplay].isVisible(): - return self.stats[self.currentDisplay].getContentHeight() + self.graphHeight - else: return 0 - - def setGraphHeight(self, newGraphHeight): - """ - Sets the preferred height used for the graph (restricted to the - MIN_GRAPH_HEIGHT minimum). - - Arguments: - newGraphHeight - new height for the graph - """ - - self.graphHeight = max(MIN_GRAPH_HEIGHT, newGraphHeight) - - def draw(self, width, height): - """ Redraws graph panel """ - - if self.currentDisplay: - param = self.stats[self.currentDisplay] - graphCol = min((width - 10) / 2, param.maxCol) - - primaryColor = uiTools.getColor(param.getColor(True)) - secondaryColor = uiTools.getColor(param.getColor(False)) - - if self.showLabel: self.addstr(0, 0, param.getTitle(width), curses.A_STANDOUT) - - # top labels - left, right = param.getHeaderLabel(width / 2, True), param.getHeaderLabel(width / 2, False) - if left: self.addstr(1, 0, left, curses.A_BOLD | primaryColor) - if right: self.addstr(1, graphCol + 5, right, curses.A_BOLD | secondaryColor) - - # determines max/min value on the graph - if self.bounds == Bounds.GLOBAL_MAX: - primaryMaxBound = int(param.maxPrimary[self.updateInterval]) - secondaryMaxBound = int(param.maxSecondary[self.updateInterval]) - else: - # both Bounds.LOCAL_MAX and Bounds.TIGHT use local maxima - if graphCol < 2: - # nothing being displayed - primaryMaxBound, secondaryMaxBound = 0, 0 - else: - primaryMaxBound = int(max(param.primaryCounts[self.updateInterval][1:graphCol + 1])) - secondaryMaxBound = int(max(param.secondaryCounts[self.updateInterval][1:graphCol + 1])) - - primaryMinBound = secondaryMinBound = 0 - if self.bounds == Bounds.TIGHT: - primaryMinBound = int(min(param.primaryCounts[self.updateInterval][1:graphCol + 1])) - secondaryMinBound = int(min(param.secondaryCounts[self.updateInterval][1:graphCol + 1])) - - # if the max = min (ie, all values are the same) then use zero lower - # bound so a graph is still displayed - if primaryMinBound == primaryMaxBound: primaryMinBound = 0 - if secondaryMinBound == secondaryMaxBound: secondaryMinBound = 0 - - # displays upper and lower bounds - self.addstr(2, 0, "%4i" % primaryMaxBound, primaryColor) - self.addstr(self.graphHeight + 1, 0, "%4i" % primaryMinBound, primaryColor) - - self.addstr(2, graphCol + 5, "%4i" % secondaryMaxBound, secondaryColor) - self.addstr(self.graphHeight + 1, graphCol + 5, "%4i" % secondaryMinBound, secondaryColor) - - # displays intermediate bounds on every other row - if CONFIG["features.graph.showIntermediateBounds"]: - ticks = (self.graphHeight - 3) / 2 - for i in range(ticks): - row = self.graphHeight - (2 * i) - 3 - if self.graphHeight % 2 == 0 and i >= (ticks / 2): row -= 1 - - if primaryMinBound != primaryMaxBound: - primaryVal = (primaryMaxBound - primaryMinBound) / (self.graphHeight - 1) * (self.graphHeight - row - 1) - if not primaryVal in (primaryMinBound, primaryMaxBound): self.addstr(row + 2, 0, "%4i" % primaryVal, primaryColor) - - if secondaryMinBound != secondaryMaxBound: - secondaryVal = (secondaryMaxBound - secondaryMinBound) / (self.graphHeight - 1) * (self.graphHeight - row - 1) - if not secondaryVal in (secondaryMinBound, secondaryMaxBound): self.addstr(row + 2, graphCol + 5, "%4i" % secondaryVal, secondaryColor) - - # creates bar graph (both primary and secondary) - for col in range(graphCol): - colCount = int(param.primaryCounts[self.updateInterval][col + 1]) - primaryMinBound - colHeight = min(self.graphHeight, self.graphHeight * colCount / (max(1, primaryMaxBound) - primaryMinBound)) - for row in range(colHeight): self.addstr(self.graphHeight + 1 - row, col + 5, " ", curses.A_STANDOUT | primaryColor) - - colCount = int(param.secondaryCounts[self.updateInterval][col + 1]) - secondaryMinBound - colHeight = min(self.graphHeight, self.graphHeight * colCount / (max(1, secondaryMaxBound) - secondaryMinBound)) - for row in range(colHeight): self.addstr(self.graphHeight + 1 - row, col + graphCol + 10, " ", curses.A_STANDOUT | secondaryColor) - - # bottom labeling of x-axis - intervalSec = 1 # seconds per labeling - for i in range(len(UPDATE_INTERVALS)): - if i == self.updateInterval: intervalSec = UPDATE_INTERVALS[i][1] - - intervalSpacing = 10 if graphCol >= WIDE_LABELING_GRAPH_COL else 5 - unitsLabel, decimalPrecision = None, 0 - for i in range((graphCol - 4) / intervalSpacing): - loc = (i + 1) * intervalSpacing - timeLabel = uiTools.getTimeLabel(loc * intervalSec, decimalPrecision) - - if not unitsLabel: unitsLabel = timeLabel[-1] - elif unitsLabel != timeLabel[-1]: - # upped scale so also up precision of future measurements - unitsLabel = timeLabel[-1] - decimalPrecision += 1 - else: - # if constrained on space then strips labeling since already provided - timeLabel = timeLabel[:-1] - - self.addstr(self.graphHeight + 2, 4 + loc, timeLabel, primaryColor) - self.addstr(self.graphHeight + 2, graphCol + 10 + loc, timeLabel, secondaryColor) - - param.draw(self, width, height) # allows current stats to modify the display - - def addStats(self, label, stats): - """ - Makes GraphStats instance available in the panel. - """ - - stats._graphPanel = self - stats.isPaused = True - self.stats[label] = stats - - def setStats(self, label): - """ - Sets the currently displayed stats instance, hiding panel if None. - """ - - if label != self.currentDisplay: - if self.currentDisplay: self.stats[self.currentDisplay].setPaused(True) - - if not label: - self.currentDisplay = None - elif label in self.stats.keys(): - self.currentDisplay = label - self.stats[label].setPaused(self.isPaused) - else: raise ValueError("Unrecognized stats label: %s" % label) - - def setPaused(self, isPause): - """ - If true, prevents bandwidth updates from being presented. - """ - - if isPause == self.isPaused: return - self.isPaused = isPause - if self.currentDisplay: self.stats[self.currentDisplay].setPaused(self.isPaused) - diff --git a/src/interface/graphing/resourceStats.py b/src/interface/graphing/resourceStats.py deleted file mode 100644 index 864957e..0000000 --- a/src/interface/graphing/resourceStats.py +++ /dev/null @@ -1,47 +0,0 @@ -""" -Tracks the system resource usage (cpu and memory) of the tor process. -""" - -from interface.graphing import graphPanel -from util import sysTools, torTools, uiTools - -class ResourceStats(graphPanel.GraphStats): - """ - System resource usage tracker. - """ - - def __init__(self): - graphPanel.GraphStats.__init__(self) - self.queryPid = torTools.getConn().getMyPid() - - def getTitle(self, width): - return "System Resources:" - - def getHeaderLabel(self, width, isPrimary): - avg = (self.primaryTotal if isPrimary else self.secondaryTotal) / max(1, self.tick) - lastAmount = self.lastPrimary if isPrimary else self.lastSecondary - - if isPrimary: - return "CPU (%0.1f%%, avg: %0.1f%%):" % (lastAmount, avg) - else: - # memory sizes are converted from MB to B before generating labels - usageLabel = uiTools.getSizeLabel(lastAmount * 1048576, 1) - avgLabel = uiTools.getSizeLabel(avg * 1048576, 1) - return "Memory (%s, avg: %s):" % (usageLabel, avgLabel) - - def eventTick(self): - """ - Fetch the cached measurement of resource usage from the ResourceTracker. - """ - - primary, secondary = 0, 0 - if self.queryPid: - resourceTracker = sysTools.getResourceTracker(self.queryPid) - - if not resourceTracker.lastQueryFailed(): - primary, _, secondary, _ = resourceTracker.getResourceUsage() - primary *= 100 # decimal percentage to whole numbers - secondary /= 1048576 # translate size to MB so axis labels are short - - self._processEvent(primary, secondary) - diff --git a/src/interface/headerPanel.py b/src/interface/headerPanel.py deleted file mode 100644 index f653299..0000000 --- a/src/interface/headerPanel.py +++ /dev/null @@ -1,474 +0,0 @@ -""" -Top panel for every page, containing basic system and tor related information. -If there's room available then this expands to present its information in two -columns, otherwise it's laid out as follows: - arm - <hostname> (<os> <sys/version>) Tor <tor/version> (<new, old, recommended, etc>) - <nickname> - <address>:<orPort>, [Dir Port: <dirPort>, ]Control Port (<open, password, cookie>): <controlPort> - cpu: <cpu%> mem: <mem> (<mem%>) uid: <uid> uptime: <upmin>:<upsec> - fingerprint: <fingerprint> - -Example: - arm - odin (Linux 2.6.24-24-generic) Tor 0.2.1.19 (recommended) - odin - 76.104.132.98:9001, Dir Port: 9030, Control Port (cookie): 9051 - cpu: 14.6% mem: 42 MB (4.2%) pid: 20060 uptime: 48:27 - fingerprint: BDAD31F6F318E0413833E8EBDA956F76E4D66788 -""" - -import os -import time -import curses -import threading - -from util import log, panel, sysTools, torTools, uiTools - -# minimum width for which panel attempts to double up contents (two columns to -# better use screen real estate) -MIN_DUAL_COL_WIDTH = 141 - -FLAG_COLORS = {"Authority": "white", "BadExit": "red", "BadDirectory": "red", "Exit": "cyan", - "Fast": "yellow", "Guard": "green", "HSDir": "magenta", "Named": "blue", - "Stable": "blue", "Running": "yellow", "Unnamed": "magenta", "Valid": "green", - "V2Dir": "cyan", "V3Dir": "white"} - -VERSION_STATUS_COLORS = {"new": "blue", "new in series": "blue", "obsolete": "red", "recommended": "green", - "old": "red", "unrecommended": "red", "unknown": "cyan"} - -DEFAULT_CONFIG = {"features.showFdUsage": False, - "log.fdUsageSixtyPercent": log.NOTICE, - "log.fdUsageNinetyPercent": log.WARN} - -class HeaderPanel(panel.Panel, threading.Thread): - """ - Top area contenting tor settings and system information. Stats are stored in - the vals mapping, keys including: - tor/ version, versionStatus, nickname, orPort, dirPort, controlPort, - exitPolicy, isAuthPassword (bool), isAuthCookie (bool), - orListenAddr, *address, *fingerprint, *flags, pid, startTime, - *fdUsed, fdLimit, isFdLimitEstimate - sys/ hostname, os, version - stat/ *%torCpu, *%armCpu, *rss, *%mem - - * volatile parameter that'll be reset on each update - """ - - def __init__(self, stdscr, startTime, config = None): - panel.Panel.__init__(self, stdscr, "header", 0) - threading.Thread.__init__(self) - self.setDaemon(True) - - self._config = dict(DEFAULT_CONFIG) - if config: config.update(self._config) - - self._isTorConnected = True - self._lastUpdate = -1 # time the content was last revised - self._isPaused = False # prevents updates if true - self._halt = False # terminates thread if true - self._cond = threading.Condition() # used for pausing the thread - - # Time when the panel was paused or tor was stopped. This is used to - # freeze the uptime statistic (uptime increments normally when None). - self._haltTime = None - - # The last arm cpu usage sampling taken. This is a tuple of the form: - # (total arm cpu time, sampling timestamp) - # - # The initial cpu total should be zero. However, at startup the cpu time - # in practice is often greater than the real time causing the initially - # reported cpu usage to be over 100% (which shouldn't be possible on - # single core systems). - # - # Setting the initial cpu total to the value at this panel's init tends to - # give smoother results (staying in the same ballpark as the second - # sampling) so fudging the numbers this way for now. - - self._armCpuSampling = (sum(os.times()[:3]), startTime) - - # Last sampling received from the ResourceTracker, used to detect when it - # changes. - self._lastResourceFetch = -1 - - # flag to indicate if we've already given file descriptor warnings - self._isFdSixtyPercentWarned = False - self._isFdNinetyPercentWarned = False - - self.vals = {} - self.valsLock = threading.RLock() - self._update(True) - - # listens for tor reload (sighup) events - torTools.getConn().addStatusListener(self.resetListener) - - def getHeight(self): - """ - Provides the height of the content, which is dynamically determined by the - panel's maximum width. - """ - - isWide = self.getParent().getmaxyx()[1] >= MIN_DUAL_COL_WIDTH - if self.vals["tor/orPort"]: return 4 if isWide else 6 - else: return 3 if isWide else 4 - - def draw(self, width, height): - self.valsLock.acquire() - isWide = width + 1 >= MIN_DUAL_COL_WIDTH - - # space available for content - if isWide: - leftWidth = max(width / 2, 77) - rightWidth = width - leftWidth - else: leftWidth = rightWidth = width - - # Line 1 / Line 1 Left (system and tor version information) - sysNameLabel = "arm - %s" % self.vals["sys/hostname"] - contentSpace = min(leftWidth, 40) - - if len(sysNameLabel) + 10 <= contentSpace: - sysTypeLabel = "%s %s" % (self.vals["sys/os"], self.vals["sys/version"]) - sysTypeLabel = uiTools.cropStr(sysTypeLabel, contentSpace - len(sysNameLabel) - 3, 4) - self.addstr(0, 0, "%s (%s)" % (sysNameLabel, sysTypeLabel)) - else: - self.addstr(0, 0, uiTools.cropStr(sysNameLabel, contentSpace)) - - contentSpace = leftWidth - 43 - if 7 + len(self.vals["tor/version"]) + len(self.vals["tor/versionStatus"]) <= contentSpace: - versionColor = VERSION_STATUS_COLORS[self.vals["tor/versionStatus"]] if \ - self.vals["tor/versionStatus"] in VERSION_STATUS_COLORS else "white" - versionStatusMsg = "<%s>%s</%s>" % (versionColor, self.vals["tor/versionStatus"], versionColor) - self.addfstr(0, 43, "Tor %s (%s)" % (self.vals["tor/version"], versionStatusMsg)) - elif 11 <= contentSpace: - self.addstr(0, 43, uiTools.cropStr("Tor %s" % self.vals["tor/version"], contentSpace, 4)) - - # Line 2 / Line 2 Left (tor ip/port information) - if self.vals["tor/orPort"]: - myAddress = "Unknown" - if self.vals["tor/orListenAddr"]: myAddress = self.vals["tor/orListenAddr"] - elif self.vals["tor/address"]: myAddress = self.vals["tor/address"] - - # acting as a relay (we can assume certain parameters are set - entry = "" - dirPortLabel = ", Dir Port: %s" % self.vals["tor/dirPort"] if self.vals["tor/dirPort"] != "0" else "" - for label in (self.vals["tor/nickname"], " - " + myAddress, ":" + self.vals["tor/orPort"], dirPortLabel): - if len(entry) + len(label) <= leftWidth: entry += label - else: break - else: - # non-relay (client only) - # TODO: not sure what sort of stats to provide... - entry = "<red>Relaying Disabled</red>" - - if self.vals["tor/isAuthPassword"]: authType = "password" - elif self.vals["tor/isAuthCookie"]: authType = "cookie" - else: authType = "open" - - if len(entry) + 19 + len(self.vals["tor/controlPort"]) + len(authType) <= leftWidth: - authColor = "red" if authType == "open" else "green" - authLabel = "<%s>%s</%s>" % (authColor, authType, authColor) - self.addfstr(1, 0, "%s, Control Port (%s): %s" % (entry, authLabel, self.vals["tor/controlPort"])) - elif len(entry) + 16 + len(self.vals["tor/controlPort"]) <= leftWidth: - self.addstr(1, 0, "%s, Control Port: %s" % (entry, self.vals["tor/controlPort"])) - else: self.addstr(1, 0, entry) - - # Line 3 / Line 1 Right (system usage info) - y, x = (0, leftWidth) if isWide else (2, 0) - if self.vals["stat/rss"] != "0": memoryLabel = uiTools.getSizeLabel(int(self.vals["stat/rss"])) - else: memoryLabel = "0" - - uptimeLabel = "" - if self.vals["tor/startTime"]: - if self._haltTime: - # freeze the uptime when paused or the tor process is stopped - uptimeLabel = uiTools.getShortTimeLabel(self._haltTime - self.vals["tor/startTime"]) - else: - uptimeLabel = uiTools.getShortTimeLabel(time.time() - self.vals["tor/startTime"]) - - sysFields = ((0, "cpu: %s%% tor, %s%% arm" % (self.vals["stat/%torCpu"], self.vals["stat/%armCpu"])), - (27, "mem: %s (%s%%)" % (memoryLabel, self.vals["stat/%mem"])), - (47, "pid: %s" % (self.vals["tor/pid"] if self._isTorConnected else "")), - (59, "uptime: %s" % uptimeLabel)) - - for (start, label) in sysFields: - if start + len(label) <= rightWidth: self.addstr(y, x + start, label) - else: break - - if self.vals["tor/orPort"]: - # Line 4 / Line 2 Right (fingerprint, and possibly file descriptor usage) - y, x = (1, leftWidth) if isWide else (3, 0) - - fingerprintLabel = uiTools.cropStr("fingerprint: %s" % self.vals["tor/fingerprint"], width) - self.addstr(y, x, fingerprintLabel) - - # if there's room and we're able to retrieve both the file descriptor - # usage and limit then it might be presented - if width - x - 59 >= 20 and self.vals["tor/fdUsed"] and self.vals["tor/fdLimit"]: - # display file descriptor usage if we're either configured to do so or - # running out - - fdPercent = 100 * self.vals["tor/fdUsed"] / self.vals["tor/fdLimit"] - - if fdPercent >= 60 or self._config["features.showFdUsage"]: - fdPercentLabel, fdPercentFormat = "%i%%" % fdPercent, curses.A_NORMAL - if fdPercent >= 95: - fdPercentFormat = curses.A_BOLD | uiTools.getColor("red") - elif fdPercent >= 90: - fdPercentFormat = uiTools.getColor("red") - elif fdPercent >= 60: - fdPercentFormat = uiTools.getColor("yellow") - - estimateChar = "?" if self.vals["tor/isFdLimitEstimate"] else "" - baseLabel = "file desc: %i / %i%s (" % (self.vals["tor/fdUsed"], self.vals["tor/fdLimit"], estimateChar) - - self.addstr(y, x + 59, baseLabel) - self.addstr(y, x + 59 + len(baseLabel), fdPercentLabel, fdPercentFormat) - self.addstr(y, x + 59 + len(baseLabel) + len(fdPercentLabel), ")") - - # Line 5 / Line 3 Left (flags) - if self._isTorConnected: - flagLine = "flags: " - for flag in self.vals["tor/flags"]: - flagColor = FLAG_COLORS[flag] if flag in FLAG_COLORS.keys() else "white" - flagLine += "<%s>%s</%s>, " % (flagColor, flag, flagColor) - - if len(self.vals["tor/flags"]) > 0: flagLine = flagLine[:-2] - else: flagLine += "<cyan>none</cyan>" - - self.addfstr(2 if isWide else 4, 0, flagLine) - else: - statusTime = torTools.getConn().getStatus()[1] - statusTimeLabel = time.strftime("%H:%M %m/%d/%Y", time.localtime(statusTime)) - self.addfstr(2 if isWide else 4, 0, "<red>Tor Disconnected</red> (%s)" % statusTimeLabel) - - # Undisplayed / Line 3 Right (exit policy) - if isWide: - exitPolicy = self.vals["tor/exitPolicy"] - - # adds note when default exit policy is appended - if exitPolicy == "": exitPolicy = "<default>" - elif not exitPolicy.endswith((" *:*", " *")): exitPolicy += ", <default>" - - # color codes accepts to be green, rejects to be red, and default marker to be cyan - isSimple = len(exitPolicy) > rightWidth - 13 - policies = exitPolicy.split(", ") - for i in range(len(policies)): - policy = policies[i].strip() - displayedPolicy = policy.replace("accept", "").replace("reject", "").strip() if isSimple else policy - if policy.startswith("accept"): policy = "<green>%s</green>" % displayedPolicy - elif policy.startswith("reject"): policy = "<red>%s</red>" % displayedPolicy - elif policy.startswith("<default>"): policy = "<cyan>%s</cyan>" % displayedPolicy - policies[i] = policy - - self.addfstr(2, leftWidth, "exit policy: %s" % ", ".join(policies)) - else: - # Client only - # TODO: not sure what information to provide here... - pass - - self.valsLock.release() - - def setPaused(self, isPause): - """ - If true, prevents updates from being presented. - """ - - if not self._isPaused == isPause: - self._isPaused = isPause - if self._isTorConnected: - if isPause: self._haltTime = time.time() - else: self._haltTime = None - - # Redraw now so we'll be displaying the state right when paused - # (otherwise the uptime might be off by a second, and change when - # the panel's redrawn for other reasons). - self.redraw(True) - - def run(self): - """ - Keeps stats updated, checking for new information at a set rate. - """ - - lastDraw = time.time() - 1 - while not self._halt: - currentTime = time.time() - - if self._isPaused or currentTime - lastDraw < 1 or not self._isTorConnected: - self._cond.acquire() - if not self._halt: self._cond.wait(0.2) - self._cond.release() - else: - # Update the volatile attributes (cpu, memory, flags, etc) if we have - # a new resource usage sampling (the most dynamic stat) or its been - # twenty seconds since last fetched (so we still refresh occasionally - # when resource fetches fail). - # - # Otherwise, just redraw the panel to change the uptime field. - - isChanged = False - if self.vals["tor/pid"]: - resourceTracker = sysTools.getResourceTracker(self.vals["tor/pid"]) - isChanged = self._lastResourceFetch != resourceTracker.getRunCount() - - if isChanged or currentTime - self._lastUpdate >= 20: - self._update() - - self.redraw(True) - lastDraw += 1 - - def stop(self): - """ - Halts further resolutions and terminates the thread. - """ - - self._cond.acquire() - self._halt = True - self._cond.notifyAll() - self._cond.release() - - def resetListener(self, conn, eventType): - """ - Updates static parameters on tor reload (sighup) events. - - Arguments: - conn - tor controller - eventType - type of event detected - """ - - if eventType == torTools.State.INIT: - self._isTorConnected = True - if self._isPaused: self._haltTime = time.time() - else: self._haltTime = None - - self._update(True) - self.redraw(True) - elif eventType == torTools.State.CLOSED: - self._isTorConnected = False - self._haltTime = time.time() - self._update() - self.redraw(True) - - def _update(self, setStatic=False): - """ - Updates stats in the vals mapping. By default this just revises volatile - attributes. - - Arguments: - setStatic - resets all parameters, including relatively static values - """ - - self.valsLock.acquire() - conn = torTools.getConn() - - if setStatic: - # version is truncated to first part, for instance: - # 0.2.2.13-alpha (git-feb8c1b5f67f2c6f) -> 0.2.2.13-alpha - self.vals["tor/version"] = conn.getInfo("version", "Unknown").split()[0] - self.vals["tor/versionStatus"] = conn.getInfo("status/version/current", "Unknown") - self.vals["tor/nickname"] = conn.getOption("Nickname", "") - self.vals["tor/orPort"] = conn.getOption("ORPort", "0") - self.vals["tor/dirPort"] = conn.getOption("DirPort", "0") - self.vals["tor/controlPort"] = conn.getOption("ControlPort", "") - self.vals["tor/isAuthPassword"] = conn.getOption("HashedControlPassword") != None - self.vals["tor/isAuthCookie"] = conn.getOption("CookieAuthentication") == "1" - - # orport is reported as zero if unset - if self.vals["tor/orPort"] == "0": self.vals["tor/orPort"] = "" - - # overwrite address if ORListenAddress is set (and possibly orPort too) - self.vals["tor/orListenAddr"] = "" - listenAddr = conn.getOption("ORListenAddress") - if listenAddr: - if ":" in listenAddr: - # both ip and port overwritten - self.vals["tor/orListenAddr"] = listenAddr[:listenAddr.find(":")] - self.vals["tor/orPort"] = listenAddr[listenAddr.find(":") + 1:] - else: - self.vals["tor/orListenAddr"] = listenAddr - - # fetch exit policy (might span over multiple lines) - policyEntries = [] - for exitPolicy in conn.getOption("ExitPolicy", [], True): - policyEntries += [policy.strip() for policy in exitPolicy.split(",")] - self.vals["tor/exitPolicy"] = ", ".join(policyEntries) - - # file descriptor limit for the process, if this can't be determined - # then the limit is None - fdLimit, fdIsEstimate = conn.getMyFileDescriptorLimit() - self.vals["tor/fdLimit"] = fdLimit - self.vals["tor/isFdLimitEstimate"] = fdIsEstimate - - # system information - unameVals = os.uname() - self.vals["sys/hostname"] = unameVals[1] - self.vals["sys/os"] = unameVals[0] - self.vals["sys/version"] = unameVals[2] - - pid = conn.getMyPid() - self.vals["tor/pid"] = pid if pid else "" - - startTime = conn.getStartTime() - self.vals["tor/startTime"] = startTime if startTime else "" - - # reverts volatile parameters to defaults - self.vals["tor/fingerprint"] = "Unknown" - self.vals["tor/flags"] = [] - self.vals["tor/fdUsed"] = 0 - self.vals["stat/%torCpu"] = "0" - self.vals["stat/%armCpu"] = "0" - self.vals["stat/rss"] = "0" - self.vals["stat/%mem"] = "0" - - # sets volatile parameters - # TODO: This can change, being reported by STATUS_SERVER -> EXTERNAL_ADDRESS - # events. Introduce caching via torTools? - self.vals["tor/address"] = conn.getInfo("address", "") - - self.vals["tor/fingerprint"] = conn.getInfo("fingerprint", self.vals["tor/fingerprint"]) - self.vals["tor/flags"] = conn.getMyFlags(self.vals["tor/flags"]) - - # Updates file descriptor usage and logs if the usage is high. If we don't - # have a known limit or it's obviously faulty (being lower than our - # current usage) then omit file descriptor functionality. - if self.vals["tor/fdLimit"]: - fdUsed = conn.getMyFileDescriptorUsage() - if fdUsed and fdUsed <= self.vals["tor/fdLimit"]: self.vals["tor/fdUsed"] = fdUsed - else: self.vals["tor/fdUsed"] = 0 - - if self.vals["tor/fdUsed"] and self.vals["tor/fdLimit"]: - fdPercent = 100 * self.vals["tor/fdUsed"] / self.vals["tor/fdLimit"] - estimatedLabel = " estimated" if self.vals["tor/isFdLimitEstimate"] else "" - msg = "Tor's%s file descriptor usage is at %i%%." % (estimatedLabel, fdPercent) - - if fdPercent >= 90 and not self._isFdNinetyPercentWarned: - self._isFdSixtyPercentWarned, self._isFdNinetyPercentWarned = True, True - msg += " If you run out Tor will be unable to continue functioning." - log.log(self._config["log.fdUsageNinetyPercent"], msg) - elif fdPercent >= 60 and not self._isFdSixtyPercentWarned: - self._isFdSixtyPercentWarned = True - log.log(self._config["log.fdUsageSixtyPercent"], msg) - - # ps or proc derived resource usage stats - if self.vals["tor/pid"]: - resourceTracker = sysTools.getResourceTracker(self.vals["tor/pid"]) - - if resourceTracker.lastQueryFailed(): - self.vals["stat/%torCpu"] = "0" - self.vals["stat/rss"] = "0" - self.vals["stat/%mem"] = "0" - else: - cpuUsage, _, memUsage, memUsagePercent = resourceTracker.getResourceUsage() - self._lastResourceFetch = resourceTracker.getRunCount() - self.vals["stat/%torCpu"] = "%0.1f" % (100 * cpuUsage) - self.vals["stat/rss"] = str(memUsage) - self.vals["stat/%mem"] = "%0.1f" % (100 * memUsagePercent) - - # determines the cpu time for the arm process (including user and system - # time of both the primary and child processes) - - totalArmCpuTime, currentTime = sum(os.times()[:3]), time.time() - armCpuDelta = totalArmCpuTime - self._armCpuSampling[0] - armTimeDelta = currentTime - self._armCpuSampling[1] - pythonCpuTime = armCpuDelta / armTimeDelta - sysCallCpuTime = sysTools.getSysCpuUsage() - self.vals["stat/%armCpu"] = "%0.1f" % (100 * (pythonCpuTime + sysCallCpuTime)) - self._armCpuSampling = (totalArmCpuTime, currentTime) - - self._lastUpdate = currentTime - self.valsLock.release() - diff --git a/src/interface/logPanel.py b/src/interface/logPanel.py deleted file mode 100644 index 86e680f..0000000 --- a/src/interface/logPanel.py +++ /dev/null @@ -1,1100 +0,0 @@ -""" -Panel providing a chronological log of events its been configured to listen -for. This provides prepopulation from the log file and supports filtering by -regular expressions. -""" - -import time -import os -import curses -import threading - -from TorCtl import TorCtl - -from version import VERSION -from util import conf, log, panel, sysTools, torTools, uiTools - -TOR_EVENT_TYPES = { - "d": "DEBUG", "a": "ADDRMAP", "k": "DESCCHANGED", "s": "STREAM", - "i": "INFO", "f": "AUTHDIR_NEWDESCS", "g": "GUARD", "r": "STREAM_BW", - "n": "NOTICE", "h": "BUILDTIMEOUT_SET", "l": "NEWCONSENSUS", "t": "STATUS_CLIENT", - "w": "WARN", "b": "BW", "m": "NEWDESC", "u": "STATUS_GENERAL", - "e": "ERR", "c": "CIRC", "p": "NS", "v": "STATUS_SERVER", - "j": "CLIENTS_SEEN", "q": "ORCONN"} - -EVENT_LISTING = """ d DEBUG a ADDRMAP k DESCCHANGED s STREAM - i INFO f AUTHDIR_NEWDESCS g GUARD r STREAM_BW - n NOTICE h BUILDTIMEOUT_SET l NEWCONSENSUS t STATUS_CLIENT - w WARN b BW m NEWDESC u STATUS_GENERAL - e ERR c CIRC p NS v STATUS_SERVER - j CLIENTS_SEEN q ORCONN - DINWE tor runlevel+ A All Events - 12345 arm runlevel+ X No Events - 67890 torctl runlevel+ U Unknown Events""" - -RUNLEVEL_EVENT_COLOR = {log.DEBUG: "magenta", log.INFO: "blue", log.NOTICE: "green", - log.WARN: "yellow", log.ERR: "red"} -DAYBREAK_EVENT = "DAYBREAK" # special event for marking when the date changes -TIMEZONE_OFFSET = time.altzone if time.localtime()[8] else time.timezone - -ENTRY_INDENT = 2 # spaces an entry's message is indented after the first line -DEFAULT_CONFIG = {"features.logFile": "", - "features.log.showDateDividers": True, - "features.log.showDuplicateEntries": False, - "features.log.entryDuration": 7, - "features.log.maxLinesPerEntry": 4, - "features.log.prepopulate": True, - "features.log.prepopulateReadLimit": 5000, - "features.log.maxRefreshRate": 300, - "cache.logPanel.size": 1000, - "log.logPanel.prepopulateSuccess": log.INFO, - "log.logPanel.prepopulateFailed": log.WARN, - "log.logPanel.logFileOpened": log.NOTICE, - "log.logPanel.logFileWriteFailed": log.ERR, - "log.logPanel.forceDoubleRedraw": log.DEBUG} - -DUPLICATE_MSG = " [%i duplicate%s hidden]" - -# The height of the drawn content is estimated based on the last time we redrew -# the panel. It's chiefly used for scrolling and the bar indicating its -# position. Letting the estimate be too inaccurate results in a display bug, so -# redraws the display if it's off by this threshold. -CONTENT_HEIGHT_REDRAW_THRESHOLD = 3 - -# static starting portion of common log entries, fetched from the config when -# needed if None -COMMON_LOG_MESSAGES = None - -# cached values and the arguments that generated it for the getDaybreaks and -# getDuplicates functions -CACHED_DAYBREAKS_ARGUMENTS = (None, None) # events, current day -CACHED_DAYBREAKS_RESULT = None -CACHED_DUPLICATES_ARGUMENTS = None # events -CACHED_DUPLICATES_RESULT = None - -# duration we'll wait for the deduplication function before giving up (in ms) -DEDUPLICATION_TIMEOUT = 100 - -def daysSince(timestamp=None): - """ - Provides the number of days since the epoch converted to local time (rounded - down). - - Arguments: - timestamp - unix timestamp to convert, current time if undefined - """ - - if timestamp == None: timestamp = time.time() - return int((timestamp - TIMEZONE_OFFSET) / 86400) - -def expandEvents(eventAbbr): - """ - Expands event abbreviations to their full names. Beside mappings provided in - TOR_EVENT_TYPES this recognizes the following special events and aliases: - U - UKNOWN events - A - all events - X - no events - DINWE - runlevel and higher - 12345 - arm runlevel and higher (ARM_DEBUG - ARM_ERR) - 67890 - torctl runlevel and higher (TORCTL_DEBUG - TORCTL_ERR) - Raises ValueError with invalid input if any part isn't recognized. - - Examples: - "inUt" -> ["INFO", "NOTICE", "UNKNOWN", "STREAM_BW"] - "N4" -> ["NOTICE", "WARN", "ERR", "ARM_WARN", "ARM_ERR"] - "cfX" -> [] - - Arguments: - eventAbbr - flags to be parsed to event types - """ - - expandedEvents, invalidFlags = set(), "" - - for flag in eventAbbr: - if flag == "A": - armRunlevels = ["ARM_" + runlevel for runlevel in log.Runlevel.values()] - torctlRunlevels = ["TORCTL_" + runlevel for runlevel in log.Runlevel.values()] - expandedEvents = set(TOR_EVENT_TYPES.values() + armRunlevels + torctlRunlevels + ["UNKNOWN"]) - break - elif flag == "X": - expandedEvents = set() - break - elif flag in "DINWE1234567890": - # all events for a runlevel and higher - if flag in "DINWE": typePrefix = "" - elif flag in "12345": typePrefix = "ARM_" - elif flag in "67890": typePrefix = "TORCTL_" - - if flag in "D16": runlevelIndex = 0 - elif flag in "I27": runlevelIndex = 1 - elif flag in "N38": runlevelIndex = 2 - elif flag in "W49": runlevelIndex = 3 - elif flag in "E50": runlevelIndex = 4 - - runlevelSet = [typePrefix + runlevel for runlevel in log.Runlevel.values()[runlevelIndex:]] - expandedEvents = expandedEvents.union(set(runlevelSet)) - elif flag == "U": - expandedEvents.add("UNKNOWN") - elif flag in TOR_EVENT_TYPES: - expandedEvents.add(TOR_EVENT_TYPES[flag]) - else: - invalidFlags += flag - - if invalidFlags: raise ValueError(invalidFlags) - else: return expandedEvents - -def getMissingEventTypes(): - """ - Provides the event types the current torctl connection supports but arm - doesn't. This provides an empty list if no event types are missing, and None - if the GETINFO query fails. - """ - - torEventTypes = torTools.getConn().getInfo("events/names") - - if torEventTypes: - torEventTypes = torEventTypes.split(" ") - armEventTypes = TOR_EVENT_TYPES.values() - return [event for event in torEventTypes if not event in armEventTypes] - else: return None # GETINFO call failed - -def loadLogMessages(): - """ - Fetches a mapping of common log messages to their runlevels from the config. - """ - - global COMMON_LOG_MESSAGES - armConf = conf.getConfig("arm") - - COMMON_LOG_MESSAGES = {} - for confKey in armConf.getKeys(): - if confKey.startswith("msg."): - eventType = confKey[4:].upper() - messages = armConf.get(confKey, []) - COMMON_LOG_MESSAGES[eventType] = messages - -def getLogFileEntries(runlevels, readLimit = None, addLimit = None, config = None): - """ - Parses tor's log file for past events matching the given runlevels, providing - a list of log entries (ordered newest to oldest). Limiting the number of read - entries is suggested to avoid parsing everything from logs in the GB and TB - range. - - Arguments: - runlevels - event types (DEBUG - ERR) to be returned - readLimit - max lines of the log file that'll be read (unlimited if None) - addLimit - maximum entries to provide back (unlimited if None) - config - configuration parameters related to this panel, uses defaults - if left as None - """ - - startTime = time.time() - if not runlevels: return [] - - if not config: config = DEFAULT_CONFIG - - # checks tor's configuration for the log file's location (if any exists) - loggingTypes, loggingLocation = None, None - for loggingEntry in torTools.getConn().getOption("Log", [], True): - # looks for an entry like: notice file /var/log/tor/notices.log - entryComp = loggingEntry.split() - - if entryComp[1] == "file": - loggingTypes, loggingLocation = entryComp[0], entryComp[2] - break - - if not loggingLocation: return [] - - # includes the prefix for tor paths - loggingLocation = torTools.getConn().getPathPrefix() + loggingLocation - - # if the runlevels argument is a superset of the log file then we can - # limit the read contents to the addLimit - runlevels = log.Runlevel.values() - loggingTypes = loggingTypes.upper() - if addLimit and (not readLimit or readLimit > addLimit): - if "-" in loggingTypes: - divIndex = loggingTypes.find("-") - sIndex = runlevels.index(loggingTypes[:divIndex]) - eIndex = runlevels.index(loggingTypes[divIndex+1:]) - logFileRunlevels = runlevels[sIndex:eIndex+1] - else: - sIndex = runlevels.index(loggingTypes) - logFileRunlevels = runlevels[sIndex:] - - # checks if runlevels we're reporting are a superset of the file's contents - isFileSubset = True - for runlevelType in logFileRunlevels: - if runlevelType not in runlevels: - isFileSubset = False - break - - if isFileSubset: readLimit = addLimit - - # tries opening the log file, cropping results to avoid choking on huge logs - lines = [] - try: - if readLimit: - lines = sysTools.call("tail -n %i %s" % (readLimit, loggingLocation)) - if not lines: raise IOError() - else: - logFile = open(loggingLocation, "r") - lines = logFile.readlines() - logFile.close() - except IOError: - msg = "Unable to read tor's log file: %s" % loggingLocation - log.log(config["log.logPanel.prepopulateFailed"], msg) - - if not lines: return [] - - loggedEvents = [] - currentUnixTime, currentLocalTime = time.time(), time.localtime() - for i in range(len(lines) - 1, -1, -1): - line = lines[i] - - # entries look like: - # Jul 15 18:29:48.806 [notice] Parsing GEOIP file. - lineComp = line.split() - eventType = lineComp[3][1:-1].upper() - - if eventType in runlevels: - # converts timestamp to unix time - timestamp = " ".join(lineComp[:3]) - - # strips the decimal seconds - if "." in timestamp: timestamp = timestamp[:timestamp.find(".")] - - # overwrites missing time parameters with the local time (ignoring wday - # and yday since they aren't used) - eventTimeComp = list(time.strptime(timestamp, "%b %d %H:%M:%S")) - eventTimeComp[0] = currentLocalTime.tm_year - eventTimeComp[8] = currentLocalTime.tm_isdst - eventTime = time.mktime(eventTimeComp) # converts local to unix time - - # The above is gonna be wrong if the logs are for the previous year. If - # the event's in the future then correct for this. - if eventTime > currentUnixTime + 60: - eventTimeComp[0] -= 1 - eventTime = time.mktime(eventTimeComp) - - eventMsg = " ".join(lineComp[4:]) - loggedEvents.append(LogEntry(eventTime, eventType, eventMsg, RUNLEVEL_EVENT_COLOR[eventType])) - - if "opening log file" in line: - break # this entry marks the start of this tor instance - - if addLimit: loggedEvents = loggedEvents[:addLimit] - msg = "Read %i entries from tor's log file: %s (read limit: %i, runtime: %0.3f)" % (len(loggedEvents), loggingLocation, readLimit, time.time() - startTime) - log.log(config["log.logPanel.prepopulateSuccess"], msg) - return loggedEvents - -def getDaybreaks(events, ignoreTimeForCache = False): - """ - Provides the input events back with special 'DAYBREAK_EVENT' markers inserted - whenever the date changed between log entries (or since the most recent - event). The timestamp matches the beginning of the day for the following - entry. - - Arguments: - events - chronologically ordered listing of events - ignoreTimeForCache - skips taking the day into consideration for providing - cached results if true - """ - - global CACHED_DAYBREAKS_ARGUMENTS, CACHED_DAYBREAKS_RESULT - if not events: return [] - - newListing = [] - currentDay = daysSince() - lastDay = currentDay - - if CACHED_DAYBREAKS_ARGUMENTS[0] == events and \ - (ignoreTimeForCache or CACHED_DAYBREAKS_ARGUMENTS[1] == currentDay): - return list(CACHED_DAYBREAKS_RESULT) - - for entry in events: - eventDay = daysSince(entry.timestamp) - if eventDay != lastDay: - markerTimestamp = (eventDay * 86400) + TIMEZONE_OFFSET - newListing.append(LogEntry(markerTimestamp, DAYBREAK_EVENT, "", "white")) - - newListing.append(entry) - lastDay = eventDay - - CACHED_DAYBREAKS_ARGUMENTS = (list(events), currentDay) - CACHED_DAYBREAKS_RESULT = list(newListing) - - return newListing - -def getDuplicates(events): - """ - Deduplicates a list of log entries, providing back a tuple listing with the - log entry and count of duplicates following it. Entries in different days are - not considered to be duplicates. This times out, returning None if it takes - longer than DEDUPLICATION_TIMEOUT. - - Arguments: - events - chronologically ordered listing of events - """ - - global CACHED_DUPLICATES_ARGUMENTS, CACHED_DUPLICATES_RESULT - if CACHED_DUPLICATES_ARGUMENTS == events: - return list(CACHED_DUPLICATES_RESULT) - - # loads common log entries from the config if they haven't been - if COMMON_LOG_MESSAGES == None: loadLogMessages() - - startTime = time.time() - eventsRemaining = list(events) - returnEvents = [] - - while eventsRemaining: - entry = eventsRemaining.pop(0) - duplicateIndices = isDuplicate(entry, eventsRemaining, True) - - # checks if the call timeout has been reached - if (time.time() - startTime) > DEDUPLICATION_TIMEOUT / 1000.0: - return None - - # drops duplicate entries - duplicateIndices.reverse() - for i in duplicateIndices: del eventsRemaining[i] - - returnEvents.append((entry, len(duplicateIndices))) - - CACHED_DUPLICATES_ARGUMENTS = list(events) - CACHED_DUPLICATES_RESULT = list(returnEvents) - - return returnEvents - -def isDuplicate(event, eventSet, getDuplicates = False): - """ - True if the event is a duplicate for something in the eventSet, false - otherwise. If the getDuplicates flag is set this provides the indices of - the duplicates instead. - - Arguments: - event - event to search for duplicates of - eventSet - set to look for the event in - getDuplicates - instead of providing back a boolean this gives a list of - the duplicate indices in the eventSet - """ - - duplicateIndices = [] - for i in range(len(eventSet)): - forwardEntry = eventSet[i] - - # if showing dates then do duplicate detection for each day, rather - # than globally - if forwardEntry.type == DAYBREAK_EVENT: break - - if event.type == forwardEntry.type: - isDuplicate = False - if event.msg == forwardEntry.msg: isDuplicate = True - elif event.type in COMMON_LOG_MESSAGES: - for commonMsg in COMMON_LOG_MESSAGES[event.type]: - # if it starts with an asterisk then check the whole message rather - # than just the start - if commonMsg[0] == "*": - isDuplicate = commonMsg[1:] in event.msg and commonMsg[1:] in forwardEntry.msg - else: - isDuplicate = event.msg.startswith(commonMsg) and forwardEntry.msg.startswith(commonMsg) - - if isDuplicate: break - - if isDuplicate: - if getDuplicates: duplicateIndices.append(i) - else: return True - - if getDuplicates: return duplicateIndices - else: return False - -class LogEntry(): - """ - Individual log file entry, having the following attributes: - timestamp - unix timestamp for when the event occurred - eventType - event type that occurred ("INFO", "BW", "ARM_WARN", etc) - msg - message that was logged - color - color of the log entry - """ - - def __init__(self, timestamp, eventType, msg, color): - self.timestamp = timestamp - self.type = eventType - self.msg = msg - self.color = color - self._displayMessage = None - - def getDisplayMessage(self, includeDate = False): - """ - Provides the entry's message for the log. - - Arguments: - includeDate - appends the event's date to the start of the message - """ - - if includeDate: - # not the common case so skip caching - entryTime = time.localtime(self.timestamp) - timeLabel = "%i/%i/%i %02i:%02i:%02i" % (entryTime[1], entryTime[2], entryTime[0], entryTime[3], entryTime[4], entryTime[5]) - return "%s [%s] %s" % (timeLabel, self.type, self.msg) - - if not self._displayMessage: - entryTime = time.localtime(self.timestamp) - self._displayMessage = "%02i:%02i:%02i [%s] %s" % (entryTime[3], entryTime[4], entryTime[5], self.type, self.msg) - - return self._displayMessage - -class TorEventObserver(TorCtl.PostEventListener): - """ - Listens for all types of events provided by TorCtl, providing an LogEntry - instance to the given callback function. - """ - - def __init__(self, callback): - """ - Tor event listener with the purpose of translating events to nicely - formatted calls of a callback function. - - Arguments: - callback - function accepting a LogEntry, called when an event of these - types occur - """ - - TorCtl.PostEventListener.__init__(self) - self.callback = callback - - def circ_status_event(self, event): - msg = "ID: %-3s STATUS: %-10s PATH: %s" % (event.circ_id, event.status, ", ".join(event.path)) - if event.purpose: msg += " PURPOSE: %s" % event.purpose - if event.reason: msg += " REASON: %s" % event.reason - if event.remote_reason: msg += " REMOTE_REASON: %s" % event.remote_reason - self._notify(event, msg, "yellow") - - def buildtimeout_set_event(self, event): - self._notify(event, "SET_TYPE: %s, TOTAL_TIMES: %s, TIMEOUT_MS: %s, XM: %s, ALPHA: %s, CUTOFF_QUANTILE: %s" % (event.set_type, event.total_times, event.timeout_ms, event.xm, event.alpha, event.cutoff_quantile)) - - def stream_status_event(self, event): - self._notify(event, "ID: %s STATUS: %s CIRC_ID: %s TARGET: %s:%s REASON: %s REMOTE_REASON: %s SOURCE: %s SOURCE_ADDR: %s PURPOSE: %s" % (event.strm_id, event.status, event.circ_id, event.target_host, event.target_port, event.reason, event.remote_reason, event.source, event.source_addr, event.purpose)) - - def or_conn_status_event(self, event): - msg = "STATUS: %-10s ENDPOINT: %-20s" % (event.status, event.endpoint) - if event.age: msg += " AGE: %-3s" % event.age - if event.read_bytes: msg += " READ: %-4i" % event.read_bytes - if event.wrote_bytes: msg += " WRITTEN: %-4i" % event.wrote_bytes - if event.reason: msg += " REASON: %-6s" % event.reason - if event.ncircs: msg += " NCIRCS: %i" % event.ncircs - self._notify(event, msg) - - def stream_bw_event(self, event): - self._notify(event, "ID: %s READ: %s WRITTEN: %s" % (event.strm_id, event.bytes_read, event.bytes_written)) - - def bandwidth_event(self, event): - self._notify(event, "READ: %i, WRITTEN: %i" % (event.read, event.written), "cyan") - - def msg_event(self, event): - self._notify(event, event.msg, RUNLEVEL_EVENT_COLOR[event.level]) - - def new_desc_event(self, event): - idlistStr = [str(item) for item in event.idlist] - self._notify(event, ", ".join(idlistStr)) - - def address_mapped_event(self, event): - self._notify(event, "%s, %s -> %s" % (event.when, event.from_addr, event.to_addr)) - - def ns_event(self, event): - # NetworkStatus params: nickname, idhash, orhash, ip, orport (int), - # dirport (int), flags, idhex, bandwidth, updated (datetime) - msg = ", ".join(["%s (%s)" % (ns.idhex, ns.nickname) for ns in event.nslist]) - self._notify(event, "Listed (%i): %s" % (len(event.nslist), msg), "blue") - - def new_consensus_event(self, event): - msg = ", ".join(["%s (%s)" % (ns.idhex, ns.nickname) for ns in event.nslist]) - self._notify(event, "Listed (%i): %s" % (len(event.nslist), msg), "magenta") - - def unknown_event(self, event): - msg = "(%s) %s" % (event.event_name, event.event_string) - self.callback(LogEntry(event.arrived_at, "UNKNOWN", msg, "red")) - - def _notify(self, event, msg, color="white"): - self.callback(LogEntry(event.arrived_at, event.event_name, msg, color)) - -class LogPanel(panel.Panel, threading.Thread): - """ - Listens for and displays tor, arm, and torctl events. This can prepopulate - from tor's log file if it exists. - """ - - def __init__(self, stdscr, loggedEvents, config=None): - panel.Panel.__init__(self, stdscr, "log", 0) - threading.Thread.__init__(self) - self.setDaemon(True) - - self._config = dict(DEFAULT_CONFIG) - - if config: - config.update(self._config, { - "features.log.maxLinesPerEntry": 1, - "features.log.prepopulateReadLimit": 0, - "features.log.maxRefreshRate": 10, - "cache.logPanel.size": 1000}) - - # collapses duplicate log entries if false, showing only the most recent - self.showDuplicates = self._config["features.log.showDuplicateEntries"] - - self.msgLog = [] # log entries, sorted by the timestamp - self.loggedEvents = loggedEvents # events we're listening to - self.regexFilter = None # filter for presented log events (no filtering if None) - self.lastContentHeight = 0 # height of the rendered content when last drawn - self.logFile = None # file log messages are saved to (skipped if None) - self.scroll = 0 - self._isPaused = False - self._pauseBuffer = [] # location where messages are buffered if paused - - self._lastUpdate = -1 # time the content was last revised - self._halt = False # terminates thread if true - self._cond = threading.Condition() # used for pausing/resuming the thread - - # restricts concurrent write access to attributes used to draw the display - # and pausing: - # msgLog, loggedEvents, regexFilter, scroll, _pauseBuffer - self.valsLock = threading.RLock() - - # cached parameters (invalidated if arguments for them change) - # last set of events we've drawn with - self._lastLoggedEvents = [] - - # _getTitle (args: loggedEvents, regexFilter pattern, width) - self._titleCache = None - self._titleArgs = (None, None, None) - - # fetches past tor events from log file, if available - torEventBacklog = [] - if self._config["features.log.prepopulate"]: - setRunlevels = list(set.intersection(set(self.loggedEvents), set(log.Runlevel.values()))) - readLimit = self._config["features.log.prepopulateReadLimit"] - addLimit = self._config["cache.logPanel.size"] - torEventBacklog = getLogFileEntries(setRunlevels, readLimit, addLimit, self._config) - - # adds arm listener and fetches past events - log.LOG_LOCK.acquire() - try: - armRunlevels = [log.DEBUG, log.INFO, log.NOTICE, log.WARN, log.ERR] - log.addListeners(armRunlevels, self._registerArmEvent) - - # gets the set of arm events we're logging - setRunlevels = [] - for i in range(len(armRunlevels)): - if "ARM_" + log.Runlevel.values()[i] in self.loggedEvents: - setRunlevels.append(armRunlevels[i]) - - armEventBacklog = [] - for level, msg, eventTime in log._getEntries(setRunlevels): - armEventEntry = LogEntry(eventTime, "ARM_" + level, msg, RUNLEVEL_EVENT_COLOR[level]) - armEventBacklog.insert(0, armEventEntry) - - # joins armEventBacklog and torEventBacklog chronologically into msgLog - while armEventBacklog or torEventBacklog: - if not armEventBacklog: - self.msgLog.append(torEventBacklog.pop(0)) - elif not torEventBacklog: - self.msgLog.append(armEventBacklog.pop(0)) - elif armEventBacklog[0].timestamp < torEventBacklog[0].timestamp: - self.msgLog.append(torEventBacklog.pop(0)) - else: - self.msgLog.append(armEventBacklog.pop(0)) - finally: - log.LOG_LOCK.release() - - # crops events that are either too old, or more numerous than the caching size - self._trimEvents(self.msgLog) - - # leaving lastContentHeight as being too low causes initialization problems - self.lastContentHeight = len(self.msgLog) - - # adds listeners for tor and torctl events - conn = torTools.getConn() - conn.addEventListener(TorEventObserver(self.registerEvent)) - conn.addTorCtlListener(self._registerTorCtlEvent) - - # opens log file if we'll be saving entries - if self._config["features.logFile"]: - logPath = self._config["features.logFile"] - - try: - # make dir if the path doesn't already exist - baseDir = os.path.dirname(logPath) - if not os.path.exists(baseDir): os.makedirs(baseDir) - - self.logFile = open(logPath, "a") - log.log(self._config["log.logPanel.logFileOpened"], "arm %s opening log file (%s)" % (VERSION, logPath)) - except (IOError, OSError), exc: - log.log(self._config["log.logPanel.logFileWriteFailed"], "Unable to write to log file: %s" % sysTools.getFileErrorMsg(exc)) - self.logFile = None - - def registerEvent(self, event): - """ - Notes event and redraws log. If paused it's held in a temporary buffer. - - Arguments: - event - LogEntry for the event that occurred - """ - - if not event.type in self.loggedEvents: return - - # strips control characters to avoid screwing up the terminal - event.msg = uiTools.getPrintable(event.msg) - - # note event in the log file if we're saving them - if self.logFile: - try: - self.logFile.write(event.getDisplayMessage(True) + "\n") - self.logFile.flush() - except IOError, exc: - log.log(self._config["log.logPanel.logFileWriteFailed"], "Unable to write to log file: %s" % sysTools.getFileErrorMsg(exc)) - self.logFile = None - - if self._isPaused: - self.valsLock.acquire() - self._pauseBuffer.insert(0, event) - self._trimEvents(self._pauseBuffer) - self.valsLock.release() - else: - self.valsLock.acquire() - self.msgLog.insert(0, event) - self._trimEvents(self.msgLog) - - # notifies the display that it has new content - if not self.regexFilter or self.regexFilter.search(event.getDisplayMessage()): - self._cond.acquire() - self._cond.notifyAll() - self._cond.release() - - self.valsLock.release() - - def _registerArmEvent(self, level, msg, eventTime): - eventColor = RUNLEVEL_EVENT_COLOR[level] - self.registerEvent(LogEntry(eventTime, "ARM_%s" % level, msg, eventColor)) - - def _registerTorCtlEvent(self, level, msg): - eventColor = RUNLEVEL_EVENT_COLOR[level] - self.registerEvent(LogEntry(time.time(), "TORCTL_%s" % level, msg, eventColor)) - - def setLoggedEvents(self, eventTypes): - """ - Sets the event types recognized by the panel. - - Arguments: - eventTypes - event types to be logged - """ - - if eventTypes == self.loggedEvents: return - - self.valsLock.acquire() - self.loggedEvents = eventTypes - self.redraw(True) - self.valsLock.release() - - def setFilter(self, logFilter): - """ - Filters log entries according to the given regular expression. - - Arguments: - logFilter - regular expression used to determine which messages are - shown, None if no filter should be applied - """ - - if logFilter == self.regexFilter: return - - self.valsLock.acquire() - self.regexFilter = logFilter - self.redraw(True) - self.valsLock.release() - - def clear(self): - """ - Clears the contents of the event log. - """ - - self.valsLock.acquire() - self.msgLog = [] - self.redraw(True) - self.valsLock.release() - - def saveSnapshot(self, path): - """ - Saves the log events currently being displayed to the given path. This - takes filers into account. This overwrites the file if it already exists, - and raises an IOError if there's a problem. - - Arguments: - path - path where to save the log snapshot - """ - - # make dir if the path doesn't already exist - baseDir = os.path.dirname(path) - if not os.path.exists(baseDir): os.makedirs(baseDir) - - snapshotFile = open(path, "w") - self.valsLock.acquire() - try: - for entry in self.msgLog: - isVisible = not self.regexFilter or self.regexFilter.search(entry.getDisplayMessage()) - if isVisible: snapshotFile.write(entry.getDisplayMessage(True) + "\n") - - self.valsLock.release() - except Exception, exc: - self.valsLock.release() - raise exc - - def handleKey(self, key): - if uiTools.isScrollKey(key): - pageHeight = self.getPreferredSize()[0] - 1 - newScroll = uiTools.getScrollPosition(key, self.scroll, pageHeight, self.lastContentHeight) - - if self.scroll != newScroll: - self.valsLock.acquire() - self.scroll = newScroll - self.redraw(True) - self.valsLock.release() - elif key in (ord('u'), ord('U')): - self.valsLock.acquire() - self.showDuplicates = not self.showDuplicates - self.redraw(True) - self.valsLock.release() - - def setPaused(self, isPause): - """ - If true, prevents message log from being updated with new events. - """ - - if isPause == self._isPaused: return - - self._isPaused = isPause - if self._isPaused: self._pauseBuffer = [] - else: - self.valsLock.acquire() - self.msgLog = (self._pauseBuffer + self.msgLog)[:self._config["cache.logPanel.size"]] - self.redraw(True) - self.valsLock.release() - - def draw(self, width, height): - """ - Redraws message log. Entries stretch to use available space and may - contain up to two lines. Starts with newest entries. - """ - - self.valsLock.acquire() - self._lastLoggedEvents, self._lastUpdate = list(self.msgLog), time.time() - - # draws the top label - self.addstr(0, 0, self._getTitle(width), curses.A_STANDOUT) - - # restricts scroll location to valid bounds - self.scroll = max(0, min(self.scroll, self.lastContentHeight - height + 1)) - - # draws left-hand scroll bar if content's longer than the height - msgIndent, dividerIndent = 1, 0 # offsets for scroll bar - isScrollBarVisible = self.lastContentHeight > height - 1 - if isScrollBarVisible: - msgIndent, dividerIndent = 3, 2 - self.addScrollBar(self.scroll, self.scroll + height - 1, self.lastContentHeight, 1) - - # draws log entries - lineCount = 1 - self.scroll - seenFirstDateDivider = False - dividerAttr, duplicateAttr = curses.A_BOLD | uiTools.getColor("yellow"), curses.A_BOLD | uiTools.getColor("green") - - isDatesShown = self.regexFilter == None and self._config["features.log.showDateDividers"] - eventLog = getDaybreaks(self.msgLog, self._isPaused) if isDatesShown else list(self.msgLog) - if not self.showDuplicates: - deduplicatedLog = getDuplicates(eventLog) - - if deduplicatedLog == None: - msg = "Deduplication took too long. Its current implementation has difficulty handling large logs so disabling it to keep the interface responsive." - log.log(log.WARN, msg) - self.showDuplicates = True - deduplicatedLog = [(entry, 0) for entry in eventLog] - else: deduplicatedLog = [(entry, 0) for entry in eventLog] - - # determines if we have the minimum width to show date dividers - showDaybreaks = width - dividerIndent >= 3 - - while deduplicatedLog: - entry, duplicateCount = deduplicatedLog.pop(0) - - if self.regexFilter and not self.regexFilter.search(entry.getDisplayMessage()): - continue # filter doesn't match log message - skip - - # checks if we should be showing a divider with the date - if entry.type == DAYBREAK_EVENT: - # bottom of the divider - if seenFirstDateDivider: - if lineCount >= 1 and lineCount < height and showDaybreaks: - self.addch(lineCount, dividerIndent, curses.ACS_LLCORNER, dividerAttr) - self.hline(lineCount, dividerIndent + 1, width - dividerIndent - 1, dividerAttr) - self.addch(lineCount, width, curses.ACS_LRCORNER, dividerAttr) - - lineCount += 1 - - # top of the divider - if lineCount >= 1 and lineCount < height and showDaybreaks: - timeLabel = time.strftime(" %B %d, %Y ", time.localtime(entry.timestamp)) - self.addch(lineCount, dividerIndent, curses.ACS_ULCORNER, dividerAttr) - self.addch(lineCount, dividerIndent + 1, curses.ACS_HLINE, dividerAttr) - self.addstr(lineCount, dividerIndent + 2, timeLabel, curses.A_BOLD | dividerAttr) - - lineLength = width - dividerIndent - len(timeLabel) - 2 - self.hline(lineCount, dividerIndent + len(timeLabel) + 2, lineLength, dividerAttr) - self.addch(lineCount, dividerIndent + len(timeLabel) + 2 + lineLength, curses.ACS_URCORNER, dividerAttr) - - seenFirstDateDivider = True - lineCount += 1 - else: - # entry contents to be displayed, tuples of the form: - # (msg, formatting, includeLinebreak) - displayQueue = [] - - msgComp = entry.getDisplayMessage().split("\n") - for i in range(len(msgComp)): - font = curses.A_BOLD if "ERR" in entry.type else curses.A_NORMAL # emphasizes ERR messages - displayQueue.append((msgComp[i].strip(), font | uiTools.getColor(entry.color), i != len(msgComp) - 1)) - - if duplicateCount: - pluralLabel = "s" if duplicateCount > 1 else "" - duplicateMsg = DUPLICATE_MSG % (duplicateCount, pluralLabel) - displayQueue.append((duplicateMsg, duplicateAttr, False)) - - cursorLoc, lineOffset = msgIndent, 0 - maxEntriesPerLine = self._config["features.log.maxLinesPerEntry"] - while displayQueue: - msg, format, includeBreak = displayQueue.pop(0) - drawLine = lineCount + lineOffset - if lineOffset == maxEntriesPerLine: break - - maxMsgSize = width - cursorLoc - if len(msg) > maxMsgSize: - # message is too long - break it up - if lineOffset == maxEntriesPerLine - 1: - msg = uiTools.cropStr(msg, maxMsgSize) - else: - msg, remainder = uiTools.cropStr(msg, maxMsgSize, 4, 4, uiTools.Ending.HYPHEN, True) - displayQueue.insert(0, (remainder.strip(), format, includeBreak)) - - includeBreak = True - - if drawLine < height and drawLine >= 1: - if seenFirstDateDivider and width - dividerIndent >= 3 and showDaybreaks: - self.addch(drawLine, dividerIndent, curses.ACS_VLINE, dividerAttr) - self.addch(drawLine, width, curses.ACS_VLINE, dividerAttr) - - self.addstr(drawLine, cursorLoc, msg, format) - - cursorLoc += len(msg) - - if includeBreak or not displayQueue: - lineOffset += 1 - cursorLoc = msgIndent + ENTRY_INDENT - - lineCount += lineOffset - - # if this is the last line and there's room, then draw the bottom of the divider - if not deduplicatedLog and seenFirstDateDivider: - if lineCount < height and showDaybreaks: - self.addch(lineCount, dividerIndent, curses.ACS_LLCORNER, dividerAttr) - self.hline(lineCount, dividerIndent + 1, width - dividerIndent - 1, dividerAttr) - self.addch(lineCount, width, curses.ACS_LRCORNER, dividerAttr) - - lineCount += 1 - - # redraw the display if... - # - lastContentHeight was off by too much - # - we're off the bottom of the page - newContentHeight = lineCount + self.scroll - 1 - contentHeightDelta = abs(self.lastContentHeight - newContentHeight) - forceRedraw, forceRedrawReason = True, "" - - if contentHeightDelta >= CONTENT_HEIGHT_REDRAW_THRESHOLD: - forceRedrawReason = "estimate was off by %i" % contentHeightDelta - elif newContentHeight > height and self.scroll + height - 1 > newContentHeight: - forceRedrawReason = "scrolled off the bottom of the page" - elif not isScrollBarVisible and newContentHeight > height - 1: - forceRedrawReason = "scroll bar wasn't previously visible" - elif isScrollBarVisible and newContentHeight <= height - 1: - forceRedrawReason = "scroll bar shouldn't be visible" - else: forceRedraw = False - - self.lastContentHeight = newContentHeight - if forceRedraw: - forceRedrawReason = "redrawing the log panel with the corrected content height (%s)" % forceRedrawReason - log.log(self._config["log.logPanel.forceDoubleRedraw"], forceRedrawReason) - self.redraw(True) - - self.valsLock.release() - - def redraw(self, forceRedraw=False, block=False): - # determines if the content needs to be redrawn or not - panel.Panel.redraw(self, forceRedraw, block) - - def run(self): - """ - Redraws the display, coalescing updates if events are rapidly logged (for - instance running at the DEBUG runlevel) while also being immediately - responsive if additions are less frequent. - """ - - lastDay = daysSince() # used to determine if the date has changed - while not self._halt: - currentDay = daysSince() - timeSinceReset = time.time() - self._lastUpdate - maxLogUpdateRate = self._config["features.log.maxRefreshRate"] / 1000.0 - - sleepTime = 0 - if (self.msgLog == self._lastLoggedEvents and lastDay == currentDay) or self._isPaused: - sleepTime = 5 - elif timeSinceReset < maxLogUpdateRate: - sleepTime = max(0.05, maxLogUpdateRate - timeSinceReset) - - if sleepTime: - self._cond.acquire() - if not self._halt: self._cond.wait(sleepTime) - self._cond.release() - else: - lastDay = currentDay - self.redraw(True) - - def stop(self): - """ - Halts further resolutions and terminates the thread. - """ - - self._cond.acquire() - self._halt = True - self._cond.notifyAll() - self._cond.release() - - def _getTitle(self, width): - """ - Provides the label used for the panel, looking like: - Events (ARM NOTICE - ERR, BW - filter: prepopulate): - - This truncates the attributes (with an ellipse) if too long, and condenses - runlevel ranges if there's three or more in a row (for instance ARM_INFO, - ARM_NOTICE, and ARM_WARN becomes "ARM_INFO - WARN"). - - Arguments: - width - width constraint the label needs to fix in - """ - - # usually the attributes used to make the label are decently static, so - # provide cached results if they're unchanged - self.valsLock.acquire() - currentPattern = self.regexFilter.pattern if self.regexFilter else None - isUnchanged = self._titleArgs[0] == self.loggedEvents - isUnchanged &= self._titleArgs[1] == currentPattern - isUnchanged &= self._titleArgs[2] == width - if isUnchanged: - self.valsLock.release() - return self._titleCache - - eventsList = list(self.loggedEvents) - if not eventsList: - if not currentPattern: - panelLabel = "Events:" - else: - labelPattern = uiTools.cropStr(currentPattern, width - 18) - panelLabel = "Events (filter: %s):" % labelPattern - else: - # does the following with all runlevel types (tor, arm, and torctl): - # - pulls to the start of the list - # - condenses range if there's three or more in a row (ex. "ARM_INFO - WARN") - # - condense further if there's identical runlevel ranges for multiple - # types (ex. "NOTICE - ERR, ARM_NOTICE - ERR" becomes "TOR/ARM NOTICE - ERR") - tmpRunlevels = [] # runlevels pulled from the list (just the runlevel part) - runlevelRanges = [] # tuple of type, startLevel, endLevel for ranges to be consensed - - # reverses runlevels and types so they're appended in the right order - reversedRunlevels = log.Runlevel.values() - reversedRunlevels.reverse() - for prefix in ("TORCTL_", "ARM_", ""): - # blank ending runlevel forces the break condition to be reached at the end - for runlevel in reversedRunlevels + [""]: - eventType = prefix + runlevel - if runlevel and eventType in eventsList: - # runlevel event found, move to the tmp list - eventsList.remove(eventType) - tmpRunlevels.append(runlevel) - elif tmpRunlevels: - # adds all tmp list entries to the start of eventsList - if len(tmpRunlevels) >= 3: - # save condense sequential runlevels to be added later - runlevelRanges.append((prefix, tmpRunlevels[-1], tmpRunlevels[0])) - else: - # adds runlevels individaully - for tmpRunlevel in tmpRunlevels: - eventsList.insert(0, prefix + tmpRunlevel) - - tmpRunlevels = [] - - # adds runlevel ranges, condensing if there's identical ranges - for i in range(len(runlevelRanges)): - if runlevelRanges[i]: - prefix, startLevel, endLevel = runlevelRanges[i] - - # check for matching ranges - matches = [] - for j in range(i + 1, len(runlevelRanges)): - if runlevelRanges[j] and runlevelRanges[j][1] == startLevel and runlevelRanges[j][2] == endLevel: - matches.append(runlevelRanges[j]) - runlevelRanges[j] = None - - if matches: - # strips underscores and replaces empty entries with "TOR" - prefixes = [entry[0] for entry in matches] + [prefix] - for k in range(len(prefixes)): - if prefixes[k] == "": prefixes[k] = "TOR" - else: prefixes[k] = prefixes[k].replace("_", "") - - eventsList.insert(0, "%s %s - %s" % ("/".join(prefixes), startLevel, endLevel)) - else: - eventsList.insert(0, "%s%s - %s" % (prefix, startLevel, endLevel)) - - # truncates to use an ellipsis if too long, for instance: - attrLabel = ", ".join(eventsList) - if currentPattern: attrLabel += " - filter: %s" % currentPattern - attrLabel = uiTools.cropStr(attrLabel, width - 10, 1) - if attrLabel: attrLabel = " (%s)" % attrLabel - panelLabel = "Events%s:" % attrLabel - - # cache results and return - self._titleCache = panelLabel - self._titleArgs = (list(self.loggedEvents), currentPattern, width) - self.valsLock.release() - return panelLabel - - def _trimEvents(self, eventListing): - """ - Crops events that have either: - - grown beyond the cache limit - - outlived the configured log duration - - Argument: - eventListing - listing of log entries - """ - - cacheSize = self._config["cache.logPanel.size"] - if len(eventListing) > cacheSize: del eventListing[cacheSize:] - - logTTL = self._config["features.log.entryDuration"] - if logTTL > 0: - currentDay = daysSince() - - breakpoint = None # index at which to crop from - for i in range(len(eventListing) - 1, -1, -1): - daysSinceEvent = currentDay - daysSince(eventListing[i].timestamp) - if daysSinceEvent > logTTL: breakpoint = i # older than the ttl - else: break - - # removes entries older than the ttl - if breakpoint != None: del eventListing[breakpoint:] - diff --git a/src/interface/torrcPanel.py b/src/interface/torrcPanel.py deleted file mode 100644 index b7cad86..0000000 --- a/src/interface/torrcPanel.py +++ /dev/null @@ -1,221 +0,0 @@ -""" -Panel displaying the torrc or armrc with the validation done against it. -""" - -import math -import curses -import threading - -from util import conf, enum, panel, torConfig, uiTools - -DEFAULT_CONFIG = {"features.config.file.showScrollbars": True, - "features.config.file.maxLinesPerEntry": 8} - -# TODO: The armrc use case is incomplete. There should be equivilant reloading -# and validation capabilities to the torrc. -Config = enum.Enum("TORRC", "ARMRC") # configuration file types that can be displayed - -class TorrcPanel(panel.Panel): - """ - Renders the current torrc or armrc with syntax highlighting in a scrollable - area. - """ - - def __init__(self, stdscr, configType, config=None): - panel.Panel.__init__(self, stdscr, "configFile", 0) - - self._config = dict(DEFAULT_CONFIG) - if config: - config.update(self._config, {"features.config.file.maxLinesPerEntry": 1}) - - self.valsLock = threading.RLock() - self.configType = configType - self.scroll = 0 - self.showLabel = True # shows top label (hides otherwise) - self.showLineNum = True # shows left aligned line numbers - self.stripComments = False # drops comments and extra whitespace - - # height of the content when last rendered (the cached value is invalid if - # _lastContentHeightArgs is None or differs from the current dimensions) - self._lastContentHeight = 1 - self._lastContentHeightArgs = None - - def handleKey(self, key): - self.valsLock.acquire() - if uiTools.isScrollKey(key): - pageHeight = self.getPreferredSize()[0] - 1 - newScroll = uiTools.getScrollPosition(key, self.scroll, pageHeight, self._lastContentHeight) - - if self.scroll != newScroll: - self.scroll = newScroll - self.redraw(True) - elif key == ord('n') or key == ord('N'): - self.showLineNum = not self.showLineNum - self._lastContentHeightArgs = None - self.redraw(True) - elif key == ord('s') or key == ord('S'): - self.stripComments = not self.stripComments - self._lastContentHeightArgs = None - self.redraw(True) - - self.valsLock.release() - - def draw(self, width, height): - self.valsLock.acquire() - - # If true, we assume that the cached value in self._lastContentHeight is - # still accurate, and stop drawing when there's nothing more to display. - # Otherwise the self._lastContentHeight is suspect, and we'll process all - # the content to check if it's right (and redraw again with the corrected - # height if not). - trustLastContentHeight = self._lastContentHeightArgs == (width, height) - - # restricts scroll location to valid bounds - self.scroll = max(0, min(self.scroll, self._lastContentHeight - height + 1)) - - renderedContents, corrections, confLocation = None, {}, None - if self.configType == Config.TORRC: - loadedTorrc = torConfig.getTorrc() - loadedTorrc.getLock().acquire() - confLocation = loadedTorrc.getConfigLocation() - - if not loadedTorrc.isLoaded(): - renderedContents = ["### Unable to load the torrc ###"] - else: - renderedContents = loadedTorrc.getDisplayContents(self.stripComments) - - # constructs a mapping of line numbers to the issue on it - corrections = dict((lineNum, (issue, msg)) for lineNum, issue, msg in loadedTorrc.getCorrections()) - - loadedTorrc.getLock().release() - else: - loadedArmrc = conf.getConfig("arm") - confLocation = loadedArmrc.path - renderedContents = list(loadedArmrc.rawContents) - - # offset to make room for the line numbers - lineNumOffset = 0 - if self.showLineNum: - if len(renderedContents) == 0: lineNumOffset = 2 - else: lineNumOffset = int(math.log10(len(renderedContents))) + 2 - - # draws left-hand scroll bar if content's longer than the height - scrollOffset = 0 - if self._config["features.config.file.showScrollbars"] and self._lastContentHeight > height - 1: - scrollOffset = 3 - self.addScrollBar(self.scroll, self.scroll + height - 1, self._lastContentHeight, 1) - - displayLine = -self.scroll + 1 # line we're drawing on - - # draws the top label - if self.showLabel: - sourceLabel = "Tor" if self.configType == Config.TORRC else "Arm" - locationLabel = " (%s)" % confLocation if confLocation else "" - self.addstr(0, 0, "%s Configuration File%s:" % (sourceLabel, locationLabel), curses.A_STANDOUT) - - isMultiline = False # true if we're in the middle of a multiline torrc entry - for lineNumber in range(0, len(renderedContents)): - lineText = renderedContents[lineNumber] - lineText = lineText.rstrip() # remove ending whitespace - - # blank lines are hidden when stripping comments - if self.stripComments and not lineText: continue - - # splits the line into its component (msg, format) tuples - lineComp = {"option": ["", curses.A_BOLD | uiTools.getColor("green")], - "argument": ["", curses.A_BOLD | uiTools.getColor("cyan")], - "correction": ["", curses.A_BOLD | uiTools.getColor("cyan")], - "comment": ["", uiTools.getColor("white")]} - - # parses the comment - commentIndex = lineText.find("#") - if commentIndex != -1: - lineComp["comment"][0] = lineText[commentIndex:] - lineText = lineText[:commentIndex] - - # splits the option and argument, preserving any whitespace around them - strippedLine = lineText.strip() - optionIndex = strippedLine.find(" ") - if isMultiline: - # part of a multiline entry started on a previous line so everything - # is part of the argument - lineComp["argument"][0] = lineText - elif optionIndex == -1: - # no argument provided - lineComp["option"][0] = lineText - else: - optionText = strippedLine[:optionIndex] - optionEnd = lineText.find(optionText) + len(optionText) - lineComp["option"][0] = lineText[:optionEnd] - lineComp["argument"][0] = lineText[optionEnd:] - - # flags following lines as belonging to this multiline entry if it ends - # with a slash - if strippedLine: isMultiline = strippedLine.endswith("\\") - - # gets the correction - if lineNumber in corrections: - lineIssue, lineIssueMsg = corrections[lineNumber] - - if lineIssue in (torConfig.ValidationError.DUPLICATE, torConfig.ValidationError.IS_DEFAULT): - lineComp["option"][1] = curses.A_BOLD | uiTools.getColor("blue") - lineComp["argument"][1] = curses.A_BOLD | uiTools.getColor("blue") - elif lineIssue == torConfig.ValidationError.MISMATCH: - lineComp["argument"][1] = curses.A_BOLD | uiTools.getColor("red") - lineComp["correction"][0] = " (%s)" % lineIssueMsg - else: - # For some types of configs the correction field is simply used to - # provide extra data (for instance, the type for tor state fields). - lineComp["correction"][0] = " (%s)" % lineIssueMsg - lineComp["correction"][1] = curses.A_BOLD | uiTools.getColor("magenta") - - # draws the line number - if self.showLineNum and displayLine < height and displayLine >= 1: - lineNumStr = ("%%%ii" % (lineNumOffset - 1)) % (lineNumber + 1) - self.addstr(displayLine, scrollOffset, lineNumStr, curses.A_BOLD | uiTools.getColor("yellow")) - - # draws the rest of the components with line wrap - cursorLoc, lineOffset = lineNumOffset + scrollOffset, 0 - maxLinesPerEntry = self._config["features.config.file.maxLinesPerEntry"] - displayQueue = [lineComp[entry] for entry in ("option", "argument", "correction", "comment")] - - while displayQueue: - msg, format = displayQueue.pop(0) - - maxMsgSize, includeBreak = width - cursorLoc, False - if len(msg) >= maxMsgSize: - # message is too long - break it up - if lineOffset == maxLinesPerEntry - 1: - msg = uiTools.cropStr(msg, maxMsgSize) - else: - includeBreak = True - msg, remainder = uiTools.cropStr(msg, maxMsgSize, 4, 4, uiTools.Ending.HYPHEN, True) - displayQueue.insert(0, (remainder.strip(), format)) - - drawLine = displayLine + lineOffset - if msg and drawLine < height and drawLine >= 1: - self.addstr(drawLine, cursorLoc, msg, format) - - # If we're done, and have added content to this line, then start - # further content on the next line. - cursorLoc += len(msg) - includeBreak |= not displayQueue and cursorLoc != lineNumOffset + scrollOffset - - if includeBreak: - lineOffset += 1 - cursorLoc = lineNumOffset + scrollOffset - - displayLine += max(lineOffset, 1) - - if trustLastContentHeight and displayLine >= height: break - - if not trustLastContentHeight: - self._lastContentHeightArgs = (width, height) - newContentHeight = displayLine + self.scroll - 1 - - if self._lastContentHeight != newContentHeight: - self._lastContentHeight = newContentHeight - self.redraw(True) - - self.valsLock.release() - diff --git a/src/starter.py b/src/starter.py index 09fc37a..c0a0270 100644 --- a/src/starter.py +++ b/src/starter.py @@ -14,8 +14,8 @@ import socket import platform import version -import interface.controller -import interface.logPanel +import cli.controller +import cli.logPanel import util.conf import util.connections import util.hostnames @@ -67,7 +67,7 @@ Terminal status monitor for Tor relays. Example: arm -b -i 1643 hide connection data, attaching to control port 1643 arm -e we -c /tmp/cfg use this configuration file with 'WARN'/'ERR' events -""" % (CONFIG["startup.interface.ipAddress"], CONFIG["startup.interface.port"], DEFAULT_CONFIG, LOG_DUMP_PATH, CONFIG["startup.events"], interface.logPanel.EVENT_LISTING) +""" % (CONFIG["startup.interface.ipAddress"], CONFIG["startup.interface.port"], DEFAULT_CONFIG, LOG_DUMP_PATH, CONFIG["startup.events"], cli.logPanel.EVENT_LISTING) # filename used for cached tor config descriptions CONFIG_DESC_FILENAME = "torConfigDesc.txt" @@ -312,7 +312,7 @@ if __name__ == '__main__': # validates and expands log event flags try: - expandedEvents = interface.logPanel.expandEvents(param["startup.events"]) + expandedEvents = cli.logPanel.expandEvents(param["startup.events"]) except ValueError, exc: for flag in str(exc): print "Unrecognized event flag: %s" % flag @@ -387,5 +387,5 @@ if __name__ == '__main__': util.log.log(CONFIG["log.savingDebugLog"], "Saving a debug log to '%s' (please check it for sensitive information before sharing)" % LOG_DUMP_PATH) _dumpConfig() - interface.controller.startTorMonitor(time.time() - initTime, expandedEvents, param["startup.blindModeEnabled"]) + cli.controller.startTorMonitor(time.time() - initTime, expandedEvents, param["startup.blindModeEnabled"])

1 0

r24680: {projects} finish light mucking. still 5 pages. i suggest a complete re (projects/articles/browser-privacy)
by Roger Dingledine 27 Apr '11

27 Apr '11

Author: arma Date: 2011-04-27 01:30:49 +0000 (Wed, 27 Apr 2011) New Revision: 24680 Modified: projects/articles/browser-privacy/W3CIdentity.tex Log: finish light mucking. still 5 pages. i suggest a complete rewrite, first by picking several key points you want to communicate, and then putting those points in the abstract, and then expanding on them. (topics you plan to talk about in the paper do not count as points.) Modified: projects/articles/browser-privacy/W3CIdentity.tex =================================================================== --- projects/articles/browser-privacy/W3CIdentity.tex 2011-04-27 01:08:34 UTC (rev 24679) +++ projects/articles/browser-privacy/W3CIdentity.tex 2011-04-27 01:30:49 UTC (rev 24680) @@ -97,11 +97,9 @@ two major sources of disconnect\footnotemark. The first direction is improving the linkability issues inherent with the multi-origin model of the web itself. The second direction is improving user cues and browser interface to suggest a -coherent concept of identity to the user by accurately reflecting the +coherent concept of identity to users by accurately reflecting the set of unique identifiers they have accumulated. Both of these directions can be pursued independently. -% "the user" is not the thing that "accurately reflects". 'which' is a -% klunky term here. -RD \footnotetext{We only consider implementations that involve privacy-by-design. Privacy-by-policy approaches such as Do Not Track will not be discussed.} @@ -142,7 +140,7 @@ pages. The default experience is such that all of this data exchange is concealed from the user. -So then what comprises the user's web identity for tracking purposes? In terms +So then what comprises a user's web identity for tracking purposes? In terms of authentication, it would at first appear to be limited to cookies, HTTP Auth tokens, and client TLS certificates. However, this identifier-based approach breaks down quickly on the modern web. High-security websites are @@ -151,15 +149,14 @@ utilize everything they can to build complete portraits of users' identities~\cite{tracking-identity}. -Despite what the user may believe, their actual web identity then is a +Despite what the user may believe, her actual web identity then is a superset of all the stored identifiers and authentication tokens used by the -browser. It is the ability to link a user's activity in one instance to their +browser. This identity can link a user's activity in one instance to her activity in another instance, be it across time, or even on the very same page due to multiple content origins. -% 'It'? Their web identity is the ability to link...? -RD -Therefore, instead of viewing the user's identity as the sum of their -identifiers, or as their relationship to individual websites, it is best to +Therefore, instead of viewing the user's identity as the sum of her +identifiers, or as her relationship to individual websites, it is best to view it as the ability to link activity from one website to activity in another website. We will call this property ``user linkability''. @@ -231,12 +228,12 @@ %It is apparent that Users experience disconnects with the technical realities of the web on two major fronts: the average user does not grasp the -privacy implications of the multi-origin model, nor are they given a clear +privacy implications of the multi-origin model, nor is she given a clear concept of browser identity to grasp the privacy implications of the union -of the linkable components of their browsers. +of the linkable components of her browser. We will now examine examples of attempts at reducing this disconnect on each -of these two fronts. Note that these to fronts are orthogonal. Approaches from +of these two fronts. Note that these two fronts are orthogonal. Approaches from them may be combined, or used independently. \subsection{Improving the Origin Model} @@ -260,7 +257,8 @@ cache to the top-level origin in the URL bar. Commonly sourced content elements are then fetched and cached repeatedly, but this is necessary to prevent linkability: each of these content elements can be -crafted to include an identifier unique to each user, thus tracking users who +crafted to include an identifier unique to each user, thus tracking even +users who attempt to avoid tracking by clearing normal cookies. The Mozilla development wiki describes an origin model improvement for @@ -285,7 +283,7 @@ of that top level origin to represent all of the browser state accumulated by that origin. The user could delete the entire set of browser state (cookies, cache, storage, cryptographic tokens) associated with a site simply by -removing its favicon from their privacy info panel. +removing its favicon from her privacy info panel. The problem with origin model improvement approaches is that individually, they do not fully address the linkability problem unless the same @@ -304,15 +302,16 @@ So, while these approaches are in fact useful for bringing the technical realities of the web closer to what the user assumes is happening, they must be deployed uniformly, with a consistent top-level origin restriction model. -This may take significant coordination and standardization efforts. Without -this, it is necessary to fill the remaining linkability gaps by presenting -the user with a visual representation of their overall web identity. +Uniform deployment will take significant coordination and standardization +efforts. Until then, +it is necessary to fill the remaining linkability gaps by presenting +the user with a visual representation of her overall web identity. \subsection{Conveying Identity to the User} Even if the origin model of identifier transmission and other linkable -attributes is altered uniformly to be more in line with what users expect, it -is likely that the average user will still experience privacy benefits if the +attributes is altered uniformly to be more in line with what users expect, +the average user can still experience privacy benefits if the browser conveys the sum of all linkable information as a single storable, mutable, and clearable user identity. % @@ -346,21 +345,24 @@ be integrated with the browsing experience. Users should be able to click a button to get a clean slate for a new identity, and should be able to log into and out of password-protected stored identities, which would -contain the entire state of the browser. The Tor Project is heading in -this direction with the Tor Browser Bundle~\cite{not-to-toggle}. +contain the entire state of the browser. % To this user, the Private Browsing Mode would be no more than a special case of this identity UI---a special identity that they can trust not to store browsing history information to disk. Such a UI also more explicitly captures what is going on with respect to the user's relationship to the web. +The Tor Project is heading in this direction with the Tor Browser +Bundle~\cite{not-to-toggle}. -However, all current private browsing modes fall short of protecting against a +Unfortunately, all current private browsing modes fall short of protecting +against a network-level adversary and fail to deal with linkability against such an adversary~\cite{private-browsing}, claiming that it is outside their threat model\footnotemark. If the user is given a new identity that is still linkable to the previous one due to shortcomings of the browser, this approach has failed as a privacy measure. % XXXX Define network-level adversary. +% I agree. You could do that here or in the footnote. -RD \footnotetext{The primary reason given to abstain from addressing a network-level @@ -382,16 +384,15 @@ associated with altering browser behavior have lulled us into accepting user deception as the norm for web use. The average user completely lacks the understanding needed to grasp how web tracking is carried out. This disconnect -%in understanding -is extreme to the point where moral issues arise about the +is so extreme that it raises moral issues about the level of consent actually involved in web use and associated tracking. In fact, standardization efforts seemed to realize this problem early on but -failed to create a feasible recommendations for improving the situation. RFC -2965 governing HTTP State Management mandated in section 3.3.6 that +failed to create feasible recommendations for improving the situation. +RFC~2965 governing HTTP State Management mandated in Section~3.3.6 that third-party origins must not cause the browser to transmit cookies unless the interaction is ``verifiable'' and readily apparent to the user~\cite{rfc2965}. -In section 6, it also strongly suggested that informed consent and user +In Section~6, it also strongly suggested that informed consent and user control should govern the interaction of users to tracking identifiers. Without changes to browser behavior, browser interface, or both, such informed @@ -400,6 +401,7 @@ the linkability issues with the web's origin model with minimal breakage. Additionally, the first steps towards providing the user with an explicit representation of their web identity have been taken. +% Taken by who? Change to active tense. -RD The pieces are in place to build robust private browsing modes based on these two approaches, and metrics exist to measure their success.

1 0

r24679: {projects} initial cleanups while reading also, it's 5 pages again (projects/articles/browser-privacy)
by Roger Dingledine 27 Apr '11

27 Apr '11

Author: arma Date: 2011-04-27 01:08:34 +0000 (Wed, 27 Apr 2011) New Revision: 24679 Modified: projects/articles/browser-privacy/W3CIdentity.tex Log: initial cleanups while reading also, it's 5 pages again Modified: projects/articles/browser-privacy/W3CIdentity.tex =================================================================== --- projects/articles/browser-privacy/W3CIdentity.tex 2011-04-26 15:06:46 UTC (rev 24678) +++ projects/articles/browser-privacy/W3CIdentity.tex 2011-04-27 01:08:34 UTC (rev 24679) @@ -31,12 +31,14 @@ There is a huge disconnect between how users perceive their online presence and the reality of their relationship with the websites they visit. This position paper explores this disconnect and provides some recommendations for -making the technical reality of the web match user perception, through both -technical improvements as well as user interface cues. We frame the core -technical problem as one of ``linkability'' -- the level of correlation +making the technical reality of the web match user perception. %, through both +%technical improvements as well as user interface cues. +We frame the core +technical problem as one of ``linkability''---the level of correlation between various online activities that the user naturally expects to be independent. We look to address the issue of unexpected linkability through -both improvements to the web's origin model, as well as through user interface +both technical improvements to the web's origin model, as well as through +user interface cues about the set of accumulated identifiers that can be said to comprise a user's online identity. @@ -50,11 +52,12 @@ advertisement is relevant to the current activity, and if possible, relevant to the current user. -The cost of this is that user privacy on the web is a nightmare. There is +The cost of this incentive structure is that user privacy on the web is a +nightmare. There is ubiquitous tracking, unseen partnership agreements and data exchange, and surreptitious attempts to uncover users' identities against their will and without their knowledge. This is not just happening in the dark, unseemly -corners of the web. It is happening everywhere\cite{facebook-like}. +corners of the web. It is happening everywhere~\cite{facebook-like}. The problem is that the revenue model of the web has incentivized companies to find ways to continue to track users against their will, even if those users @@ -62,9 +65,9 @@ Starting with the infamous ``Flash cookies'', we have progressed through a seemingly endless arms race of secondary identifiers and tracking information: visited history, cache, font and system data, desktop resolution, keystroke -timing, and so on and so forth\cite{wsj-fingerprinting}. +timing, and so on and so forth~\cite{wsj-fingerprinting}. -These efforts have lead to an even wider disconnect between a user's +These efforts have led to an even wider disconnect between users' perception of their privacy and the reality of their privacy. Users simply can't keep up with the ways they are being tracked. @@ -87,23 +90,25 @@ To this end, the rest of this document is structured as follows: First, we -examine how the user perceives their privacy on the web, comparing the average +examine how users perceive their privacy on the web, comparing the average user's perspective to what actually is happening technically behind the -scenes, and noting the major disconnects. We then examine solutions attempting -to bridge this disconnect from two different directions, corresponding to the +scenes, and noting the major disconnects. We then examine solutions that +bridge this disconnect from two different directions, corresponding to the two major sources of disconnect\footnotemark. The first direction is improving the linkability issues inherent with the multi-origin model of the web itself. The second direction is improving user cues and browser interface to suggest a -coherent concept of identity to the user, which more accurately reflects the +coherent concept of identity to the user by accurately reflecting the set of unique identifiers they have accumulated. Both of these directions can be pursued independently. +% "the user" is not the thing that "accurately reflects". 'which' is a +% klunky term here. -RD \footnotetext{We only consider implementations that involve privacy-by-design. Privacy-by-policy approaches such as Do Not Track will not be discussed.} \section{User Privacy on the Web} -To properly examine the privacy problem, we must probe both the average user's +To properly examine the privacy problem, we must probe both the average users' perception of what their ``web identity'' is, as well as the technical realities of web authentication and tracking. @@ -111,12 +116,13 @@ Instinctively, users define their privacy in terms of their identity: in terms of how they have interacted with a site in order to inform it of who they are. -Typically, the user's perception of their identity on the web is usually a direct +Typically, the users' perception of their identity on the web is usually a direct function of the identifiers used for strong authentication for particular sites. For example, users expect that logging in to Facebook creates a relationship in their browsers when facebook.com is present in the URL bar, but they are -typically not aware that this also extends to their activity on other, arbitrary +typically not aware that this relationship also extends to their activity +on other, arbitrary sites that happen to include ``Like this on Facebook'' buttons or Facebook-sourced advertising content. @@ -141,15 +147,16 @@ Auth tokens, and client TLS certificates. However, this identifier-based approach breaks down quickly on the modern web. High-security websites are already using fingerprinting as an auxiliary second factor of -authentication\cite{security-fingerprinting}, and online data aggregators +authentication~\cite{security-fingerprinting}, and online data aggregators utilize everything they can to build complete portraits of users' -identities\cite{tracking-identity}. +identities~\cite{tracking-identity}. Despite what the user may believe, their actual web identity then is a superset of all the stored identifiers and authentication tokens used by the browser. It is the ability to link a user's activity in one instance to their activity in another instance, be it across time, or even on the very same page due to multiple content origins. +% 'It'? Their web identity is the ability to link...? -RD Therefore, instead of viewing the user's identity as the sum of their identifiers, or as their relationship to individual websites, it is best to @@ -159,12 +166,12 @@ \subsection{User Privacy as Linkability} In terms of what the user actually expects, user privacy is more accurately -modeled as the level of linkability between subsequent actions on the web, as -opposed to the mere sum of their unique identifiers and authentication tokens. +modeled as the level of linkability between subsequent actions on the web---not +just the sum of her unique identifiers and authentication tokens. When privacy is expanded to cover all items that enable or substantially contribute to linkability, a lot more components of the browser are now in -scope. We will briefly enumerate these components. +scope. We will briefly enumerate these four categories of components. First, the obvious properties are found in the state of the browser: cookies, DOM storage, cache, cryptographic tokens and cryptographic state, and @@ -192,7 +199,7 @@ various browser properties that extend beyond any stored origin-related state. The Panopticlick project by the EFF provides us with exactly this -metric\cite{panopticlick}. The researchers conducted a survey of volunteers +metric~\cite{panopticlick}. The researchers conducted a survey of volunteers who were asked to visit an experiment page that harvested many of the above components. They then computed the Shannon entropy of the resulting distribution of each of several key attributes to determine how many bits of @@ -201,7 +208,7 @@ While not perfect\footnotemark, this metric allows us to prioritize our efforts on the components that have the most potential for linkability. - +% \footnotetext{In particular, the test does not take in all aspects of resolution information. It did not calculate the size of widgets, window decoration, or toolbar size. We believe these resolution-related properties @@ -209,19 +216,20 @@ measure clock offset and other time-based fingerprints. Furthermore, as new browser features are added, the experiment should be repeated to include them.} - -This metric also indicates that it is beneficial to standardize on +% +It also shows the benefits of standardizing on implementations of fingerprinting resistance where possible. More implementations using the same defenses means more users with similar fingerprints, which means less entropy in the metric. Similarly, uniform -feature deployment leads to less entropy in the metric. +feature deployment leads to less entropy. \section{Matching User Perception with Reality} For users to have privacy, and for private browsing modes to function, the relationship between a user and a site must be understood by that user. - -It is apparent that the user experiences disconnect with the technical +% +%It is apparent that +Users experience disconnects with the technical realities of the web on two major fronts: the average user does not grasp the privacy implications of the multi-origin model, nor are they given a clear concept of browser identity to grasp the privacy implications of the union @@ -236,7 +244,7 @@ The current identifier origin model used by the web is fundamentally flawed when viewed from the perspective of meeting the expectations of the user. Unique, globally linkable identifiers can be transmitted for arbitrary content -elements on any page, which can be sourced from anywhere without user +elements on any page, and they can be sourced from anywhere without user interaction or awareness. However, the behavior of identifiers and linkable attributes can be improved @@ -246,26 +254,26 @@ content origin. Where linkability attributes exist, they should be obfuscated on a per-origin basis. -An early relevant example of this idea is SafeCache\cite{safecache}. +An early relevant example of this idea is SafeCache~\cite{safecache}. SafeCache seeks to reduce the ability for third-party content elements to use the cache to store identifiers. It does this by limiting the scope of the -cache to the top-level origin in the URL bar. This has the effect that -commonly sourced content elements are fetched and cached repeatedly, but this +cache to the top-level origin in the URL bar. Commonly sourced content +elements are then fetched and cached repeatedly, but this is necessary to prevent linkability: each of these content elements can be crafted to include an identifier unique to each user, thus tracking users who attempt to avoid tracking by clearing normal cookies. The Mozilla development wiki describes an origin model improvement for cookie transmission -written by Dan Witte\cite{thirdparty}. Dan describes a new +written by Dan Witte~\cite{thirdparty}. He describes a new dual-keyed origin for cookies, so that cookies would only be transmitted if -they matched both the top-level origin and the third-party origin involved in +they match both the top-level origin and the third-party origin involved in their creation. This approach would go a long way towards preventing implicit tracking across multiple websites, and has some interesting properties that make user interaction with content elements more explicitly tied to the current site. % XXXX I can't tell what this paragraph is supposed to mean. --RR - +% Similarly, one could imagine this two-level dual-keyed origin isolation being deployed to improve similar issues with DOM Storage and cryptographic tokens. @@ -292,7 +300,7 @@ amenable to improvement under this model. In particular, one can imagine per-origin plugin loading permissions, per-origin limits on the number of fonts that can be used, and randomized window-specific time offsets. - +% So, while these approaches are in fact useful for bringing the technical realities of the web closer to what the user assumes is happening, they must be deployed uniformly, with a consistent top-level origin restriction model. @@ -305,9 +313,9 @@ Even if the origin model of identifier transmission and other linkable attributes is altered uniformly to be more in line with what users expect, it is likely that the average user will still experience privacy benefits if the -browser conveys the sum of all linkable information as a single, storable, +browser conveys the sum of all linkable information as a single storable, mutable, and clearable user identity. - +% Providing this concept of identity to the user is also simpler than origin improvements, as it does not require extensive compatibility testing or standards coordination. @@ -321,34 +329,34 @@ The better UI appears to lead to less mode error (in which the user forgets whether private browsing is enabled) than other browsers' private browsing -modes\cite{private-browsing}. -% XXXX ‘mode error’? +modes~\cite{private-browsing}. -The Mozilla Weave project appears to be proposing an identity-oriented method +The Mozilla Weave project~\cite{weave-manager} appears to be proposing +an identity-oriented method of managing, syncing, and storing authentication tokens, and also has use -cases described for multiple users of a single browser\cite{weave-manager}. It +cases described for multiple users of a single browser. It is the closest idea on paper to what we envision as the way to bridge user assumptions with reality. We believe that the user interface of the browser should convey a sense of persistent identity prominently to the user in the form of a visual cue. This cue can either be an abstract image, graphic or theme (such as the user's -choice of Firefox Persona\cite{firefox-personas}), or it can be a text area +choice of Firefox Persona~\cite{firefox-personas}), or it can be a text area with the user's current favored pseudonym. This idea of identity should then be integrated with the browsing experience. Users should be able to click a button to get a clean slate for a new identity, and should be able to log into and out of password-protected stored identities, which would -contain the entire state of the browser. This is the direction the Tor Project -intends to head in with the Tor Browser Bundle\cite{not-to-toggle}. - +contain the entire state of the browser. The Tor Project is heading in +this direction with the Tor Browser Bundle~\cite{not-to-toggle}. +% To this user, the Private Browsing Mode would be no more than a special case -of this identity UI - a special identity that they can trust not to store +of this identity UI---a special identity that they can trust not to store browsing history information to disk. Such a UI also more explicitly captures what is going on with respect to the user's relationship to the web. However, all current private browsing modes fall short of protecting against a network-level adversary and fail to deal with linkability against such an -adversary\cite{private-browsing}, claiming that it is outside their threat +adversary~\cite{private-browsing}, claiming that it is outside their threat model\footnotemark. If the user is given a new identity that is still linkable to the previous one due to shortcomings of the browser, this approach has failed as a privacy measure. @@ -358,10 +366,11 @@ network-level adversary is IP-address linkability. However, we believe this to be a red herring. Users are quite capable of using alternate Internet connections, and -it is common practice for ISPs in many parts of the world to rotate user IP +it is common practice for ISPs (especially cellular IP networks) +%in many parts of the world +to rotate user IP addresses daily, to discourage users from operating servers and to impede the -spread of malware. -This is especially true of cellular IP networks.} +spread of malware.} Linkability solutions within the identity framework would be similar to the origin model solutions, except they would be properties of the entire browser @@ -369,18 +378,19 @@ \section{Conclusions} -The appeal of the prevailing revenue model of the web and the difficulties +The appeal of the web's prevailing revenue model and the difficulties associated with altering browser behavior have lulled us into accepting user deception as the norm for web use. The average user completely lacks the understanding needed to grasp how web tracking is carried out. This disconnect -in understanding is extreme to the point where moral issues arise about the +%in understanding +is extreme to the point where moral issues arise about the level of consent actually involved in web use and associated tracking. In fact, standardization efforts seemed to realize this problem early on but failed to create a feasible recommendations for improving the situation. RFC 2965 governing HTTP State Management mandated in section 3.3.6 that third-party origins must not cause the browser to transmit cookies unless the -interaction is ``verifiable'' and readily apparent to the user\cite{rfc2965}. +interaction is ``verifiable'' and readily apparent to the user~\cite{rfc2965}. In section 6, it also strongly suggested that informed consent and user control should govern the interaction of users to tracking identifiers. @@ -396,7 +406,8 @@ \bibliographystyle{plain} \bibliography{W3CIdentity} -\clearpage -\appendix +%\clearpage +%\appendix \end{document} +

1 0