[or-cvs] r22148: {arm} Utility and service rewrite (refactored roughly a third of t (in arm/trunk: . init interface util)
Damian Johnson
atagar1 at gmail.com
Thu Apr 8 16:25:15 UTC 2010
Author: atagar
Date: 2010-04-08 16:25:14 +0000 (Thu, 08 Apr 2010)
New Revision: 22148
Added:
arm/trunk/util/connections.py
arm/trunk/util/hostnames.py
arm/trunk/util/log.py
Removed:
arm/trunk/interface/connResolver.py
arm/trunk/interface/hostnameResolver.py
Modified:
arm/trunk/ChangeLog
arm/trunk/README
arm/trunk/TODO
arm/trunk/init/starter.py
arm/trunk/interface/__init__.py
arm/trunk/interface/bandwidthMonitor.py
arm/trunk/interface/confPanel.py
arm/trunk/interface/connCountMonitor.py
arm/trunk/interface/connPanel.py
arm/trunk/interface/controller.py
arm/trunk/interface/descriptorPopup.py
arm/trunk/interface/fileDescriptorPopup.py
arm/trunk/interface/graphPanel.py
arm/trunk/interface/headerPanel.py
arm/trunk/interface/logPanel.py
arm/trunk/util/__init__.py
arm/trunk/util/panel.py
arm/trunk/util/uiTools.py
Log:
Utility and service rewrite (refactored roughly a third of the codebase, including revised APIs and much better documentation).
added: centralized logging utility for handling arm events, simplifying several parts of the interface
added: rewrote connection resolver, including:
- fallback support for 'ss' and 'lsof' (requested by dun, John Case, and Christopher Davis)
- readjusts resolution rate if calls prove burdensome
- ui option for selecting mode of resolution
added: rewrote hostname resolver, including:
- optional resolution via socket module (seems worse so disabled by default... pity)
- non-blocking thread safety
- extra error info
change: revised curses wrapper utilities (plus some hacks of the interface to accommodate it)
fix: issuing resets via RELOAD signal rather than sighup (thanks to Sebastian for pointing this out)
fix: taking into account potential None values when running get_option on arbitrary values (caught by pipe and enki)
fix: crashing problem if use_default_colors() calls failed (caught by sid77)
fix: removed workaround for mysterious torrc validation bug (was accidentally already fixed - thanks to dun for lending a test environment)
fix: size and time labels weren't doing integer truncation (rounding was unintended and frustratingly difficult to get rid of)
fix: hack to prevent log panel from drawing before being positioned
fix: arm crashed if torrc was an empty file
fix: wasn't consistently bolding help keys
Modified: arm/trunk/ChangeLog
===================================================================
--- arm/trunk/ChangeLog 2010-04-08 12:22:36 UTC (rev 22147)
+++ arm/trunk/ChangeLog 2010-04-08 16:25:14 UTC (rev 22148)
@@ -1,5 +1,27 @@
CHANGE LOG
+4/8/10 - version 1.3.5
+Utility and service rewrite (refactored roughly a third of the codebase, including revised APIs and much better documentation).
+
+ * added: centralized logging utility for handling arm events, simplifying several parts of the interface
+ * added: rewrote connection resolver, including:
+ o fallback support for 'ss' and 'lsof' (requested by dun, John Case, and Christopher Davis)
+ o readjusts resolution rate if calls prove burdensome
+ o ui option for selecting mode of resolution
+ * added: rewrote hostname resolver, including:
+ o optional resolution via socket module (seems worse so disabled by default... pity)
+ o non-blocking thread safety
+ o extra error info
+ * change: revised curses wrapper utilities (plus some hacks of the interface to accommodate it)
+ * fix: issuing resets via RELOAD signal rather than sighup (thanks to Sebastian for pointing this out)
+ * fix: taking into account potential None values when running get_option on arbitrary values (caught by pipe and enki)
+ * fix: crashing problem if use_default_colors() calls failed (caught by sid77)
+ * fix: removed workaround for mysterious torrc validation bug (was accidentally already fixed - thanks to dun for lending a test environment)
+ * fix: size and time labels weren't doing integer truncation (rounding was unintended and frustratingly difficult to get rid of)
+ * fix: hack to prevent log panel from drawing before being positioned
+ * fix: arm crashed if torrc was an empty file
+ * fix: wasn't consistently bolding help keys
+
3/7/10 - version 1.3.4 (r21852)
Weekend bugfix bundle.
@@ -12,6 +34,8 @@
* fix: several uncaught exceptions when the network consensus couldn't be fetched
* fix: torrc comment stripping wasn't removing comments on the same lines as commands
* fix: torrc validation was failing under some conditions for CSV values (like ExitPolicy)
+ * fix (3/9/10, r21888): initializing error when processing family connections (caught by dun)
+ * fix (4/7/10, r22134): scrubbing wrong data for inbound connections (caught by waltman)
2/27/10 - version 1.3.3 (r21772)
Hiding client/exit information to address privacy concerns and fixes for numerous issues brought up in irc.
Modified: arm/trunk/README
===================================================================
--- arm/trunk/README 2010-04-08 12:22:36 UTC (rev 22147)
+++ arm/trunk/README 2010-04-08 16:25:14 UTC (rev 22148)
@@ -26,10 +26,10 @@
... or including 'ControlPort <PORT>' in your torrc
For full functionality this also needs:
-Common *nix commands including: ps, pidof, tail, pwdx, host, netstat, lsof, and
- ulimit
-To be ran with the same user as tor to avoid permission issues with netstat,
- lsof, and reading the torrc
+Common *nix commands including: ps, pidof, tail, pwdx, host, ulimit, and a
+ method of connection resolution (netstat, ss, or lsof)
+To be ran with the same user as tor to avoid permission issues with connection
+ resolution and reading the torrc
This is started via 'arm' (use the '--help' argument for usage).
@@ -46,18 +46,10 @@
> If you're listing connections then what about exit nodes? Won't this include
people's traffic?
-While arm isn't intended to be a sniffer it does provide real time connection
-data which, for exit nodes, includes the endpoints being visited through you.
-Unfortunately this is pretty unavoidable. The control port doesn't provide a
-means of distinguishing those connections and trying to figure it out by
-correlating against consensus data has proved pretty inaccurate.
+No. Potential client and exit connections are specifically scrubbed of
+identifying information. Be aware that it's highly discouraged for relay
+operators to fetch this data, so please don't.
-That said, this really isn't much of a concern. For Tor users the real threats
-come from things like Wireshark and MITM attacks on their unencrypted traffic.
-Simply seeing an unknown individual's endpoints is no great feat in itself.
-Just attach netstat to a cron job and voilà! You've got a sniffer that's just
-as mighty as arm.
-
> Is it harmful to share the information provided by arm?
Not really, but it's discouraged. The original plan for arm included a special
@@ -106,15 +98,13 @@
init/
__init__.py
- arm.py - parses and validates commandline parameters
- prereq.py - checks python version and for required packages
+ starter.py - parses and validates commandline parameters
+ prereq.py - checks python version and for required packages
interface/
__init__.py
controller.py - main display loop, handling input and layout
headerPanel.py - top of all pages, providing general information
- connResolver.py - (daemon thread) periodic netstat lookups
- hostnameResolver.py - (daemon thread) nonblocking reverse dns lookups
graphPanel.py - (page 1) presents graphs for data instances
@@ -131,6 +121,9 @@
util/
__init__.py
- panel.py - wrapper for safely working with curses subwindows
- uiTools.py - helper functions for interface
+ connections.py - service providing periodic connection lookups
+ hostnames.py - service providing nonblocking reverse dns lookups
+ log.py - aggregator for application events
+ panel.py - wrapper for safely working with curses subwindows
+ uiTools.py - helper functions for interface
Modified: arm/trunk/TODO
===================================================================
--- arm/trunk/TODO 2010-04-08 12:22:36 UTC (rev 22147)
+++ arm/trunk/TODO 2010-04-08 16:25:14 UTC (rev 22148)
@@ -6,11 +6,6 @@
fallback if an issue's detected.
notify John Case <case at sdf.lonestar.org>
caught by Christopher Davis
- * torrc validation bug reported (appears to be os specific)
- Reported instances were with Gentoo, OpenSuse, and OpenBSD. Change should be in:
- interface/confPanel.py lines 104-130
- setting up a Gentoo vm proved to be an absurd pain in the ass so gonna
- try repro in OpenSuse instead.
* torrc validation doesn't catch if parameters are missing
* revise multikey sort of connections
Currently using a pretty ugly hack. Look at:
@@ -30,6 +25,9 @@
be given the "UNKNOWN" type.
* regex fails for multiline log entries
* when logging no events still showing brackets
+ The current code for dynamically sizing the events label is kinda
+ tricky. Putting this off until I've made a utility to handle this
+ uglyness.
* scrolling in the torrc isn't working properly when comments are stripped
Current method of displaying torrc is pretty stupid (lots of repeated
work in display loop). When rewritten fixing this bug should be trivial.
@@ -48,9 +46,11 @@
concerning the utilities (/util). Migrating the following to util:
- os calls (to provide transparent platform independence)
- torrc validation
- - arm logging (static interface with listener design)
- wrapper for tor connection, state, and data parsing (abstracting
TorCtl connection should allow for arm to be resumed if tor restarts)
+ * provide bridge statistics
+ Include bridge related data via GETINFO option (feature request by
+ waltman).
* provide performance ARM-DEBUG events
Help with diagnosing performance bottlenecks. This is pending the
codebase revisions to figure out the low hanging fruit for caching.
@@ -67,9 +67,6 @@
- check that all connections are properly related to a circuit, for
instance no outbound connections without a corresponding inbound (not
possible yet due to being unable to correlate connections to circuts)
- * abstract away netstat calls
- In preparation for drop in replacement of lsof or calls to tor's
- GETINFO.
* add page that allows raw control port access
Piggyback on the arm connection, providing something like an interactive
prompt. In addition, provide:
@@ -89,9 +86,10 @@
* show advertised bandwidth
if set and there's extra room available show 'MaxAdvertisedBandwidth'
* check family connections to see if they're alive (VERSION cell handshake?)
- * update site's screenshots (pretty out of date...)
* look into providing UPnP support
This might be provided by tor itself so wait and see...
+ * unit tests
+ Primarily for util, for instance 'addfstr' woudl be a good candidate.
- Ideas (low priority)
* python 3 compatability
Modified: arm/trunk/init/starter.py
===================================================================
--- arm/trunk/init/starter.py 2010-04-08 12:22:36 UTC (rev 22147)
+++ arm/trunk/init/starter.py 2010-04-08 16:25:14 UTC (rev 22148)
@@ -17,8 +17,8 @@
from TorCtl import TorCtl, TorUtil
from interface import controller, logPanel
-VERSION = "1.3.4"
-LAST_MODIFIED = "Mar 7, 2010"
+VERSION = "1.3.5"
+LAST_MODIFIED = "Apr 8, 2010"
DEFAULT_CONTROL_ADDR = "127.0.0.1"
DEFAULT_CONTROL_PORT = 9051
@@ -39,7 +39,7 @@
Example:
arm -b -i 1643 hide connection data, attaching to control port 1643
-arm -e=we -p=nemesis use password 'nemesis' with 'WARN'/'ERR' events
+arm -e we -p nemesis use password 'nemesis' with 'WARN'/'ERR' events
""" % (DEFAULT_CONTROL_ADDR, DEFAULT_CONTROL_PORT, DEFAULT_LOGGED_EVENTS, logPanel.EVENT_LISTING)
def isValidIpAddr(ipStr):
Modified: arm/trunk/interface/__init__.py
===================================================================
--- arm/trunk/interface/__init__.py 2010-04-08 12:22:36 UTC (rev 22147)
+++ arm/trunk/interface/__init__.py 2010-04-08 16:25:14 UTC (rev 22148)
@@ -2,5 +2,5 @@
Panels, popups, and handlers comprising the arm user interface.
"""
-__all__ = ["bandwidthMonitor", "confPanel", "connCountMonitor", "connPanel", "connResolver", "controller", "cpuMemMonitor", "descriptorPopup", "fileDescriptorPopup", "graphPanel", "headerPanel", "hostnameResolver", "logPanel"]
+__all__ = ["bandwidthMonitor", "confPanel", "connCountMonitor", "connPanel", "controller", "cpuMemMonitor", "descriptorPopup", "fileDescriptorPopup", "graphPanel", "headerPanel", "logPanel"]
Modified: arm/trunk/interface/bandwidthMonitor.py
===================================================================
--- arm/trunk/interface/bandwidthMonitor.py 2010-04-08 12:22:36 UTC (rev 22147)
+++ arm/trunk/interface/bandwidthMonitor.py 2010-04-08 16:25:14 UTC (rev 22148)
@@ -30,7 +30,7 @@
# dummy values for static data
self.isAccounting = False
- self.bwRate, self.bwBurst = -1, -1
+ self.bwRate, self.bwBurst = None, None
self.resetOptions()
def resetOptions(self):
@@ -97,7 +97,18 @@
def getTitle(self, width):
# provides label, dropping stats if there's not enough room
- labelContents = "Bandwidth (cap: %s, burst: %s):" % (self.bwRate, self.bwBurst)
+ capLabel = "cap: %s" % self.bwRate if self.bwRate else ""
+ burstLabel = "burst: %s" % self.bwBurst if self.bwBurst else ""
+
+ if capLabel and burstLabel:
+ bwLabel = " (%s, %s)" % (capLabel, burstLabel)
+ elif capLabel or burstLabel:
+ # only one is set - use whatever's avaialble
+ bwLabel = " (%s%s)" % (capLabel, burstLabel)
+ else:
+ bwLabel = ""
+
+ labelContents = "Bandwidth%s:" % bwLabel
if width < len(labelContents):
labelContents = "%s):" % labelContents[:labelContents.find(",")] # removes burst measure
if width < len(labelContents): labelContents = "Bandwidth:" # removes both
Modified: arm/trunk/interface/confPanel.py
===================================================================
--- arm/trunk/interface/confPanel.py 2010-04-08 12:22:36 UTC (rev 22147)
+++ arm/trunk/interface/confPanel.py 2010-04-08 16:25:14 UTC (rev 22148)
@@ -6,8 +6,9 @@
import curses
import socket
+import controller
from TorCtl import TorCtl
-from util import panel, uiTools
+from util import log, panel, uiTools
# torrc parameters that can be defined multiple times without overwriting
# from src/or/config.c (entries with LINELIST or LINELIST_S)
@@ -35,8 +36,8 @@
Presents torrc with syntax highlighting in a scroll-able area.
"""
- def __init__(self, confLocation, conn, logPanel):
- panel.Panel.__init__(self, -1)
+ def __init__(self, stdscr, confLocation, conn):
+ panel.Panel.__init__(self, stdscr, 0)
self.confLocation = confLocation
self.showLineNum = True
self.stripComments = False
@@ -50,7 +51,6 @@
# is of line numbers (one-indexed) to tor's actual values
self.corrections = {}
self.conn = conn
- self.logger = logPanel
self.reset()
@@ -118,6 +118,12 @@
else:
# general case - fetch all valid values
for key, val in self.conn.get_option(command):
+ if val == None:
+ # TODO: investigate situations where this might occure
+ # (happens if trying to parse HIDDEN_SERVICE_PARAM)
+ if logErrors: log.log(log.WARN, "BUG: Failed to find torrc value for %s" % key)
+ continue
+
# TODO: check for a better way of figuring out CSV parameters
# (kinda doubt this is right... in config.c its listed as being
# a 'LINELIST') - still, good enough for common cases
@@ -139,16 +145,8 @@
for entry in arguments:
if not entry in actualValues:
self.corrections[lineNumber + 1] = ", ".join(actualValues)
- except (TypeError, socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed):
- # TODO: for some reason the above provided:
- # TypeError: sequence item 0: expected string, NoneType found
- #
- # for the corrections setting. This issue seems to be specific to
- # Gentoo, OpenSuse, and OpenBSD but haven't yet managed to
- # reproduce. Catching the TypeError to just drop the torrc
- # validation for those systems
-
- if logErrors: self.logger.monitor_event("WARN", "Unable to validate torrc")
+ except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed):
+ if logErrors: log.log(log.WARN, "Unable to validate torrc")
# logs issues that arose
if self.irrelevantLines and logErrors:
@@ -156,21 +154,20 @@
else: first, second, third = "Entry", "is", " on line"
baseMsg = "%s in your torrc %s ignored due to duplication%s" % (first, second, third)
- self.logger.monitor_event("NOTICE", "%s: %s (highlighted in blue)" % (baseMsg, ", ".join([str(val) for val in self.irrelevantLines])))
+ log.log(log.NOTICE, "%s: %s (highlighted in blue)" % (baseMsg, ", ".join([str(val) for val in self.irrelevantLines])))
if self.corrections and logErrors:
- self.logger.monitor_event("WARN", "Tor's state differs from loaded torrc")
+ log.log(log.WARN, "Tor's state differs from loaded torrc")
except IOError, exc:
resetSuccessful = False
self.confContents = ["### Unable to load torrc ###"]
- if logErrors: self.logger.monitor_event("WARN", "Unable to load torrc (%s)" % str(exc))
+ if logErrors: log.log(log.WARN, "Unable to load torrc (%s)" % str(exc))
self.scroll = 0
return resetSuccessful
def handleKey(self, key):
- self._resetBounds()
- pageHeight = self.maxY - 1
+ pageHeight = self.getPreferredSize()[0] - 1
if key == curses.KEY_UP: self.scroll = max(self.scroll - 1, 0)
elif key == curses.KEY_DOWN: self.scroll = max(0, min(self.scroll + 1, len(self.confContents) - pageHeight))
elif key == curses.KEY_PPAGE: self.scroll = max(self.scroll - pageHeight, 0)
@@ -181,11 +178,12 @@
self.scroll = 0
self.redraw()
- def draw(self):
- self.addstr(0, 0, "Tor Config (%s):" % self.confLocation, uiTools.LABEL_ATTR)
+ def draw(self, subwindow, width, height):
+ self.addstr(0, 0, "Tor Config (%s):" % self.confLocation, curses.A_STANDOUT)
- pageHeight = self.maxY - 1
- numFieldWidth = int(math.log10(len(self.confContents))) + 1
+ pageHeight = height - 1
+ if self.confContents: numFieldWidth = int(math.log10(len(self.confContents))) + 1
+ else: numFieldWidth = 0 # torrc is blank
lineNum, displayLineNum = self.scroll + 1, 1 # lineNum corresponds to torrc, displayLineNum concerns what's presented
# determine the ending line in the display (prevents us from going to the
@@ -243,10 +241,10 @@
numOffset = numFieldWidth + 1
xLoc = 0
- displayLineNum, xLoc = self.addstr_wrap(displayLineNum, xLoc, command, curses.A_BOLD | uiTools.getColor(commandColor), numOffset)
- displayLineNum, xLoc = self.addstr_wrap(displayLineNum, xLoc, argument, curses.A_BOLD | uiTools.getColor(argumentColor), numOffset)
- displayLineNum, xLoc = self.addstr_wrap(displayLineNum, xLoc, correction, curses.A_BOLD | uiTools.getColor(correctionColor), numOffset)
- displayLineNum, xLoc = self.addstr_wrap(displayLineNum, xLoc, comment, uiTools.getColor(commentColor), numOffset)
+ displayLineNum, xLoc = controller.addstr_wrap(self, displayLineNum, xLoc, command, curses.A_BOLD | uiTools.getColor(commandColor), numOffset)
+ displayLineNum, xLoc = controller.addstr_wrap(self, displayLineNum, xLoc, argument, curses.A_BOLD | uiTools.getColor(argumentColor), numOffset)
+ displayLineNum, xLoc = controller.addstr_wrap(self, displayLineNum, xLoc, correction, curses.A_BOLD | uiTools.getColor(correctionColor), numOffset)
+ displayLineNum, xLoc = controller.addstr_wrap(self, displayLineNum, xLoc, comment, uiTools.getColor(commentColor), numOffset)
displayLineNum += 1
Modified: arm/trunk/interface/connCountMonitor.py
===================================================================
--- arm/trunk/interface/connCountMonitor.py 2010-04-08 12:22:36 UTC (rev 22147)
+++ arm/trunk/interface/connCountMonitor.py 2010-04-08 16:25:14 UTC (rev 22148)
@@ -6,6 +6,7 @@
from TorCtl import TorCtl
import graphPanel
+from util import connections
class ConnCountMonitor(graphPanel.GraphStats, TorCtl.PostEventListener):
"""
@@ -13,11 +14,10 @@
outbound.
"""
- def __init__(self, conn, connResolver):
+ def __init__(self, conn):
graphPanel.GraphStats.__init__(self)
TorCtl.PostEventListener.__init__(self)
graphPanel.GraphStats.initialize(self, "green", "cyan", 10)
- self.connResolver = connResolver # thread performing netstat queries
self.orPort = "0"
self.dirPort = "0"
@@ -39,15 +39,10 @@
# (and so it stops if Tor stops - used to use a separate thread but this
# is better)
inbound, outbound, control = 0, 0, 0
- results = self.connResolver.getConnections()
- for line in results:
- if not line.startswith("tcp"): continue
- param = line.split()
- localPort = param[3][param[3].find(":") + 1:]
-
- if localPort in (self.orPort, self.dirPort): inbound += 1
- elif localPort == self.controlPort: control += 1
+ for lIp, lPort, fIp, fPort in connections.getResolver("tor").getConnections():
+ if lPort in (self.orPort, self.dirPort): inbound += 1
+ elif lPort == self.controlPort: control += 1
else: outbound += 1
self._processEvent(inbound, outbound)
Modified: arm/trunk/interface/connPanel.py
===================================================================
--- arm/trunk/interface/connPanel.py 2010-04-08 12:22:36 UTC (rev 22147)
+++ arm/trunk/interface/connPanel.py 2010-04-08 16:25:14 UTC (rev 22148)
@@ -8,8 +8,7 @@
from threading import RLock
from TorCtl import TorCtl
-import hostnameResolver
-from util import panel, uiTools
+from util import log, connections, hostnames, panel, uiTools
# Scrubs private data from any connection that might belong to client or exit
# traffic. This is a little overly conservative, hiding anything that isn't
@@ -107,16 +106,14 @@
class ConnPanel(TorCtl.PostEventListener, panel.Panel):
"""
- Lists netstat provided network data of tor.
+ Lists tor related connection data.
"""
- def __init__(self, conn, connResolver, logger):
+ def __init__(self, stdscr, conn):
TorCtl.PostEventListener.__init__(self)
- panel.Panel.__init__(self, -1)
+ panel.Panel.__init__(self, stdscr, 0)
self.scroll = 0
self.conn = conn # tor connection for querrying country codes
- self.connResolver = connResolver # thread performing netstat queries
- self.logger = logger # notified in case of problems
self.listingType = LIST_IP # information used in listing entries
self.allowDNS = True # permits hostname resolutions if true
self.showLabel = True # shows top label if true, hides otherwise
@@ -124,7 +121,6 @@
self.lastUpdate = -1 # time last stats was retrived
self.localhostEntry = None # special connection - tuple with (entry for this node, fingerprint)
self.sortOrdering = [ORD_TYPE, ORD_FOREIGN_LISTING, ORD_FOREIGN_PORT]
- self.resolver = hostnameResolver.HostnameResolver()
self.fingerprintLookupCache = {} # cache of (ip, port) -> fingerprint
self.nicknameLookupCache = {} # cache of (ip, port) -> nickname
self.fingerprintMappings = _getFingerprintMappings(self.conn) # mappings of ip -> [(port, fingerprint, nickname), ...]
@@ -134,7 +130,7 @@
self.clientConnectionCache = None # listing of nicknames for our client connections
self.clientConnectionLock = RLock() # lock for clientConnectionCache
self.isDisabled = False # prevent panel from updating entirely
- self.lastNetstatResults = None # used to check if raw netstat results have changed
+ self.lastConnResults = None # used to check if connection results have changed
self.isCursorEnabled = True
self.cursorSelection = None
@@ -163,7 +159,7 @@
self.resetOptions()
- # netstat results are tuples of the form:
+ # connection results are tuples of the form:
# (type, local IP, local port, foreign IP, foreign port, country code)
self.connections = []
self.connectionsLock = RLock() # limits modifications of connections
@@ -177,7 +173,7 @@
self.familyResolutions = {}
try:
- self.address = ""
+ self.address = "" # fetched when needed if unset
self.nickname = self.conn.get_option("Nickname")[0][1]
self.orPort = self.conn.get_option("ORPort")[0][1]
@@ -245,7 +241,7 @@
if len(nsData) > 1:
# multiple records for fingerprint (shouldn't happen)
- self.logger.monitor_event("WARN", "Multiple consensus entries for fingerprint: %s" % fingerprint)
+ log.log(log.WARN, "Multiple consensus entries for fingerprint: %s" % fingerprint)
return
nsEntry = nsData[0]
@@ -268,7 +264,7 @@
def reset(self):
"""
- Reloads netstat results.
+ Reloads connection results.
"""
# inaccessable during startup so might need to be refetched
@@ -302,27 +298,22 @@
for entry in (self.connections if not self.isPaused else self.connectionsBuffer):
connTimes[(entry[CONN_F_IP], entry[CONN_F_PORT])] = entry[CONN_TIME]
- results = self.connResolver.getConnections()
- if results == self.lastNetstatResults: return # contents haven't changed
+ results = connections.getResolver("tor").getConnections()
+ if results == self.lastConnResults: return # contents haven't changed
- for line in results:
- if not line.startswith("tcp"): continue
- param = line.split()
- local, foreign = param[3], param[4]
- localIP, foreignIP = local[:local.find(":")], foreign[:foreign.find(":")]
- localPort, foreignPort = local[len(localIP) + 1:], foreign[len(foreignIP) + 1:]
- fingerprint = self.getFingerprint(foreignIP, foreignPort)
+ for lIp, lPort, fIp, fPort in results:
+ fingerprint = self.getFingerprint(fIp, fPort)
isPrivate = False
- if localPort in (self.listenPort, self.dirPort):
+ if lPort in (self.listenPort, self.dirPort):
type = "inbound"
connectionCountTmp[0] += 1
- if SCRUB_PRIVATE_DATA and foreignIP not in self.fingerprintMappings.keys(): isPrivate = isGuard or self.isBridge
- elif localPort == self.controlPort:
+ if SCRUB_PRIVATE_DATA and fIp not in self.fingerprintMappings.keys(): isPrivate = isGuard or self.isBridge
+ elif lPort == self.controlPort:
type = "control"
connectionCountTmp[4] += 1
else:
- nickname = self.getNickname(foreignIP, foreignPort)
+ nickname = self.getNickname(fIp, fPort)
isClient = False
for clientName in self.clientConnectionCache:
@@ -333,30 +324,30 @@
if isClient:
type = "client"
connectionCountTmp[2] += 1
- elif (foreignIP, foreignPort) in DIR_SERVERS:
+ elif (fIp, fPort) in DIR_SERVERS:
type = "directory"
connectionCountTmp[3] += 1
else:
type = "outbound"
connectionCountTmp[1] += 1
- if SCRUB_PRIVATE_DATA and foreignIP not in self.fingerprintMappings.keys(): isPrivate = isExitAllowed(foreignIP, foreignPort, self.exitPolicy, self.exitRejectPrivate, self.logger)
+ if SCRUB_PRIVATE_DATA and fIp not in self.fingerprintMappings.keys(): isPrivate = isExitAllowed(fIp, fPort, self.exitPolicy, self.exitRejectPrivate)
# replace nat address with external version if available
- if self.address and type != "control": localIP = self.address
+ if self.address and type != "control": lIp = self.address
try:
- countryCodeQuery = "ip-to-country/%s" % foreign[:foreign.find(":")]
+ countryCodeQuery = "ip-to-country/%s" % fIp
countryCode = self.conn.get_info(countryCodeQuery)[countryCodeQuery]
except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed):
countryCode = "??"
if not self.providedGeoipWarning:
- self.logger.monitor_event("WARN", "Tor geoip database is unavailable.")
+ log.log(log.WARN, "Tor geoip database is unavailable.")
self.providedGeoipWarning = True
- if (foreignIP, foreignPort) in connTimes: connTime = connTimes[(foreignIP, foreignPort)]
+ if (fIp, fPort) in connTimes: connTime = connTimes[(fIp, fPort)]
else: connTime = time.time()
- connectionsTmp.append((type, localIP, localPort, foreignIP, foreignPort, countryCode, connTime, isPrivate))
+ connectionsTmp.append((type, lIp, lPort, fIp, fPort, countryCode, connTime, isPrivate))
# appends localhost connection to allow user to look up their own consensus entry
selfFingerprint = None
@@ -418,7 +409,7 @@
# hostnames are sorted at draw - otherwise now's a good time
if self.listingType != LIST_HOSTNAME: self.sortConnections()
- self.lastNetstatResults = results
+ self.lastConnResults = results
finally:
self.connectionsLock.release()
self.clientConnectionLock.release()
@@ -426,8 +417,7 @@
def handleKey(self, key):
# cursor or scroll movement
if key in (curses.KEY_UP, curses.KEY_DOWN, curses.KEY_PPAGE, curses.KEY_NPAGE):
- self._resetBounds()
- pageHeight = self.maxY - 1
+ pageHeight = self.getPreferredSize()[0] - 1
if self.showingDetails: pageHeight -= 8
self.connectionsLock.acquire()
@@ -457,12 +447,12 @@
self.connectionsLock.release()
elif key == ord('r') or key == ord('R'):
self.allowDNS = not self.allowDNS
- if not self.allowDNS: self.resolver.setPaused(True)
- elif self.listingType == LIST_HOSTNAME: self.resolver.setPaused(False)
+ if not self.allowDNS: hostnames.setPaused(True)
+ elif self.listingType == LIST_HOSTNAME: hostnames.setPaused(False)
else: return # skip following redraw
self.redraw()
- def draw(self):
+ def draw(self, subwindow, width, height):
self.connectionsLock.acquire()
try:
# hostnames frequently get updated so frequent sorting needed
@@ -474,18 +464,18 @@
for i in range(len(self.connectionCount)):
if self.connectionCount[i] > 0: countLabel += "%i %s, " % (self.connectionCount[i], CONN_COUNT_LABELS[i])
if countLabel: countLabel = " (%s)" % countLabel[:-2] # strips ending ", " and encases in parentheses
- self.addstr(0, 0, "Connections%s:" % countLabel, uiTools.LABEL_ATTR)
+ self.addstr(0, 0, "Connections%s:" % countLabel, curses.A_STANDOUT)
if self.connections:
- listingHeight = self.maxY - 1
+ listingHeight = height - 1
currentTime = time.time() if not self.isPaused else self.pauseTime
if self.showingDetails:
listingHeight -= 8
- isScrollBarVisible = len(self.connections) > self.maxY - 9
- if self.maxX > 80: self.win.hline(8, 80, curses.ACS_HLINE, self.maxX - 81)
+ isScrollBarVisible = len(self.connections) > height - 9
+ if width > 80: subwindow.hline(8, 80, curses.ACS_HLINE, width - 81)
else:
- isScrollBarVisible = len(self.connections) > self.maxY - 1
+ isScrollBarVisible = len(self.connections) > height - 1
xOffset = 3 if isScrollBarVisible else 0 # content offset for scroll bar
# ensure cursor location and scroll top are within bounds
@@ -520,14 +510,14 @@
src, dst = "%-21s" % src, "%-26s" % dst
etc = ""
- if self.maxX > 115 + xOffset:
+ if width > 115 + xOffset:
# show fingerprint (column width: 42 characters)
etc += "%-40s " % self.getFingerprint(entry[CONN_F_IP], entry[CONN_F_PORT])
- if self.maxX > 127 + xOffset:
+ if width > 127 + xOffset:
# show nickname (column width: remainder)
nickname = self.getNickname(entry[CONN_F_IP], entry[CONN_F_PORT])
- nicknameSpace = self.maxX - 118 - xOffset
+ nicknameSpace = width - 118 - xOffset
# truncates if too long
if len(nickname) > nicknameSpace: nickname = "%s..." % nickname[:nicknameSpace - 3]
@@ -538,10 +528,10 @@
src = "localhost:%-5s" % entry[CONN_L_PORT]
# space available for foreign hostname (stretched to claim any free space)
- foreignHostnameSpace = self.maxX - 42 - xOffset
+ foreignHostnameSpace = width - 42 - xOffset
etc = ""
- if self.maxX > 102 + xOffset:
+ if width > 102 + xOffset:
# shows ip/locale (column width: 22 characters)
foreignHostnameSpace -= 22
@@ -549,15 +539,15 @@
else: ipEntry = "%s %s" % (entry[CONN_F_IP], "" if type == "control" else "(%s)" % entry[CONN_COUNTRY])
etc += "%-20s " % ipEntry
- if self.maxX > 134 + xOffset:
+ if width > 134 + xOffset:
# show fingerprint (column width: 42 characters)
foreignHostnameSpace -= 42
etc += "%-40s " % self.getFingerprint(entry[CONN_F_IP], entry[CONN_F_PORT])
- if self.maxX > 151 + xOffset:
+ if width > 151 + xOffset:
# show nickname (column width: min 17 characters, uses half of the remainder)
nickname = self.getNickname(entry[CONN_F_IP], entry[CONN_F_PORT])
- nicknameSpace = 15 + (self.maxX - xOffset - 151) / 2
+ nicknameSpace = 15 + (width - xOffset - 151) / 2
foreignHostnameSpace -= (nicknameSpace + 2)
if len(nickname) > nicknameSpace: nickname = "%s..." % nickname[:nicknameSpace - 3]
@@ -565,7 +555,8 @@
if isPrivate: dst = "<scrubbed>"
else:
- hostname = self.resolver.resolve(entry[CONN_F_IP])
+ try: hostname = hostnames.resolve(entry[CONN_F_IP])
+ except ValueError: hostname = None
# truncates long hostnames
portDigits = len(str(entry[CONN_F_PORT]))
@@ -583,14 +574,14 @@
dst = "%-40s" % dst
etc = ""
- if self.maxX > 92 + xOffset:
+ if width > 92 + xOffset:
# show nickname (column width: min 17 characters, uses remainder if extra room's available)
nickname = self.getNickname(entry[CONN_F_IP], entry[CONN_F_PORT])
- nicknameSpace = self.maxX - 78 - xOffset if self.maxX < 126 else self.maxX - 106 - xOffset
+ nicknameSpace = width - 78 - xOffset if width < 126 else width - 106 - xOffset
if len(nickname) > nicknameSpace: nickname = "%s..." % nickname[:nicknameSpace - 3]
etc += ("%%-%is " % nicknameSpace) % nickname
- if self.maxX > 125 + xOffset:
+ if width > 125 + xOffset:
# shows ip/port/locale (column width: 28 characters)
if isPrivate: ipEntry = "<scrubbed>"
else: ipEntry = "%s:%s %s" % (entry[CONN_F_IP], entry[CONN_F_PORT], "" if type == "control" else "(%s)" % entry[CONN_COUNTRY])
@@ -602,15 +593,15 @@
else: dst = self.getNickname(entry[CONN_F_IP], entry[CONN_F_PORT])
# space available for foreign nickname
- foreignNicknameSpace = self.maxX - len(self.nickname) - 27 - xOffset
+ foreignNicknameSpace = width - len(self.nickname) - 27 - xOffset
etc = ""
- if self.maxX > 92 + xOffset:
+ if width > 92 + xOffset:
# show fingerprint (column width: 42 characters)
foreignNicknameSpace -= 42
etc += "%-40s " % self.getFingerprint(entry[CONN_F_IP], entry[CONN_F_PORT])
- if self.maxX > 120 + xOffset:
+ if width > 120 + xOffset:
# shows ip/port/locale (column width: 28 characters)
foreignNicknameSpace -= 28
@@ -637,7 +628,7 @@
ipStart = etc.find("256")
if ipStart > -1: etc = etc[:ipStart] + ("%%-%is" % len(etc[ipStart:])) % "UNKNOWN"
- padding = self.maxX - (len(src) + len(dst) + len(etc) + 27) - xOffset # padding needed to fill full line
+ padding = width - (len(src) + len(dst) + len(etc) + 27) - xOffset # padding needed to fill full line
lineEntry = "<%s>%s --> %s %s%s%5s (<b>%s</b>)%s</%s>" % (color, src, dst, etc, " " * padding, timeLabel, type.upper(), " " * (9 - len(type)), color)
if self.isCursorEnabled and entry == self.cursorSelection:
@@ -649,8 +640,8 @@
if isScrollBarVisible:
topY = 9 if self.showingDetails else 1
- bottomEntry = self.scroll + self.maxY - 9 if self.showingDetails else self.scroll + self.maxY - 1
- uiTools.drawScrollBar(self, topY, self.maxY - 1, self.scroll, bottomEntry, len(self.connections))
+ bottomEntry = self.scroll + height - 9 if self.showingDetails else self.scroll + height - 1
+ self.addScrollBar(self.scroll, bottomEntry, len(self.connections), topY)
finally:
self.connectionsLock.release()
@@ -789,7 +780,7 @@
listingWrapper = lambda ip, port: _ipToInt(ip)
elif self.listingType == LIST_HOSTNAME:
# alphanumeric hostnames followed by unresolved IP addresses
- listingWrapper = lambda ip, port: self.resolver.resolve(ip).upper() if self.resolver.resolve(ip) else "zzzzz%099i" % _ipToInt(ip)
+ listingWrapper = lambda ip, port: _getHostname(ip).upper() if _getHostname(ip) else "zzzzz%099i" % _ipToInt(ip)
elif self.listingType == LIST_FINGERPRINT:
# alphanumeric fingerprints followed by UNKNOWN entries
listingWrapper = lambda ip, port: self.getFingerprint(ip, port) if self.getFingerprint(ip, port) != "UNKNOWN" else "zzzzz%099i" % _ipToInt(ip)
@@ -816,6 +807,10 @@
if comp or len(sorts) == 1: return comp
else: return _multisort(conn1, conn2, sorts[1:])
+def _getHostname(ipAddr):
+ try: return hostnames.resolve(ipAddr)
+ except ValueError: return None
+
# provides comparison int for sorting IP addresses
def _ipToInt(ipAddr):
total = 0
@@ -852,7 +847,7 @@
return clients
-def isExitAllowed(ip, port, exitPolicy, isPrivateRejected, logger):
+def isExitAllowed(ip, port, exitPolicy, isPrivateRejected):
"""
Determines if a given connection is a permissable exit with the given
policy or not (True if it's allowed to be an exit connection, False
@@ -904,6 +899,6 @@
if isIPMatch and isPortMatch: return isAccept
# we shouldn't ever fall through due to default exit policy
- logger.monitor_event("WARN", "Exit policy left connection uncategorized: %s:%i" % (ip, port))
+ log.log(log.WARN, "Exit policy left connection uncategorized: %s:%i" % (ip, port))
return False
Deleted: arm/trunk/interface/connResolver.py
===================================================================
--- arm/trunk/interface/connResolver.py 2010-04-08 12:22:36 UTC (rev 22147)
+++ arm/trunk/interface/connResolver.py 2010-04-08 16:25:14 UTC (rev 22148)
@@ -1,96 +0,0 @@
-#!/usr/bin/env python
-# connResolver.py -- Background thread for retrieving tor's connections.
-# Released under the GPL v3 (http://www.gnu.org/licenses/gpl.html)
-
-import os
-import time
-from threading import Thread
-from threading import RLock
-
-MIN_LOOKUP_WAIT = 5 # minimum seconds between lookups
-SLEEP_INTERVAL = 1 # period to sleep when not making a netstat call
-FAILURE_TOLERANCE = 3 # number of subsiquent failures tolerated before pausing thread
-FAILURE_MSG = "Unable to query netstat for new connections"
-SERIAL_FAILURE_MSG = "Failing to query netstat (connection related portions of the monitor won't function)"
-
-class ConnResolver(Thread):
- """
- Service that periodically queries Tor's current connections to allow for a
- best effort, non-blocking lookup. This is currently implemented via netstat.
- In case of failure this gives an INFO level warning and provides the last
- known results. This process provides an WARN level warning and pauses itself
- if there's several subsiquent failures (probably indicating that netstat
- isn't available).
- """
-
- def __init__(self, conn, pid, logPanel):
- Thread.__init__(self)
- self.conn = conn # used to stop querring netstat if tor's closed
- self.pid = pid # tor process ID to make sure we've got the right instance
- self.logger = logPanel # used to notify of lookup failures
-
- self.connections = [] # unprocessed lines from netstat results
- self.connectionsLock = RLock() # limits concurrent access to connections
- self.isPaused = False
- self.halt = False # terminates thread if true
- self.lastLookup = -1 # time of last lookup (reguardless of success)
- self.subsiquentFailures = 0 # number of failed netstat calls in a row
- self.setDaemon(True)
-
- def getConnections(self):
- """
- Provides the last querried connection results, empty list if tor's closed.
- """
-
- if self.conn._closed == 1: return []
- connectionsTmp = None
-
- self.connectionsLock.acquire()
- try: connectionsTmp = list(self.connections)
- finally: self.connectionsLock.release()
-
- return connectionsTmp
-
- def run(self):
- if not self.pid: return
-
- while not self.halt:
- if self.isPaused or time.time() - MIN_LOOKUP_WAIT < self.lastLookup or self.conn._closed == 1: time.sleep(SLEEP_INTERVAL)
- else:
- try:
- netstatStart = time.time()
-
- # looks at netstat for tor with stderr redirected to /dev/null, options are:
- # n = prevents dns lookups, p = include process (say if it's tor), t = tcp only
- netstatCall = os.popen("netstat -npt 2> /dev/null | grep %s/tor 2> /dev/null" % self.pid)
-
- self.logger.monitor_event("DEBUG", "netstat queried in %.4f seconds" % (time.time() - netstatStart))
- results = netstatCall.readlines()
- if not results: raise IOError
-
- # assign obtained results
- self.connectionsLock.acquire()
- try: self.connections = results
- finally: self.connectionsLock.release()
-
- self.subsiquentFailures = 0
- except IOError:
- # netstat call failed
- self.subsiquentFailures += 1
- self.logger.monitor_event("INFO", "%s (%i)" % (FAILURE_MSG, self.subsiquentFailures))
-
- if self.subsiquentFailures >= FAILURE_TOLERANCE:
- self.logger.monitor_event("WARN", SERIAL_FAILURE_MSG)
- self.setPaused(True)
- finally:
- self.lastLookup = time.time()
- netstatCall.close()
-
- def setPaused(self, isPause):
- """
- If true, prevents further netstat lookups.
- """
-
- if isPause == self.isPaused: return
- self.isPaused = isPause
-
Modified: arm/trunk/interface/controller.py
===================================================================
--- arm/trunk/interface/controller.py 2010-04-08 12:22:36 UTC (rev 22147)
+++ arm/trunk/interface/controller.py 2010-04-08 16:25:14 UTC (rev 22148)
@@ -23,8 +23,7 @@
import descriptorPopup
import fileDescriptorPopup
-from util import panel, uiTools
-import connResolver
+from util import log, connections, hostnames, panel, uiTools
import bandwidthMonitor
import cpuMemMonitor
import connCountMonitor
@@ -50,12 +49,11 @@
class ControlPanel(panel.Panel):
""" Draws single line label for interface controls. """
- def __init__(self, resolver, isBlindMode):
- panel.Panel.__init__(self, 1)
+ def __init__(self, stdscr, isBlindMode):
+ panel.Panel.__init__(self, stdscr, 0, 1)
self.msgText = CTL_HELP # message text to be displyed
self.msgAttr = curses.A_NORMAL # formatting attributes
self.page = 1 # page number currently being displayed
- self.resolver = resolver # dns resolution thread
self.resolvingCounter = -1 # count of resolver when starting (-1 if we aren't working on a batch)
self.isBlindMode = isBlindMode
@@ -68,7 +66,7 @@
self.msgText = msgText
self.msgAttr = msgAttr
- def draw(self):
+ def draw(self, subwindow, width, height):
msgText = self.msgText
msgAttr = self.msgAttr
barTab = 2 # space between msgText and progress bar
@@ -80,13 +78,13 @@
msgAttr = curses.A_NORMAL
if self.resolvingCounter != -1:
- if self.resolver.unresolvedQueue.empty() or self.resolver.isPaused:
+ if hostnames.isPaused() or not hostnames.isResolving():
# done resolving dns batch
self.resolvingCounter = -1
curses.halfdelay(REFRESH_RATE * 10) # revert to normal refresh rate
else:
- batchSize = self.resolver.totalResolves - self.resolvingCounter
- entryCount = batchSize - self.resolver.unresolvedQueue.qsize()
+ batchSize = hostnames.getRequestCount() - self.resolvingCounter
+ entryCount = batchSize - hostnames.getPendingCount()
if batchSize > 0: progress = 100 * entryCount / batchSize
else: progress = 0
@@ -96,7 +94,7 @@
#msgText = "Resolving hostnames (%i / %i, %i%%) - press esc %sto cancel" % (entryCount, batchSize, progress, additive)
msgText = "Resolving hostnames (press esc %sto cancel) - %s / %i, %2i%%" % (additive, entryCountLabel, batchSize, progress)
- barWidth = min(barWidthMax, self.maxX - len(msgText) - 3 - barTab)
+ barWidth = min(barWidthMax, width - len(msgText) - 3 - barTab)
barProgress = barWidth * entryCount / batchSize
if self.resolvingCounter == -1:
@@ -119,6 +117,72 @@
self.addstr(0, xLoc + 1, " " * barProgress, curses.A_STANDOUT | uiTools.getColor("red"))
self.addstr(0, xLoc + barWidth + 1, "]", curses.A_BOLD)
+class Popup(panel.Panel):
+ """
+ Temporarily providing old panel methods until permanent workaround for popup
+ can be derrived (this passive drawing method is horrible - I'll need to
+ provide a version using the more active repaint design later in the
+ revision).
+ """
+
+ def __init__(self, stdscr, height):
+ panel.Panel.__init__(self, stdscr, 0, height)
+
+ # The following methods are to emulate old panel functionality (this was the
+ # only implementations to use these methods and will require a complete
+ # rewrite when refactoring gets here)
+ def clear(self):
+ if self.win:
+ self.isDisplaced = self.top > self.win.getparyx()[0]
+ if not self.isDisplaced: self.win.erase()
+
+ def refresh(self):
+ if self.win and not self.isDisplaced: self.win.refresh()
+
+ def recreate(self, stdscr, newWidth=-1, newTop=None):
+ self.setParent(stdscr)
+ self.setWidth(newWidth)
+ if newTop != None: self.setTop(newTop)
+
+ newHeight, newWidth = self.getPreferredSize()
+ if newHeight > 0:
+ self.win = self.parent.subwin(newHeight, newWidth, self.top, 0)
+ elif self.win == None:
+ # don't want to leave the window as none (in very edge cases could cause
+ # problems) - rather, create a displaced instance
+ self.win = self.parent.subwin(1, newWidth, 0, 0)
+
+ self.maxY, self.maxX = self.win.getmaxyx()
+
+def addstr_wrap(panel, y, x, text, formatting, startX = 0, endX = -1, maxY = -1):
+ """
+ Writes text with word wrapping, returning the ending y/x coordinate.
+ y: starting write line
+ x: column offset from startX
+ text / formatting: content to be written
+ startX / endX: column bounds in which text may be written
+ """
+
+ # moved out of panel (trying not to polute new code!)
+ # TODO: unpleaseantly complex usage - replace with something else when
+ # rewriting confPanel and descriptorPopup (the only places this is used)
+ if not text: return (y, x) # nothing to write
+ if endX == -1: endX = panel.maxX # defaults to writing to end of panel
+ if maxY == -1: maxY = panel.maxY + 1 # defaults to writing to bottom of panel
+ lineWidth = endX - startX # room for text
+ while True:
+ if len(text) > lineWidth - x - 1:
+ chunkSize = text.rfind(" ", 0, lineWidth - x)
+ writeText = text[:chunkSize]
+ text = text[chunkSize:].strip()
+
+ panel.addstr(y, x + startX, writeText, formatting)
+ y, x = y + 1, 0
+ if y >= maxY: return (y, x)
+ else:
+ panel.addstr(y, x + startX, text, formatting)
+ return (y, x + len(text))
+
class sighupListener(TorCtl.PostEventListener):
"""
Listens for reload signal (hup), which is produced by:
@@ -154,6 +218,7 @@
if popup.win:
if not panel.CURSES_LOCK.acquire(False): return -1
try:
+ # TODO: should pause interface (to avoid event accumilation)
curses.cbreak() # wait indefinitely for key presses (no timeout)
# uses smaller dimentions more fitting for small content
@@ -166,7 +231,7 @@
while key not in (curses.KEY_ENTER, 10, ord(' ')):
popup.clear()
popup.win.box()
- popup.addstr(0, 0, title, uiTools.LABEL_ATTR)
+ popup.addstr(0, 0, title, curses.A_STANDOUT)
for i in range(len(options)):
label = options[i]
@@ -191,7 +256,7 @@
return selection
-def setEventListening(loggedEvents, conn, logListener):
+def setEventListening(loggedEvents, conn):
"""
Tries to set events being listened for, displaying error for any event
types that aren't supported (possibly due to version issues). This returns
@@ -229,8 +294,8 @@
if eventType == "BW": msg = "(bandwidth graph won't function)"
elif eventType in ("NEWDESC", "NEWCONSENSUS") and not isBlindMode: msg = "(connections listing can't register consensus changes)"
else: msg = ""
- logListener.monitor_event("ERR", "Unsupported event type: %s %s" % (eventType, msg))
- else: logListener.monitor_event("WARN", "Unsupported event type: %s" % eventType)
+ log.log(log.ERR, "Unsupported event type: %s %s" % (eventType, msg))
+ else: log.log(log.WARN, "Unsupported event type: %s" % eventType)
except TorCtl.TorCtlClosed:
return []
@@ -248,9 +313,9 @@
otherwise unrecognized events)
"""
- curses.use_default_colors() # allows things like semi-transparent backgrounds
- uiTools.initColors() # initalizes color pairs for colored text
curses.halfdelay(REFRESH_RATE * 10) # uses getch call as timer for REFRESH_RATE seconds
+ try: curses.use_default_colors() # allows things like semi-transparent backgrounds (call can fail with ERR)
+ except curses.error: pass
# attempts to make the cursor invisible (not supported in all terminals)
try: curses.curs_set(0)
@@ -302,30 +367,40 @@
except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed):
confLocation = ""
+ # minor refinements for connection resolver
+ resolver = connections.getResolver("tor")
+ if torPid: resolver.processPid = torPid # helps narrow connection results
+
+ # hack to display a better (arm specific) notice if all resolvers fail
+ connections.RESOLVER_FINAL_FAILURE_MSG += " (connection related portions of the monitor won't function)"
+
panels = {
- "header": headerPanel.HeaderPanel(conn, torPid),
- "popup": panel.Panel(9),
- "graph": graphPanel.GraphPanel(),
- "log": logPanel.LogMonitor(conn, loggedEvents)}
+ "header": headerPanel.HeaderPanel(stdscr, conn, torPid),
+ "popup": Popup(stdscr, 9),
+ "graph": graphPanel.GraphPanel(stdscr),
+ "log": logPanel.LogMonitor(stdscr, conn, loggedEvents)}
- # starts thread for processing netstat queries
- connResolutionThread = connResolver.ConnResolver(conn, torPid, panels["log"])
- if not isBlindMode: connResolutionThread.start()
+ # TODO: later it would be good to set the right 'top' values during initialization,
+ # but for now this is just necessary for the log panel (and a hack in the log...)
- panels["conn"] = connPanel.ConnPanel(conn, connResolutionThread, panels["log"])
- panels["control"] = ControlPanel(panels["conn"].resolver, isBlindMode)
- panels["torrc"] = confPanel.ConfPanel(confLocation, conn, panels["log"])
+ # TODO: bug from not setting top is that the log panel might attempt to draw
+ # before being positioned - the following is a quick hack til rewritten
+ panels["log"].setPaused(True)
- # prevents netstat calls by connPanel if not being used
+ panels["conn"] = connPanel.ConnPanel(stdscr, conn)
+ panels["control"] = ControlPanel(stdscr, isBlindMode)
+ panels["torrc"] = confPanel.ConfPanel(stdscr, confLocation, conn)
+
+ # prevents connection resolution via the connPanel if not being used
if isBlindMode: panels["conn"].isDisabled = True
# provides error if pid coulnd't be determined (hopefully shouldn't happen...)
- if not torPid: panels["log"].monitor_event("WARN", "Unable to resolve tor pid, abandoning connection listing")
+ if not torPid: log.log(log.WARN, "Unable to resolve tor pid, abandoning connection listing")
# statistical monitors for graph
panels["graph"].addStats("bandwidth", bandwidthMonitor.BandwidthMonitor(conn))
panels["graph"].addStats("system resources", cpuMemMonitor.CpuMemMonitor(panels["header"]))
- if not isBlindMode: panels["graph"].addStats("connections", connCountMonitor.ConnCountMonitor(conn, connResolutionThread))
+ if not isBlindMode: panels["graph"].addStats("connections", connCountMonitor.ConnCountMonitor(conn))
panels["graph"].setStats("bandwidth")
# listeners that update bandwidth and log panels with Tor status
@@ -338,7 +413,7 @@
conn.add_event_listener(sighupTracker)
# tells Tor to listen to the events we're interested
- loggedEvents = setEventListening(loggedEvents, conn, panels["log"])
+ loggedEvents = setEventListening(loggedEvents, conn)
panels["log"].loggedEvents = loggedEvents # strips any that couldn't be set
# directs logged TorCtl events to log panel
@@ -352,7 +427,7 @@
" a. 'FetchUselessDescriptors 1' is set in your torrc", \
" b. the directory service is provided ('DirPort' defined)", \
" c. or tor is used as a client"]
- panels["log"].monitor_event("WARN", warning)
+ log.log(log.WARN, warning)
except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed): pass
isUnresponsive = False # true if it's been over ten seconds since the last BW event (probably due to Tor closing)
@@ -360,6 +435,7 @@
overrideKey = None # immediately runs with this input rather than waiting for the user if set
page = 0
regexFilters = [] # previously used log regex filters
+ panels["popup"].redraw() # hack to make sure popup has a window instance (not entirely sure why...)
while True:
# tried only refreshing when the screen was resized but it caused a
@@ -388,9 +464,16 @@
# originally this checked in the bounds changed but 'recreate' is a no-op
# if panel properties are unchanged and checking every redraw is more
# resilient in case of funky changes (such as resizing during popups)
+
+ # hack to make sure header picks layout before using the dimensions below
+ panels["header"].getPreferredSize()
+
startY = 0
for panelKey in PAGE_S[:2]:
- panels[panelKey].recreate(stdscr, -1, startY)
+ #panels[panelKey].recreate(stdscr, -1, startY)
+ panels[panelKey].setParent(stdscr)
+ panels[panelKey].setWidth(-1)
+ panels[panelKey].setTop(startY)
startY += panels[panelKey].height
panels["popup"].recreate(stdscr, 80, startY)
@@ -399,30 +482,31 @@
tmpStartY = startY
for panelKey in panelSet:
- panels[panelKey].recreate(stdscr, -1, tmpStartY)
+ #panels[panelKey].recreate(stdscr, -1, tmpStartY)
+ panels[panelKey].setParent(stdscr)
+ panels[panelKey].setWidth(-1)
+ panels[panelKey].setTop(tmpStartY)
tmpStartY += panels[panelKey].height
# if it's been at least ten seconds since the last BW event Tor's probably done
if not isUnresponsive and not panels["log"].controlPortClosed and panels["log"].getHeartbeat() >= 10:
isUnresponsive = True
- panels["log"].monitor_event("NOTICE", "Relay unresponsive (last heartbeat: %s)" % time.ctime(panels["log"].lastHeartbeat))
+ log.log(log.NOTICE, "Relay unresponsive (last heartbeat: %s)" % time.ctime(panels["log"].lastHeartbeat))
elif not panels["log"].controlPortClosed and (isUnresponsive and panels["log"].getHeartbeat() < 10):
# shouldn't happen unless Tor freezes for a bit - BW events happen every second...
isUnresponsive = False
- panels["log"].monitor_event("NOTICE", "Relay resumed")
+ log.log(log.NOTICE, "Relay resumed")
panels["conn"].reset()
+ # TODO: part two of hack to prevent premature drawing by log panel
+ if page == 0 and not isPaused: panels["log"].setPaused(False)
+
# I haven't the foggiest why, but doesn't work if redrawn out of order...
+ for panelKey in (PAGE_S + PAGES[page]):
+ # redrawing popup can result in display flicker when it should be hidden
+ if panelKey != "popup": panels[panelKey].redraw()
- # TODO: temporary hack to prevent popup redraw until a valid replacement is implemented (ick!)
- tmpSubwin = panels["popup"].win
- panels["popup"].win = None
-
- for panelKey in (PAGE_S + PAGES[page]): panels[panelKey].redraw()
-
- panels["popup"].win = tmpSubwin
-
stdscr.refresh()
finally:
panel.CURSES_LOCK.release()
@@ -461,16 +545,8 @@
# quits arm
# very occasionally stderr gets "close failed: [Errno 11] Resource temporarily unavailable"
# this appears to be a python bug: http://bugs.python.org/issue3014
- daemonThreads = panels["conn"].resolver.threadPool
-
- # sets halt flags for all worker daemon threads
- for worker in daemonThreads: worker.halt = True
-
- # joins on workers (prevents noisy termination)
- for worker in daemonThreads: worker.join()
-
+ hostnames.stop()
conn.close() # joins on TorCtl event thread
-
break
elif key == curses.KEY_LEFT or key == curses.KEY_RIGHT:
# switch page
@@ -487,7 +563,11 @@
setPauseState(panels, isPaused, page)
panels["control"].page = page + 1
- panels["control"].refresh()
+
+ # TODO: this redraw doesn't seem necessary (redraws anyway after this
+ # loop) - look into this when refactoring
+ panels["control"].redraw()
+
elif key == ord('p') or key == ord('P'):
# toggles update freezing
panel.CURSES_LOCK.acquire()
@@ -507,7 +587,7 @@
popup = panels["popup"]
popup.clear()
popup.win.box()
- popup.addstr(0, 0, "Page %i Commands:" % (page + 1), uiTools.LABEL_ATTR)
+ popup.addstr(0, 0, "Page %i Commands:" % (page + 1), curses.A_STANDOUT)
pageOverrideKeys = ()
@@ -516,7 +596,7 @@
if not graphedStats: graphedStats = "none"
popup.addfstr(1, 2, "<b>s</b>: graphed stats (<b>%s</b>)" % graphedStats)
popup.addfstr(1, 41, "<b>i</b>: graph update interval (<b>%s</b>)" % panels["graph"].updateInterval)
- popup.addfstr(2, 2, "b: graph bounds (<b>%s</b>)" % graphPanel.BOUND_LABELS[panels["graph"].bounds])
+ popup.addfstr(2, 2, "<b>b</b>: graph bounds (<b>%s</b>)" % graphPanel.BOUND_LABELS[panels["graph"].bounds])
popup.addfstr(2, 41, "<b>d</b>: file descriptors")
popup.addfstr(3, 2, "<b>e</b>: change logged events")
@@ -525,38 +605,44 @@
pageOverrideKeys = (ord('s'), ord('i'), ord('d'), ord('e'), ord('r'), ord('f'))
if page == 1:
- popup.addstr(1, 2, "up arrow: scroll up a line")
- popup.addstr(1, 41, "down arrow: scroll down a line")
- popup.addstr(2, 2, "page up: scroll up a page")
- popup.addstr(2, 41, "page down: scroll down a page")
- popup.addstr(3, 2, "enter: connection details")
+ popup.addfstr(1, 2, "<b>up arrow</b>: scroll up a line")
+ popup.addfstr(1, 41, "<b>down arrow</b>: scroll down a line")
+ popup.addfstr(2, 2, "<b>page up</b>: scroll up a page")
+ popup.addfstr(2, 41, "<b>page down</b>: scroll down a page")
+ popup.addfstr(3, 2, "<b>enter</b>: connection details")
popup.addfstr(3, 41, "<b>d</b>: raw consensus descriptor")
listingType = connPanel.LIST_LABEL[panels["conn"].listingType].lower()
popup.addfstr(4, 2, "<b>l</b>: listed identity (<b>%s</b>)" % listingType)
+ resolverUtil = connections.getResolver("tor").overwriteResolver
+ if resolverUtil == None: resolverUtil = "auto"
+ else: resolverUtil = connections.CMD_STR[resolverUtil]
+ popup.addfstr(4, 41, "<b>u</b>: resolving utility (<b>%s</b>)" % resolverUtil)
+
allowDnsLabel = "allow" if panels["conn"].allowDNS else "disallow"
- popup.addfstr(4, 41, "r: permit DNS resolution (<b>%s</b>)" % allowDnsLabel)
+ popup.addfstr(5, 2, "<b>r</b>: permit DNS resolution (<b>%s</b>)" % allowDnsLabel)
- popup.addfstr(5, 2, "<b>s</b>: sort ordering")
- popup.addfstr(5, 41, "<b>c</b>: client circuits")
+ popup.addfstr(5, 41, "<b>s</b>: sort ordering")
+ popup.addfstr(6, 2, "<b>c</b>: client circuits")
+
#popup.addfstr(5, 41, "c: toggle cursor (<b>%s</b>)" % ("on" if panels["conn"].isCursorEnabled else "off"))
pageOverrideKeys = (ord('d'), ord('l'), ord('s'), ord('c'))
elif page == 2:
- popup.addstr(1, 2, "up arrow: scroll up a line")
- popup.addstr(1, 41, "down arrow: scroll down a line")
- popup.addstr(2, 2, "page up: scroll up a page")
- popup.addstr(2, 41, "page down: scroll down a page")
+ popup.addfstr(1, 2, "<b>up arrow</b>: scroll up a line")
+ popup.addfstr(1, 41, "<b>down arrow</b>: scroll down a line")
+ popup.addfstr(2, 2, "<b>page up</b>: scroll up a page")
+ popup.addfstr(2, 41, "<b>page down</b>: scroll down a page")
strippingLabel = "on" if panels["torrc"].stripComments else "off"
- popup.addfstr(3, 2, "s: comment stripping (<b>%s</b>)" % strippingLabel)
+ popup.addfstr(3, 2, "<b>s</b>: comment stripping (<b>%s</b>)" % strippingLabel)
lineNumLabel = "on" if panels["torrc"].showLineNum else "off"
- popup.addfstr(3, 41, "n: line numbering (<b>%s</b>)" % lineNumLabel)
+ popup.addfstr(3, 41, "<b>n</b>: line numbering (<b>%s</b>)" % lineNumLabel)
- popup.addstr(4, 2, "r: reload torrc")
- popup.addstr(4, 41, "x: reset tor (issue sighup)")
+ popup.addfstr(4, 2, "<b>r</b>: reload torrc")
+ popup.addfstr(4, 41, "<b>x</b>: reset tor (issue sighup)")
popup.addstr(7, 2, "Press any key...")
popup.refresh()
@@ -662,7 +748,7 @@
popup.clear()
popup.win.box()
- popup.addstr(0, 0, "Event Types:", uiTools.LABEL_ATTR)
+ popup.addstr(0, 0, "Event Types:", curses.A_STANDOUT)
lineNum = 1
for line in logPanel.EVENT_LISTING.split("\n"):
line = line[6:]
@@ -684,7 +770,7 @@
if eventsInput != "":
try:
expandedEvents = logPanel.expandEvents(eventsInput)
- loggedEvents = setEventListening(expandedEvents, conn, panels["log"])
+ loggedEvents = setEventListening(expandedEvents, conn)
panels["log"].loggedEvents = loggedEvents
except ValueError, exc:
panels["control"].setMsg("Invalid flags: %s" % str(exc), curses.A_STANDOUT)
@@ -759,7 +845,7 @@
del regexFilters[selection]
except re.error, exc:
# shouldn't happen since we've already checked validity
- panels["log"].monitor_event("WARN", "Invalid regular expression ('%s': %s) - removing from listing" % (regexFilters[selection - 1], str(exc)))
+ log.log(log.WARN, "Invalid regular expression ('%s': %s) - removing from listing" % (regexFilters[selection - 1], str(exc)))
del regexFilters[selection - 1]
if len(regexFilters) > MAX_REGEX_FILTERS: del regexFilters[MAX_REGEX_FILTERS:]
@@ -771,7 +857,7 @@
# canceling hostname resolution (esc on any page)
panels["conn"].listingType = connPanel.LIST_IP
panels["control"].resolvingCounter = -1
- panels["conn"].resolver.setPaused(True)
+ hostnames.setPaused(True)
panels["conn"].sortConnections()
elif page == 1 and panels["conn"].isCursorEnabled and key in (curses.KEY_ENTER, 10, ord(' ')):
# provides details on selected connection
@@ -785,8 +871,7 @@
panels["conn"].showingDetails = True
panels["conn"].redraw()
- resolver = panels["conn"].resolver
- resolver.setPaused(not panels["conn"].allowDNS)
+ hostnames.setPaused(not panels["conn"].allowDNS)
relayLookupCache = {} # temporary cache of entry -> (ns data, desc data)
curses.cbreak() # wait indefinitely for key presses (no timeout)
@@ -795,7 +880,7 @@
while key not in (curses.KEY_ENTER, 10, ord(' ')):
popup.clear()
popup.win.box()
- popup.addstr(0, 0, "Connection Details:", uiTools.LABEL_ATTR)
+ popup.addstr(0, 0, "Connection Details:", curses.A_STANDOUT)
selection = panels["conn"].cursorSelection
if not selection or not panels["conn"].connections: break
@@ -813,17 +898,16 @@
addrLabel = "address: unknown"
if selectedIsPrivate: hostname = None
- else: hostname = resolver.resolve(selectedIp)
+ else:
+ try: hostname = hostnames.resolve(selectedIp)
+ except ValueError: hostname = "unknown" # hostname couldn't be resolved
if hostname == None:
- if resolver.isPaused or selectedIsPrivate: hostname = "DNS resolution disallowed"
- elif selectedIp not in resolver.resolvedCache.keys():
+ if hostnames.isPaused() or selectedIsPrivate: hostname = "DNS resolution disallowed"
+ else:
# if hostname is still being resolved refresh panel every half-second until it's completed
curses.halfdelay(5)
hostname = "resolving..."
- else:
- # hostname couldn't be resolved
- hostname = "unknown"
elif len(hostname) > 73 - len(addrLabel):
# hostname too long - truncate
hostname = "%s..." % hostname[:70 - len(addrLabel)]
@@ -871,7 +955,7 @@
if not lookupErrored and nsCall:
if len(nsCall) > 1:
# multiple records for fingerprint (shouldn't happen)
- panels["log"].monitor_event("WARN", "Multiple consensus entries for fingerprint: %s" % fingerprint)
+ log.log(log.WARN, "Multiple consensus entries for fingerprint: %s" % fingerprint)
nsEntry = nsCall[0]
@@ -919,7 +1003,7 @@
panels["conn"].showLabel = True
panels["conn"].showingDetails = False
- resolver.setPaused(not panels["conn"].allowDNS and panels["conn"].listingType == connPanel.LIST_HOSTNAME)
+ hostnames.setPaused(not panels["conn"].allowDNS and panels["conn"].listingType == connPanel.LIST_HOSTNAME)
setPauseState(panels, isPaused, page)
curses.halfdelay(REFRESH_RATE * 10) # reset normal pausing behavior
finally:
@@ -963,16 +1047,39 @@
if panels["conn"].listingType == connPanel.LIST_HOSTNAME:
curses.halfdelay(10) # refreshes display every second until done resolving
- panels["control"].resolvingCounter = panels["conn"].resolver.totalResolves - panels["conn"].resolver.unresolvedQueue.qsize()
+ panels["control"].resolvingCounter = hostnames.getRequestCount() - hostnames.getPendingCount()
- resolver = panels["conn"].resolver
- resolver.setPaused(not panels["conn"].allowDNS)
- for connEntry in panels["conn"].connections: resolver.resolve(connEntry[connPanel.CONN_F_IP])
+ hostnames.setPaused(not panels["conn"].allowDNS)
+ for connEntry in panels["conn"].connections:
+ try: hostnames.resolve(connEntry[connPanel.CONN_F_IP])
+ except ValueError: pass
else:
panels["control"].resolvingCounter = -1
- panels["conn"].resolver.setPaused(True)
+ hostnames.setPaused(True)
panels["conn"].sortConnections()
+ elif page == 1 and (key == ord('u') or key == ord('U')):
+ # provides menu to pick identification resolving utility
+ optionTypes = [None, connections.CMD_NETSTAT, connections.CMD_SS, connections.CMD_LSOF]
+ options = ["auto"] + [connections.CMD_STR[util] for util in optionTypes[1:]]
+
+ initialSelection = connections.getResolver("tor").overwriteResolver # enums correspond to indices
+ if initialSelection == None: initialSelection = 0
+
+ # hides top label of conn panel and pauses panels
+ panels["conn"].showLabel = False
+ panels["conn"].redraw()
+ setPauseState(panels, isPaused, page, True)
+
+ selection = showMenu(stdscr, panels["popup"], "Resolver Util:", options, initialSelection)
+
+ # reverts changes made for popup
+ panels["conn"].showLabel = True
+ setPauseState(panels, isPaused, page)
+
+ # applies new setting
+ if selection != -1 and optionTypes[selection] != connections.getResolver("tor").overwriteResolver:
+ connections.getResolver("tor").overwriteResolver = optionTypes[selection]
elif page == 1 and (key == ord('s') or key == ord('S')):
# set ordering for connection listing
panel.CURSES_LOCK.acquire()
@@ -998,7 +1105,7 @@
while len(selections) < 3:
popup.clear()
popup.win.box()
- popup.addstr(0, 0, "Connection Ordering:", uiTools.LABEL_ATTR)
+ popup.addstr(0, 0, "Connection Ordering:", curses.A_STANDOUT)
popup.addfstr(1, 2, prevOrdering)
# provides new ordering
@@ -1057,15 +1164,14 @@
# makes sure there's room for the longest entry
popup = panels["popup"]
- popup._resetBounds()
- if clientCircuits and maxEntryLength + 4 > popup.maxX:
+ if clientCircuits and maxEntryLength + 4 > popup.getPreferredSize()[1]:
popup.height = max(popup.height, len(clientCircuits) + 3)
popup.recreate(stdscr, maxEntryLength + 4)
# lists commands
popup.clear()
popup.win.box()
- popup.addstr(0, 0, "Client Circuits:", uiTools.LABEL_ATTR)
+ popup.addstr(0, 0, "Client Circuits:", curses.A_STANDOUT)
if clientCircuits == None:
popup.addstr(1, 2, "Unable to retireve current circuits")
@@ -1116,6 +1222,20 @@
confirmationKey = stdscr.getch()
if confirmationKey in (ord('x'), ord('X')):
try:
+ conn.send_signal("RELOAD")
+ except Exception, err:
+ # new torrc parameters caused an error (tor's likely shut down)
+ # BUG: this doesn't work - torrc errors still cause TorCtl to crash... :(
+ # http://bugs.noreply.org/flyspray/index.php?do=details&id=1329
+ log.log(log.ERR, "Error detected when reloading tor: %s" % str(err))
+ pass
+
+ # The following issues a sighup via a system command (Sebastian
+ # mentioned later that sending a RELOAD signal is equivilant, which
+ # is of course far preferable).
+
+ """
+ try:
# Redirects stderr to stdout so we can check error status (output
# should be empty if successful). Example error:
# pkill: 5592 - Operation not permitted
@@ -1152,6 +1272,7 @@
panels["control"].setMsg("Sighup failed%s" % errorMsg, curses.A_STANDOUT)
panels["control"].redraw()
time.sleep(2)
+ """
# reverts display settings
curses.halfdelay(REFRESH_RATE * 10)
Modified: arm/trunk/interface/descriptorPopup.py
===================================================================
--- arm/trunk/interface/descriptorPopup.py 2010-04-08 12:22:36 UTC (rev 22147)
+++ arm/trunk/interface/descriptorPopup.py 2010-04-08 16:25:14 UTC (rev 22148)
@@ -7,6 +7,7 @@
import curses
from TorCtl import TorCtl
+import controller
import connPanel
from util import panel, uiTools
@@ -98,8 +99,7 @@
# tracks number of extra lines that will be taken due to text wrap
height += (lineWidth - 2) / connectionPanel.maxX
- popup._resetBounds()
- popup.height = min(len(properties.text) + height + 2, connectionPanel.maxY)
+ popup.setHeight(min(len(properties.text) + height + 2, connectionPanel.maxY))
popup.recreate(stdscr, width)
while isVisible:
@@ -115,7 +115,7 @@
break
else: properties.handleKey(key, popup.height - 2)
- popup.height = 9
+ popup.setHeight(9)
popup.recreate(stdscr, 80)
finally:
panel.CURSES_LOCK.release()
@@ -126,8 +126,8 @@
xOffset = 2
if properties.text:
- if properties.fingerprint: popup.addstr(0, 0, "Consensus Descriptor (%s):" % properties.fingerprint, uiTools.LABEL_ATTR)
- else: popup.addstr(0, 0, "Consensus Descriptor:", uiTools.LABEL_ATTR)
+ if properties.fingerprint: popup.addstr(0, 0, "Consensus Descriptor (%s):" % properties.fingerprint, curses.A_STANDOUT)
+ else: popup.addstr(0, 0, "Consensus Descriptor:", curses.A_STANDOUT)
isEncryption = False # true if line is part of an encryption block
@@ -171,8 +171,8 @@
keyword, remainder = lineText, ""
keywordFormat = uiTools.getColor(SIG_COLOR)
- lineNum, xLoc = popup.addstr_wrap(lineNum, 0, keyword, keywordFormat, xOffset + numOffset, popup.maxX - 1, popup.maxY - 1)
- lineNum, xLoc = popup.addstr_wrap(lineNum, xLoc, remainder, remainderFormat, xOffset + numOffset, popup.maxX - 1, popup.maxY - 1)
+ lineNum, xLoc = controller.addstr_wrap(popup, lineNum, 0, keyword, keywordFormat, xOffset + numOffset, popup.maxX - 1, popup.maxY - 1)
+ lineNum, xLoc = controller.addstr_wrap(popup, lineNum, xLoc, remainder, remainderFormat, xOffset + numOffset, popup.maxX - 1, popup.maxY - 1)
lineNum += 1
if lineNum > pageHeight: break
Modified: arm/trunk/interface/fileDescriptorPopup.py
===================================================================
--- arm/trunk/interface/fileDescriptorPopup.py 2010-04-08 12:22:36 UTC (rev 22147)
+++ arm/trunk/interface/fileDescriptorPopup.py 2010-04-08 16:25:14 UTC (rev 22148)
@@ -116,8 +116,7 @@
popupHeight = len(properties.fdFile) + len(properties.fdConn) + len(properties.fdMisc) + 4
- popup._resetBounds()
- popup.height = popupHeight
+ popup.setHeight(popupHeight)
popup.recreate(stdscr, popupWidth)
while True:
Modified: arm/trunk/interface/graphPanel.py
===================================================================
--- arm/trunk/interface/graphPanel.py 2010-04-08 12:22:36 UTC (rev 22147)
+++ arm/trunk/interface/graphPanel.py 2010-04-08 16:25:14 UTC (rev 22148)
@@ -163,8 +163,8 @@
implementations.
"""
- def __init__(self):
- panel.Panel.__init__(self, 0) # height is overwritten with current module
+ def __init__(self, stdscr):
+ panel.Panel.__init__(self, stdscr, 0, 0) # height is overwritten with current module
self.updateInterval = DEFAULT_UPDATE_INTERVAL
self.isPaused = False
self.showLabel = True # shows top label if true, hides otherwise
@@ -172,20 +172,20 @@
self.currentDisplay = None # label of the stats currently being displayed
self.stats = {} # available stats (mappings of label -> instance)
- def draw(self):
+ def draw(self, subwindow, width, height):
""" Redraws graph panel """
- graphCol = min((self.maxX - 10) / 2, MAX_GRAPH_COL)
+ graphCol = min((width - 10) / 2, MAX_GRAPH_COL)
if self.currentDisplay:
param = self.stats[self.currentDisplay]
primaryColor = uiTools.getColor(param.primaryColor)
secondaryColor = uiTools.getColor(param.secondaryColor)
- if self.showLabel: self.addstr(0, 0, param.getTitle(self.maxX), uiTools.LABEL_ATTR)
+ if self.showLabel: self.addstr(0, 0, param.getTitle(width), curses.A_STANDOUT)
# top labels
- left, right = param.getHeaderLabel(self.maxX / 2, True), param.getHeaderLabel(self.maxX / 2, False)
+ left, right = param.getHeaderLabel(width / 2, True), param.getHeaderLabel(width / 2, False)
if left: self.addstr(1, 0, left, curses.A_BOLD | primaryColor)
if right: self.addstr(1, graphCol + 5, right, curses.A_BOLD | secondaryColor)
Modified: arm/trunk/interface/headerPanel.py
===================================================================
--- arm/trunk/interface/headerPanel.py 2010-04-08 12:22:36 UTC (rev 22147)
+++ arm/trunk/interface/headerPanel.py 2010-04-08 16:25:14 UTC (rev 22148)
@@ -37,8 +37,8 @@
fingerprint: BDAD31F6F318E0413833E8EBDA956F76E4D66788
"""
- def __init__(self, conn, torPid):
- panel.Panel.__init__(self, 6)
+ def __init__(self, stdscr, conn, torPid):
+ panel.Panel.__init__(self, stdscr, 0, 6)
self.vals = {"pid": torPid} # mapping of information to be presented
self.conn = conn # Tor control port connection
self.isPaused = False
@@ -46,28 +46,29 @@
self.rightParamX = 0 # offset used for doubled up parameters
self.lastUpdate = -1 # time last stats was retrived
self._updateParams()
+ self.getPreferredSize() # hack to force properly initialize size (when using wide version)
- def recreate(self, stdscr, maxX=-1, newTop=None):
- # might need to recreate twice so we have a window to get width
- if not self.win: panel.Panel.recreate(self, stdscr, maxX, newTop)
-
- self._resetBounds()
- self.isWide = self.maxX >= MIN_DUAL_ROW_WIDTH
- self.rightParamX = max(self.maxX / 2, 75) if self.isWide else 0
- self.height = 4 if self.isWide else 6
-
- panel.Panel.recreate(self, stdscr, maxX, newTop)
+ def getPreferredSize(self):
+ # width partially determines height (panel has two layouts)
+ panelHeight, panelWidth = panel.Panel.getPreferredSize(self)
+ self.isWide = panelWidth >= MIN_DUAL_ROW_WIDTH
+ self.rightParamX = max(panelWidth / 2, 75) if self.isWide else 0
+ self.setHeight(4 if self.isWide else 6)
+ return panel.Panel.getPreferredSize(self)
- def draw(self):
+ def draw(self, subwindow, width, height):
if not self.isPaused: self._updateParams()
+ # TODO: remove after a few revisions if this issue can't be reproduced
+ # (seemed to be a freak ui problem...)
+
# extra erase/refresh is needed to avoid internal caching screwing up and
# refusing to redisplay content in the case of graphical glitches - probably
# an obscure curses bug...
- self.win.erase()
- self.win.refresh()
+ #self.win.erase()
+ #self.win.refresh()
- self.clear()
+ #self.clear()
# Line 1 (system and tor version information)
systemNameLabel = "arm - %s " % self.vals["sys-name"]
@@ -93,7 +94,7 @@
# truncates torVersionLabel if too long
torVersionLabel = self.vals["version"]
- versionLabelMaxWidth = (self.rightParamX if self.isWide else self.maxX) - 51 - len(versionStatus)
+ versionLabelMaxWidth = (self.rightParamX if self.isWide else width) - 51 - len(versionStatus)
if len(torVersionLabel) > versionLabelMaxWidth:
torVersionLabel = torVersionLabel[:versionLabelMaxWidth - 1].strip() + "-"
@@ -143,6 +144,7 @@
policies = exitPolicy.split(", ")
# color codes accepts to be green, rejects to be red, and default marker to be cyan
+ # TODO: instead base this on if there's space available for the full verbose version
isSimple = len(policies) <= 2 # if policy is short then it's kept verbose, otherwise 'accept' and 'reject' keywords removed
for i in range(len(policies)):
policy = policies[i].strip()
@@ -197,7 +199,7 @@
# fetch exit policy (might span over multiple lines)
exitPolicyEntries = []
for (key, value) in self.conn.get_option("ExitPolicy"):
- exitPolicyEntries.append(value)
+ if value: exitPolicyEntries.append(value)
self.vals["ExitPolicy"] = ", ".join(exitPolicyEntries)
Deleted: arm/trunk/interface/hostnameResolver.py
===================================================================
--- arm/trunk/interface/hostnameResolver.py 2010-04-08 12:22:36 UTC (rev 22147)
+++ arm/trunk/interface/hostnameResolver.py 2010-04-08 16:25:14 UTC (rev 22148)
@@ -1,150 +0,0 @@
-#!/usr/bin/env python
-# hostnameResolver.py -- Background thread for performing reverse DNS resolution.
-# Released under the GPL v3 (http://www.gnu.org/licenses/gpl.html)
-
-import os
-import time
-import itertools
-import Queue
-from threading import Thread
-
-RESOLVER_THREAD_POOL_SIZE = 5 # upping to around 30 causes the program to intermittently seize
-RESOLVER_MAX_CACHE_SIZE = 5000
-RESOLVER_CACHE_TRIM_SIZE = 2000 # entries removed when max cache size reached
-DNS_ERROR_CODES = ("1(FORMERR)", "2(SERVFAIL)", "3(NXDOMAIN)", "4(NOTIMP)", "5(REFUSED)", "6(YXDOMAIN)", "7(YXRRSET)", "8(NXRRSET)", "9(NOTAUTH)", "10(NOTZONE)", "16(BADVERS)")
-
-class HostnameResolver():
- """
- Provides background threads that quietly performs reverse DNS lookup of
- address with caching. This is non-blocking, providing None in the case of
- errors or new requests.
- """
-
- # Resolutions are made using os 'host' calls as opposed to 'gethostbyaddr' in
- # the socket module because the later appears to be a blocking call (ie, serial
- # requests which vastly reduces performance). In theory this shouldn't be the
- # case if your system has the gethostbyname_r function, which you can check
- # for with:
- # import distutils.sysconfig
- # distutils.sysconfig.get_config_var("HAVE_GETHOSTBYNAME_R")
- # however, I didn't find this to be the case. As always, suggestions welcome!
-
- def __init__(self):
- self.resolvedCache = {} # IP Address => (hostname, age) (None if couldn't be resolved)
- self.unresolvedQueue = Queue.Queue()
- self.recentQueries = [] # recent resolution requests to prevent duplicate requests
- self.counter = itertools.count() # atomic counter to track age of entries (for trimming)
- self.threadPool = [] # worker threads that process requests
- self.totalResolves = 0 # counter for the total number of addresses querried to be resolved
- self.isPaused = True
-
- for i in range(RESOLVER_THREAD_POOL_SIZE):
- t = _ResolverWorker(self.resolvedCache, self.unresolvedQueue, self.counter)
- t.setDaemon(True)
- t.setPaused(self.isPaused)
- t.start()
- self.threadPool.append(t)
-
- def resolve(self, ipAddr, blockTime = 0):
- """
- Provides hostname associated with an IP address. If not found this returns
- None and performs a reverse DNS lookup for future reference. This also
- provides None if the address couldn't be resolved. This can be made to block
- if some delay is tolerable.
- """
-
- # if outstanding requests are done then clear recentQueries so we can run erronious requests again
- if self.unresolvedQueue.empty(): self.recentQueries = []
-
- if ipAddr in self.resolvedCache.keys():
- return self.resolvedCache[ipAddr][0]
- elif ipAddr not in self.recentQueries:
- self.totalResolves += 1
- self.recentQueries.append(ipAddr)
- self.unresolvedQueue.put(ipAddr)
-
- if len(self.resolvedCache) > RESOLVER_MAX_CACHE_SIZE:
- # trims cache (clean out oldest entries)
- currentCount = self.counter.next()
- threshold = currentCount - (RESOLVER_MAX_CACHE_SIZE - RESOLVER_CACHE_TRIM_SIZE) # max count of entries being removed
- toDelete = []
-
- # checks age of each entry, adding to toDelete if too old
- for ipAddr in self.resolvedCache.keys():
- if self.resolvedCache[ipAddr][1] < threshold: toDelete.append(ipAddr)
-
- for entryAddr in toDelete: del self.resolvedCache[entryAddr]
-
- if blockTime > 0 and not self.isPaused:
- timeWaited = 0
-
- while ipAddr not in self.resolvedCache.keys() and timeWaited < blockTime:
- time.sleep(0.1)
- timeWaited += 0.1
-
- if ipAddr in self.resolvedCache.keys(): return self.resolvedCache[ipAddr][0]
- else: return None
-
- def setPaused(self, isPause):
- """
- If true, prevents further dns requests.
- """
-
- if isPause == self.isPaused: return
- self.isPaused = isPause
- for t in self.threadPool: t.setPaused(self.isPaused)
-
-class _ResolverWorker(Thread):
- """
- Helper thread for HostnameResolver, performing lookups on unresolved IP
- addresses and adding the results to the resolvedCache.
- """
-
- def __init__(self, resolvedCache, unresolvedQueue, counter):
- Thread.__init__(self)
- self.resolvedCache = resolvedCache
- self.unresolvedQueue = unresolvedQueue
- self.counter = counter
- self.isPaused = False
- self.halt = False # terminates thread if true
-
- def run(self):
- while not self.halt:
- while self.isPaused and not self.halt: time.sleep(0.25)
- if self.halt: break
-
- # snags next available ip, timeout is because queue can't be woken up
- # when 'halt' is set
- try: ipAddr = self.unresolvedQueue.get(True, 0.25)
- except Queue.Empty: continue
-
- resolutionFailed = False # if true don't cache results
- hostCall = os.popen("host %s 2> /dev/null" % ipAddr)
-
- try:
- hostname = hostCall.read()
- if hostname: hostname = hostname.split()[-1:][0]
- else: raise IOError # call failed ('host' command probably unavailable)
-
- if hostname == "reached":
- # got message: ";; connection timed out; no servers could be reached"
- resolutionFailed = True
- elif hostname in DNS_ERROR_CODES:
- # got error response (can't do resolution on address)
- hostname = None
- else:
- # strips off ending period
- hostname = hostname[:-1]
- except IOError: resolutionFailed = True # host call failed
-
- hostCall.close()
- if not resolutionFailed: self.resolvedCache[ipAddr] = (hostname, self.counter.next())
- self.unresolvedQueue.task_done() # signals that job's done
-
- def setPaused(self, isPause):
- """
- Puts further work on hold if true.
- """
-
- self.isPaused = isPause
-
Modified: arm/trunk/interface/logPanel.py
===================================================================
--- arm/trunk/interface/logPanel.py 2010-04-08 12:22:36 UTC (rev 22147)
+++ arm/trunk/interface/logPanel.py 2010-04-08 16:25:14 UTC (rev 22148)
@@ -8,7 +8,7 @@
from curses.ascii import isprint
from TorCtl import TorCtl
-from util import panel, uiTools
+from util import log, panel, uiTools
PRE_POPULATE_LOG = True # attempts to retrieve events from log file if available
@@ -89,9 +89,9 @@
Tor event listener, noting messages, the time, and their type in a panel.
"""
- def __init__(self, conn, loggedEvents):
+ def __init__(self, stdscr, conn, loggedEvents):
TorCtl.PostEventListener.__init__(self)
- panel.Panel.__init__(self, -1)
+ panel.Panel.__init__(self, stdscr, 0)
self.scroll = 0
self.msgLog = [] # tuples of (logText, color)
self.isPaused = False
@@ -102,6 +102,12 @@
self.eventTimeOverwrite = None # replaces time for further events with this (uses time it occures if None)
self.controlPortClosed = False # flag set if TorCtl provided notice that control port is closed
+ # prevents attempts to redraw while processing batch of events
+ previousPauseState = self.isPaused
+ self.setPaused(True)
+ log.addListeners([log.DEBUG, log.INFO, log.NOTICE, log.WARN, log.ERR], self.arm_event_wrapper, True)
+ self.setPaused(previousPauseState)
+
# attempts to process events from log file
if PRE_POPULATE_LOG:
previousPauseState = self.isPaused
@@ -151,8 +157,7 @@
def handleKey(self, key):
# scroll movement
if key in (curses.KEY_UP, curses.KEY_DOWN, curses.KEY_PPAGE, curses.KEY_NPAGE):
- self._resetBounds()
- pageHeight, shift = self.maxY - 1, 0
+ pageHeight, shift = self.getPreferredSize()[0] - 1, 0
# location offset
if key == curses.KEY_UP: shift = -1
@@ -229,6 +234,12 @@
def unknown_event(self, event):
if "UNKNOWN" in self.loggedEvents: self.registerEvent("UNKNOWN", event.event_string, "red")
+ def arm_event_wrapper(self, level, msg, eventTime):
+ # temporary adaptor hack to use the new logging functions until I'm sure they'll work
+ # TODO: insert into log according to the event's timestamp (harder part
+ # here will be interpreting tor's event timestamps...)
+ self.monitor_event(level, msg)
+
def monitor_event(self, level, msg):
# events provided by the arm monitor
if "ARM_" + level in self.loggedEvents: self.registerEvent("ARM-%s" % level, msg, RUNLEVEL_EVENT_COLOR[level])
@@ -251,7 +262,7 @@
if TOR_CTL_CLOSE_MSG in msg:
# TorCtl providing notice that control port is closed
self.controlPortClosed = True
- self.monitor_event("NOTICE", "Tor control port closed")
+ log.log(log.NOTICE, "Tor control port closed")
self.tor_ctl_event(level, msg)
def flush(self): pass
@@ -287,13 +298,13 @@
if len(self.msgLog) > MAX_LOG_ENTRIES: del self.msgLog[MAX_LOG_ENTRIES:]
self.redraw()
- def draw(self):
+ def draw(self, subwindow, width, height):
"""
Redraws message log. Entries stretch to use available space and may
contain up to two lines. Starts with newest entries.
"""
- isScrollBarVisible = self.getLogDisplayLength() > self.maxY - 1
+ isScrollBarVisible = self.getLogDisplayLength() > height - 1
xOffset = 3 if isScrollBarVisible else 0 # content offset for scroll bar
# draws label - uses ellipsis if too long, for instance:
@@ -315,22 +326,22 @@
if firstLabelLen == -1: firstLabelLen = len(eventsListing)
else: firstLabelLen += 3
- if self.maxX > 10 + firstLabelLen:
+ if width > 10 + firstLabelLen:
eventsLabel += " ("
- if len(eventsListing) > self.maxX - 11:
- labelBreak = eventsListing[:self.maxX - 12].rfind(", ")
+ if len(eventsListing) > width - 11:
+ labelBreak = eventsListing[:width - 12].rfind(", ")
eventsLabel += "%s..." % eventsListing[:labelBreak]
- elif len(eventsListing) + len(filterLabel) > self.maxX - 11:
+ elif len(eventsListing) + len(filterLabel) > width - 11:
eventsLabel += eventsListing
else: eventsLabel += eventsListing + filterLabel
eventsLabel += ")"
eventsLabel += ":"
- self.addstr(0, 0, eventsLabel, uiTools.LABEL_ATTR)
+ self.addstr(0, 0, eventsLabel, curses.A_STANDOUT)
# log entries
- maxLoc = self.getLogDisplayLength() - self.maxY + 1
+ maxLoc = self.getLogDisplayLength() - height + 1
self.scroll = max(0, min(self.scroll, maxLoc))
lineCount = 1 - self.scroll
@@ -339,18 +350,18 @@
continue # filter doesn't match log message - skip
# splits over too lines if too long
- if len(line) < self.maxX:
+ if len(line) < width:
if lineCount >= 1: self.addstr(lineCount, xOffset, line, uiTools.getColor(color))
lineCount += 1
else:
- (line1, line2) = splitLine(line, self.maxX - xOffset)
+ (line1, line2) = splitLine(line, width - xOffset)
if lineCount >= 1: self.addstr(lineCount, xOffset, line1, uiTools.getColor(color))
if lineCount >= 0: self.addstr(lineCount + 1, xOffset, line2, uiTools.getColor(color))
lineCount += 2
- if lineCount >= self.maxY: break # further log messages wouldn't fit
+ if lineCount >= height: break # further log messages wouldn't fit
- if isScrollBarVisible: uiTools.drawScrollBar(self, 1, self.maxY - 1, self.scroll, self.scroll + self.maxY - 1, self.getLogDisplayLength())
+ if isScrollBarVisible: self.addScrollBar(self.scroll, self.scroll + height - 1, self.getLogDisplayLength(), 1)
def getLogDisplayLength(self):
"""
@@ -360,10 +371,9 @@
logLength = len(self.msgLog)
# takes into account filtered and wrapped messages
- self._resetBounds()
for (line, color) in self.msgLog:
if self.regexFilter and not self.regexFilter.search(line): logLength -= 1
- elif len(line) >= self.maxX: logLength += 1
+ elif len(line) >= self.getPreferredSize()[1]: logLength += 1
return logLength
@@ -378,7 +388,7 @@
if self.isPaused: self.pauseBuffer = []
else:
self.msgLog = (self.pauseBuffer + self.msgLog)[:MAX_LOG_ENTRIES]
- self.redraw()
+ if self.win: self.redraw() # hack to avoid redrawing during init
def getHeartbeat(self):
"""
Modified: arm/trunk/util/__init__.py
===================================================================
--- arm/trunk/util/__init__.py 2010-04-08 12:22:36 UTC (rev 22147)
+++ arm/trunk/util/__init__.py 2010-04-08 16:25:14 UTC (rev 22148)
@@ -4,5 +4,5 @@
and safely working with curses (hiding some of the gory details).
"""
-__all__ = ["panel", "uiTools"]
+__all__ = ["connections", "hostnames", "log", "panel", "uiTools"]
Added: arm/trunk/util/connections.py
===================================================================
--- arm/trunk/util/connections.py (rev 0)
+++ arm/trunk/util/connections.py 2010-04-08 16:25:14 UTC (rev 22148)
@@ -0,0 +1,327 @@
+"""
+Fetches connection data (IP addresses and ports) associated with a given
+process. This sort of data can be retrieved via a variety of common *nix
+utilities:
+- netstat netstat -npt | grep <pid>/<process>
+- ss ss -p | grep "\"<process>\",<pid>"
+- lsof lsof -nPi | grep "<process>\s*<pid>.*(ESTABLISHED)"
+
+all queries dump its stderr (directing it to /dev/null).
+"""
+
+import os
+import sys
+import time
+import threading
+
+import log
+
+# enums for connection resolution utilities
+CMD_NETSTAT, CMD_SS, CMD_LSOF = range(1, 4)
+CMD_STR = {CMD_NETSTAT: "netstat", CMD_SS: "ss", CMD_LSOF: "lsof"}
+
+# formatted strings for the commands to be executed with the various resolvers
+# options are:
+# n = prevents dns lookups, p = include process, t = tcp only
+# output:
+# tcp 0 0 127.0.0.1:9051 127.0.0.1:53308 ESTABLISHED 9912/tor
+RUN_NETSTAT = "netstat -npt 2> /dev/null | grep %s/%s 2> /dev/null"
+
+# p = include process
+# output:
+# ESTAB 0 0 127.0.0.1:9051 127.0.0.1:53308 users:(("tor",9912,20))
+RUN_SS = "ss -p 2> /dev/null | grep \"\\\"%s\\\",%s\" 2> /dev/null"
+
+# n = prevent dns lookups, P = show port numbers (not names), i = ip only
+# output:
+# tor 9912 atagar 20u IPv4 33453 TCP 127.0.0.1:9051->127.0.0.1:53308
+RUN_LSOF = "lsof -nPi 2> /dev/null | grep \"%s\s*%s.*(ESTABLISHED)\" 2> /dev/null"
+
+RESOLVERS = [] # connection resolvers available via the singleton constructor
+RESOLVER_MIN_DEFAULT_LOOKUP = 5 # minimum seconds between lookups (unless overwritten)
+RESOLVER_SLEEP_INTERVAL = 1 # period to sleep when not resolving
+RESOLVER_FAILURE_TOLERANCE = 3 # number of subsequent failures before moving on to another resolver
+RESOLVER_SERIAL_FAILURE_MSG = "Querying connections with %s failed, trying %s"
+RESOLVER_FINAL_FAILURE_MSG = "All connection resolvers failed"
+
+def getConnections(resolutionCmd, processName, processPid = ""):
+ """
+ Retrieves a list of the current connections for a given process, providing a
+ tuple list of the form:
+ [(local_ipAddr1, local_port1, foreign_ipAddr1, foreign_port1), ...]
+ this raises an IOError if no connections are available or resolution fails
+ (in most cases these appear identical). Common issues include:
+ - insufficient permissions
+ - resolution command is unavailable
+ - usage of the command is non-standard (particularly an issue for BSD)
+
+ Arguments:
+ resolutionCmd - command to use in resolving the address
+ processName - name of the process for which connections are fetched
+ processPid - process ID (this helps improve accuracy)
+ """
+
+ if resolutionCmd == CMD_NETSTAT: cmd = RUN_NETSTAT % (processPid, processName)
+ elif resolutionCmd == CMD_SS: cmd = RUN_SS % (processName, processPid)
+ else: cmd = RUN_LSOF % (processName, processPid)
+
+ resolutionCall = os.popen(cmd)
+ results = resolutionCall.readlines()
+ resolutionCall.close()
+
+ if not results: raise IOError("Unable to resolve connections using: %s" % cmd)
+
+ # parses results for the resolution command
+ conn = []
+ for line in results:
+ comp = line.split()
+
+ if resolutionCmd == CMD_NETSTAT or resolutionCmd == CMD_SS:
+ localIp, localPort = comp[3].split(":")
+ foreignIp, foreignPort = comp[4].split(":")
+ else:
+ local, foreign = comp[7].split("->")
+ localIp, localPort = local.split(":")
+ foreignIp, foreignPort = foreign.split(":")
+
+ conn.append((localIp, localPort, foreignIp, foreignPort))
+
+ return conn
+
+def getResolver(processName, processPid = "", newInit = True):
+ """
+ Singleton constructor for resolver instances. If a resolver already exists
+ for the process then it's returned. Otherwise one is created and started.
+
+ Arguments:
+ processName - name of the process being resolved
+ processPid - pid of the process being resolved, if undefined this matches
+ against any resolver with the process name
+ newInit - if a resolver isn't available then one's created if true,
+ otherwise this returns None
+ """
+
+ # check if one's already been created
+ for resolver in RESOLVERS:
+ if resolver.processName == processName and (not processPid or resolver.processPid == processPid):
+ return resolver
+
+ # make a new resolver
+ if newInit:
+ r = ConnectionResolver(processName, processPid)
+ r.start()
+ RESOLVERS.append(r)
+ return r
+ else: return None
+
+def _isAvailable(command):
+ """
+ Checks the current PATH to see if a command is available or not. This returns
+ True if an accessible executable by the name is found and False otherwise.
+
+ Arguments:
+ command - name of the command for which to search
+ """
+
+ for path in os.environ["PATH"].split(os.pathsep):
+ cmdPath = os.path.join(path, command)
+ if os.path.exists(cmdPath) and os.access(cmdPath, os.X_OK): return True
+
+ return False
+
+if __name__ == '__main__':
+ # quick method for testing connection resolution
+ userInput = raw_input("Enter query (RESOLVER PROCESS_NAME [PID]: ").split()
+
+ # checks if there's enough arguments
+ if len(userInput) == 0: sys.exit(0)
+ elif len(userInput) == 1:
+ print "no process name provided"
+ sys.exit(1)
+
+ # translates resolver string to enum
+ userInput[0] = userInput[0].lower()
+ if userInput[0] == "ss": userInput[0] = CMD_SS
+ elif userInput[0] == "netstat": userInput[0] = CMD_NETSTAT
+ elif userInput[0] == "lsof": userInput[0] = CMD_LSOF
+ else:
+ print "unrecognized type of resolver: %s" % userInput[2]
+ sys.exit(1)
+
+ # resolves connections
+ try:
+ if len(userInput) == 2: connections = getConnections(userInput[0], userInput[1])
+ else: connections = getConnections(userInput[0], userInput[1], userInput[2])
+ except IOError, exc:
+ print exc
+ sys.exit(1)
+
+ # prints results
+ print "-" * 40
+ for lIp, lPort, fIp, fPort in connections:
+ print "%s:%s -> %s:%s" % (lIp, lPort, fIp, fPort)
+
+class ConnectionResolver(threading.Thread):
+ """
+ Service that periodically queries for a process' current connections. This
+ provides several benefits over on-demand queries:
+ - queries are non-blocking (providing cached results)
+ - falls back to use different resolution methods in case of repeated failures
+ - avoids overly frequent querying of connection data, which can be demanding
+ in terms of system resources
+
+ Unless an overriding method of resolution is requested this defaults to
+ choosing a resolver the following way:
+
+ - Checks the current PATH to determine which resolvers are available. This
+ uses the first of the following that's available:
+ netstat, ss, lsof (picks netstat if none are found)
+
+ - Attempts to resolve using the selection. Single failures are logged at the
+ INFO level, and a series of failures at NOTICE. In the later case this
+ blacklists the resolver, moving on to the next. If all resolvers fail this
+ way then resolution's abandoned and logs a WARN message.
+
+ The time between resolving connections, unless overwritten, is set to be
+ either five seconds or ten times the runtime of the resolver (whichever is
+ larger). This is to prevent systems either strapped for resources or with a
+ vast number of connections from being burdened too heavily by this daemon.
+
+ Parameters:
+ processName - name of the process being resolved
+ processPid - pid of the process being resolved
+ resolveRate - minimum time between resolving connections (in seconds,
+ None if using the default)
+ * defaultRate - default time between resolving connections
+ lastLookup - time connections were last resolved (unix time, -1 if
+ no resolutions have yet been successful)
+ overwriteResolver - method of resolution (uses default if None)
+ * defaultResolver - resolver used by default (None if all resolution
+ methods have been exhausted)
+
+ * read-only
+ """
+
+ def __init__(self, processName, processPid = "", resolveRate = None):
+ """
+ Initializes a new resolver daemon. When no longer needed it's suggested
+ that this is stopped.
+
+ Arguments:
+ processName - name of the process being resolved
+ processPid - pid of the process being resolved
+ resolveRate - time between resolving connections (in seconds, None if
+ chosen dynamically)
+ """
+
+ threading.Thread.__init__(self)
+ self.setDaemon(True)
+
+ self.processName = processName
+ self.processPid = processPid
+ self.resolveRate = resolveRate
+ self.defaultRate = RESOLVER_MIN_DEFAULT_LOOKUP
+ self.lastLookup = -1
+ self.overwriteResolver = None
+ self.defaultResolver = CMD_NETSTAT
+
+ # sets the default resolver to be the first found in the system's PATH
+ # (left as netstat if none are found)
+ for resolver in [CMD_NETSTAT, CMD_SS, CMD_LSOF]:
+ if _isAvailable(CMD_STR[resolver]):
+ self.defaultResolve = resolver
+ break
+
+ self._connections = [] # connection cache (latest results)
+ self._isPaused = False
+ self._halt = False # terminates thread if true
+ self._subsiquentFailures = 0 # number of failed resolutions with the default in a row
+ self._resolverBlacklist = [] # resolvers that have failed to resolve
+
+ def run(self):
+ while not self._halt:
+ minWait = self.resolveRate if self.resolveRate else self.defaultRate
+
+ if self._isPaused or time.time() - self.lastLookup < minWait:
+ time.sleep(RESOLVER_SLEEP_INTERVAL)
+ continue # done waiting, try again
+
+ isDefault = self.overwriteResolver == None
+ resolver = self.defaultResolver if isDefault else self.overwriteResolver
+
+ # checks if there's nothing to resolve with
+ if not resolver:
+ self.lastLookup = time.time() # avoids a busy wait in this case
+ continue
+
+ try:
+ resolveStart = time.time()
+ connResults = getConnections(resolver, self.processName, self.processPid)
+ lookupTime = time.time() - resolveStart
+
+ log.log(log.DEBUG, "%s queried in %.4f seconds (%i results)" % (CMD_STR[resolver], lookupTime, len(connResults)))
+
+ self._connections = connResults
+ self.defaultRate = max(5, 10 % lookupTime)
+ if isDefault: self._subsiquentFailures = 0
+ except IOError, exc:
+ log.log(log.INFO, str(exc)) # notice that a single resolution has failed
+
+ if isDefault:
+ self._subsiquentFailures += 1
+
+ if self._subsiquentFailures >= RESOLVER_FAILURE_TOLERANCE:
+ # failed several times in a row - abandon resolver and move on to another
+ self._resolverBlacklist.append(resolver)
+ self._subsiquentFailures = 0
+
+ # pick another (non-blacklisted) resolver
+ newResolver = None
+ for r in [CMD_NETSTAT, CMD_SS, CMD_LSOF]:
+ if not r in self._resolverBlacklist:
+ newResolver = r
+ break
+
+ if newResolver:
+ # provide notice that failures have occured and resolver is changing
+ log.log(log.NOTICE, RESOLVER_SERIAL_FAILURE_MSG % (CMD_STR[resolver], CMD_STR[newResolver]))
+ else:
+ # exhausted all resolvers, give warning
+ log.log(log.WARN, RESOLVER_FINAL_FAILURE_MSG)
+
+ self.defaultResolver = newResolver
+ finally:
+ self.lastLookup = time.time()
+
+ def getConnections(self):
+ """
+ Provides the last queried connection results, an empty list if resolver
+ has been halted.
+ """
+
+ if self._halt: return []
+ else: return list(self._connections)
+
+ def setPaused(self, isPause):
+ """
+ Allows or prevents further connection resolutions (this still makes use of
+ cached results).
+
+ Arguments:
+ isPause - puts a freeze on further resolutions if true, allows them to
+ continue otherwise
+ """
+
+ if isPause == self._isPaused: return
+ self._isPaused = isPause
+
+ def stop(self):
+ """
+ Halts further resolutions and terminates the thread.
+ """
+
+ self._halt = True
+
+ # removes this from consideration among active singleton instances
+ if self in RESOLVERS: RESOLVERS.remove(self)
+
Added: arm/trunk/util/hostnames.py
===================================================================
--- arm/trunk/util/hostnames.py (rev 0)
+++ arm/trunk/util/hostnames.py 2010-04-08 16:25:14 UTC (rev 22148)
@@ -0,0 +1,365 @@
+"""
+Service providing hostname resolution via reverse DNS lookups. This provides
+both resolution via a thread pool (looking up several addresses at a time) and
+caching of the results. If used, it's advisable that this service is stopped
+when it's no longer needed. All calls are both non-blocking and thread safe.
+
+Be aware that this relies on querying the system's DNS servers, possibly
+leaking the requested addresses to third parties.
+"""
+
+# The only points of concern in terms of concurrent calls are the RESOLVER and
+# RESOLVER.resolvedCache. This services provides (mostly) non-locking thread
+# safety via the following invariants:
+# - Resolver and cache instances are non-destructible
+# Nothing can be removed or invalidated. Rather, halting resolvers and
+# trimming the cache are done via reassignment (pointing the RESOLVER or
+# RESOLVER.resolvedCache to another copy).
+# - Functions create and use local references to the resolver and its cache
+# This is for consistency (ie, all operations are done on the same resolver
+# or cache instance regardless of concurrent assignments). Usually it's
+# assigned to a local variable called 'resolverRef' or 'cacheRef'.
+# - Locks aren't necessary, but used to help in the following cases:
+# - When assigning to the RESOLVER (to avoid orphaned instances with
+# running thread pools).
+# - When adding/removing from the cache (prevents workers from updating
+# an outdated cache reference).
+
+import os
+import time
+import socket
+import threading
+import itertools
+import Queue
+import distutils.sysconfig
+
+RESOLVER = None # hostname resolver (service is stopped if None)
+RESOLVER_LOCK = threading.RLock() # regulates assignment to the RESOLVER
+RESOLVER_CACHE_SIZE = 700000 # threshold for when cached results are discarded
+RESOLVER_CACHE_TRIM_SIZE = 200000 # number of entries discarded when the limit's reached
+RESOLVER_THREAD_POOL_SIZE = 5 # upping to around 30 causes the program to intermittently seize
+RESOLVER_COUNTER = itertools.count() # atomic counter, providing the age for new entries (for trimming)
+DNS_ERROR_CODES = ("1(FORMERR)", "2(SERVFAIL)", "3(NXDOMAIN)", "4(NOTIMP)", "5(REFUSED)", "6(YXDOMAIN)",
+ "7(YXRRSET)", "8(NXRRSET)", "9(NOTAUTH)", "10(NOTZONE)", "16(BADVERS)")
+
+# If true this allows for the use of socket.gethostbyaddr to resolve addresses
+# (this seems to be far slower, but would seem preferable if I'm wrong...).
+ALLOW_SOCKET_RESOLUTION = False
+
+def start():
+ """
+ Primes the service to start resolving addresses. Calling this explicitly is
+ not necessary since resolving any address will start the service if it isn't
+ already running.
+ """
+
+ global RESOLVER
+ RESOLVER_LOCK.acquire()
+ if not isRunning(): RESOLVER = _Resolver()
+ RESOLVER_LOCK.release()
+
+def stop():
+ """
+ Halts further resolutions and stops the service. This joins on the resolver's
+ thread pool and clears its lookup cache.
+ """
+
+ global RESOLVER
+ RESOLVER_LOCK.acquire()
+ if isRunning():
+ # Releases resolver instance. This is done first so concurrent calls to the
+ # service won't try to use it. However, using a halted instance is fine and
+ # all calls currently in progress can still proceed on the RESOLVER's local
+ # references.
+ resolverRef, RESOLVER = RESOLVER, None
+
+ # joins on its worker thread pool
+ resolverRef.halt = True
+ for t in resolverRef.threadPool: t.join()
+ RESOLVER_LOCK.release()
+
+def setPaused(isPause):
+ """
+ Allows or prevents further hostname resolutions (resolutions still make use of
+ cached entries if available). This starts the service if it isn't already
+ running.
+
+ Arguments:
+ isPause - puts a freeze on further resolutions if true, allows them to
+ continue otherwise
+ """
+
+ # makes sure a running resolver is set with the pausing setting
+ RESOLVER_LOCK.acquire()
+ start()
+ RESOLVER.isPaused = isPause
+ RESOLVER_LOCK.release()
+
+def isRunning():
+ """
+ Returns True if the service is currently running, False otherwise.
+ """
+
+ return bool(RESOLVER)
+
+def isPaused():
+ """
+ Returns True if the resolver is paused, False otherwise.
+ """
+
+ resolverRef = RESOLVER
+ if resolverRef: return resolverRef.isPaused
+ else: return False
+
+def isResolving():
+ """
+ Returns True if addresses are currently waiting to be resolved, False
+ otherwise.
+ """
+
+ resolverRef = RESOLVER
+ if resolverRef: return not resolverRef.unresolvedQueue.empty()
+ else: return False
+
+def resolve(ipAddr, timeout = 0, suppressIOExc = True):
+ """
+ Provides the hostname associated with a given IP address. By default this is
+ a non-blocking call, fetching cached results if available and queuing the
+ lookup if not. This provides None if the lookup fails (with a suppressed
+ exception) or timeout is reached without resolution. This starts the service
+ if it isn't already running.
+
+ If paused this simply returns the cached reply (no request is queued and
+ returns immediately regardless of the timeout argument).
+
+ Requests may raise the following exceptions:
+ - ValueError - address was unresolvable (includes the DNS error response)
+ - IOError - lookup failed due to os or network issues (suppressed by default)
+
+ Arguments:
+ ipAddr - ip address to be resolved
+ timeout - maximum duration to wait for a resolution (blocks to
+ completion if None)
+ suppressIOExc - suppresses lookup errors and re-runs failed calls if true,
+ raises otherwise
+ """
+
+ # starts the service if it isn't already running (making sure we have an
+ # instance in a thread safe fashion before continuing)
+ resolverRef = RESOLVER
+ if resolverRef == None:
+ RESOLVER_LOCK.acquire()
+ start()
+ resolverRef = RESOLVER
+ RESOLVER_LOCK.release()
+
+ if resolverRef.isPaused:
+ # get cache entry, raising if an exception and returning if a hostname
+ cacheRef = resolverRef.resolvedCache
+
+ if ipAddr in cacheRef.keys():
+ entry = cacheRef[ipAddr][0]
+ if suppressIOExc and type(entry) == IOError: return None
+ elif isinstance(entry, Exception): raise entry
+ else: return entry
+ else: return None
+ elif suppressIOExc:
+ # if resolver has cached an IOError then flush the entry (this defaults to
+ # suppression since these error may be transient)
+ cacheRef = resolverRef.resolvedCache
+ flush = ipAddr in cacheRef.keys() and type(cacheRef[ipAddr]) == IOError
+
+ try: return resolverRef.getHostname(ipAddr, timeout, flush)
+ except IOError: return None
+ else: return resolverRef.getHostname(ipAddr, timeout)
+
+def getPendingCount():
+ """
+ Provides an approximate count of the number of addresses still pending
+ resolution.
+ """
+
+ resolverRef = RESOLVER
+ if resolverRef: return resolverRef.unresolvedQueue.qsize()
+ else: return 0
+
+def getRequestCount():
+ """
+ Provides the number of resolutions requested since starting the service.
+ """
+
+ resolverRef = RESOLVER
+ if resolverRef: return resolverRef.totalResolves
+ else: return 0
+
+def _resolveViaSocket(ipAddr):
+ """
+ Performs hostname lookup via the socket module's gethostbyaddr function. This
+ raises an IOError if the lookup fails (network issue) and a ValueError in
+ case of DNS errors (address unresolvable).
+
+ Arguments:
+ ipAddr - ip address to be resolved
+ """
+
+ try:
+ # provides tuple like: ('localhost', [], ['127.0.0.1'])
+ return socket.gethostbyaddr(ipAddr)[0]
+ except socket.herror, exc:
+ if exc[0] == 2: raise IOError(exc[1]) # "Host name lookup failure"
+ else: raise ValueError(exc[1]) # usually "Unknown host"
+ except socket.error, exc: raise ValueError(exc[1])
+
+def _resolveViaHost(ipAddr):
+ """
+ Performs a host lookup for the given IP, returning the resolved hostname.
+ This raises an IOError if the lookup fails (os or network issue), and a
+ ValueError in the case of DNS errors (address is unresolvable).
+
+ Arguments:
+ ipAddr - ip address to be resolved
+ """
+
+ hostCall = os.popen("host %s 2> /dev/null" % ipAddr)
+ hostname = hostCall.read()
+ hostCall.close()
+
+ if hostname: hostname = hostname.split()[-1:][0]
+ else: raise IOError("lookup failed - is the host command available?")
+
+ if hostname == "reached":
+ # got message: ";; connection timed out; no servers could be reached"
+ raise IOError("lookup timed out")
+ elif hostname in DNS_ERROR_CODES:
+ # got error response (can't do resolution on address)
+ raise ValueError("address is unresolvable: %s" % hostname)
+ else:
+ # strips off ending period and returns hostname
+ return hostname[:-1]
+
+class _Resolver():
+ """
+ Performs reverse DNS resolutions. Lookups are a network bound operation so
+ this spawns a pool of worker threads to do several at a time in parallel.
+ """
+
+ def __init__(self):
+ # IP Address => (hostname/error, age), resolution failures result in a
+ # ValueError with the lookup's status
+ self.resolvedCache = {}
+
+ self.resolvedLock = threading.RLock() # governs concurrent access when modifying resolvedCache
+ self.unresolvedQueue = Queue.Queue() # unprocessed lookup requests
+ self.recentQueries = [] # recent resolution requests to prevent duplicate requests
+ self.threadPool = [] # worker threads that process requests
+ self.totalResolves = 0 # counter for the total number of addresses queried to be resolved
+ self.isPaused = False # prevents further resolutions if true
+ self.halt = False # if true, tells workers to stop
+
+ # Determines if resolutions are made using os 'host' calls or python's
+ # 'socket.gethostbyaddr'. The following checks if the system has the
+ # gethostbyname_r function, which determines if python resolutions can be
+ # done in parallel or not. If so, this is preferable.
+ isSocketResolutionParallel = distutils.sysconfig.get_config_var("HAVE_GETHOSTBYNAME_R")
+ self.useSocketResolution = ALLOW_SOCKET_RESOLUTION and isSocketResolutionParallel
+
+ for _ in range(RESOLVER_THREAD_POOL_SIZE):
+ t = threading.Thread(target = self._workerLoop)
+ t.setDaemon(True)
+ t.start()
+ self.threadPool.append(t)
+
+ def getHostname(self, ipAddr, timeout, flushCache = False):
+ """
+ Provides the hostname, queuing the request and returning None if the
+ timeout is reached before resolution. If a problem's encountered then this
+ either raises an IOError (for os and network issues) or ValueError (for DNS
+ resolution errors).
+
+ Arguments:
+ ipAddr - ip address to be resolved
+ timeout - maximum duration to wait for a resolution (blocks to
+ completion if None)
+ flushCache - if true the cache is skipped and address re-resolved
+ """
+
+ # if outstanding requests are done then clear recentQueries to allow
+ # entries removed from the cache to be re-run
+ if self.unresolvedQueue.empty(): self.recentQueries = []
+
+ # copies reference cache (this is important in case the cache is trimmed
+ # during this call)
+ cacheRef = self.resolvedCache
+
+ if not flushCache and ipAddr in cacheRef.keys():
+ # cached response is available - raise if an error, return if a hostname
+ response = cacheRef[ipAddr][0]
+ if isinstance(response, Exception): raise response
+ else: return response
+ elif flushCache or ipAddr not in self.recentQueries:
+ # new request - queue for resolution
+ self.totalResolves += 1
+ self.recentQueries.append(ipAddr)
+ self.unresolvedQueue.put(ipAddr)
+
+ # periodically check cache if requester is willing to wait
+ if timeout == None or timeout > 0:
+ startTime = time.time()
+
+ while timeout == None or time.time() - startTime < timeout:
+ if ipAddr in cacheRef.keys():
+ # address was resolved - raise if an error, return if a hostname
+ response = cacheRef[ipAddr][0]
+ if isinstance(response, Exception): raise response
+ else: return response
+ else: time.sleep(0.1)
+
+ return None # timeout reached without resolution
+
+ def _workerLoop(self):
+ """
+ Simple producer-consumer loop followed by worker threads. This takes
+ addresses from the unresolvedQueue, attempts to look up its hostname, and
+ adds its results or the error to the resolved cache. Resolver reference
+ provides shared resources used by the thread pool.
+ """
+
+ while not self.halt:
+ # if resolver is paused then put a hold on further resolutions
+ while self.isPaused and not self.halt: time.sleep(0.25)
+ if self.halt: break
+
+ # snags next available ip, timeout is because queue can't be woken up
+ # when 'halt' is set
+ try: ipAddr = self.unresolvedQueue.get(True, 0.25)
+ except Queue.Empty: continue
+ if self.halt: break
+
+ try:
+ if self.useSocketResolution: result = _resolveViaSocket(ipAddr)
+ else: result = _resolveViaHost(ipAddr)
+ except IOError, exc: result = exc # lookup failed
+ except ValueError, exc: result = exc # dns error
+
+ self.resolvedLock.acquire()
+ self.resolvedCache[ipAddr] = (result, RESOLVER_COUNTER.next())
+
+ # trim cache if excessively large (clearing out oldest entries)
+ if len(self.resolvedCache) > RESOLVER_CACHE_SIZE:
+ # Providing for concurrent, non-blocking calls require that entries are
+ # never removed from the cache, so this creates a new, trimmed version
+ # instead.
+
+ # determines minimum age of entries to be kept
+ currentCount = RESOLVER_COUNTER.next()
+ threshold = currentCount - (RESOLVER_CACHE_SIZE - RESOLVER_CACHE_TRIM_SIZE)
+ newCache = {}
+
+ # checks age of each entry, adding to toDelete if too old
+ for ipAddr, entry in self.resolvedCache.iteritems():
+ if entry[1] >= threshold: newCache[ipAddr] = entry
+
+ self.resolvedCache = newCache
+
+ self.resolvedLock.release()
+
Added: arm/trunk/util/log.py
===================================================================
--- arm/trunk/util/log.py (rev 0)
+++ arm/trunk/util/log.py 2010-04-08 16:25:14 UTC (rev 22148)
@@ -0,0 +1,156 @@
+"""
+Tracks application events, both directing them to attached listeners and
+keeping a record of them. A limited space is provided for old events, keeping
+and trimming them on a per-runlevel basis (ie, too many DEBUG events will only
+result in entries from that runlevel being dropped). All functions are thread
+safe.
+"""
+
+import time
+from sys import maxint
+from threading import RLock
+
+# logging runlevels
+DEBUG, INFO, NOTICE, WARN, ERR = range(1, 6)
+RUNLEVEL_STR = {DEBUG: "DEBUG", INFO: "INFO", NOTICE: "NOTICE", WARN: "WARN", ERR: "ERR"}
+
+LOG_LIMIT = 1000 # threshold (per runlevel) at which entries are discarded
+LOG_TRIM_SIZE = 200 # number of entries discarded when the limit's reached
+LOG_LOCK = RLock() # provides thread safety for logging operations
+
+# chronologically ordered records of events for each runlevel, stored as tuples
+# consisting of: (time, message)
+_backlog = dict([(level, []) for level in range(1, 6)])
+
+# mapping of runlevels to the listeners interested in receiving events from it
+_listeners = dict([(level, []) for level in range(1, 6)])
+
+def log(level, msg, eventTime = None):
+ """
+ Registers an event, directing it to interested listeners and preserving it in
+ the backlog.
+
+ Arguments:
+ level - runlevel coresponding to the message severity
+ msg - string associated with the message
+ eventTime - unix time at which the event occured, current time if undefined
+ """
+
+ if eventTime == None: eventTime = time.time()
+
+ LOG_LOCK.acquire()
+ try:
+ newEvent = (eventTime, msg)
+ eventBacklog = _backlog[level]
+
+ # inserts the new event into the backlog
+ if not eventBacklog or eventTime >= eventBacklog[-1][0]:
+ # newest event - append to end
+ eventBacklog.append(newEvent)
+ elif eventTime <= eventBacklog[0][0]:
+ # oldest event - insert at start
+ eventBacklog.insert(0, newEvent)
+ else:
+ # somewhere in the middle - start checking from the end
+ for i in range(len(eventBacklog) - 1, -1, -1):
+ if eventBacklog[i][0] <= eventTime:
+ eventBacklog.insert(i + 1, newEvent)
+ break
+
+ # turncates backlog if too long
+ toDelete = len(eventBacklog) - LOG_LIMIT
+ if toDelete >= 0: del eventBacklog[: toDelete + LOG_TRIM_SIZE]
+
+ # notifies listeners
+ for callback in _listeners[level]:
+ callback(RUNLEVEL_STR[level], msg, eventTime)
+ finally:
+ LOG_LOCK.release()
+
+def addListener(level, callback):
+ """
+ Directs future events to the given fallback function. The runlevels passed on
+ to listeners are provided as the corresponding strings ("DEBUG", "INFO",
+ "NOTICE", etc), and times in POSIX (unix) time.
+
+ Arguments:
+ level - event runlevel the listener should be notified of
+ callback - functor that'll accept the events, expected to be of the form:
+ myFunction(level, msg, time)
+ """
+
+ if not callback in _listeners[level]:
+ _listeners[level].append(callback)
+
+def addListeners(levels, callback, dumpBacklog = False):
+ """
+ Directs future events of multiple runlevels to the given callback function.
+
+ Arguments:
+ levels - list of runlevel events the listener should be notified of
+ callback - functor that'll accept the events, expected to be of the
+ form: myFunction(level, msg, time)
+ dumpBacklog - if true, any past events of the designated runlevels will be
+ provided to the listener before returning (in chronological
+ order)
+ """
+
+ LOG_LOCK.acquire()
+ try:
+ for level in levels: addListener(level, callback)
+
+ if dumpBacklog:
+ for level, msg, eventTime in _getEntries(levels):
+ callback(RUNLEVEL_STR[level], msg, eventTime)
+ finally:
+ LOG_LOCK.release()
+
+def removeListener(level, callback):
+ """
+ Stops listener from being notified of further events. This returns true if a
+ listener's removed, false otherwise.
+
+ Arguments:
+ level - runlevel the listener to be removed
+ callback - functor to be removed
+ """
+
+ if callback in _listeners[level]:
+ _listeners[level].remove(callback)
+ return True
+ else: return False
+
+def _getEntries(levels):
+ """
+ Generator for providing past events belonging to the given runlevels (in
+ chronological order). This should be used under the LOG_LOCK to prevent
+ concurrent modifications.
+
+ Arguments:
+ levels - runlevels for which events are provided
+ """
+
+ # drops any runlevels if there aren't entries in it
+ toRemove = [level for level in levels if not _backlog[level]]
+ for level in toRemove: levels.remove(level)
+
+ # tracks where unprocessed entries start in the backlog
+ backlogPtr = dict([(level, 0) for level in levels])
+
+ while levels:
+ earliestLevel, earliestMsg, earliestTime = None, "", maxint
+
+ # finds the earliest unprocessed event
+ for level in levels:
+ entry = _backlog[level][backlogPtr[level]]
+
+ if entry[0] < earliestTime:
+ earliestLevel, earliestMsg, earliestTime = level, entry[1], entry[0]
+
+ yield (earliestLevel, earliestMsg, earliestTime)
+
+ # removes runlevel if there aren't any more entries
+ backlogPtr[earliestLevel] += 1
+ if len(_backlog[earliestLevel]) <= backlogPtr[earliestLevel]:
+ levels.remove(earliestLevel)
+
Modified: arm/trunk/util/panel.py
===================================================================
--- arm/trunk/util/panel.py 2010-04-08 12:22:36 UTC (rev 22147)
+++ arm/trunk/util/panel.py 2010-04-08 16:25:14 UTC (rev 22148)
@@ -3,17 +3,22 @@
"""
import curses
-from sys import maxint
from threading import RLock
import uiTools
-# TODO: external usage of clear and refresh are only used by popup (remove when alternative is used)
-
# global ui lock governing all panel instances (curses isn't thread save and
# concurrency bugs produce especially sinister glitches)
CURSES_LOCK = RLock()
+# tags used by addfstr - this maps to functor/argument combinations since the
+# actual values (color attributes - grr...) might not yet be initialized
+def _noOp(arg): return arg
+FORMAT_TAGS = {"<b>": (_noOp, curses.A_BOLD),
+ "<u>": (_noOp, curses.A_UNDERLINE),
+ "<h>": (_noOp, curses.A_STANDOUT)}
+for colorLabel in uiTools.COLOR_LIST.keys(): FORMAT_TAGS["<%s>" % colorLabel] = (uiTools.getColor, colorLabel)
+
class Panel():
"""
Wrapper for curses subwindows. This hides most of the ugliness in common
@@ -23,123 +28,200 @@
- clip text that falls outside the panel
- convenience methods for word wrap, inline formatting, etc
- This can't be used until it has a subwindow instance, which is done via the
- recreate() function. Until this is done the top, maxX, and maxY parameters
- are defaulted to -1.
-
- Parameters:
- win - current curses subwindow
- height - preferred (max) height of panel, -1 if infinite
- top - upper Y-coordinate within parent window
- maxX, maxY - cached bounds of subwindow
+ This uses a design akin to Swing where panel instances provide their display
+ implementation by overwriting the draw() method, and are redrawn with
+ redraw().
"""
- def __init__(self, height):
- self.win = None
+ def __init__(self, parent, top, height=-1, width=-1):
+ """
+ Creates a durable wrapper for a curses subwindow in the given parent.
+
+ Arguments:
+ parent - parent curses window
+ top - positioning of top within parent
+ height - maximum height of panel (uses all available space if -1)
+ width - maximum width of panel (uses all available space if -1)
+ """
+
+ # The not-so-pythonic getters for these parameters are because some
+ # implementations aren't entirely deterministic (for instance panels
+ # might chose their height based on its parent's current width).
+
+ self.parent = parent
+ self.top = top
self.height = height
- self.top = -1
- self.maxY, self.maxX = -1, -1
+ self.width = width
- # when the terminal is shrank then expanded curses attempts to draw
- # displaced windows in the wrong location - this results in graphical
- # glitches if we let the panel be redrawn
- self.isDisplaced = True
+ # The panel's subwindow instance. This is made available to implementors
+ # via their draw method and shouldn't be accessed directly.
+ #
+ # This is None if either the subwindow failed to be created or needs to be
+ # remade before it's used. The later could be for a couple reasons:
+ # - The subwindow was never initialized.
+ # - Any of the parameters used for subwindow initialization have changed.
+ self.win = None
+
+ self.maxY, self.maxX = -1, -1 # subwindow dimensions when last redrawn
- def draw(self):
+ def getParent(self):
"""
- Draws display's content. This is meant to be overwriten by
- impelementations.
+ Provides the parent used to create subwindows.
"""
- pass
+ return self.parent
- def redraw(self, block=False):
+ def setParent(self, parent):
"""
- Clears display and redraws.
+ Changes the parent used to create subwindows.
+
+ Arguments:
+ parent - parent curses window
"""
- if self.win:
- if not CURSES_LOCK.acquire(block): return
- try:
- self.clear()
- self.draw()
- self.refresh()
- finally:
- CURSES_LOCK.release()
+ if self.parent != parent:
+ self.parent = parent
+ self.win = None
- def recreate(self, stdscr, newWidth=-1, newTop=None):
+ def getTop(self):
"""
- Creates a new subwindow for the panel if:
- - panel currently doesn't have a subwindow
- - the panel is being moved (top is different from newTop)
- - there's room for the panel to grow
+ Provides the position subwindows are placed at within its parent.
+ """
- Returns True if subwindow's created, False otherwise.
+ return self.top
+
+ def setTop(self, top):
"""
+ Changes the position where subwindows are placed within its parent.
- if newTop == None: newTop = self.top
+ Arguments:
+ top - positioning of top within parent
+ """
- # I'm not sure if recreating subwindows is some sort of memory leak but the
- # Python curses bindings seem to lack all of the following:
- # - subwindow deletion (to tell curses to free the memory)
- # - subwindow moving/resizing (to restore the displaced windows)
- # so this is the only option (besides removing subwindows entirly which
- # would mean more complicated code and no more selective refreshing)
+ if self.top != top:
+ self.top = top
+ self.win = None
+
+ def getHeight(self):
+ """
+ Provides the height used for subwindows (-1 if it isn't limited).
+ """
- y, x = stdscr.getmaxyx()
- self._resetBounds()
+ return self.height
+
+ def setHeight(self, height):
+ """
+ Changes the height used for subwindows. This uses all available space if -1.
- if self.win and newTop > y:
- return False # trying to make panel out of bounds
+ Arguments:
+ height - maximum height of panel (uses all available space if -1)
+ """
- newHeight = max(0, y - newTop)
- if self.height != -1: newHeight = min(newHeight, self.height)
+ if self.height != height:
+ self.height = height
+ self.win = None
+
+ def getWidth(self):
+ """
+ Provides the width used for subwindows (-1 if it isn't limited).
+ """
- recreate = False
- recreate |= self.top != newTop # position has shifted
- recreate |= newHeight != self.maxY # subwindow can grow (vertically)
- recreate |= self.isDisplaced # resizing has bumped subwindow out of position
- recreate |= self.maxX != newWidth and newWidth != -1 # set to use a new width
+ return self.width
+
+ def setWidth(self, width):
+ """
+ Changes the width used for subwindows. This uses all available space if -1.
- if recreate:
- if newWidth == -1: newWidth = x
- else: newWidth = min(newWidth, x)
-
- self.top = newTop
- newTop = min(newTop, y - 1) # better create a displaced window than leave it as None
-
- self.win = stdscr.subwin(newHeight, newWidth, newTop, 0)
- return True
- else: return False
+ Arguments:
+ width - maximum width of panel (uses all available space if -1)
+ """
+
+ if self.width != width:
+ self.width = width
+ self.win = None
- # TODO: merge into repaint when no longer needed
- def clear(self):
+ def getPreferredSize(self):
"""
- Erases window and resets bounds used in writting to it.
+ Provides the dimensions the subwindow would use when next redrawn, given
+ that none of the properties of the panel or parent change before then. This
+ returns a tuple of (height, width).
"""
- if self.win:
- self.isDisplaced = self.top > self.win.getparyx()[0]
- if not self.isDisplaced: self.win.erase()
- self._resetBounds()
+ newHeight, newWidth = self.parent.getmaxyx()
+ newHeight = max(0, newHeight - self.top)
+ if self.height != -1: newHeight = min(self.height, newHeight)
+ if self.width != -1: newWidth = min(self.width, newWidth)
+ return (newHeight, newWidth)
- # TODO: merge into repaint when no longer needed
- def refresh(self):
+ def draw(self, subwindow, width, height):
"""
- Proxy for window refresh.
+ Draws display's content. This is meant to be overwritten by
+ implementations and not called directly (use redraw() instead).
+
+ Arguments:
+ sudwindow - panel's current subwindow instance, providing raw access to
+ its curses functions
+ width - horizontal space available for content
+ height - vertical space available for content
"""
- if self.win and not self.isDisplaced: self.win.refresh()
+ pass
+ def redraw(self, refresh=False, block=False):
+ """
+ Clears display and redraws its content.
+
+ Arguments:
+ refresh - skips redrawing content if able (ie, the subwindow's
+ unchanged), instead just refreshing the display
+ block - if drawing concurrently with other panels this determines if
+ the request is willing to wait its turn or should be abandoned
+ """
+
+ # if the panel's completely outside its parent then this is a no-op
+ newHeight, newWidth = self.getPreferredSize()
+ if newHeight == 0:
+ self.win = None
+ return
+
+ # recreates the subwindow if necessary
+ isNewWindow = self._resetSubwindow()
+
+ # The reset argument is disregarded in a couple of situations:
+ # - The subwindow's been recreated (obviously it then doesn't have the old
+ # content to refresh).
+ # - The subwindow's dimensions have changed since last drawn (this will
+ # likely change the content's layout)
+
+ subwinMaxY, subwinMaxX = self.win.getmaxyx()
+ if isNewWindow or subwinMaxY != self.maxY or subwinMaxX != self.maxX:
+ refresh = False
+
+ self.maxY, self.maxX = subwinMaxY, subwinMaxX
+ if not CURSES_LOCK.acquire(block): return
+ try:
+ self.win.erase() # clears any old contents
+ if not refresh: self.draw(self.win, self.maxX, self.maxY)
+ self.win.refresh()
+ finally:
+ CURSES_LOCK.release()
+
def addstr(self, y, x, msg, attr=curses.A_NORMAL):
"""
Writes string to subwindow if able. This takes into account screen bounds
- to avoid making curses upset.
+ to avoid making curses upset. This should only be called from the context
+ of a panel's draw method.
+
+ Arguments:
+ y - vertical location
+ x - horizontal location
+ msg - text to be added
+ attr - text attributes
"""
# subwindows need a single character buffer (either in the x or y
# direction) from actual content to prevent crash when shrank
- if self.win and self.maxX > x and self.maxY > y and not self.isDisplaced:
+ if self.win and self.maxX > x and self.maxY > y:
self.win.addstr(y, x, msg[:self.maxX - x - 1], attr)
def addfstr(self, y, x, msg):
@@ -149,82 +231,162 @@
<b>text</b> bold
<u>text</u> underline
<h>text</h> highlight
- <[color]>text</[color]> use color (see COLOR_LIST for constants)
+ <[color]>text</[color]> use color (see uiTools.getColor() for constants)
- Tag nexting is supported and tag closing is not strictly enforced. This
- does not valididate input and unrecognized tags are treated as normal text.
- Currently this funtion has the following restrictions:
- - Duplicate tags nested (such as "<b><b>foo</b></b>") is invalid and may
- throw an error.
- - Color tags shouldn't be nested in each other (results are undefined).
+ Tag nesting is supported and tag closing is strictly enforced (raising an
+ exception for invalid formatting). Unrecognized tags are treated as normal
+ text. This should only be called from the context of a panel's draw method.
+
+ Text in multiple color tags (for instance "<blue><red>hello</red></blue>")
+ uses the bitwise OR of those flags (hint: that's probably not what you
+ want).
+
+ Arguments:
+ y - vertical location
+ x - horizontal location
+ msg - formatted text to be added
"""
- if self.win and self.maxY > y and not self.isDisplaced:
+ if self.win and self.maxY > y:
formatting = [curses.A_NORMAL]
expectedCloseTags = []
+ unusedMsg = msg
- while self.maxX > x and len(msg) > 0:
- # finds next consumeable tag
- nextTag, nextTagIndex = None, maxint
+ while self.maxX > x and len(unusedMsg) > 0:
+ # finds next consumeable tag (left as None if there aren't any left)
+ nextTag, tagStart, tagEnd = None, -1, -1
- for tag in uiTools.FORMAT_TAGS.keys() + expectedCloseTags:
- tagLoc = msg.find(tag)
- if tagLoc != -1 and tagLoc < nextTagIndex:
- nextTag, nextTagIndex = tag, tagLoc
+ tmpChecked = 0 # portion of the message cleared for having any valid tags
+ expectedTags = FORMAT_TAGS.keys() + expectedCloseTags
+ while nextTag == None:
+ tagStart = unusedMsg.find("<", tmpChecked)
+ tagEnd = unusedMsg.find(">", tagStart) + 1 if tagStart != -1 else -1
+
+ if tagStart == -1 or tagEnd == -1: break # no more tags to consume
+ else:
+ # check if the tag we've found matches anything being expected
+ if unusedMsg[tagStart:tagEnd] in expectedTags:
+ nextTag = unusedMsg[tagStart:tagEnd]
+ break # found a tag to use
+ else:
+ # not a valid tag - narrow search to everything after it
+ tmpChecked = tagEnd
# splits into text before and after tag
if nextTag:
- msgSegment = msg[:nextTagIndex]
- msg = msg[nextTagIndex + len(nextTag):]
+ msgSegment = unusedMsg[:tagStart]
+ unusedMsg = unusedMsg[tagEnd:]
else:
- msgSegment = msg
- msg = ""
+ msgSegment = unusedMsg
+ unusedMsg = ""
# adds text before tag with current formatting
attr = 0
for format in formatting: attr |= format
self.win.addstr(y, x, msgSegment[:self.maxX - x - 1], attr)
+ x += len(msgSegment)
# applies tag attributes for future text
if nextTag:
+ formatTag = "<" + nextTag[2:] if nextTag.startswith("</") else nextTag
+ formatMatch = FORMAT_TAGS[formatTag][0](FORMAT_TAGS[formatTag][1])
+
if not nextTag.startswith("</"):
# open tag - add formatting
expectedCloseTags.append("</" + nextTag[1:])
- formatting.append(uiTools.FORMAT_TAGS[nextTag])
+ formatting.append(formatMatch)
else:
# close tag - remove formatting
expectedCloseTags.remove(nextTag)
- formatting.remove(uiTools.FORMAT_TAGS["<" + nextTag[2:]])
-
- x += len(msgSegment)
+ formatting.remove(formatMatch)
+
+ # only check for unclosed tags if we processed the whole message (if we
+ # stopped processing prematurely it might still be valid)
+ if expectedCloseTags and not unusedMsg:
+ # if we're done then raise an exception for any unclosed tags (tisk, tisk)
+ baseMsg = "Unclosed formatting tag%s:" % ("s" if len(expectedCloseTags) > 1 else "")
+ raise ValueError("%s: '%s'\n \"%s\"" % (baseMsg, "', '".join(expectedCloseTags), msg))
- def addstr_wrap(self, y, x, text, formatting, startX = 0, endX = -1, maxY = -1):
+ def addScrollBar(self, top, bottom, size, drawTop = 0, drawBottom = -1):
"""
- Writes text with word wrapping, returning the ending y/x coordinate.
- y: starting write line
- x: column offset from startX
- text / formatting: content to be written
- startX / endX: column bounds in which text may be written
+ Draws a left justified scroll bar reflecting position within a vertical
+ listing. This is shorted if necessary, and left undrawn if no space is
+ available. The bottom is squared off, having a layout like:
+ |
+ *|
+ *|
+ *|
+ |
+ -+
+
+ This should only be called from the context of a panel's draw method.
+
+ Arguments:
+ top - list index for the top-most visible element
+ bottom - list index for the bottom-most visible element
+ size - size of the list in which the listed elements are contained
+ drawTop - starting row where the scroll bar should be drawn
+ drawBottom - ending row where the scroll bar should end, -1 if it should
+ span to the bottom of the panel
"""
- if not text: return (y, x) # nothing to write
- if endX == -1: endX = self.maxX # defaults to writing to end of panel
- if maxY == -1: maxY = self.maxY + 1 # defaults to writing to bottom of panel
- lineWidth = endX - startX # room for text
- while True:
- if len(text) > lineWidth - x - 1:
- chunkSize = text.rfind(" ", 0, lineWidth - x)
- writeText = text[:chunkSize]
- text = text[chunkSize:].strip()
-
- self.addstr(y, x + startX, writeText, formatting)
- y, x = y + 1, 0
- if y >= maxY: return (y, x)
- else:
- self.addstr(y, x + startX, text, formatting)
- return (y, x + len(text))
+ if (self.maxY - drawTop) < 2: return # not enough room
+
+ # sets drawBottom to be the actual row on which the scrollbar should end
+ if drawBottom == -1: drawBottom = self.maxY - 1
+ else: drawBottom = min(drawBottom, self.maxY - 1)
+
+ # determines scrollbar dimensions
+ scrollbarHeight = drawBottom - drawTop
+ sliderTop = scrollbarHeight * top / size
+ sliderSize = scrollbarHeight * (bottom - top) / size
+
+ # ensures slider isn't at top or bottom unless really at those extreme bounds
+ if top > 0: sliderTop = max(sliderTop, 1)
+ if bottom != size: sliderTop = min(sliderTop, scrollbarHeight - sliderSize - 2)
+
+ # draws scrollbar slider
+ for i in range(scrollbarHeight):
+ if i >= sliderTop and i <= sliderTop + sliderSize:
+ self.addstr(i + drawTop, 0, " ", curses.A_STANDOUT)
+
+ # draws box around the scroll bar
+ self.win.vline(drawTop, 1, curses.ACS_VLINE, self.maxY - 2)
+ self.win.vline(drawBottom, 1, curses.ACS_LRCORNER, 1)
+ self.win.hline(drawBottom, 0, curses.ACS_HLINE, 1)
- def _resetBounds(self):
- if self.win: self.maxY, self.maxX = self.win.getmaxyx()
- else: self.maxY, self.maxX = -1, -1
-
+ def _resetSubwindow(self):
+ """
+ Create a new subwindow instance for the panel if:
+ - Panel currently doesn't have a subwindow (was uninitialized or
+ invalidated).
+ - There's room for the panel to grow vertically (curses automatically
+ lets subwindows regrow horizontally, but not vertically).
+ - The subwindow has been displaced. This is a curses display bug that
+ manifests if the terminal's shrank then re-expanded. Displaced
+ subwindows are never restored to their proper position, resulting in
+ graphical glitches if we draw to them.
+
+ This returns True if a new subwindow instance was created, False otherwise.
+ """
+
+ newHeight, newWidth = self.getPreferredSize()
+ if newHeight == 0: return False # subwindow would be outside its parent
+
+ # determines if a new subwindow should be recreated
+ recreate = self.win == None
+ if self.win:
+ subwinMaxY, subwinMaxX = self.win.getmaxyx()
+ recreate |= subwinMaxY < newHeight # check for vertical growth
+ recreate |= self.top > self.win.getparyx()[0] # check for displacement
+
+ # I'm not sure if recreating subwindows is some sort of memory leak but the
+ # Python curses bindings seem to lack all of the following:
+ # - subwindow deletion (to tell curses to free the memory)
+ # - subwindow moving/resizing (to restore the displaced windows)
+ # so this is the only option (besides removing subwindows entirely which
+ # would mean far more complicated code and no more selective refreshing)
+
+ if recreate: self.win = self.parent.subwin(newHeight, newWidth, self.top, 0)
+ return recreate
+
Modified: arm/trunk/util/uiTools.py
===================================================================
--- arm/trunk/util/uiTools.py 2010-04-08 12:22:36 UTC (rev 22147)
+++ arm/trunk/util/uiTools.py 2010-04-08 16:25:14 UTC (rev 22148)
@@ -1,115 +1,174 @@
-#!/usr/bin/env python
-# util.py -- support functions common for arm user interface.
-# Released under the GPL v3 (http://www.gnu.org/licenses/gpl.html)
+"""
+Toolkit for common ui tasks when working with curses. This provides a quick and
+easy method of providing the following interface components:
+- preinitialized curses color attributes
+- unit conversion for labels
+"""
import curses
-LABEL_ATTR = curses.A_STANDOUT # default formatting constant
-
# colors curses can handle
-COLOR_LIST = (("red", curses.COLOR_RED),
- ("green", curses.COLOR_GREEN),
- ("yellow", curses.COLOR_YELLOW),
- ("blue", curses.COLOR_BLUE),
- ("cyan", curses.COLOR_CYAN),
- ("magenta", curses.COLOR_MAGENTA),
- ("black", curses.COLOR_BLACK),
- ("white", curses.COLOR_WHITE))
+COLOR_LIST = {"red": curses.COLOR_RED, "green": curses.COLOR_GREEN,
+ "yellow": curses.COLOR_YELLOW, "blue": curses.COLOR_BLUE,
+ "cyan": curses.COLOR_CYAN, "magenta": curses.COLOR_MAGENTA,
+ "black": curses.COLOR_BLACK, "white": curses.COLOR_WHITE}
-FORMAT_TAGS = {"<b>": curses.A_BOLD,
- "<u>": curses.A_UNDERLINE,
- "<h>": curses.A_STANDOUT}
-for (colorLabel, cursesAttr) in COLOR_LIST: FORMAT_TAGS["<%s>" % colorLabel] = curses.A_NORMAL
-
-# foreground color mappings (starts uninitialized - all colors associated with default white fg / black bg)
+# mappings for getColor() - this uses the default terminal color scheme if
+# color support is unavailable
COLOR_ATTR_INITIALIZED = False
-COLOR_ATTR = dict([(color[0], 0) for color in COLOR_LIST])
+COLOR_ATTR = dict([(color, 0) for color in COLOR_LIST.keys()])
-def initColors():
- """
- Initializes color mappings for the current curses. This needs to be called
- after curses.initscr().
- """
-
- global COLOR_ATTR_INITIALIZED
- if not COLOR_ATTR_INITIALIZED:
- COLOR_ATTR_INITIALIZED = True
-
- # if color support is available initializes color mappings
- if curses.has_colors():
- colorpair = 0
-
- for name, fgColor in COLOR_LIST:
- colorpair += 1
- curses.init_pair(colorpair, fgColor, -1) # -1 allows for default (possibly transparent) background
- COLOR_ATTR[name] = curses.color_pair(colorpair)
-
- # maps color tags to initialized attributes
- for colorLabel in COLOR_ATTR.keys(): FORMAT_TAGS["<%s>" % colorLabel] = COLOR_ATTR[colorLabel]
+# value tuples for label conversions (bytes / seconds, short label, long label)
+SIZE_UNITS = [(1125899906842624.0, " PB", " Petabyte"), (1099511627776.0, " TB", " Terabyte"),
+ (1073741824.0, " GB", " Gigabyte"), (1048576.0, " MB", " Megabyte"),
+ (1024.0, " KB", " Kilobyte"), (1.0, " B", " Byte")]
+TIME_UNITS = [(86400.0, "d", " day"), (3600.0, "h", " hour"),
+ (60.0, "m", " minute"), (1.0, "s", " second")]
def getColor(color):
"""
Provides attribute corresponding to a given text color. Supported colors
include:
- red, green, yellow, blue, cyan, magenta, black, and white
+ red green yellow blue
+ cyan magenta black white
- If color support isn't available then this uses the default terminal coloring
- scheme.
+ If color support isn't available or colors can't be initialized then this uses the
+ terminal's default coloring scheme.
+
+ Arguments:
+ color - name of the foreground color to be returned
"""
+ if not COLOR_ATTR_INITIALIZED: _initColors()
return COLOR_ATTR[color]
-def getSizeLabel(bytes, decimal = 0):
+def getSizeLabel(bytes, decimal = 0, isLong = False):
"""
Converts byte count into label in its most significant units, for instance
- 7500 bytes would return "7 KB".
+ 7500 bytes would return "7 KB". If the isLong option is used this expands
+ unit labels to be the properly pluralised full word (for instance 'Kilobytes'
+ rather than 'KB'). Units go up through PB.
+
+ Example Usage:
+ getSizeLabel(2000000) = '1 MB'
+ getSizeLabel(1050, 2) = '1.02 KB'
+ getSizeLabel(1050, 3, True) = '1.025 Kilobytes'
+
+ Arguments:
+ bytes - source number of bytes for conversion
+ decimal - number of decimal digits to be included
+ isLong - expands units label
"""
- format = "%%.%if" % decimal
- if bytes >= 1073741824: return (format + " GB") % (bytes / 1073741824.0)
- elif bytes >= 1048576: return (format + " MB") % (bytes / 1048576.0)
- elif bytes >= 1024: return (format + " KB") % (bytes / 1024.0)
- else: return "%i bytes" % bytes
+ return _getLabel(SIZE_UNITS, bytes, decimal, isLong)
-def getTimeLabel(seconds, decimal = 0):
+def getTimeLabel(seconds, decimal = 0, isLong = False):
"""
- Concerts seconds into a time label truncated to its most significant units,
+ Converts seconds into a time label truncated to its most significant units,
for instance 7500 seconds would return "2h". Units go up through days.
+
+ This defaults to presenting single character labels, but if the isLong option
+ is used this expands labels to be the full word (space included and properly
+ pluralised). For instance, "4h" would be "4 hours" and "1m" would become
+ "1 minute".
+
+ Example Usage:
+ getTimeLabel(10000) = '2h'
+ getTimeLabel(61, 1, True) = '1.0 minute'
+ getTimeLabel(61, 2, True) = '1.01 minutes'
+
+ Arguments:
+ seconds - source number of seconds for conversion
+ decimal - number of decimal digits to be included
+ isLong - expands units label
"""
- format = "%%.%if" % decimal
- if seconds >= 86400: return (format + "d") % (seconds / 86400.0)
- elif seconds >= 3600: return (format + "h") % (seconds / 3600.0)
- elif seconds >= 60: return (format + "m") % (seconds / 60.0)
- else: return "%is" % seconds
+ return _getLabel(TIME_UNITS, seconds, decimal, isLong)
-def drawScrollBar(panel, drawTop, drawBottom, top, bottom, size):
+def getTimeLabels(seconds, isLong = False):
"""
- Draws scroll bar reflecting position within a vertical listing. This is
- squared off at the bottom, having a layout like:
- |
- *|
- *|
- *|
- |
- -+
+ Provides a list containing label conversions for each time unit, starting
+ with its most significant units on down. Any counts that evaluate to zero are
+ omitted.
+
+ Example Usage:
+ getTimeLabels(400) = ['6m', '40s']
+ getTimeLabels(3640, True) = ['1 hour', '40 seconds']
+
+ Arguments:
+ seconds - source number of seconds for conversion
+ isLong - expands units label
"""
- if panel.maxY < 2: return # not enough room
+ timeLabels = []
- barTop = (drawBottom - drawTop) * top / size
- barSize = (drawBottom - drawTop) * (bottom - top) / size
+ for countPerUnit, shortLabel, longLabel in TIME_UNITS:
+ if seconds >= countPerUnit:
+ timeLabels.append(_getLabel(TIME_UNITS, seconds, 0, isLong))
+ seconds %= countPerUnit
- # makes sure bar isn't at top or bottom unless really at those extreme bounds
- if top > 0: barTop = max(barTop, 1)
- if bottom != size: barTop = min(barTop, drawBottom - drawTop - barSize - 2)
+ return timeLabels
+
+def _getLabel(units, count, decimal, isLong):
+ """
+ Provides label corresponding to units of the highest significance in the
+ provided set. This rounds down (ie, integer truncation after visible units).
- for i in range(drawBottom - drawTop):
- if i >= barTop and i <= barTop + barSize:
- panel.addstr(i + drawTop, 0, " ", curses.A_STANDOUT)
+ Arguments:
+ units - type of units to be used for conversion, a tuple containing
+ (countPerUnit, shortLabel, longLabel)
+ count - number of base units being converted
+ decimal - decimal precision of label
+ isLong - uses the long label if true, short label otherwise
+ """
- # draws box around scroll bar
- panel.win.vline(drawTop, 1, curses.ACS_VLINE, panel.maxY - 2)
- panel.win.vline(drawBottom, 1, curses.ACS_LRCORNER, 1)
- panel.win.hline(drawBottom, 0, curses.ACS_HLINE, 1)
+ format = "%%.%if" % decimal
+ if count < 1:
+ unitsLabel = units[-1][2] + "s" if isLong else units[-1][1]
+ return "%s%s" % (format % count, unitsLabel)
+
+ for countPerUnit, shortLabel, longLabel in units:
+ if count >= countPerUnit:
+ if count * 10 ** decimal % countPerUnit * 10 ** decimal == 0:
+ # even division, keep it simple
+ countLabel = format % (count / countPerUnit)
+ else:
+ # unfortunately the %f formatting has no method of rounding down, so
+ # reducing value to only concern the digits that are visible - note
+ # that this doesn't work with miniscule values (starts breaking down at
+ # around eight decimal places) or edge cases when working with powers
+ # of two
+ croppedCount = count - (count % (countPerUnit / (10 ** decimal)))
+ countLabel = format % (croppedCount / countPerUnit)
+
+ if isLong:
+ # plural if any of the visible units make it greater than one (for
+ # instance 1.0003 is plural but 1.000 isn't)
+ if decimal > 0: isPlural = count >= (countPerUnit + countPerUnit / (10 ** decimal))
+ else: isPlural = count >= countPerUnit * 2
+ return countLabel + longLabel + ("s" if isPlural else "")
+ else: return countLabel + shortLabel
+def _initColors():
+ """
+ Initializes color mappings usable by curses. This can only be done after
+ calling curses.initscr().
+ """
+
+ global COLOR_ATTR_INITIALIZED
+ if not COLOR_ATTR_INITIALIZED:
+ try: hasColorSupport = curses.has_colors()
+ except curses.error: return # initscr hasn't been called yet
+
+ # initializes color mappings if color support is available
+ COLOR_ATTR_INITIALIZED = True
+ if hasColorSupport:
+ colorpair = 0
+
+ for colorName in COLOR_LIST.keys():
+ fgColor = COLOR_LIST[colorName]
+ bgColor = -1 # allows for default (possibly transparent) background
+ colorpair += 1
+ curses.init_pair(colorpair, fgColor, bgColor)
+ COLOR_ATTR[colorName] = curses.color_pair(colorpair)
+
More information about the tor-commits
mailing list