[or-cvs] r23196: {arm} change: full rewrite of the log panel, providing: - an optio (in arm/trunk: . src/interface src/interface/graphing src/util)
Damian Johnson
atagar1 at gmail.com
Tue Sep 14 22:21:48 UTC 2010
Author: atagar
Date: 2010-09-14 22:21:48 +0000 (Tue, 14 Sep 2010)
New Revision: 23196
Modified:
arm/trunk/README
arm/trunk/TODO
arm/trunk/armrc.sample
arm/trunk/src/interface/confPanel.py
arm/trunk/src/interface/connPanel.py
arm/trunk/src/interface/controller.py
arm/trunk/src/interface/graphing/bandwidthStats.py
arm/trunk/src/interface/graphing/connStats.py
arm/trunk/src/interface/graphing/graphPanel.py
arm/trunk/src/interface/graphing/psStats.py
arm/trunk/src/interface/headerPanel.py
arm/trunk/src/interface/logPanel.py
arm/trunk/src/util/sysTools.py
arm/trunk/src/util/torTools.py
arm/trunk/src/util/uiTools.py
Log:
change: full rewrite of the log panel, providing:
- an option to clear the event log
- coalescing of updates if they're numerous (such as at the DEBUG runlevel)
- extra restriction of the read portion of tor's log file if it contains a subset of the events being logged
- several performance fixes (for log prepopulation, duplicate work when determining the content length, etc)
- minor fixes including:
- dropping brackets from lable if no events are being logged
- merging tor and arm backlogs according to timestamps
- regex matches were failing for multiline log entries
added: caching for static GETINFO parameter
change: home/end keys jump to start/end of all scroll areas (request by dun)
change: measuring by bits for transfer rates (config can set it back to bytes)
change: dropping the 'frequentRefresh' parameter in favor of just doing refreshes when there's new graph stats available
fix: adding additional refreshes to make graph panel seem more responsive during popups
fix: refactoring mistake resulting in a crash during sighup
fix: bug with cache invalidation making use of the fingerprint if running as a clinet
Modified: arm/trunk/README
===================================================================
--- arm/trunk/README 2010-09-14 22:04:58 UTC (rev 23195)
+++ arm/trunk/README 2010-09-14 22:21:48 UTC (rev 23196)
@@ -86,17 +86,25 @@
./
arm - startup script
+ uninstall - installation script
armrc.sample - example arm configuration file with defaults
ChangeLog - revision history
LICENSE - copy of the gpl v3
README - um... guess you figured this one out
TODO - known issues, future plans, etc
+ setup.py - distutils installation script for arm
- src/arm/
+ debian/ - resources for generating debs and rpms (most is metadata)
+ make-deb - script for generating debian installer
+ make-rpm - script for generating red hat installer
+ arm.1.gz - man page
+
+ src/
__init__.py
- starter.py - parses and validates commandline parameters
- prereq.py - checks python version and for required packages
+ starter.py - parses and validates commandline parameters
+ prereq.py - checks python version and for required packages
+ uninstall - removal script
interface/
graphing/
Modified: arm/trunk/TODO
===================================================================
--- arm/trunk/TODO 2010-09-14 22:04:58 UTC (rev 23195)
+++ arm/trunk/TODO 2010-09-14 22:21:48 UTC (rev 23196)
@@ -9,19 +9,17 @@
progress - /init and /util are done and /interface is partly done. Known
bugs are being fixed while refactoring.
[ ] log panel
- - option to clear log
- - allow home/end keys to jump to start/end
- also do this for the conn panel and conf panel (request by dun)
- - make log parsing script stand alone, with syntax hilighting, regex,
- sorting, etc
- provide notice if tor supports events that arm doesn't
getInfo("events/names") provides the space-separated listing
- check what events TorCtl can provide us, and give notice if any are
missing
- - log to file? Is this acceptable? It would allow tor's non-runlevel
- events to be easily saved.
+ - log to file, allowing non-runlevel events to be saved (provide both
+ a continuous option and snapshots taking into account the current
+ filter)
- condense tor/arm log listing types if they're the same
Ie, make default "TOR/ARM NOTICE - ERR"
+ - provide daily dividers (otherwise there's no indicator for the day)
+ - log cropping based on time (idea by voidzero)
- drop duplicate or overly verbose messages (feature request by asn)
check if asn wants to implement this when refactoring is done
[ ] conf panel
@@ -49,6 +47,12 @@
[ ] provide performance ARM-DEBUG events
Help with diagnosing performance bottlenecks. This is pending the
codebase revisions to figure out the low hanging fruit for caching.
+ [ ] email alerts for changes to the relay's status (similar to tor-weather)
+ [ ] simple alert if tor shuts down
+ [ ] accounting and alerts for if the bandwidth drops to zero
+ [ ] daily/weekly/etc alerts for basic status (log output, bandwidth
+ history, etc), borrowing from the consensus tracker for some of the
+ formatting
[ ] tor util
[X] wrapper for accessing torctl
[ ] allow arm to resume after restarting tor (attaching to a new torctl
@@ -64,7 +68,14 @@
- dmg (http://en.wikipedia.org/wiki/Apple_Disk_Image)
Most conventional method of software distribution on mac. This is
just a container (no updating/removal support), but could contain
- an icon for the dock that starts a terminal with arm.
+ an icon for the dock that starts a terminal with arm. This might
+ include a pkg installer.
+
+ - mpkg (http://pypi.python.org/pypi/bdist_mpkg/)
+ Plugin for distutils. Like most mac packaging, this can only run
+ on a mac. It also requires setuptools:
+ http://www.errorhelp.com/search/details/74034/importerror-no-module-named-setuptools
+
[ ] updater (checks for a new tarball and installs it automatically)
[ ] look into CAPs to get around permission issues for connection
listing sudo wrapper for arm to help arm run as the same user as
@@ -90,10 +101,11 @@
* not catching events unexpected by arm
Future tor and TorCtl revisions could provide new events - these should
be given the "UNKNOWN" type.
- * regex fails for multiline log entries (works for two lines, but not more)
- * when logging no events still showing brackets
- The current code for dynamically sizing the events label is kinda
- tricky. Putting this off until revising this section.
+ * log prepopulation fails to limit entries to the current tor instance if
+ the file isn't logged to at the NOTICE level. Use timestamps as a
+ backup method, or just switch entirely?
+ * arm is triggering DEBUG level events
+ https://trac.torproject.org/projects/tor/ticket/1933
* conf panel:
* torrc validation doesn't catch if parameters are missing
@@ -124,7 +136,7 @@
* connection uptimes shouldn't show fractions of a second
* connections aren't cleared when control port closes
-- Features / Site
+- Features
* client mode use cases
* not sure what sort of information would be useful in the header (to
replace the orport, fingerprint, flags, etc)
@@ -156,7 +168,6 @@
* attempt to clear controller password from memory
http://www.codexon.com/posts/clearing-passwords-in-memory-with-python
* escaping function for uiTools' formatted strings
- * tor-weather like functionality (email notices)
* make update rates configurable via the ui
Also provide option for saving these settings to the config
* config option to cap resource usage
@@ -164,6 +175,9 @@
* switch check of ip address validity to regex?
match = re.match("(\d*)\.(\d*)\.(\d*)\.(\d*)", ip)
http://wang.yuxuan.org/blog/2009/4/2/python_script_to_convert_from_ip_range_to_ip_mask
+ * look into using Enum2
+ TorCtl uses a specail class for enumerations. It's probably better than
+ my current range(1, X) approach.
* audit tor connections
Provide warnings if tor misbehaves, checks possibly including:
- ensuring ExitPolicyRejectPrivate is being obeyed
Modified: arm/trunk/armrc.sample
===================================================================
--- arm/trunk/armrc.sample 2010-09-14 22:04:58 UTC (rev 23195)
+++ arm/trunk/armrc.sample 2010-09-14 22:21:48 UTC (rev 23196)
@@ -9,17 +9,17 @@
# log panel parameters
# prepopulate: attempts to read past events from the log file if true
-# prepopulateAddLimit: maximum entries added from the log file
# prepopulateReadLimit: maximum entries read from the log file
+# maxRefreshRate: rate limiting (in milliseconds) for drawing the log if
+# updates are made rapidly (for instance, when at the DEBUG runlevel)
#
# Limits are to prevent big log files from causing a slow startup time. For
# instance, if arm's only listening for ERR entries but the log has all
-# runlevels then this will add the first <prepopulateAddLimit> ERR entries and
-# stop reading after <prepopulateReadLimit> lines.
+# runlevels then this will stop reading after <prepopulateReadLimit> lines.
features.log.prepopulate true
-features.log.prepopulateAddLimit 1000
features.log.prepopulateReadLimit 5000
+features.log.maxRefreshRate 300
# general graph parameters
# height: height of graphed stats
@@ -29,7 +29,6 @@
# bound: 0 -> global maxima, 1 -> local maxima, 2 -> tight
# type: 0 -> None, 1 -> Bandwidth, 2 -> Connections, 3 -> System Resources
# showIntermediateBounds: shows y-axis increments between the top/bottom bounds
-# frequentRefrsh: updates stats each second if true, otherwise matches interval
features.graph.height 7
features.graph.maxWidth 150
@@ -37,7 +36,6 @@
features.graph.bound 1
features.graph.type 1
features.graph.showIntermediateBounds true
-features.graph.frequentRefresh true
# ps graph parameters
# primary/secondaryStat: any numeric field provided by the ps command
@@ -49,6 +47,7 @@
features.graph.ps.cachedOnly true
features.graph.bw.prepopulate true
+features.graph.bw.transferInBytes false
features.graph.bw.accounting.show true
features.graph.bw.accounting.rate 10
features.graph.bw.accounting.isTimeLong false
@@ -95,6 +94,8 @@
log.graph.ps.abandon WARN
log.graph.bw.prepopulateSuccess NOTICE
log.graph.bw.prepopulateFailure NOTICE
+log.logPanel.prepopulateSuccess INFO
+log.logPanel.prepopulateFailed WARN
log.connLookupFailed INFO
log.connLookupFailover NOTICE
log.connLookupAbandon WARN
Modified: arm/trunk/src/interface/confPanel.py
===================================================================
--- arm/trunk/src/interface/confPanel.py 2010-09-14 22:04:58 UTC (rev 23195)
+++ arm/trunk/src/interface/confPanel.py 2010-09-14 22:21:48 UTC (rev 23196)
@@ -27,7 +27,7 @@
# GETCONF aliases (from the _option_abbrevs struct of src/or/config.c)
# fix for: https://trac.torproject.org/projects/tor/ticket/1798
-# TODO: remove if/when fixed in tor
+# TODO: this has been fixed in tor- wait for a while then retest and remove
# TODO: the following alias entry doesn't work on Tor 0.2.1.19:
# "HashedControlPassword": "__HashedControlSessionPassword"
CONF_ALIASES = {"l": "Log",
@@ -203,11 +203,10 @@
return resetSuccessful
def handleKey(self, key):
- pageHeight = self.getPreferredSize()[0] - 1
- if key == curses.KEY_UP: self.scroll = max(self.scroll - 1, 0)
- elif key == curses.KEY_DOWN: self.scroll = max(0, min(self.scroll + 1, len(self.confContents) - pageHeight))
- elif key == curses.KEY_PPAGE: self.scroll = max(self.scroll - pageHeight, 0)
- elif key == curses.KEY_NPAGE: self.scroll = max(0, min(self.scroll + pageHeight, len(self.confContents) - pageHeight))
+ if uiTools.isScrollKey(key):
+ pageHeight = self.getPreferredSize()[0] - 1
+ contentHeight = len(self.confContents)
+ self.scroll = uiTools.getScrollPosition(key, self.scroll, pageHeight, contentHeight)
elif key == ord('n') or key == ord('N'): self.showLineNum = not self.showLineNum
elif key == ord('s') or key == ord('S'):
self.stripComments = not self.stripComments
Modified: arm/trunk/src/interface/connPanel.py
===================================================================
--- arm/trunk/src/interface/connPanel.py 2010-09-14 22:04:58 UTC (rev 23195)
+++ arm/trunk/src/interface/connPanel.py 2010-09-14 22:21:48 UTC (rev 23196)
@@ -440,7 +440,9 @@
def handleKey(self, key):
# cursor or scroll movement
- if key in (curses.KEY_UP, curses.KEY_DOWN, curses.KEY_PPAGE, curses.KEY_NPAGE):
+
+ #if key in (curses.KEY_UP, curses.KEY_DOWN, curses.KEY_PPAGE, curses.KEY_NPAGE):
+ if uiTools.isScrollKey(key):
pageHeight = self.getPreferredSize()[0] - 1
if self.showingDetails: pageHeight -= 8
@@ -457,6 +459,8 @@
elif key == curses.KEY_DOWN: shift = 1
elif key == curses.KEY_PPAGE: shift = -pageHeight + 1 if self.isCursorEnabled else -pageHeight
elif key == curses.KEY_NPAGE: shift = pageHeight - 1 if self.isCursorEnabled else pageHeight
+ elif key == curses.KEY_HOME: shift = -currentLoc
+ elif key == curses.KEY_END: shift = len(self.connections) # always below the lower bound
newLoc = currentLoc + shift
# restricts to valid bounds
Modified: arm/trunk/src/interface/controller.py
===================================================================
--- arm/trunk/src/interface/controller.py 2010-09-14 22:04:58 UTC (rev 23195)
+++ arm/trunk/src/interface/controller.py 2010-09-14 22:21:48 UTC (rev 23196)
@@ -357,7 +357,7 @@
"header": headerPanel.HeaderPanel(stdscr, config),
"popup": Popup(stdscr, 9),
"graph": graphing.graphPanel.GraphPanel(stdscr),
- "log": logPanel.LogMonitor(stdscr, conn, loggedEvents)}
+ "log": logPanel.LogPanel(stdscr, loggedEvents, config)}
# TODO: later it would be good to set the right 'top' values during initialization,
# but for now this is just necessary for the log panel (and a hack in the log...)
@@ -387,7 +387,7 @@
# listeners that update bandwidth and log panels with Tor status
sighupTracker = sighupListener()
- conn.add_event_listener(panels["log"])
+ #conn.add_event_listener(panels["log"])
conn.add_event_listener(panels["graph"].stats["bandwidth"])
conn.add_event_listener(panels["graph"].stats["system resources"])
if not isBlindMode: conn.add_event_listener(panels["graph"].stats["connections"])
@@ -401,24 +401,26 @@
# tells Tor to listen to the events we're interested
loggedEvents = setEventListening(loggedEvents, isBlindMode)
- panels["log"].loggedEvents = loggedEvents # strips any that couldn't be set
+ #panels["log"].loggedEvents = loggedEvents # strips any that couldn't be set
+ panels["log"].setLoggedEvents(loggedEvents) # strips any that couldn't be set
# directs logged TorCtl events to log panel
#TorUtil.loglevel = "DEBUG"
#TorUtil.logfile = panels["log"]
- torTools.getConn().addTorCtlListener(panels["log"].tor_ctl_event)
+ #torTools.getConn().addTorCtlListener(panels["log"].tor_ctl_event)
# tells revised panels to run as daemons
panels["header"].start()
+ panels["log"].start()
# warns if tor isn't updating descriptors
try:
if conn.get_option("FetchUselessDescriptors")[0][1] == "0" and conn.get_option("DirPort")[0][1] == "0":
- warning = ["Descriptors won't be updated (causing some connection information to be stale) unless:", \
- " a. 'FetchUselessDescriptors 1' is set in your torrc", \
- " b. the directory service is provided ('DirPort' defined)", \
- " c. or tor is used as a client"]
+ warning = """Descriptors won't be updated (causing some connection information to be stale) unless:
+ a. 'FetchUselessDescriptors 1' is set in your torrc
+ b. the directory service is provided ('DirPort' defined)
+ c. or tor is used as a client"""
log.log(log.WARN, warning)
except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed): pass
@@ -458,7 +460,7 @@
# if bandwidth graph is being shown then height might have changed
if panels["graph"].currentDisplay == "bandwidth":
- panels["graph"].setHeight(panels["graph"].stats["bandwidth"].getPreferredHeight())
+ panels["graph"].setHeight(panels["graph"].stats["bandwidth"].getContentHeight())
panels["torrc"].reset()
sighupTracker.isReset = False
@@ -511,7 +513,7 @@
for panelKey in (PAGE_S + PAGES[page]):
# redrawing popup can result in display flicker when it should be hidden
if panelKey != "popup":
- if panelKey in ("header", "graph"):
+ if panelKey in ("header", "graph", "log"):
# revised panel (handles its own content refreshing)
panels[panelKey].redraw()
else:
@@ -571,7 +573,10 @@
# stops panel daemons
panels["header"].stop()
+ panels["log"].stop()
+
panels["header"].join()
+ panels["log"].join()
conn.close() # joins on TorCtl event thread
break
@@ -636,8 +641,9 @@
regexLabel = "enabled" if panels["log"].regexFilter else "disabled"
popup.addfstr(5, 41, "<b>f</b>: log regex filter (<b>%s</b>)" % regexLabel)
+ popup.addfstr(6, 2, "<b>x</b>: clear event log")
- pageOverrideKeys = (ord('m'), ord('n'), ord('s'), ord('i'), ord('d'), ord('e'), ord('r'), ord('f'))
+ pageOverrideKeys = (ord('m'), ord('n'), ord('s'), ord('i'), ord('d'), ord('e'), ord('r'), ord('f'), ord('x'))
if page == 1:
popup.addfstr(1, 2, "<b>up arrow</b>: scroll up a line")
popup.addfstr(1, 41, "<b>down arrow</b>: scroll down a line")
@@ -725,6 +731,9 @@
else: panels["graph"].setStats(options[selection].lower())
selectiveRefresh(panels, page)
+
+ # TODO: this shouldn't be necessary with the above refresh, but doesn't seem responsive otherwise...
+ panels["graph"].redraw(True)
elif page == 0 and (key == ord('i') or key == ord('I')):
# provides menu to pick graph panel update interval
options = [label for (label, intervalTime) in graphing.graphPanel.UPDATE_INTERVALS]
@@ -769,6 +778,8 @@
curses.halfdelay(REFRESH_RATE * 10) # reset normal pausing behavior
finally:
panel.CURSES_LOCK.release()
+
+ panels["graph"].redraw(True)
elif page == 0 and (key == ord('e') or key == ord('E')):
# allow user to enter new types of events to log - unchanged if left blank
panel.CURSES_LOCK.acquire()
@@ -814,7 +825,7 @@
try:
expandedEvents = logPanel.expandEvents(eventsInput)
loggedEvents = setEventListening(expandedEvents, isBlindMode)
- panels["log"].loggedEvents = loggedEvents
+ panels["log"].setLoggedEvents(loggedEvents)
except ValueError, exc:
panels["control"].setMsg("Invalid flags: %s" % str(exc), curses.A_STANDOUT)
panels["control"].redraw(True)
@@ -828,6 +839,8 @@
setPauseState(panels, isPaused, page)
finally:
panel.CURSES_LOCK.release()
+
+ panels["graph"].redraw(True)
elif page == 0 and (key == ord('f') or key == ord('F')):
# provides menu to pick previous regular expression filters or to add a new one
# for syntax see: http://docs.python.org/library/re.html#regular-expression-syntax
@@ -844,7 +857,7 @@
# applies new setting
if selection == 0:
- panels["log"].regexFilter = None
+ panels["log"].setFilter(None)
elif selection == len(options) - 1:
# selected 'New...' option - prompt user to input regular expression
panel.CURSES_LOCK.acquire()
@@ -869,7 +882,7 @@
if regexInput != "":
try:
- panels["log"].regexFilter = re.compile(regexInput)
+ panels["log"].setFilter(re.compile(regexInput))
if regexInput in regexFilters: regexFilters.remove(regexInput)
regexFilters = [regexInput] + regexFilters
except re.error, exc:
@@ -881,7 +894,7 @@
panel.CURSES_LOCK.release()
elif selection != -1:
try:
- panels["log"].regexFilter = re.compile(regexFilters[selection - 1])
+ panels["log"].setFilter(re.compile(regexFilters[selection - 1]))
# move selection to top
regexFilters = [regexFilters[selection - 1]] + regexFilters
@@ -896,6 +909,7 @@
# reverts changes made for popup
panels["graph"].showLabel = True
setPauseState(panels, isPaused, page)
+ panels["graph"].redraw(True)
elif page == 0 and key in (ord('n'), ord('N'), ord('m'), ord('M')):
# Unfortunately modifier keys don't work with the up/down arrows (sending
# multiple keycodes. The only exception to this is shift + left/right,
@@ -911,6 +925,26 @@
if currentHeight < maxHeight + 1:
panels["graph"].setGraphHeight(panels["graph"].graphHeight + 1)
+ elif page == 0 and (key == ord('x') or key == ord('X')):
+ # provides prompt to confirm that arm should clear the log
+ panel.CURSES_LOCK.acquire()
+ try:
+ setPauseState(panels, isPaused, page, True)
+
+ # provides prompt
+ panels["control"].setMsg("This will clear the log. Are you sure (x again to confirm)?", curses.A_BOLD)
+ panels["control"].redraw(True)
+
+ curses.cbreak()
+ confirmationKey = stdscr.getch()
+ if confirmationKey in (ord('x'), ord('X')): panels["log"].clear()
+
+ # reverts display settings
+ curses.halfdelay(REFRESH_RATE * 10)
+ panels["control"].setMsg(CTL_PAUSED if isPaused else CTL_HELP)
+ setPauseState(panels, isPaused, page)
+ finally:
+ panel.CURSES_LOCK.release()
elif key == 27 and panels["conn"].listingType == connPanel.LIST_HOSTNAME and panels["control"].resolvingCounter != -1:
# canceling hostname resolution (esc on any page)
panels["conn"].listingType = connPanel.LIST_IP
Modified: arm/trunk/src/interface/graphing/bandwidthStats.py
===================================================================
--- arm/trunk/src/interface/graphing/bandwidthStats.py 2010-09-14 22:04:58 UTC (rev 23195)
+++ arm/trunk/src/interface/graphing/bandwidthStats.py 2010-09-14 22:21:48 UTC (rev 23196)
@@ -20,7 +20,7 @@
PREPOPULATE_SUCCESS_MSG = "Read the last day of bandwidth history from the state file"
PREPOPULATE_FAILURE_MSG = "Unable to prepopulate bandwidth information (%s)"
-DEFAULT_CONFIG = {"features.graph.bw.accounting.show": True, "features.graph.bw.accounting.rate": 10, "features.graph.bw.accounting.isTimeLong": False, "log.graph.bw.prepopulateSuccess": log.NOTICE, "log.graph.bw.prepopulateFailure": log.NOTICE}
+DEFAULT_CONFIG = {"features.graph.bw.transferInBytes": False, "features.graph.bw.accounting.show": True, "features.graph.bw.accounting.rate": 10, "features.graph.bw.accounting.isTimeLong": False, "log.graph.bw.prepopulateSuccess": log.NOTICE, "log.graph.bw.prepopulateFailure": log.NOTICE}
class BandwidthStats(graphPanel.GraphStats):
"""
@@ -254,7 +254,7 @@
stats[1] = "- %s" % self._getAvgLabel(isPrimary)
stats[2] = ", %s" % self._getTotalLabel(isPrimary)
- stats[0] = "%-14s" % ("%s/sec" % uiTools.getSizeLabel((self.lastPrimary if isPrimary else self.lastSecondary) * 1024, 1))
+ stats[0] = "%-14s" % ("%s/sec" % uiTools.getSizeLabel((self.lastPrimary if isPrimary else self.lastSecondary) * 1024, 1, False, self._config["features.graph.bw.transferInBytes"]))
# drops label's components if there's not enough space
labeling = graphType + " (" + "".join(stats).strip() + "):"
@@ -280,39 +280,40 @@
conn = torTools.getConn()
if not conn.isAlive(): return # keep old values
- myFingerprint = conn.getMyFingerprint()
+ myFingerprint = conn.getInfo("fingerprint")
if not self._titleStats or not myFingerprint or (event and myFingerprint in event.idlist):
stats = []
bwRate = conn.getMyBandwidthRate()
bwBurst = conn.getMyBandwidthBurst()
bwObserved = conn.getMyBandwidthObserved()
bwMeasured = conn.getMyBandwidthMeasured()
+ labelInBytes = self._config["features.graph.bw.transferInBytes"]
if bwRate and bwBurst:
- bwRateLabel = uiTools.getSizeLabel(bwRate, 1)
- bwBurstLabel = uiTools.getSizeLabel(bwBurst, 1)
+ bwRateLabel = uiTools.getSizeLabel(bwRate, 1, False, labelInBytes)
+ bwBurstLabel = uiTools.getSizeLabel(bwBurst, 1, False, labelInBytes)
# if both are using rounded values then strip off the ".0" decimal
if ".0" in bwRateLabel and ".0" in bwBurstLabel:
bwRateLabel = bwRateLabel.replace(".0", "")
bwBurstLabel = bwBurstLabel.replace(".0", "")
- stats.append("limit: %s" % bwRateLabel)
- stats.append("burst: %s" % bwBurstLabel)
+ stats.append("limit: %s/s" % bwRateLabel)
+ stats.append("burst: %s/s" % bwBurstLabel)
# Provide the observed bandwidth either if the measured bandwidth isn't
# available or if the measured bandwidth is the observed (this happens
# if there isn't yet enough bandwidth measurements).
if bwObserved and (not bwMeasured or bwMeasured == bwObserved):
- stats.append("observed: %s" % uiTools.getSizeLabel(bwObserved, 1))
+ stats.append("observed: %s/s" % uiTools.getSizeLabel(bwObserved, 1, False, labelInBytes))
elif bwMeasured:
- stats.append("measured: %s" % uiTools.getSizeLabel(bwMeasured, 1))
+ stats.append("measured: %s/s" % uiTools.getSizeLabel(bwMeasured, 1, False, labelInBytes))
self._titleStats = stats
def _getAvgLabel(self, isPrimary):
total = self.primaryTotal if isPrimary else self.secondaryTotal
- return "avg: %s/sec" % uiTools.getSizeLabel((total / max(1, self.tick)) * 1024, 1)
+ return "avg: %s/sec" % uiTools.getSizeLabel((total / max(1, self.tick)) * 1024, 1, False, self._config["features.graph.bw.transferInBytes"])
def _getTotalLabel(self, isPrimary):
total = self.primaryTotal if isPrimary else self.secondaryTotal
Modified: arm/trunk/src/interface/graphing/connStats.py
===================================================================
--- arm/trunk/src/interface/graphing/connStats.py 2010-09-14 22:04:58 UTC (rev 23195)
+++ arm/trunk/src/interface/graphing/connStats.py 2010-09-14 22:21:48 UTC (rev 23196)
@@ -48,4 +48,7 @@
avg = (self.primaryTotal if isPrimary else self.secondaryTotal) / max(1, self.tick)
if isPrimary: return "Inbound (%s, avg: %s):" % (self.lastPrimary, avg)
else: return "Outbound (%s, avg: %s):" % (self.lastSecondary, avg)
+
+ def getRefreshRate(self):
+ return 5
Modified: arm/trunk/src/interface/graphing/graphPanel.py
===================================================================
--- arm/trunk/src/interface/graphing/graphPanel.py 2010-09-14 22:04:58 UTC (rev 23195)
+++ arm/trunk/src/interface/graphing/graphPanel.py 2010-09-14 22:21:48 UTC (rev 23196)
@@ -41,7 +41,7 @@
WIDE_LABELING_GRAPH_COL = 50 # minimum graph columns to use wide spacing for x-axis labels
# used for setting defaults when initializing GraphStats and GraphPanel instances
-CONFIG = {"features.graph.height": 7, "features.graph.interval": 0, "features.graph.bound": 1, "features.graph.maxWidth": 150, "features.graph.showIntermediateBounds": True, "features.graph.frequentRefresh": True}
+CONFIG = {"features.graph.height": 7, "features.graph.interval": 0, "features.graph.bound": 1, "features.graph.maxWidth": 150, "features.graph.showIntermediateBounds": True}
def loadConfig(config):
config.update(CONFIG)
@@ -107,12 +107,10 @@
"""
if self._graphPanel and not self.isPauseBuffer and not self.isPaused:
- if CONFIG["features.graph.frequentRefresh"]: return True
- else:
- updateRate = UPDATE_INTERVALS[self._graphPanel.updateInterval][1]
- if (self.tick + 1) % updateRate == 0: return True
-
- return False
+ # use the minimum of the current refresh rate and the panel's
+ updateRate = UPDATE_INTERVALS[self._graphPanel.updateInterval][1]
+ return (self.tick + 1) % min(updateRate, self.getRefreshRate()) == 0
+ else: return False
def getTitle(self, width):
"""
@@ -142,6 +140,14 @@
return DEFAULT_CONTENT_HEIGHT
+ def getRefreshRate(self):
+ """
+ Provides the number of ticks between when the stats have new values to be
+ redrawn.
+ """
+
+ return 1
+
def isVisible(self):
"""
True if the stat has content to present, false if it should be hidden.
Modified: arm/trunk/src/interface/graphing/psStats.py
===================================================================
--- arm/trunk/src/interface/graphing/psStats.py 2010-09-14 22:04:58 UTC (rev 23195)
+++ arm/trunk/src/interface/graphing/psStats.py 2010-09-14 22:21:48 UTC (rev 23196)
@@ -4,7 +4,7 @@
"""
import graphPanel
-from util import log, sysTools, torTools, uiTools
+from util import conf, log, sysTools, torTools, uiTools
# number of subsequent failed queries before giving up
FAILURE_THRESHOLD = 5
@@ -43,6 +43,12 @@
def getTitle(self, width):
return "System Resources:"
+ def getRefreshRate(self):
+ # provides the rate at which the panel has new stats to display
+ if self._config["features.graph.ps.cachedOnly"]:
+ return int(conf.getConfig("arm").get("queries.ps.rate"))
+ else: return 1
+
def getHeaderLabel(self, width, isPrimary):
avg = (self.primaryTotal if isPrimary else self.secondaryTotal) / max(1, self.tick)
lastAmount = self.lastPrimary if isPrimary else self.lastSecondary
Modified: arm/trunk/src/interface/headerPanel.py
===================================================================
--- arm/trunk/src/interface/headerPanel.py 2010-09-14 22:04:58 UTC (rev 23195)
+++ arm/trunk/src/interface/headerPanel.py 2010-09-14 22:21:48 UTC (rev 23196)
@@ -328,7 +328,7 @@
if self.vals["tor/address"] == "Unknown":
volatile["tor/address"] = conn.getInfo("address", self.vals["tor/address"])
- volatile["tor/fingerprint"] = conn.getMyFingerprint(self.vals["tor/fingerprint"])
+ volatile["tor/fingerprint"] = conn.getInfo("fingerprint", self.vals["tor/fingerprint"])
volatile["tor/flags"] = conn.getMyFlags(self.vals["tor/flags"])
# ps derived stats
Modified: arm/trunk/src/interface/logPanel.py
===================================================================
--- arm/trunk/src/interface/logPanel.py 2010-09-14 22:04:58 UTC (rev 23195)
+++ arm/trunk/src/interface/logPanel.py 2010-09-14 22:21:48 UTC (rev 23196)
@@ -1,22 +1,18 @@
-#!/usr/bin/env python
-# logPanel.py -- Resources related to Tor event monitoring.
-# Released under the GPL v3 (http://www.gnu.org/licenses/gpl.html)
+"""
+Panel providing a chronological log of events its been configured to listen
+for. This provides prepopulation from the log file and supports filtering by
+regular expressions.
+"""
import time
import curses
+import threading
from curses.ascii import isprint
+
from TorCtl import TorCtl
from util import log, panel, sysTools, torTools, uiTools
-PRE_POPULATE_LOG = True # attempts to retrieve events from log file if available
-
-# truncates to the last X log lines (needed to start in a decent time if the log's big)
-PRE_POPULATE_MIN_LIMIT = 1000 # limit in case of verbose logging
-PRE_POPULATE_MAX_LIMIT = 5000 # limit for NOTICE - ERR (since most lines are skipped)
-MAX_LOG_ENTRIES = 1000 # size of log buffer (max number of entries)
-RUNLEVEL_EVENT_COLOR = {"DEBUG": "magenta", "INFO": "blue", "NOTICE": "green", "WARN": "yellow", "ERR": "red"}
-
TOR_EVENT_TYPES = {
"d": "DEBUG", "a": "ADDRMAP", "k": "DESCCHANGED", "s": "STREAM",
"i": "INFO", "f": "AUTHDIR_NEWDESCS", "g": "GUARD", "r": "STREAM_BW",
@@ -35,11 +31,14 @@
12345 arm runlevel+ X No Events
67890 torctl runlevel+ U Unknown Events"""
-TOR_CTL_CLOSE_MSG = "Tor closed control connection. Exiting event thread."
+RUNLEVELS = ["DEBUG", "INFO", "NOTICE", "WARN", "ERR"]
+RUNLEVEL_EVENT_COLOR = {"DEBUG": "magenta", "INFO": "blue", "NOTICE": "green", "WARN": "yellow", "ERR": "red"}
+DEFAULT_CONFIG = {"features.log.prepopulate": True, "features.log.prepopulateReadLimit": 5000, "features.log.maxRefreshRate": 300, "cache.logPanel.size": 1000, "log.logPanel.prepopulateSuccess": log.INFO, "log.logPanel.prepopulateFailed": log.WARN}
+
def expandEvents(eventAbbr):
"""
- Expands event abbreviations to their full names. Beside mappings privided in
+ Expands event abbreviations to their full names. Beside mappings provided in
TOR_EVENT_TYPES this recognizes the following special events and aliases:
U - UKNOWN events
A - all events
@@ -53,33 +52,38 @@
"inUt" -> ["INFO", "NOTICE", "UNKNOWN", "STREAM_BW"]
"N4" -> ["NOTICE", "WARN", "ERR", "ARM_WARN", "ARM_ERR"]
"cfX" -> []
+
+ Arguments:
+ eventAbbr - flags to be parsed to event types
"""
- expandedEvents = set()
- invalidFlags = ""
+ expandedEvents, invalidFlags = set(), ""
+
for flag in eventAbbr:
if flag == "A":
- expandedEvents = set(TOR_EVENT_TYPES.values() + ["ARM_DEBUG", "ARM_INFO", "ARM_NOTICE", "ARM_WARN", "ARM_ERR"])
+ armRunlevels = ["ARM_" + runlevel for runlevel in RUNLEVELS]
+ torctlRunlevels = ["TORCTL_" + runlevel for runlevel in RUNLEVELS]
+ expandedEvents = set(TOR_EVENT_TYPES.values() + armRunlevels + torctlRunlevels + ["UNKNOWN"])
break
elif flag == "X":
expandedEvents = set()
break
- elif flag == "U": expandedEvents.add("UNKNOWN")
- elif flag == "D": expandedEvents = expandedEvents.union(set(["DEBUG", "INFO", "NOTICE", "WARN", "ERR"]))
- elif flag == "I": expandedEvents = expandedEvents.union(set(["INFO", "NOTICE", "WARN", "ERR"]))
- elif flag == "N": expandedEvents = expandedEvents.union(set(["NOTICE", "WARN", "ERR"]))
- elif flag == "W": expandedEvents = expandedEvents.union(set(["WARN", "ERR"]))
- elif flag == "E": expandedEvents.add("ERR")
- elif flag == "1": expandedEvents = expandedEvents.union(set(["ARM_DEBUG", "ARM_INFO", "ARM_NOTICE", "ARM_WARN", "ARM_ERR"]))
- elif flag == "2": expandedEvents = expandedEvents.union(set(["ARM_INFO", "ARM_NOTICE", "ARM_WARN", "ARM_ERR"]))
- elif flag == "3": expandedEvents = expandedEvents.union(set(["ARM_NOTICE", "ARM_WARN", "ARM_ERR"]))
- elif flag == "4": expandedEvents = expandedEvents.union(set(["ARM_WARN", "ARM_ERR"]))
- elif flag == "5": expandedEvents.add("ARM_ERR")
- elif flag == "6": expandedEvents = expandedEvents.union(set(["TORCTL_DEBUG", "TORCTL_INFO", "TORCTL_NOTICE", "TORCTL_WARN", "TORCTL_ERR"]))
- elif flag == "7": expandedEvents = expandedEvents.union(set(["TORCTL_INFO", "TORCTL_NOTICE", "TORCTL_WARN", "TORCTL_ERR"]))
- elif flag == "8": expandedEvents = expandedEvents.union(set(["TORCTL_NOTICE", "TORCTL_WARN", "TORCTL_ERR"]))
- elif flag == "9": expandedEvents = expandedEvents.union(set(["TORCTL_WARN", "TORCTL_ERR"]))
- elif flag == "0": expandedEvents.add("TORCTL_ERR")
+ elif flag in "DINWE1234567890":
+ # all events for a runlevel and higher
+ if flag in "DINWE": typePrefix = ""
+ elif flag in "12345": typePrefix = "ARM_"
+ elif flag in "67890": typePrefix = "TORCTL_"
+
+ if flag in "D16": runlevelIndex = 0
+ elif flag in "I27": runlevelIndex = 1
+ elif flag in "N38": runlevelIndex = 2
+ elif flag in "W49": runlevelIndex = 3
+ elif flag in "E50": runlevelIndex = 4
+
+ runlevelSet = [typePrefix + runlevel for runlevel in RUNLEVELS[runlevelIndex:]]
+ expandedEvents = expandedEvents.union(set(runlevelSet))
+ elif flag == "U":
+ expandedEvents.add("UNKNOWN")
elif flag in TOR_EVENT_TYPES:
expandedEvents.add(TOR_EVENT_TYPES[flag])
else:
@@ -88,234 +92,408 @@
if invalidFlags: raise ValueError(invalidFlags)
else: return expandedEvents
-class LogMonitor(TorCtl.PostEventListener, panel.Panel):
+def getLogFileEntries(runlevels, readLimit = None, addLimit = None):
"""
- Tor event listener, noting messages, the time, and their type in a panel.
+ Parses tor's log file for past events matching the given runlevels, providing
+ a list of log entries (ordered newest to oldest). Limiting the number of read
+ entries is suggested to avoid parsing everything from logs in the GB and TB
+ range.
+
+ Arguments:
+ runlevels - event types (DEBUG - ERR) to be returned
+ readLimit - max lines of the log file that'll be read (unlimited if None)
+ addLimit - maximum entries to provide back (unlimited if None)
"""
- def __init__(self, stdscr, conn, loggedEvents):
- TorCtl.PostEventListener.__init__(self)
- panel.Panel.__init__(self, stdscr, "log", 0)
- self.scroll = 0
- self.msgLog = [] # tuples of (logText, color)
- self.isPaused = False
- self.pauseBuffer = [] # location where messages are buffered if paused
- self.loggedEvents = loggedEvents # events we're listening to
- self.lastHeartbeat = time.time() # time of last event
- self.regexFilter = None # filter for presented log events (no filtering if None)
- self.eventTimeOverwrite = None # replaces time for further events with this (uses time it occures if None)
- self.controlPortClosed = False # flag set if TorCtl provided notice that control port is closed
+ startTime = time.time()
+ if not runlevels: return []
+
+ # checks tor's configuration for the log file's location (if any exists)
+ loggingTypes, loggingLocation = None, None
+ for loggingEntry in torTools.getConn().getOption("Log", [], True):
+ # looks for an entry like: notice file /var/log/tor/notices.log
+ entryComp = loggingEntry.split()
- # prevents attempts to redraw while processing batch of events
- previousPauseState = self.isPaused
- self.setPaused(True)
- log.addListeners([log.DEBUG, log.INFO, log.NOTICE, log.WARN, log.ERR], self.arm_event_wrapper, True)
- self.setPaused(previousPauseState)
+ if entryComp[1] == "file":
+ loggingTypes, loggingLocation = entryComp[0], entryComp[2]
+ break
+
+ if not loggingLocation: return []
+
+ # if the runlevels argument is a superset of the log file then we can
+ # limit the read contents to the addLimit
+ if addLimit and (not readLimit or readLimit > addLimit):
+ if "-" in loggingTypes:
+ divIndex = loggingTypes.find("-")
+ sIndex = RUNLEVELS.index(loggingTypes[:divIndex])
+ eIndex = RUNLEVELS.index(loggingTypes[divIndex+1:])
+ logFileRunlevels = RUNLEVELS[sIndex:eIndex+1]
+ else:
+ sIndex = RUNLEVELS.index(loggingTypes)
+ logFileRunlevels = RUNLEVELS[sIndex:]
- # attempts to process events from log file
- if PRE_POPULATE_LOG:
- previousPauseState = self.isPaused
-
- try:
- logFileLoc = None
- loggingLocations = conn.get_option("Log")
-
- for entry in loggingLocations:
- entryComp = entry[1].split()
- if entryComp[1] == "file":
- logFileLoc = entryComp[2]
- break
-
- if logFileLoc:
- # prevents attempts to redraw while processing batch of events
- self.setPaused(True)
-
- # trims log to last entries to deal with logs when they're in the GB or TB range
- # throws IOError if tail fails (falls to the catch-all later)
- # TODO: now that this is using sysTools figure out if we can do away with the catch-all...
- limit = PRE_POPULATE_MIN_LIMIT if ("DEBUG" in self.loggedEvents or "INFO" in self.loggedEvents) else PRE_POPULATE_MAX_LIMIT
-
- # truncates to entries for this tor instance
- lines = sysTools.call("tail -n %i %s" % (limit, logFileLoc))
- instanceStart = 0
- for i in range(len(lines) - 1, -1, -1):
- if "opening log file" in lines[i]:
- instanceStart = i
- break
-
- for line in lines[instanceStart:]:
- lineComp = line.split()
- eventType = lineComp[3][1:-1].upper()
-
- if eventType in self.loggedEvents:
- timeComp = lineComp[2][:lineComp[2].find(".")].split(":")
- self.eventTimeOverwrite = (0, 0, 0, int(timeComp[0]), int(timeComp[1]), int(timeComp[2]))
- self.listen(TorCtl.LogEvent(eventType, " ".join(lineComp[4:])))
- except Exception: pass # disreguard any issues that might arise
- finally:
- self.setPaused(previousPauseState)
- self.eventTimeOverwrite = None
+ # checks if runlevels we're reporting are a superset of the file's contents
+ isFileSubset = True
+ for runlevelType in logFileRunlevels:
+ if runlevelType not in runlevels:
+ isFileSubset = False
+ break
+
+ if isFileSubset: readLimit = addLimit
- def handleKey(self, key):
- # scroll movement
- if key in (curses.KEY_UP, curses.KEY_DOWN, curses.KEY_PPAGE, curses.KEY_NPAGE):
- pageHeight, shift = self.getPreferredSize()[0] - 1, 0
+ # tries opening the log file, cropping results to avoid choking on huge logs
+ lines = []
+ try:
+ if readLimit:
+ lines = sysTools.call("tail -n %i %s" % (readLimit, loggingLocation))
+ if not lines: raise IOError()
+ else:
+ logFile = open(loggingLocation, "r")
+ lines = logFile.readlines()
+ logFile.close()
+ except IOError:
+ msg = "Unable to read tor's log file: %s" % loggingLocation
+ log.log(DEFAULT_CONFIG["log.logPanel.prepopulateFailed"], msg)
+
+ if not lines: return []
+
+ loggedEvents = []
+ currentUnixTime, currentLocalTime = time.time(), time.localtime()
+ for i in range(len(lines) - 1, -1, -1):
+ line = lines[i]
+
+ # entries look like:
+ # Jul 15 18:29:48.806 [notice] Parsing GEOIP file.
+ lineComp = line.split()
+ eventType = lineComp[3][1:-1].upper()
+
+ if eventType in runlevels:
+ # converts timestamp to unix time
+ timestamp = " ".join(lineComp[:3])
- # location offset
- if key == curses.KEY_UP: shift = -1
- elif key == curses.KEY_DOWN: shift = 1
- elif key == curses.KEY_PPAGE: shift = -pageHeight
- elif key == curses.KEY_NPAGE: shift = pageHeight
+ # strips the decimal seconds
+ if "." in timestamp: timestamp = timestamp[:timestamp.find(".")]
- # restricts to valid bounds and applies
- maxLoc = self.getLogDisplayLength() - pageHeight
- self.scroll = max(0, min(self.scroll + shift, maxLoc))
+ # overwrites missing time parameters with the local time (ignoring wday
+ # and yday since they aren't used)
+ eventTimeComp = list(time.strptime(timestamp, "%b %d %H:%M:%S"))
+ eventTimeComp[0] = currentLocalTime.tm_year
+ eventTimeComp[8] = currentLocalTime.tm_isdst
+ eventTime = time.mktime(eventTimeComp) # converts local to unix time
+
+ # The above is gonna be wrong if the logs are for the previous year. If
+ # the event's in the future then correct for this.
+ if eventTime > currentUnixTime + 60:
+ eventTimeComp[0] -= 1
+ eventTime = time.mktime(eventTimeComp)
+
+ eventMsg = " ".join(lineComp[4:])
+ loggedEvents.append(LogEntry(eventTime, eventType, eventMsg, RUNLEVEL_EVENT_COLOR[eventType]))
+
+ if "opening log file" in line:
+ break # this entry marks the start of this tor instance
- # Listens for all event types and redirects to registerEvent
+ if addLimit: loggedEvents = loggedEvents[:addLimit]
+ msg = "Read %i entries from tor's log file: %s (read limit: %i, runtime: %0.3f)" % (len(loggedEvents), loggingLocation, readLimit, time.time() - startTime)
+ log.log(DEFAULT_CONFIG["log.logPanel.prepopulateSuccess"], msg)
+ return loggedEvents
+
+class LogEntry():
+ """
+ Individual log file entry, having the following attributes:
+ timestamp - unix timestamp for when the event occurred
+ eventType - event type that occurred ("INFO", "BW", "ARM_WARN", etc)
+ msg - message that was logged
+ color - color of the log entry
+ """
+
+ def __init__(self, timestamp, eventType, msg, color):
+ self.timestamp = timestamp
+ self.type = eventType
+ self.msg = msg
+ self.color = color
+ self._displayMessage = None
+
+ def getDisplayMessage(self):
+ """
+ Provides the entry's message for the log.
+ """
+
+ if not self._displayMessage:
+ entryTime = time.localtime(self.timestamp)
+ self._displayMessage = "%02i:%02i:%02i [%s] %s" % (entryTime[3], entryTime[4], entryTime[5], self.type, self.msg)
+
+ return self._displayMessage
+
+class TorEventObserver(TorCtl.PostEventListener):
+ """
+ Listens for all types of events provided by TorCtl, providing an LogEntry
+ instance to the given callback function.
+ """
+
+ def __init__(self, callback):
+ """
+ Tor event listener with the purpose of translating events to nicely
+ formatted calls of a callback function.
+
+ Arguments:
+ callback - function accepting a LogEntry, called when an event of these
+ types occur
+ """
+
+ TorCtl.PostEventListener.__init__(self)
+ self.callback = callback
+
def circ_status_event(self, event):
- if "CIRC" in self.loggedEvents:
- optionalParams = ""
- if event.purpose: optionalParams += " PURPOSE: %s" % event.purpose
- if event.reason: optionalParams += " REASON: %s" % event.reason
- if event.remote_reason: optionalParams += " REMOTE_REASON: %s" % event.remote_reason
- self.registerEvent("CIRC", "ID: %-3s STATUS: %-10s PATH: %s%s" % (event.circ_id, event.status, ", ".join(event.path), optionalParams), "yellow")
+ msg = "ID: %-3s STATUS: %-10s PATH: %s" % (event.circ_id, event.status, ", ".join(event.path))
+ if event.purpose: msg += " PURPOSE: %s" % event.purpose
+ if event.reason: msg += " REASON: %s" % event.reason
+ if event.remote_reason: msg += " REMOTE_REASON: %s" % event.remote_reason
+ self._notify(event, msg, "yellow")
def buildtimeout_set_event(self, event):
- # TODO: not sure how to stimulate event - needs sanity check
- try:
- self.registerEvent("BUILDTIMEOUT_SET", "SET_TYPE: %s, TOTAL_TIMES: %s, TIMEOUT_MS: %s, XM: %s, ALPHA: %s, CUTOFF_QUANTILE: %s" % (event.set_type, event.total_times, event.timeout_ms, event.xm, event.alpha, event.cutoff_quantile), "white")
- except TypeError:
- self.registerEvent("BUILDTIMEOUT_SET", "DEBUG -> SET_TYPE: %s, TOTAL_TIMES: %s, TIMEOUT_MS: %s, XM: %s, ALPHA: %s, CUTOFF_QUANTILE: %s" % (type(event.set_type), type(event.total_times), type(event.timeout_ms), type(event.xm), type(event.alpha), type(event.cutoff_quantile)), "white")
+ self._notify(event, "SET_TYPE: %s, TOTAL_TIMES: %s, TIMEOUT_MS: %s, XM: %s, ALPHA: %s, CUTOFF_QUANTILE: %s" % (event.set_type, event.total_times, event.timeout_ms, event.xm, event.alpha, event.cutoff_quantile))
def stream_status_event(self, event):
- # TODO: not sure how to stimulate event - needs sanity check
- try:
- self.registerEvent("STREAM", "ID: %s STATUS: %s CIRC_ID: %s TARGET: %s:%s REASON: %s REMOTE_REASON: %s SOURCE: %s SOURCE_ADDR: %s PURPOSE: %s" % (event.strm_id, event.status, event.circ_id, event.target_host, event.target_port, event.reason, event.remote_reason, event.source, event.source_addr, event.purpose), "white")
- except TypeError:
- self.registerEvent("STREAM", "DEBUG -> ID: %s STATUS: %s CIRC_ID: %s TARGET: %s:%s REASON: %s REMOTE_REASON: %s SOURCE: %s SOURCE_ADDR: %s PURPOSE: %s" % (type(event.strm_id), type(event.status), type(event.circ_id), type(event.target_host), type(event.target_port), type(event.reason), type(event.remote_reason), type(event.source), type(event.source_addr), type(event.purpose)), "white")
+ self._notify(event, "ID: %s STATUS: %s CIRC_ID: %s TARGET: %s:%s REASON: %s REMOTE_REASON: %s SOURCE: %s SOURCE_ADDR: %s PURPOSE: %s" % (event.strm_id, event.status, event.circ_id, event.target_host, event.target_port, event.reason, event.remote_reason, event.source, event.source_addr, event.purpose))
def or_conn_status_event(self, event):
- optionalParams = ""
- if event.age: optionalParams += " AGE: %-3s" % event.age
- if event.read_bytes: optionalParams += " READ: %-4i" % event.read_bytes
- if event.wrote_bytes: optionalParams += " WRITTEN: %-4i" % event.wrote_bytes
- if event.reason: optionalParams += " REASON: %-6s" % event.reason
- if event.ncircs: optionalParams += " NCIRCS: %i" % event.ncircs
- self.registerEvent("ORCONN", "STATUS: %-10s ENDPOINT: %-20s%s" % (event.status, event.endpoint, optionalParams), "white")
+ msg = "STATUS: %-10s ENDPOINT: %-20s" % (event.status, event.endpoint)
+ if event.age: msg += " AGE: %-3s" % event.age
+ if event.read_bytes: msg += " READ: %-4i" % event.read_bytes
+ if event.wrote_bytes: msg += " WRITTEN: %-4i" % event.wrote_bytes
+ if event.reason: msg += " REASON: %-6s" % event.reason
+ if event.ncircs: msg += " NCIRCS: %i" % event.ncircs
+ self._notify(event, msg)
def stream_bw_event(self, event):
- # TODO: not sure how to stimulate event - needs sanity check
- try:
- self.registerEvent("STREAM_BW", "ID: %s READ: %i WRITTEN: %i" % (event.strm_id, event.bytes_read, event.bytes_written), "white")
- except TypeError:
- self.registerEvent("STREAM_BW", "DEBUG -> ID: %s READ: %s WRITTEN: %s" % (type(event.strm_id), type(event.bytes_read), type(event.bytes_written)), "white")
+ self._notify(event, "ID: %s READ: %s WRITTEN: %s" % (event.strm_id, event.bytes_read, event.bytes_written))
def bandwidth_event(self, event):
- self.lastHeartbeat = time.time() # ensures heartbeat at least once a second
- if "BW" in self.loggedEvents: self.registerEvent("BW", "READ: %i, WRITTEN: %i" % (event.read, event.written), "cyan")
+ self._notify(event, "READ: %i, WRITTEN: %i" % (event.read, event.written), "cyan")
def msg_event(self, event):
- if event.level in self.loggedEvents:
- self.registerEvent(event.level, event.msg, RUNLEVEL_EVENT_COLOR[event.level])
+ self._notify(event, event.msg, RUNLEVEL_EVENT_COLOR[event.level])
def new_desc_event(self, event):
- if "NEWDESC" in self.loggedEvents:
- idlistStr = [str(item) for item in event.idlist]
- self.registerEvent("NEWDESC", ", ".join(idlistStr), "white")
+ idlistStr = [str(item) for item in event.idlist]
+ self._notify(event, ", ".join(idlistStr))
def address_mapped_event(self, event):
- self.registerEvent("ADDRMAP", "%s, %s -> %s" % (event.when, event.from_addr, event.to_addr), "white")
+ self._notify(event, "%s, %s -> %s" % (event.when, event.from_addr, event.to_addr))
def ns_event(self, event):
- # NetworkStatus params: nickname, idhash, orhash, ip, orport (int), dirport (int), flags, idhex, bandwidth, updated (datetime)
- if "NS" in self.loggedEvents:
- msg = ""
- for ns in event.nslist:
- msg += ", %s (%s:%i)" % (ns.nickname, ns.ip, ns.orport)
- if len(msg) > 1: msg = msg[2:]
- self.registerEvent("NS", "Listed (%i): %s" % (len(event.nslist), msg), "blue")
+ # NetworkStatus params: nickname, idhash, orhash, ip, orport (int),
+ # dirport (int), flags, idhex, bandwidth, updated (datetime)
+ msg = ", ".join(["%s (%s)" % (ns.idhex, ns.nickname) for ns in event.nslist])
+ self._notify(event, "Listed (%i): %s" % (len(event.nslist), msg), "blue")
def new_consensus_event(self, event):
- if "NEWCONSENSUS" in self.loggedEvents:
- msg = ""
- for ns in event.nslist:
- msg += ", %s (%s:%i)" % (ns.nickname, ns.ip, ns.orport)
- self.registerEvent("NEWCONSENSUS", "Listed (%i): %s" % (len(event.nslist), msg), "magenta")
+ msg = ", ".join(["%s (%s)" % (ns.idhex, ns.nickname) for ns in event.nslist])
+ self._notify(event, "Listed (%i): %s" % (len(event.nslist), msg), "magenta")
def unknown_event(self, event):
- if "UNKNOWN" in self.loggedEvents: self.registerEvent("UNKNOWN", event.event_string, "red")
+ msg = "(%s) %s" % (event.event_name, event.event_string)
+ self.callback(LogEntry(event.arrived_at, "UNKNOWN", msg, "red"))
- def arm_event_wrapper(self, level, msg, eventTime):
- # temporary adaptor hack to use the new logging functions until I'm sure they'll work
- # TODO: insert into log according to the event's timestamp (harder part
- # here will be interpreting tor's event timestamps...)
- self.monitor_event(level, msg)
+ def _notify(self, event, msg, color="white"):
+ self.callback(LogEntry(event.arrived_at, event.event_name, msg, color))
+
+class LogPanel(panel.Panel, threading.Thread):
+ """
+ Listens for and displays tor, arm, and torctl events. This can prepopulate
+ from tor's log file if it exists.
+ """
- def monitor_event(self, level, msg):
- # events provided by the arm monitor
- if "ARM_" + level in self.loggedEvents:
- self.registerEvent("ARM-%s" % level, msg, RUNLEVEL_EVENT_COLOR[level])
+ def __init__(self, stdscr, loggedEvents, config=None):
+ panel.Panel.__init__(self, stdscr, "log", 0)
+ threading.Thread.__init__(self)
+
+ self._config = dict(DEFAULT_CONFIG)
+
+ if config:
+ config.update(self._config)
+
+ # ensures prepopulation and cache sizes are sane
+ self._config["features.log.prepopulateReadLimit"] = max(self._config["features.log.prepopulateReadLimit"], 0)
+ self._config["features.log.maxRefreshRate"] = max(self._config["features.log.maxRefreshRate"], 10)
+ self._config["cache.logPanel.size"] = max(self._config["cache.logPanel.size"], 50)
+
+ self.msgLog = [] # log entries, sorted by the timestamp
+ self.loggedEvents = loggedEvents # events we're listening to
+ self.regexFilter = None # filter for presented log events (no filtering if None)
+ self.scroll = 0
+ self._isPaused = False
+ self._pauseBuffer = [] # location where messages are buffered if paused
+
+ self._isChanged = False # if true, has new event(s) since last drawn if true
+ self._lastUpdate = -1 # time the content was last revised
+ self._halt = False # terminates thread if true
+ self._cond = threading.Condition() # used for pausing/resuming the thread
+
+ # restricts concurrent write access to attributes used to draw the display:
+ # msgLog, loggedEvents, regexFilter, scroll
+ self.valsLock = threading.RLock()
+
+ # cached parameters (invalidated if arguments for them change)
+ # _getTitle (args: loggedEvents, regexFilter pattern, width)
+ self._titleCache = None
+ self._titleArgs = (None, None, None)
+
+ # _getContentLength (args: msgLog, regexFilter pattern, height, width)
+ self._contentLengthCache = None
+ self._contentLengthArgs = (None, None, None, None)
+
+ # fetches past tor events from log file, if available
+ torEventBacklog = []
+ if self._config["features.log.prepopulate"]:
+ setRunlevels = list(set.intersection(set(self.loggedEvents), set(RUNLEVELS)))
+ readLimit = self._config["features.log.prepopulateReadLimit"]
+ addLimit = self._config["cache.logPanel.size"]
+ torEventBacklog = getLogFileEntries(setRunlevels, readLimit, addLimit)
+
+ # adds arm listener and fetches past events
+ log.LOG_LOCK.acquire()
+ try:
+ armRunlevels = [log.DEBUG, log.INFO, log.NOTICE, log.WARN, log.ERR]
+ log.addListeners(armRunlevels, self._registerArmEvent)
+
+ # gets the set of arm events we're logging
+ setRunlevels = []
+ for i in range(len(armRunlevels)):
+ if "ARM_" + RUNLEVELS[i] in self.loggedEvents:
+ setRunlevels.append(armRunlevels[i])
+
+ armEventBacklog = []
+ for level, msg, eventTime in log._getEntries(setRunlevels):
+ runlevelStr = log.RUNLEVEL_STR[level]
+ armEventEntry = LogEntry(eventTime, "ARM_" + runlevelStr, msg, RUNLEVEL_EVENT_COLOR[runlevelStr])
+ armEventBacklog.append(armEventEntry)
+
+ # joins armEventBacklog and torEventBacklog chronologically into msgLog
+ while armEventBacklog or torEventBacklog:
+ if not armEventBacklog:
+ self.msgLog.append(torEventBacklog.pop(0))
+ elif not torEventBacklog:
+ self.msgLog.append(armEventBacklog.pop(0))
+ elif armEventBacklog[0].timestamp > torEventBacklog[0].timestamp:
+ self.msgLog.append(torEventBacklog.pop(0))
+ else:
+ self.msgLog.append(armEventBacklog.pop(0))
+ finally:
+ log.LOG_LOCK.release()
+
+ # adds listeners for tor and torctl events
+ conn = torTools.getConn()
+ conn.addEventListener(TorEventObserver(self.registerEvent))
+ conn.addTorCtlListener(self._registerTorCtlEvent)
- def tor_ctl_event(self, level, msg):
- # events provided by TorCtl
- if "TORCTL_" + level in self.loggedEvents:
- self.registerEvent("TORCTL-%s" % level, msg, RUNLEVEL_EVENT_COLOR[level])
-
- # TODO: deprecated (remove later)
- def write(self, msg):
+ def registerEvent(self, event):
"""
- Tracks TorCtl events. Ugly hack since TorCtl/TorUtil.py expects a file.
+ Notes event and redraws log. If paused it's held in a temporary buffer.
+
+ Arguments:
+ event - LogEntry for the event that occurred
"""
- timestampStart = msg.find("[")
- timestampEnd = msg.find("]")
+ if not event.type in self.loggedEvents: return
- level = msg[:timestampStart]
- msg = msg[timestampEnd + 2:].strip()
+ # strips control characters to avoid screwing up the terminal
+ event.msg = "".join([char for char in event.msg if (isprint(char) or char == "\n")])
- if TOR_CTL_CLOSE_MSG in msg:
- # TorCtl providing notice that control port is closed
- self.controlPortClosed = True
- #log.log(log.NOTICE, "Tor control port closed")
+ cacheSize = self._config["cache.logPanel.size"]
+ if self._isPaused:
+ self._pauseBuffer.insert(0, event)
+ if len(self._pauseBuffer) > cacheSize: del self._pauseBuffer[cacheSize:]
+ else:
+ self.valsLock.acquire()
+ self.msgLog.insert(0, event)
+ if len(self.msgLog) > cacheSize: del self.msgLog[cacheSize:]
- # Allows the Controller to notice that tor's shut down.
- # TODO: should make the controller the torctl event listener rather than
- # this log panel (it'll also make this less hacky)
- torTools.getConn().isAlive()
- self.tor_ctl_event(level, msg)
+ # notifies the display that it has new content
+ if not self.regexFilter or self.regexFilter.search(event.getDisplayMessage()):
+ self._isChanged = True
+ self._cond.acquire()
+ self._cond.notifyAll()
+ self._cond.release()
+
+ self.valsLock.release()
- def flush(self): pass
+ def _registerArmEvent(self, level, msg, eventTime):
+ eventColor = RUNLEVEL_EVENT_COLOR[level]
+ self.registerEvent(LogEntry(eventTime, "ARM_%s" % level, msg, eventColor))
- def registerEvent(self, type, msg, color):
+ def _registerTorCtlEvent(self, level, msg):
+ eventColor = RUNLEVEL_EVENT_COLOR[level]
+ self.registerEvent(LogEntry(time.time(), "TORCTL_%s" % level, msg, eventColor))
+
+ def setLoggedEvents(self, eventTypes):
"""
- Notes event and redraws log. If paused it's held in a temporary buffer. If
- msg is a list then this is expanded to multiple lines.
+ Sets the event types recognized by the panel.
+
+ Arguments:
+ eventTypes - event types to be logged
"""
- if not type.startswith("ARM"): self.lastHeartbeat = time.time()
- eventTime = self.eventTimeOverwrite if self.eventTimeOverwrite else time.localtime()
- toAdd = []
+ if eventTypes == self.loggedEvents: return
- # wraps if a single line message
- if isinstance(msg, str): msg = [msg]
+ self.valsLock.acquire()
+ self.loggedEvents = eventTypes
+ self.redraw(True)
+ self.valsLock.release()
+
+ def setFilter(self, logFilter):
+ """
+ Filters log entries according to the given regular expression.
- firstLine = True
- for msgLine in msg:
- # strips control characters to avoid screwing up the terminal
- msgLine = "".join([char for char in msgLine if isprint(char)])
+ Arguments:
+ logFilter - regular expression used to determine which messages are
+ shown, None if no filter should be applied
+ """
+
+ if logFilter == self.regexFilter: return
+
+ self.valsLock.acquire()
+ self.regexFilter = logFilter
+ self.redraw(True)
+ self.valsLock.release()
+
+ def clear(self):
+ """
+ Clears the contents of the event log.
+ """
+
+ self.valsLock.acquire()
+ self.msgLog = []
+ self.redraw(True)
+ self.valsLock.release()
+
+ def handleKey(self, key):
+ if uiTools.isScrollKey(key):
+ pageHeight = self.getPreferredSize()[0] - 1
+ contentHeight = self._getContentLength()
+ newScroll = uiTools.getScrollPosition(key, self.scroll, pageHeight, contentHeight)
- header = "%02i:%02i:%02i %s" % (eventTime[3], eventTime[4], eventTime[5], "[%s]" % type) if firstLine else ""
- toAdd.append("%s %s" % (header, msgLine))
- firstLine = False
+ if self.scroll != newScroll:
+ self.valsLock.acquire()
+ self.scroll = newScroll
+ self.redraw(True)
+ self.valsLock.release()
+
+ def setPaused(self, isPause):
+ """
+ If true, prevents message log from being updated with new events.
+ """
- toAdd.reverse()
- if self.isPaused:
- for msgLine in toAdd: self.pauseBuffer.insert(0, (msgLine, color))
- if len(self.pauseBuffer) > MAX_LOG_ENTRIES: del self.pauseBuffer[MAX_LOG_ENTRIES:]
+ if isPause == self._isPaused: return
+
+ self._isPaused = isPause
+ if self._isPaused: self._pauseBuffer = []
else:
- for msgLine in toAdd: self.msgLog.insert(0, (msgLine, color))
- if len(self.msgLog) > MAX_LOG_ENTRIES: del self.msgLog[MAX_LOG_ENTRIES:]
+ self.valsLock.acquire()
+ self.msgLog = (self._pauseBuffer + self.msgLog)[:self._config["cache.logPanel.size"]]
self.redraw(True)
+ self.valsLock.release()
def draw(self, subwindow, width, height):
"""
@@ -323,167 +501,184 @@
contain up to two lines. Starts with newest entries.
"""
- isScrollBarVisible = self.getLogDisplayLength() > height - 1
- xOffset = 3 if isScrollBarVisible else 0 # content offset for scroll bar
+ self.valsLock.acquire()
+ self._isChanged, self._lastUpdate = False, time.time()
- # draws label - uses ellipsis if too long, for instance:
- # Events (DEBUG, INFO, NOTICE, WARN...):
- eventsLabel = "Events"
+ # draws the top label
+ self.addstr(0, 0, self._getTitle(width), curses.A_STANDOUT)
- # separates tor and arm runlevels (might be able to show as range)
- eventsList = list(self.loggedEvents)
- torRunlevelLabel = ", ".join(parseRunlevelRanges(eventsList, ""))
- armRunlevelLabel = ", ".join(parseRunlevelRanges(eventsList, "ARM_"))
- torctlRunlevelLabel = ", ".join(parseRunlevelRanges(eventsList, "TORCTL_"))
+ # restricts scroll location to valid bounds
+ contentHeight = self._getContentLength()
+ self.scroll = max(0, min(self.scroll, contentHeight - height + 1))
- if torctlRunlevelLabel: eventsList = ["TORCTL " + torctlRunlevelLabel] + eventsList
- if armRunlevelLabel: eventsList = ["ARM " + armRunlevelLabel] + eventsList
- if torRunlevelLabel: eventsList = [torRunlevelLabel] + eventsList
+ # draws left-hand scroll bar if content's longer than the height
+ xOffset = 0 # offset for scroll bar
+ if contentHeight > height - 1:
+ xOffset = 3
+ self.addScrollBar(self.scroll, self.scroll + height - 1, contentHeight, 1)
- eventsListing = ", ".join(eventsList)
- filterLabel = "" if not self.regexFilter else " - filter: %s" % self.regexFilter.pattern
-
- firstLabelLen = eventsListing.find(", ")
- if firstLabelLen == -1: firstLabelLen = len(eventsListing)
- else: firstLabelLen += 3
-
- if width > 10 + firstLabelLen:
- eventsLabel += " ("
-
- if len(eventsListing) > width - 11:
- labelBreak = eventsListing[:width - 12].rfind(", ")
- eventsLabel += "%s..." % eventsListing[:labelBreak]
- elif len(eventsListing) + len(filterLabel) > width - 11:
- eventsLabel += eventsListing
- else: eventsLabel += eventsListing + filterLabel
- eventsLabel += ")"
- eventsLabel += ":"
-
- self.addstr(0, 0, eventsLabel, curses.A_STANDOUT)
-
- # log entries
- maxLoc = self.getLogDisplayLength() - height + 1
- self.scroll = max(0, min(self.scroll, maxLoc))
+ # draws log entries
lineCount = 1 - self.scroll
-
- for (line, color) in self.msgLog:
- if self.regexFilter and not self.regexFilter.search(line):
+ for entry in self.msgLog:
+ if self.regexFilter and not self.regexFilter.search(entry.getDisplayMessage()):
continue # filter doesn't match log message - skip
- # splits over too lines if too long
- if len(line) < width:
- if lineCount >= 1: self.addstr(lineCount, xOffset, line, uiTools.getColor(color))
- lineCount += 1
- else:
- (line1, line2) = splitLine(line, width - xOffset)
- if lineCount >= 1: self.addstr(lineCount, xOffset, line1, uiTools.getColor(color))
- if lineCount >= 0: self.addstr(lineCount + 1, xOffset, line2, uiTools.getColor(color))
- lineCount += 2
+ for line in entry.getDisplayMessage().split("\n"):
+ # splits over too lines if too long
+ if len(line) < width:
+ if lineCount >= 1: self.addstr(lineCount, xOffset, line, uiTools.getColor(entry.color))
+ lineCount += 1
+ else:
+ (line1, line2) = uiTools.splitLine(line, width - xOffset)
+ if lineCount >= 1: self.addstr(lineCount, xOffset, line1, uiTools.getColor(entry.color))
+ if lineCount >= 0: self.addstr(lineCount + 1, xOffset, line2, uiTools.getColor(entry.color))
+ lineCount += 2
if lineCount >= height: break # further log messages wouldn't fit
- if isScrollBarVisible: self.addScrollBar(self.scroll, self.scroll + height - 1, self.getLogDisplayLength(), 1)
+ self.valsLock.release()
- def getLogDisplayLength(self):
+ def redraw(self, forceRedraw=False, block=False):
+ # determines if the content needs to be redrawn or not
+ panel.Panel.redraw(self, forceRedraw, block)
+
+ def run(self):
"""
- Provides the number of lines the log would currently occupy.
+ Redraws the display, coalescing updates if events are rapidly logged (for
+ instance running at the DEBUG runlevel) while also being immediately
+ responsive if additions are less frequent.
"""
- logLength = len(self.msgLog)
-
- # takes into account filtered and wrapped messages
- for (line, color) in self.msgLog:
- if self.regexFilter and not self.regexFilter.search(line): logLength -= 1
- elif len(line) >= self.getPreferredSize()[1]: logLength += 1
-
- return logLength
+ while not self._halt:
+ timeSinceReset = time.time() - self._lastUpdate
+ maxLogUpdateRate = self._config["features.log.maxRefreshRate"] / 1000.0
+
+ sleepTime = 0
+ if not self._isChanged or self._isPaused:
+ sleepTime = 10
+ elif timeSinceReset < maxLogUpdateRate:
+ sleepTime = max(0.05, maxLogUpdateRate - timeSinceReset)
+
+ if sleepTime:
+ self._cond.acquire()
+ if not self._halt: self._cond.wait(sleepTime)
+ self._cond.release()
+ else:
+ self.redraw(True)
- def setPaused(self, isPause):
+ def stop(self):
"""
- If true, prevents message log from being updated with new events.
+ Halts further resolutions and terminates the thread.
"""
- if isPause == self.isPaused: return
-
- self.isPaused = isPause
- if self.isPaused: self.pauseBuffer = []
- else:
- self.msgLog = (self.pauseBuffer + self.msgLog)[:MAX_LOG_ENTRIES]
- if self.win: self.redraw(True) # hack to avoid redrawing during init
+ self._cond.acquire()
+ self._halt = True
+ self._cond.notifyAll()
+ self._cond.release()
- def getHeartbeat(self):
+ def _getTitle(self, width):
"""
- Provides the number of seconds since the last registered event (this always
- listens to BW events so should be less than a second if relay's still
- responsive).
+ Provides the label used for the panel, looking like:
+ Events (ARM NOTICE - ERR, BW - filter: prepopulate):
+
+ This truncates the attributes (with an ellipse) if too long, and condenses
+ runlevel ranges if there's three or more in a row (for instance ARM_INFO,
+ ARM_NOTICE, and ARM_WARN becomes "ARM_INFO - WARN").
+
+ Arguments:
+ width - width constraint the label needs to fix in
"""
- return time.time() - self.lastHeartbeat
-
-def parseRunlevelRanges(eventsList, searchPrefix):
- """
- This parses a list of events to provide an ordered list of runlevels,
- condensed if three or more are in a contiguous range. This removes parsed
- runlevels from the eventsList. For instance:
-
- eventsList = ["BW", "ARM_WARN", "ERR", "ARM_ERR", "ARM_DEBUG", "ARM_NOTICE"]
- searchPrefix = "ARM_"
-
- results in:
- eventsList = ["BW", "ERR"]
- return value is ["DEBUG", "NOTICE - ERR"]
-
- """
-
- # blank ending runlevel forces the break condition to be reached at the end
- runlevels = ["DEBUG", "INFO", "NOTICE", "WARN", "ERR", ""]
- runlevelLabels = []
- start, end = "", ""
- rangeLength = 0
-
- for level in runlevels:
- if searchPrefix + level in eventsList:
- eventsList.remove(searchPrefix + level)
-
- if start:
- end = level
- rangeLength += 1
+ # usually the attributes used to make the label are decently static, so
+ # provide cached results if they're unchanged
+ self.valsLock.acquire()
+ currentPattern = self.regexFilter.pattern if self.regexFilter else None
+ isUnchanged = self._titleArgs[0] == self.loggedEvents
+ isUnchanged &= self._titleArgs[1] == currentPattern
+ isUnchanged &= self._titleArgs[2] == width
+ if isUnchanged:
+ self.valsLock.release()
+ return self._titleCache
+
+ eventsList = list(self.loggedEvents)
+ if not eventsList:
+ if not currentPattern:
+ panelLabel = "Events:"
else:
- start = level
- rangeLength = 1
- elif rangeLength > 0:
- # reached a break in the runlevels
- if rangeLength == 1: runlevelLabels += [start]
- elif rangeLength == 2: runlevelLabels += [start, end]
- else: runlevelLabels += ["%s - %s" % (start, end)]
+ labelPattern = uiTools.cropStr(currentPattern, width - 18)
+ panelLabel = "Events (filter: %s):" % labelPattern
+ else:
+ # does the following with all runlevel types (tor, arm, and torctl):
+ # - pulls to the start of the list
+ # - condenses range if there's three or more in a row (ex. "ARM_INFO - WARN")
+ tmpRunlevels = [] # runlevels pulled from the list (just the runlevel part)
+ for prefix in ("TORCTL_", "ARM_", ""):
+ # blank ending runlevel forces the break condition to be reached at the end
+ for runlevel in RUNLEVELS + [""]:
+ eventType = prefix + runlevel
+ if eventType in eventsList:
+ # runlevel event found, move to the tmp list
+ eventsList.remove(eventType)
+ tmpRunlevels.append(runlevel)
+ elif tmpRunlevels:
+ # adds all tmp list entries to the start of eventsList
+ if len(tmpRunlevels) >= 3:
+ # condense sequential runlevels
+ startLevel, endLevel = tmpRunlevels[0], tmpRunlevels[-1]
+ eventsList.insert(0, "%s%s - %s" % (prefix, startLevel, endLevel))
+ else:
+ # adds runlevels individaully
+ tmpRunlevels.reverse()
+ for tmpRunlevel in tmpRunlevels:
+ eventsList.insert(0, prefix + tmpRunlevel)
+
+ tmpRunlevels = []
- start, end = "", ""
- rangeLength = 0
+ # truncates to use an ellipsis if too long, for instance:
+ attrLabel = ", ".join(eventsList)
+ if currentPattern: attrLabel += " - filter: %s" % currentPattern
+ attrLabel = uiTools.cropStr(attrLabel, width - 10, -1)
+ panelLabel = "Events (%s):" % attrLabel
+
+ # cache results and return
+ self._titleCache = panelLabel
+ self._titleArgs = (list(self.loggedEvents), currentPattern, width)
+ self.valsLock.release()
+ return panelLabel
- return runlevelLabels
-
-def splitLine(message, x):
- """
- Divides message into two lines, attempting to do it on a wordbreak.
- """
-
- lastWordbreak = message[:x].rfind(" ")
- if x - lastWordbreak < 10:
- line1 = message[:lastWordbreak]
- line2 = " %s" % message[lastWordbreak:].strip()
- else:
- # over ten characters until the last word - dividing
- line1 = "%s-" % message[:x - 2]
- line2 = " %s" % message[x - 2:].strip()
-
- # ends line with ellipsis if too long
- if len(line2) > x:
- lastWordbreak = line2[:x - 4].rfind(" ")
+ def _getContentLength(self):
+ """
+ Provides the number of lines the log's contents would currently occupy,
+ taking into account filtered/wrapped lines, the scroll bar, etc.
+ """
- # doesn't use wordbreak if it's a long word or the whole line is one
- # word (picking up on two space indent to have index 1)
- if x - lastWordbreak > 10 or lastWordbreak == 1: lastWordbreak = x - 4
- line2 = "%s..." % line2[:lastWordbreak]
-
- return (line1, line2)
+ # if the arguments haven't changed then we can use cached results
+ self.valsLock.acquire()
+ height, width = self.getPreferredSize()
+ currentPattern = self.regexFilter.pattern if self.regexFilter else None
+ isUnchanged = self._contentLengthArgs[0] == self.msgLog
+ isUnchanged &= self._contentLengthArgs[1] == currentPattern
+ isUnchanged &= self._contentLengthArgs[2] == height
+ isUnchanged &= self._contentLengthArgs[3] == width
+ if isUnchanged:
+ self.valsLock.release()
+ return self._contentLengthCache
+
+ contentLengths = [0, 0] # length of the content without and with a scroll bar
+ for entry in self.msgLog:
+ if not self.regexFilter or self.regexFilter.search(entry.getDisplayMessage()):
+ for line in entry.getDisplayMessage().split("\n"):
+ if len(line) >= width: contentLengths[0] += 2
+ else: contentLengths[0] += 1
+
+ if len(line) >= width - 3: contentLengths[1] += 2
+ else: contentLengths[1] += 1
+
+ # checks if the scroll bar would be displayed to determine the actual length
+ actualLength = contentLengths[0] if contentLengths[0] <= height - 1 else contentLengths[1]
+
+ self._contentLengthCache = actualLength
+ self._contentLengthArgs = (list(self.msgLog), currentPattern, height, width)
+ self.valsLock.release()
+ return actualLength
Modified: arm/trunk/src/util/sysTools.py
===================================================================
--- arm/trunk/src/util/sysTools.py 2010-09-14 22:04:58 UTC (rev 23195)
+++ arm/trunk/src/util/sysTools.py 2010-09-14 22:21:48 UTC (rev 23196)
@@ -110,7 +110,7 @@
if isinstance(cachedResults, IOError):
if IS_FAILURES_CACHED:
- msg = "system call (cached failure): %s (age: %0.1f seconds, error: %s)" % (command, cacheAge, str(cachedResults))
+ msg = "system call (cached failure): %s (age: %0.1f, error: %s)" % (command, cacheAge, str(cachedResults))
log.log(CONFIG["log.sysCallCached"], msg)
if suppressExc: return None
@@ -119,7 +119,7 @@
# flag was toggled after a failure was cached - reissue call, ignoring the cache
return call(command, 0, suppressExc, quiet)
else:
- msg = "system call (cached): %s (age: %0.1f seconds)" % (command, cacheAge)
+ msg = "system call (cached): %s (age: %0.1f)" % (command, cacheAge)
log.log(CONFIG["log.sysCallCached"], msg)
return cachedResults
@@ -160,7 +160,7 @@
else: raise errorExc
else:
# log call information and if we're caching then save the results
- msg = "system call: %s (runtime: %0.2f seconds)" % (command, time.time() - startTime)
+ msg = "system call: %s (runtime: %0.2f)" % (command, time.time() - startTime)
log.log(CONFIG["log.sysCallMade"], msg)
if cacheAge > 0:
Modified: arm/trunk/src/util/torTools.py
===================================================================
--- arm/trunk/src/util/torTools.py 2010-09-14 22:04:58 UTC (rev 23195)
+++ arm/trunk/src/util/torTools.py 2010-09-14 22:21:48 UTC (rev 23196)
@@ -12,7 +12,6 @@
import os
import time
import socket
-import getpass
import thread
import threading
@@ -39,9 +38,12 @@
CONTROLLER = None # singleton Controller instance
-# valid keys for the controller's getInfo cache
-CACHE_ARGS = ("nsEntry", "descEntry", "bwRate", "bwBurst", "bwObserved",
- "bwMeasured", "flags", "fingerprint", "pid")
+# Valid keys for the controller's getInfo cache. This includes static GETINFO
+# options (unchangable, even with a SETCONF) and other useful stats
+CACHE_ARGS = ("version", "config-file", "exit-policy/default", "fingerprint",
+ "config/names", "info/names", "features/names", "events/names",
+ "nsEntry", "descEntry", "bwRate", "bwBurst", "bwObserved",
+ "bwMeasured", "flags", "pid")
TOR_CTL_CLOSE_MSG = "Tor closed control connection. Exiting event thread."
UNKNOWN = "UNKNOWN" # value used by cached information if undefined
@@ -90,7 +92,7 @@
pidFile.close()
if pidEntry.isdigit(): return pidEntry
- except Exception: pass
+ except: pass
# attempts to resolve using pidof, failing if:
# - tor's running under a different name
@@ -160,6 +162,7 @@
# directs TorCtl to notify us of events
TorUtil.logger = self
+ TorUtil.loglevel = "DEBUG"
def init(self, conn=None):
"""
@@ -172,7 +175,7 @@
"""
if conn == None:
- conn = connect()
+ conn = TorCtl.connect()
if conn == None: raise ValueError("Unable to initialize TorCtl instance.")
@@ -268,16 +271,24 @@
self.connLock.acquire()
startTime = time.time()
- result, raisedExc = default, None
+ result, raisedExc, isFromCache = default, None, False
if self.isAlive():
- try:
- getInfoVal = self.conn.get_info(param)[param]
- if getInfoVal != None: result = getInfoVal
- except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed), exc:
- if type(exc) == TorCtl.TorCtlClosed: self.close()
- raisedExc = exc
+ if param in CACHE_ARGS and self._cachedParam[param]:
+ result = self._cachedParam[param]
+ isFromCache = True
+ else:
+ try:
+ getInfoVal = self.conn.get_info(param)[param]
+ if getInfoVal != None: result = getInfoVal
+ except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed), exc:
+ if type(exc) == TorCtl.TorCtlClosed: self.close()
+ raisedExc = exc
- msg = "tor control call: GETINFO %s (runtime: %0.4f)" % (param, time.time() - startTime)
+ if not isFromCache and result and param in CACHE_ARGS:
+ self._cachedParam[param] = result
+
+ runtimeLabel = "cache fetch" if isFromCache else "runtime: %0.4f" % (time.time() - startTime)
+ msg = "GETINFO %s (%s)" % (param, runtimeLabel)
log.log(CONFIG["log.torGetInfo"], msg)
self.connLock.release()
@@ -285,6 +296,9 @@
if not suppressExc and raisedExc: raise raisedExc
else: return result
+ # TODO: This could have client side caching if there were events to indicate
+ # SETCONF events. Ask if these can be added to tor (then ask mike if he wants
+ # client side caching included in TorCtl?).
def getOption(self, param, default = None, multiple = False, suppressExc = True):
"""
Queries the control port for the given configuration option, providing the
@@ -317,7 +331,7 @@
if type(exc) == TorCtl.TorCtlClosed: self.close()
result, raisedExc = default, exc
- msg = "tor control call: GETCONF %s (runtime: %0.4f)" % (param, time.time() - startTime)
+ msg = "GETCONF %s (runtime: %0.4f)" % (param, time.time() - startTime)
log.log(CONFIG["log.torGetConf"], msg)
self.connLock.release()
@@ -404,16 +418,6 @@
return self._getRelayAttr("bwMeasured", default)
- def getMyFingerprint(self, default = None):
- """
- Provides the fingerprint for this relay.
-
- Arguments:
- default - result if the query fails
- """
-
- return self._getRelayAttr("fingerprint", default, False)
-
def getMyFlags(self, default = None):
"""
Provides the flags held by this relay.
@@ -611,6 +615,7 @@
if not issueSighup:
try:
self.conn.send_signal("RELOAD")
+ self._cachedParam = dict([(arg, "") for arg in CACHE_ARGS])
except Exception, exc:
# new torrc parameters caused an error (tor's likely shut down)
# BUG: this doesn't work - torrc errors still cause TorCtl to crash... :(
@@ -653,6 +658,8 @@
if errorLine: raise IOError(" ".join(errorLine.split()[3:]))
else: raise IOError("failed silently")
+
+ self._cachedParam = dict([(arg, "") for arg in CACHE_ARGS])
except IOError, exc:
raisedException = exc
@@ -677,7 +684,7 @@
def ns_event(self, event):
self._updateHeartbeat()
- myFingerprint = self.getMyFingerprint()
+ myFingerprint = self.getInfo("fingerprint")
if myFingerprint:
for ns in event.nslist:
if ns.idhex == myFingerprint:
@@ -700,7 +707,7 @@
def new_desc_event(self, event):
self._updateHeartbeat()
- myFingerprint = self.getMyFingerprint()
+ myFingerprint = self.getInfo("fingerprint")
if not myFingerprint or myFingerprint in event.idlist:
self._cachedParam["descEntry"] = None
self._cachedParam["bwObserved"] = None
@@ -772,7 +779,7 @@
if not currentVal and self.isAlive():
# still unset - fetch value
if key in ("nsEntry", "descEntry"):
- myFingerprint = self.getMyFingerprint()
+ myFingerprint = self.getInfo("fingerprint")
if myFingerprint:
queryType = "ns" if key == "nsEntry" else "desc"
@@ -819,13 +826,6 @@
bwValue = line[12:]
if bwValue.isdigit(): result = int(bwValue)
break
- elif key == "fingerprint":
- # Fingerprints are kept until sighup if set (most likely not even a
- # setconf can change it since it's in the data directory). If orport is
- # unset then no fingerprint will be set.
- orPort = self.getOption("ORPort", "0")
- if orPort == "0": result = UNKNOWN
- else: result = self.getInfo("fingerprint")
elif key == "flags":
for line in self.getMyNetworkStatus([]):
if line.startswith("s "):
Modified: arm/trunk/src/util/uiTools.py
===================================================================
--- arm/trunk/src/util/uiTools.py 2010-09-14 22:04:58 UTC (rev 23195)
+++ arm/trunk/src/util/uiTools.py 2010-09-14 22:21:48 UTC (rev 23196)
@@ -21,13 +21,17 @@
COLOR_ATTR_INITIALIZED = False
COLOR_ATTR = dict([(color, 0) for color in COLOR_LIST])
-# value tuples for label conversions (bytes / seconds, short label, long label)
-SIZE_UNITS = [(1125899906842624.0, " PB", " Petabyte"), (1099511627776.0, " TB", " Terabyte"),
- (1073741824.0, " GB", " Gigabyte"), (1048576.0, " MB", " Megabyte"),
- (1024.0, " KB", " Kilobyte"), (1.0, " B", " Byte")]
+# value tuples for label conversions (bits / bytes / seconds, short label, long label)
+SIZE_UNITS_BITS = [(140737488355328.0, " Pb", " Petabit"), (137438953472.0, " Tb", " Terabit"),
+ (134217728.0, " Gb", " Gigabit"), (131072.0, " Mb", " Megabit"),
+ (128.0, " Kb", " Kilobit"), (0.125, " b", " Bit")]
+SIZE_UNITS_BYTES = [(1125899906842624.0, " PB", " Petabyte"), (1099511627776.0, " TB", " Terabyte"),
+ (1073741824.0, " GB", " Gigabyte"), (1048576.0, " MB", " Megabyte"),
+ (1024.0, " KB", " Kilobyte"), (1.0, " B", " Byte")]
TIME_UNITS = [(86400.0, "d", " day"), (3600.0, "h", " hour"),
(60.0, "m", " minute"), (1.0, "s", " second")]
+SCROLL_KEYS = (curses.KEY_UP, curses.KEY_DOWN, curses.KEY_PPAGE, curses.KEY_NPAGE, curses.KEY_HOME, curses.KEY_END)
CONFIG = {"features.colorInterface": True, "log.cursesColorSupport": log.INFO}
def loadConfig(config):
@@ -103,8 +107,81 @@
if addEllipse: returnMsg += "..."
return returnMsg
-def getSizeLabel(bytes, decimal = 0, isLong = False):
+def splitLine(message, width, indent = " "):
"""
+ Divides message into two lines, attempting to do it on a wordbreak. This
+ adds an ellipse if the second line is too long.
+
+ Arguments:
+ message - string being divided
+ width - maximum width constraint for the split
+ indent - addition made to the start of the second line
+ """
+
+ if len(message) < width: return (message, "")
+
+ lastWordbreak = message[:width].rfind(" ")
+ if width - lastWordbreak < 10:
+ line1 = message[:lastWordbreak]
+ line2 = "%s%s" % (indent, message[lastWordbreak:].strip())
+ else:
+ # over ten characters until the last word - dividing
+ line1 = "%s-" % message[:width - 2]
+ line2 = "%s%s" % (indent, message[width - 2:].strip())
+
+ # ends line with ellipsis if too long
+ if len(line2) > width:
+ lastWordbreak = line2[:width - 4].rfind(" ")
+
+ # doesn't use wordbreak if it's a long word or the whole line is one
+ # word (picking up on two space indent to have index 1)
+ if width - lastWordbreak > 10 or lastWordbreak == 1: lastWordbreak = width - 4
+ line2 = "%s..." % line2[:lastWordbreak]
+
+ return (line1, line2)
+
+def isScrollKey(key):
+ """
+ Returns true if the keycode is recognized by the getScrollPosition function
+ for scrolling.
+ """
+
+ return key in SCROLL_KEYS
+
+def getScrollPosition(key, position, pageHeight, contentHeight):
+ """
+ Parses navigation keys, providing the new scroll possition the panel should
+ use. Position is always between zero and (contentHeight - pageHeight). This
+ handles the following keys:
+ Up / Down - scrolls a position up or down
+ Page Up / Page Down - scrolls by the pageHeight
+ Home - top of the content
+ End - bottom of the content
+
+ This provides the input position if the key doesn't correspond to the above.
+
+ Arguments:
+ key - keycode for the user's input
+ position - starting position
+ pageHeight - size of a single screen's worth of content
+ contentHeight - total lines of content that can be scrolled
+ """
+
+ if isScrollKey(key):
+ shift = 0
+ if key == curses.KEY_UP: shift = -1
+ elif key == curses.KEY_DOWN: shift = 1
+ elif key == curses.KEY_PPAGE: shift = -pageHeight
+ elif key == curses.KEY_NPAGE: shift = pageHeight
+ elif key == curses.KEY_HOME: shift = -contentHeight
+ elif key == curses.KEY_END: shift = contentHeight
+
+ # returns the shift, restricted to valid bounds
+ return max(0, min(position + shift, contentHeight - pageHeight))
+ else: return position
+
+def getSizeLabel(bytes, decimal = 0, isLong = False, isBytes=True):
+ """
Converts byte count into label in its most significant units, for instance
7500 bytes would return "7 KB". If the isLong option is used this expands
unit labels to be the properly pluralized full word (for instance 'Kilobytes'
@@ -119,9 +196,11 @@
bytes - source number of bytes for conversion
decimal - number of decimal digits to be included
isLong - expands units label
+ isBytes - provides units in bytes if true, bits otherwise
"""
- return _getLabel(SIZE_UNITS, bytes, decimal, isLong)
+ if isBytes: return _getLabel(SIZE_UNITS_BYTES, bytes, decimal, isLong)
+ else: return _getLabel(SIZE_UNITS_BITS, bytes, decimal, isLong)
def getTimeLabel(seconds, decimal = 0, isLong = False):
"""
More information about the tor-commits
mailing list