[or-cvs] r20814: {arm} Substantial bundle of changes including torrc validation, im (in arm/trunk: . interface)

atagar at seul.org atagar at seul.org
Thu Oct 22 03:03:13 UTC 2009


Author: atagar
Date: 2009-10-21 23:03:12 -0400 (Wed, 21 Oct 2009)
New Revision: 20814

Modified:
   arm/trunk/ChangeLog
   arm/trunk/README
   arm/trunk/TODO
   arm/trunk/arm.py
   arm/trunk/interface/bandwidthMonitor.py
   arm/trunk/interface/confPanel.py
   arm/trunk/interface/connCountMonitor.py
   arm/trunk/interface/connPanel.py
   arm/trunk/interface/connResolver.py
   arm/trunk/interface/controller.py
   arm/trunk/interface/descriptorPopup.py
   arm/trunk/interface/headerPanel.py
   arm/trunk/interface/logPanel.py
Log:
Substantial bundle of changes including torrc validation, improved arm event logging, and numerous bug fixes.
added: verifies loaded torrc consistency against tor's actual state (gives warning and providing corrections)
added: checks for torrc entries that are irrelevant due to duplication (gives notices and highlights)
added: log provides TorCtl events (hack... so ugly...)
added: option for logging runlevel events of arm, tor, or both
added: ARM-DEBUG event for netstat query time
added: providing progress bar when resolving a batch of hostnames
change: providing prompt notice when tor's control port is closed
fix: limiting pre-loaded events to this tor instance
fix: limits log entries used to pre-load events (big logs caused issue with startup time)
fix: properly closing TorCtl when quitting (was occasionally screwing up terminal)
fix: at several points TorCtlClosed exceptions were uncaught, causing crashes when tor was closed
fix: netstat and geoip failures were being noisy when tor quits
fix: bug in tracking connection counts if tor quits when paused
fix: sighup wasn't resetting all relevant internal variables
fix: pausing bypassed connection sorting



Modified: arm/trunk/ChangeLog
===================================================================
--- arm/trunk/ChangeLog	2009-10-21 23:57:31 UTC (rev 20813)
+++ arm/trunk/ChangeLog	2009-10-22 03:03:12 UTC (rev 20814)
@@ -1,6 +1,25 @@
 CHANGE LOG
 
-10/16/09 - version 1.2.0
+10/21/09 - version 1.2.1
+Substantial bundle of changes including torrc validation, improved arm event logging, and numerous bug fixes.
+
+    * added: verifies loaded torrc consistency against tor's actual state (gives warning and providing corrections)
+    * added: checks for torrc entries that are irrelevant due to duplication (gives notices and highlights)
+    * added: log provides TorCtl events (hack... so ugly...)
+    * added: option for logging runlevel events of arm, tor, or both
+    * added: ARM-DEBUG event for netstat query time
+    * added: providing progress bar when resolving a batch of hostnames
+    * change: providing prompt notice when tor's control port is closed
+    * fix: limiting pre-loaded events to this tor instance
+    * fix: limits log entries used to pre-load events (big logs caused issue with startup time)
+    * fix: properly closing TorCtl when quitting (was occasionally screwing up terminal)
+    * fix: at several points TorCtlClosed exceptions were uncaught, causing crashes when tor was closed
+    * fix: netstat and geoip failures were being noisy when tor quits
+    * fix: bug in tracking connection counts if tor quits when paused
+    * fix: sighup wasn't resetting all relevant internal variables
+    * fix: pausing bypassed connection sorting
+
+10/16/09 - version 1.2.0 (r20798)
 Resolving a few small issues that bugged me.
 
     * change: using log file to pre-populate events if available

Modified: arm/trunk/README
===================================================================
--- arm/trunk/README	2009-10-21 23:57:31 UTC (rev 20813)
+++ arm/trunk/README	2009-10-22 03:03:12 UTC (rev 20814)
@@ -21,7 +21,7 @@
 Requirements:
 Python 2.5
 TorCtl (retrieved in svn checkout)
-Common *nix commands including: ps, pidof, host, and netstat
+Common *nix commands including: ps, pidof, tail, host, and netstat
 Tor is running with an available control port. This means either...
   ... starting Tor with '--controlport <PORT>'
   ... or including 'ControlPort <PORT>' in your torrc

Modified: arm/trunk/TODO
===================================================================
--- arm/trunk/TODO	2009-10-21 23:57:31 UTC (rev 20813)
+++ arm/trunk/TODO	2009-10-22 03:03:12 UTC (rev 20814)
@@ -14,6 +14,10 @@
 			caught by hexa
 
 - Features / Site
+	* abstract away netstat calls
+			In preparation for drop in replacement of lsof or calls to tor's
+			GETINFO.
+	* file descriptor stats
 	* add page that allows raw control port access
 			Piggyback on the arm connection, providing something like an interactive
 			prompt. In addition, provide:
@@ -23,6 +27,7 @@
 				- warn and get confirmation if command would disrupt arm (for instance
 				'SETEVENTS')
 				- 'guard' option that restricts to GETINFO only	(start with this)
+				- issue sighup reset
 	* provide observed bandwidth
 			Newer relays have a 'w' entry that states the bandwidth and old versions
 			have client side measurements (third argument in 'Bandwidth' of
@@ -34,12 +39,10 @@
 	* when help popup is showing options let them be directly opened
 			requested by arma
 	* update site's screenshots (pretty out of date...)
-	* add arm to listings of support programs
-			https://wiki.torproject.org/noreply/TheOnionRouter/SupportPrograms
-			https://www.torproject.org/projects/
-	* add svn / tarball fingerprint to site
 
 - Ideas (low priority)
+	* write up a proposal for the control protocol wishlist
+	* look into providing UPnP support
 	* bundle script that dumps relay stats to stdout
 			Django has a small terminal coloring module that could be nice for
 			formatting. Could possibly include:
@@ -124,4 +127,5 @@
 				BandwidthRate/Burst)
 			list of directory authorities recognized by that instance of tor
 			total data relayed by tor - this is already kinda tracked for accounting
+			file descriptor limit (return value of the getrlimit() function)
 

Modified: arm/trunk/arm.py
===================================================================
--- arm/trunk/arm.py	2009-10-21 23:57:31 UTC (rev 20813)
+++ arm/trunk/arm.py	2009-10-22 03:03:12 UTC (rev 20814)
@@ -19,8 +19,8 @@
 from interface import controller
 from interface import logPanel
 
-VERSION = "1.2.0"
-LAST_MODIFIED = "Oct 16, 2009"
+VERSION = "1.2.1"
+LAST_MODIFIED = "Oct 21, 2009"
 
 DEFAULT_CONTROL_ADDR = "127.0.0.1"
 DEFAULT_CONTROL_PORT = 9051
@@ -186,9 +186,6 @@
       print "Unrecognized event flag: %s" % flag
     sys.exit()
   
-  # disables TorCtl from logging events (noisy and can possibly interrupt curses)
-  TorUtil.loglevel = "NONE"
-  
   # attempts to open a socket to the tor server
   s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
   try:

Modified: arm/trunk/interface/bandwidthMonitor.py
===================================================================
--- arm/trunk/interface/bandwidthMonitor.py	2009-10-21 23:57:31 UTC (rev 20813)
+++ arm/trunk/interface/bandwidthMonitor.py	2009-10-22 03:03:12 UTC (rev 20814)
@@ -3,6 +3,7 @@
 # Released under the GPL v3 (http://www.gnu.org/licenses/gpl.html)
 
 import time
+import socket
 from TorCtl import TorCtl
 
 import graphPanel
@@ -30,9 +31,9 @@
     # dummy values for static data
     self.isAccounting = False
     self.bwRate, self.bwBurst = -1, -1
-    self.resetStaticData()
+    self.resetOptions()
   
-  def resetStaticData(self):
+  def resetOptions(self):
     """
     Checks with tor for static bandwidth parameters (rates, accounting
     information, etc).
@@ -48,7 +49,7 @@
       
       self.bwRate = util.getSizeLabel(int(bwStats[0][1] if relayStats[0][1] == "0" else relayStats[0][1]))
       self.bwBurst = util.getSizeLabel(int(bwStats[1][1] if relayStats[1][1] == "0" else relayStats[1][1]))
-    except (ValueError, TorCtl.TorCtlClosed):
+    except (ValueError, socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed):
       pass # keep old values
     
     # this doesn't track accounting stats when paused so doesn't need a custom pauseBuffer
@@ -159,6 +160,6 @@
       self.accountingInfo["written"] = util.getSizeLabel(written)
       self.accountingInfo["readLimit"] = util.getSizeLabel(read + readLeft)
       self.accountingInfo["writtenLimit"] = util.getSizeLabel(written + writtenLeft)
-    except TorCtl.TorCtlClosed:
+    except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed):
       self.accountingInfo = None
 

Modified: arm/trunk/interface/confPanel.py
===================================================================
--- arm/trunk/interface/confPanel.py	2009-10-21 23:57:31 UTC (rev 20813)
+++ arm/trunk/interface/confPanel.py	2009-10-22 03:03:12 UTC (rev 20814)
@@ -7,18 +7,33 @@
 
 import util
 
+# torrc parameters that can be defined multiple times without overwriting
+# from src/or/config.c (entries with LINELIST or LINELIST_S)
+# last updated for tor version 0.2.1.19
+MULTI_LINE_PARAM = ["AlternateBridgeAuthority", "AlternateDirAuthority", "AlternateHSAuthority", "AuthDirBadDir", "AuthDirBadExit", "AuthDirInvalid", "AuthDirReject", "Bridge", "ControlListenAddress", "ControlSocket", "DirListenAddress", "DirPolicy", "DirServer", "DNSListenAddress", "ExitPolicy", "HashedControlPassword", "HiddenServiceDir", "HiddenServiceOptions", "HiddenServicePort", "HiddenServiceVersion", "HiddenServiceAuthorizeClient", "HidServAuth", "Log", "MapAddress", "NatdListenAddress", "NodeFamily", "ORListenAddress", "ReachableAddresses", "ReachableDirAddresses", "ReachableORAddresses", "RecommendedVersions", "RecommendedClientVersions", "RecommendedServerVersions", "SocksListenAddress", "SocksPolicy", "TransListenAddress", "__HashedControlSessionPassword"]
+
 class ConfPanel(util.Panel):
   """
   Presents torrc with syntax highlighting in a scroll-able area.
   """
   
-  def __init__(self, lock, confLocation):
+  def __init__(self, lock, confLocation, conn, logPanel):
     util.Panel.__init__(self, lock, -1)
     self.confLocation = confLocation
     self.showLineNum = True
     self.stripComments = False
     self.confContents = []
     self.scroll = 0
+    
+    # lines that don't matter due to duplicates
+    self.irrelevantLines = []
+    
+    # used to check consistency with tor's actual values - corrections mapping
+    # is of line numbers (one-indexed) to tor's actual values
+    self.corrections = {}
+    self.conn = conn
+    self.logger = logPanel
+    
     self.reset()
   
   def reset(self):
@@ -29,6 +44,49 @@
       confFile = open(self.confLocation, "r")
       self.confContents = confFile.readlines()
       confFile.close()
+      
+      # checks if torrc differs from get_option data
+      self.irrelevantLines = []
+      self.corrections = {}
+      correctedCmd = {}       # mapping of corrected commands to line numbers
+      
+      for lineNumber in range(len(self.confContents)):
+        lineText = self.confContents[lineNumber].strip()
+        
+        if lineText and lineText[0] != "#":
+          # relevant to tor (not blank nor comment)
+          ctlEnd = lineText.find(" ")   # end of command
+          argEnd = lineText.find("#")   # end of argument (start of comment or end of line)
+          if argEnd == -1: argEnd = len(lineText)
+          command, argument = lineText[:ctlEnd], lineText[ctlEnd:argEnd].strip()
+          
+          # most parameters are overwritten if defined multiple times, if so
+          # it's erased from corrections and noted as duplicate instead
+          if not command in MULTI_LINE_PARAM and command in correctedCmd.keys():
+            self.irrelevantLines.append(correctedCmd[command])
+            del self.corrections[correctedCmd[command]]
+          
+          # check validity against tor's actual state
+          try:
+            actualValues = []
+            for key, val in self.conn.get_option(command):
+              actualValues.append(val)
+            
+            if not argument in actualValues:
+              self.corrections[lineNumber + 1] = ", ".join(actualValues)
+              correctedCmd[command] = lineNumber + 1
+          except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed):
+            pass # unable to load tor parameter to validate... weird
+      
+      # logs issues that arose
+      if self.irrelevantLines:
+        if len(self.irrelevantLines) > 1: first, second, third = "Entries", "are", ", including lines"
+        else: first, second, third = "Entry", "is", " on line"
+        baseMsg = "%s in your torrc %s ignored due to duplication%s" % (first, second, third)
+        
+        self.logger.monitor_event("NOTICE", "%s: %s (highlighted in blue)" % (baseMsg, ", ".join([str(val) for val in self.irrelevantLines])))
+      if self.corrections:
+        self.logger.monitor_event("WARN", "Tor's state differs from loaded torrc")
     except IOError:
       self.confContents = ["### Unable to load torrc ###"]
     self.scroll = 0
@@ -53,33 +111,24 @@
         self.clear()
         self.addstr(0, 0, "Tor Config (%s):" % self.confLocation, util.LABEL_ATTR)
         
-        if self.stripComments:
-          displayText = []
-          
-          for line in self.confContents:
-            commentStart = line.find("#")
-            if commentStart != -1: line = line[:commentStart]
-            
-            line = line.strip()
-            if line: displayText.append(line)
-        else: displayText = self.confContents
+        pageHeight = self.maxY - 1
+        numFieldWidth = int(math.log10(len(self.confContents))) + 1
+        lineNum, displayLineNum = self.scroll + 1, 1 # lineNum corresponds to torrc, displayLineNum concerns what's presented
         
-        pageHeight = self.maxY - 1
-        numFieldWidth = int(math.log10(len(displayText))) + 1
-        lineNum = 1
-        for i in range(self.scroll, min(len(displayText), self.scroll + pageHeight)):
-          lineText = displayText[i].strip()
+        for i in range(self.scroll, min(len(self.confContents), self.scroll + pageHeight)):
+          lineText = self.confContents[i].strip()
+          skipLine = False # true if we're not presenting line due to stripping
           
-          numOffset = 0     # offset for line numbering
-          if self.showLineNum:
-            self.addstr(lineNum, 0, ("%%%ii" % numFieldWidth) % (i + 1), curses.A_BOLD | util.getColor("yellow"))
-            numOffset = numFieldWidth + 1
+          command, argument, correction, comment = "", "", "", ""
+          commandColor, argumentColor, correctionColor, commentColor = "green", "cyan", "cyan", "white"
           
-          command, argument, comment = "", "", ""
-          if not lineText: pass # no text
+          if not lineText:
+            # no text
+            if self.stripComments: skipLine = True
           elif lineText[0] == "#":
             # whole line is commented out
             comment = lineText
+            if self.stripComments: skipLine = True
           else:
             # parse out command, argument, and possible comment
             ctlEnd = lineText.find(" ")   # end of command
@@ -87,11 +136,29 @@
             if argEnd == -1: argEnd = len(lineText)
             
             command, argument, comment = lineText[:ctlEnd], lineText[ctlEnd:argEnd], lineText[argEnd:]
+            
+            # changes presentation if value's incorrect or irrelevant
+            if lineNum in self.corrections.keys():
+              argumentColor = "red"
+              correction = " (%s)" % self.corrections[lineNum]
+            elif lineNum in self.irrelevantLines:
+              commandColor = "blue"
+              argumentColor = "blue"
           
-          xLoc = 0
-          lineNum, xLoc = self.addstr_wrap(lineNum, xLoc, command, curses.A_BOLD | util.getColor("green"), numOffset)
-          lineNum, xLoc = self.addstr_wrap(lineNum, xLoc, argument, curses.A_BOLD | util.getColor("cyan"), numOffset)
-          lineNum, xLoc = self.addstr_wrap(lineNum, xLoc, comment, util.getColor("white"), numOffset)
+          if not skipLine:
+            numOffset = 0     # offset for line numbering
+            if self.showLineNum:
+              self.addstr(displayLineNum, 0, ("%%%ii" % numFieldWidth) % lineNum, curses.A_BOLD | util.getColor("yellow"))
+              numOffset = numFieldWidth + 1
+            
+            xLoc = 0
+            displayLineNum, xLoc = self.addstr_wrap(displayLineNum, xLoc, command, curses.A_BOLD | util.getColor(commandColor), numOffset)
+            displayLineNum, xLoc = self.addstr_wrap(displayLineNum, xLoc, argument, curses.A_BOLD | util.getColor(argumentColor), numOffset)
+            displayLineNum, xLoc = self.addstr_wrap(displayLineNum, xLoc, correction, curses.A_BOLD | util.getColor(correctionColor), numOffset)
+            displayLineNum, xLoc = self.addstr_wrap(displayLineNum, xLoc, comment, util.getColor(commentColor), numOffset)
+            
+            displayLineNum += 1
+          
           lineNum += 1
           
         self.refresh()

Modified: arm/trunk/interface/connCountMonitor.py
===================================================================
--- arm/trunk/interface/connCountMonitor.py	2009-10-21 23:57:31 UTC (rev 20813)
+++ arm/trunk/interface/connCountMonitor.py	2009-10-22 03:03:12 UTC (rev 20814)
@@ -2,6 +2,7 @@
 # connCountMonitor.py -- Tracks the number of connections made by Tor.
 # Released under the GPL v3 (http://www.gnu.org/licenses/gpl.html)
 
+import socket
 from TorCtl import TorCtl
 
 import graphPanel
@@ -17,10 +18,22 @@
     TorCtl.PostEventListener.__init__(self)
     graphPanel.GraphStats.initialize(self, "green", "cyan", 10)
     self.connResolver = connResolver    # thread performing netstat queries
-    self.orPort = conn.get_option("ORPort")[0][1]
-    self.dirPort = conn.get_option("DirPort")[0][1]
-    self.controlPort = conn.get_option("ControlPort")[0][1]
+    
+    self.orPort = "0"
+    self.dirPort = "0"
+    self.controlPort = "0"
+    self.resetOptions(conn)
   
+  def resetOptions(self, conn):
+    try:
+      self.orPort = conn.get_option("ORPort")[0][1]
+      self.dirPort = conn.get_option("DirPort")[0][1]
+      self.controlPort = conn.get_option("ControlPort")[0][1]
+    except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed):
+      self.orPort = "0"
+      self.dirPort = "0"
+      self.controlPort = "0"
+  
   def bandwidth_event(self, event):
     # doesn't use events but this keeps it in sync with the bandwidth panel
     # (and so it stops if Tor stops - used to use a separate thread but this

Modified: arm/trunk/interface/connPanel.py
===================================================================
--- arm/trunk/interface/connPanel.py	2009-10-21 23:57:31 UTC (rev 20813)
+++ arm/trunk/interface/connPanel.py	2009-10-22 03:03:12 UTC (rev 20814)
@@ -115,7 +115,6 @@
     self.fingerprintLookupCache = {}                              # chache of (ip, port) -> fingerprint
     self.nicknameLookupCache = {}                                 # chache of (ip, port) -> nickname
     self.fingerprintMappings = _getFingerprintMappings(self.conn) # mappings of ip -> [(port, fingerprint, nickname), ...]
-    self.nickname = self.conn.get_option("Nickname")[0][1]
     self.providedGeoipWarning = False
     self.orconnStatusCache = []           # cache for 'orconn-status' calls
     self.orconnStatusCacheValid = False   # indicates if cache has been invalidated
@@ -132,11 +131,13 @@
     self.isPaused = False
     self.pauseTime = 0              # time when paused
     self.connectionsBuffer = []     # location where connections are stored while paused
+    self.connectionCountBuffer = []
     
-    # uses ports to identify type of connections
-    self.orPort = self.conn.get_option("ORPort")[0][1]
-    self.dirPort = self.conn.get_option("DirPort")[0][1]
-    self.controlPort = self.conn.get_option("ControlPort")[0][1]
+    self.nickname = ""
+    self.orPort = "0"
+    self.dirPort = "0"
+    self.controlPort = "0"
+    self.resetOptions()
     
     # netstat results are tuples of the form:
     # (type, local IP, local port, foreign IP, foreign port, country code)
@@ -148,6 +149,20 @@
     
     self.reset()
   
+  def resetOptions(self):
+    try:
+      self.nickname = self.conn.get_option("Nickname")[0][1]
+      
+      # uses ports to identify type of connections
+      self.orPort = self.conn.get_option("ORPort")[0][1]
+      self.dirPort = self.conn.get_option("DirPort")[0][1]
+      self.controlPort = self.conn.get_option("ControlPort")[0][1]
+    except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed):
+      self.nickname = ""
+      self.orPort = "0"
+      self.dirPort = "0"
+      self.controlPort = "0"
+  
   # change in client circuits
   def circ_status_event(self, event):
     self.clientConnectionLock.acquire()
@@ -176,7 +191,7 @@
       
       # gets consensus data for the new description
       try: nsData = self.conn.get_network_status("id/%s" % fingerprint)
-      except (TorCtl.ErrorReply, TorCtl.TorCtlClosed): return
+      except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed): return
       
       if len(nsData) > 1:
         # multiple records for fingerprint (shouldn't happen)
@@ -261,7 +276,7 @@
         try:
           countryCodeQuery = "ip-to-country/%s" % foreign[:foreign.find(":")]
           countryCode = self.conn.get_info(countryCodeQuery)[countryCodeQuery]
-        except (socket.error, TorCtl.ErrorReply):
+        except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed):
           countryCode = "??"
           if not self.providedGeoipWarning:
             self.logger.monitor_event("WARN", "Tor geoip database is unavailable.")
@@ -273,24 +288,23 @@
         connectionsTmp.append((type, localIP, localPort, foreignIP, foreignPort, countryCode, connTime))
       
       # appends localhost connection to allow user to look up their own consensus entry
-      selfAddress, selfPort, selfFingerprint = None, None, None
+      selfAddress, selfFingerprint = None, None
       try:
         selfAddress = self.conn.get_info("address")["address"]
-        selfPort = self.conn.get_option("ORPort")[0][1]
         selfFingerprint = self.conn.get_info("fingerprint")["fingerprint"]
-      except (TorCtl.ErrorReply, TorCtl.TorCtlClosed, socket.error): pass
+      except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed): pass
       
-      if selfAddress and selfPort and selfFingerprint:
+      if selfAddress and selfFingerprint:
         try:
           countryCodeQuery = "ip-to-country/%s" % selfAddress
           selfCountryCode = self.conn.get_info(countryCodeQuery)[countryCodeQuery]
-        except (socket.error, TorCtl.ErrorReply):
+        except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed):
           selfCountryCode = "??"
         
-        if (selfAddress, selfPort) in connTimes: connTime = connTimes[(selfAddress, selfPort)]
+        if (selfAddress, self.orPort) in connTimes: connTime = connTimes[(selfAddress, self.orPort)]
         else: connTime = time.time()
         
-        self.localhostEntry = (("localhost", selfAddress, selfPort, selfAddress, selfPort, selfCountryCode, connTime), selfFingerprint)
+        self.localhostEntry = (("localhost", selfAddress, self.orPort, selfAddress, self.orPort, selfCountryCode, connTime), selfFingerprint)
         connectionsTmp.append(self.localhostEntry[0])
       else:
         self.localhostEntry = None
@@ -300,6 +314,7 @@
       # assigns results
       if self.isPaused:
         self.connectionsBuffer = connectionsTmp
+        self.connectionCountBuffer = connectionCountTmp
       else:
         self.connections = connectionsTmp
         self.connectionCount = connectionCountTmp
@@ -540,7 +555,7 @@
           for entry in self.conn.get_info("orconn-status")["orconn-status"].split():
             if isOdd: self.orconnStatusCache.append(entry)
             isOdd = not isOdd
-        except (TorCtl.TorCtlClosed, TorCtl.ErrorReply): self.orconnStatusCache = None
+        except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed): self.orconnStatusCache = None
       
       if ipAddr in self.fingerprintMappings.keys():
         potentialMatches = self.fingerprintMappings[ipAddr]
@@ -571,7 +586,7 @@
               descLookupCmd = "desc/id/%s" % entryFingerprint
               descEntry = TorCtl.Router.build_from_desc(self.conn.get_info(descLookupCmd)[descLookupCmd].split("\n"), nsEntry)
               toRemove = descEntry.down
-            except TorCtl.ErrorReply: pass # ns or desc lookup fails... also weird
+            except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed): pass # ns or desc lookup fails... also weird
             
             # eliminates connections not reported by orconn-status -
             # this has *very* little impact since few ips have multiple relays
@@ -599,7 +614,7 @@
       
       try:
         if match != "UNKNOWN": match = self.conn.get_network_status("id/%s" % match)[0].nickname
-      except TorCtl.ErrorReply: return "UNKNOWN" # don't cache result
+      except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed): return "UNKNOWN" # don't cache result
       
       self.nicknameLookupCache[(ipAddr, port)] = match
       return match
@@ -615,7 +630,13 @@
     if isPause:
       self.pauseTime = time.time()
       self.connectionsBuffer = list(self.connections)
-    else: self.connections = list(self.connectionsBuffer)
+      self.connectionCountBuffer = list(self.connectionCount)
+    else:
+      self.connections = list(self.connectionsBuffer)
+      self.connectionCount = list(self.connectionCountBuffer)
+      
+      # pause buffer connections may be unsorted
+      if self.listingType != LIST_HOSTNAME: self.sortConnections()
   
   def sortConnections(self):
     """
@@ -675,7 +696,7 @@
   
   if not nsList:
     try: nsList = conn.get_network_status()
-    except (TorCtl.TorCtlClosed, TorCtl.ErrorReply): nsList = []
+    except (socket.error, TorCtl.TorCtlClosed, TorCtl.ErrorReply): nsList = []
     except TypeError: nsList = [] # TODO: temporary workaround for a TorCtl bug, remove when fixed
   
   for entry in nsList:
@@ -693,7 +714,7 @@
     for line in conn.get_info("circuit-status")["circuit-status"].split("\n"):
       components = line.split()
       if len(components) > 3: clients += [components[2].split(",")[0]]
-  except (TorCtl.ErrorReply, TorCtl.TorCtlClosed, socket.error): pass
+  except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed): pass
   
   return clients
 

Modified: arm/trunk/interface/connResolver.py
===================================================================
--- arm/trunk/interface/connResolver.py	2009-10-21 23:57:31 UTC (rev 20813)
+++ arm/trunk/interface/connResolver.py	2009-10-22 03:03:12 UTC (rev 20814)
@@ -23,8 +23,9 @@
   isn't available).
   """
   
-  def __init__(self, pid, logPanel):
+  def __init__(self, conn, pid, logPanel):
     Thread.__init__(self)
+    self.conn = conn                  # used to stop querring netstat if tor's closed
     self.pid = pid                    # tor process ID to make sure we've got the right instance
     self.logger = logPanel            # used to notify of lookup failures
     
@@ -38,9 +39,10 @@
   
   def getConnections(self):
     """
-    Provides the last querried connection results.
+    Provides the last querried connection results, empty list if tor's closed.
     """
     
+    if self.conn._closed == 1: return []
     connectionsTmp = None
     
     self.connectionsLock.acquire()
@@ -53,12 +55,16 @@
     if not self.pid: return
     
     while not self.halt:
-      if self.isPaused or time.time() - MIN_LOOKUP_WAIT < self.lastLookup: time.sleep(SLEEP_INTERVAL)
+      if self.isPaused or time.time() - MIN_LOOKUP_WAIT < self.lastLookup or self.conn._closed == 1: time.sleep(SLEEP_INTERVAL)
       else:
         try:
+          netstatStart = time.time()
+          
           # looks at netstat for tor with stderr redirected to /dev/null, options are:
           # n = prevents dns lookups, p = include process (say if it's tor), t = tcp only
           netstatCall = os.popen("netstat -npt 2> /dev/null | grep %s/tor 2> /dev/null" % self.pid)
+          
+          self.logger.monitor_event("DEBUG", "netstat queried in %.4f seconds" % (time.time() - netstatStart))
           results = netstatCall.readlines()
           if not results: raise IOError
           

Modified: arm/trunk/interface/controller.py
===================================================================
--- arm/trunk/interface/controller.py	2009-10-21 23:57:31 UTC (rev 20813)
+++ arm/trunk/interface/controller.py	2009-10-22 03:03:12 UTC (rev 20814)
@@ -8,11 +8,13 @@
 
 import re
 import os
+import math
 import time
 import curses
 import socket
 from threading import RLock
 from TorCtl import TorCtl
+from TorCtl import TorUtil
 
 import headerPanel
 import graphPanel
@@ -74,6 +76,10 @@
       
       msgText = self.msgText
       msgAttr = self.msgAttr
+      barTab = 2                # space between msgText and progress bar
+      barWidthMax = 40          # max width to progress bar
+      barWidth = -1             # space between "[ ]" in progress bar (not visible if -1)
+      barProgress = 0           # cells to fill
       
       if msgText == CTL_HELP:
         msgAttr = curses.A_NORMAL
@@ -89,8 +95,14 @@
             if batchSize > 0: progress = 100 * entryCount / batchSize
             else: progress = 0
             
-            additive = "(or l) " if self.page == 2 else ""
-            msgText = "Resolving hostnames (%i / %i, %i%%) - press esc %sto cancel" % (entryCount, batchSize, progress, additive)
+            additive = "or l " if self.page == 2 else ""
+            batchSizeDigits = int(math.log10(batchSize)) + 1
+            entryCountLabel = ("%%%ii" % batchSizeDigits) % entryCount
+            #msgText = "Resolving hostnames (%i / %i, %i%%) - press esc %sto cancel" % (entryCount, batchSize, progress, additive)
+            msgText = "Resolving hostnames (press esc %sto cancel) - %s / %i, %2i%%" % (additive, entryCountLabel, batchSize, progress)
+            
+            barWidth = min(barWidthMax, self.maxX - len(msgText) - 3 - barTab)
+            barProgress = barWidth * entryCount / batchSize
         
         if self.resolvingCounter == -1:
           currentPage = self.page
@@ -106,6 +118,12 @@
         msgAttr = curses.A_STANDOUT
       
       self.addstr(0, 0, msgText, msgAttr)
+      if barWidth > -1:
+        xLoc = len(msgText) + barTab
+        self.addstr(0, xLoc, "[", curses.A_BOLD)
+        self.addstr(0, xLoc + 1, " " * barProgress, curses.A_STANDOUT | util.getColor("red"))
+        self.addstr(0, xLoc + barWidth + 1, "]", curses.A_BOLD)
+      
       self.refresh()
 
 class sighupListener(TorCtl.PostEventListener):
@@ -262,7 +280,7 @@
       if len(results) == 1:
         results = results[0].split()[6] # process field (ex. "7184/tor")
         torPid = results[:results.find("/")]
-    except IOError: pass # netstat call failed
+    except (IOError, socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed): pass # netstat or control port calls failed
     netstatCall.close()
   
   if not torPid:
@@ -274,29 +292,32 @@
     except IOError: pass # ps call failed
     psCall.close()
   
-  confLocation = conn.get_info("config-file")["config-file"]
-  if confLocation[0] != "/":
-    # relative path - attempt to add process pwd
-    try:
-      pwdxCall = os.popen("pwdx %s 2> /dev/null" % torPid)
-      results = pwdxCall.readlines()
-      if len(results) == 1 and len(results[0].split()) == 2: confLocation = "%s/%s" % (results[0].split()[1], confLocation)
-    except IOError: pass # pwdx call failed
-    pwdxCall.close()
+  try:
+    confLocation = conn.get_info("config-file")["config-file"]
+    if confLocation[0] != "/":
+      # relative path - attempt to add process pwd
+      try:
+        pwdxCall = os.popen("pwdx %s 2> /dev/null" % torPid)
+        results = pwdxCall.readlines()
+        if len(results) == 1 and len(results[0].split()) == 2: confLocation = "%s/%s" % (results[0].split()[1], confLocation)
+      except IOError: pass # pwdx call failed
+      pwdxCall.close()
+  except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed):
+    confLocation = ""
   
   panels = {
     "header": headerPanel.HeaderPanel(cursesLock, conn, torPid),
     "popup": util.Panel(cursesLock, 9),
     "graph": graphPanel.GraphPanel(cursesLock),
-    "log": logPanel.LogMonitor(cursesLock, conn, loggedEvents),
-    "torrc": confPanel.ConfPanel(cursesLock, confLocation)}
+    "log": logPanel.LogMonitor(cursesLock, conn, loggedEvents)}
   
   # starts thread for processing netstat queries
-  connResolutionThread = connResolver.ConnResolver(torPid, panels["log"])
+  connResolutionThread = connResolver.ConnResolver(conn, torPid, panels["log"])
   connResolutionThread.start()
   
   panels["conn"] = connPanel.ConnPanel(cursesLock, conn, connResolutionThread, panels["log"])
   panels["control"] = ControlPanel(cursesLock, panels["conn"].resolver)
+  panels["torrc"] = confPanel.ConfPanel(cursesLock, confLocation, conn, panels["log"])
   
   # prevents netstat calls by connPanel if not being used
   if DISABLE_CONNECTIONS_PAGE: panels["conn"].isDisabled = True
@@ -306,16 +327,16 @@
   
   # statistical monitors for graph
   panels["graph"].addStats("bandwidth", bandwidthMonitor.BandwidthMonitor(conn))
-  panels["graph"].addStats("cpu / memory", cpuMemMonitor.CpuMemMonitor(panels["header"]))
-  panels["graph"].addStats("connection count", connCountMonitor.ConnCountMonitor(conn, connResolutionThread))
+  panels["graph"].addStats("system resources", cpuMemMonitor.CpuMemMonitor(panels["header"]))
+  panels["graph"].addStats("connections", connCountMonitor.ConnCountMonitor(conn, connResolutionThread))
   panels["graph"].setStats("bandwidth")
   
   # listeners that update bandwidth and log panels with Tor status
   sighupTracker = sighupListener()
   conn.add_event_listener(panels["log"])
   conn.add_event_listener(panels["graph"].stats["bandwidth"])
-  conn.add_event_listener(panels["graph"].stats["cpu / memory"])
-  conn.add_event_listener(panels["graph"].stats["connection count"])
+  conn.add_event_listener(panels["graph"].stats["system resources"])
+  conn.add_event_listener(panels["graph"].stats["connections"])
   conn.add_event_listener(panels["conn"])
   conn.add_event_listener(sighupTracker)
   
@@ -323,15 +344,19 @@
   loggedEvents = setEventListening(loggedEvents, conn, panels["log"])
   panels["log"].loggedEvents = loggedEvents # strips any that couldn't be set
   
+  # directs logged TorCtl events to log panel
+  TorUtil.loglevel = "DEBUG"
+  TorUtil.logfile = panels["log"]
+  
   # warns if tor isn't updating descriptors
   try:
     if conn.get_option("FetchUselessDescriptors")[0][1] == "0" and conn.get_option("DirPort")[0][1] == "0":
       warning = ["Descriptors won't be updated (causing some connection information to be stale) unless:", \
                 "  a. 'FetchUselessDescriptors 1' is set in your torrc", \
                 "  b. the directory service is provided ('DirPort' defined)", \
-                "  c. tor is used as a client"]
+                "  c. or tor is used as a client"]
       panels["log"].monitor_event("WARN", warning)
-  except (TorCtl.ErrorReply, TorCtl.TorCtlClosed, socket.error): pass
+  except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed): pass
   
   isUnresponsive = False    # true if it's been over ten seconds since the last BW event (probably due to Tor closing)
   isPaused = False          # if true updates are frozen
@@ -348,12 +373,16 @@
       # if sighup received then reload related information
       if sighupTracker.isReset:
         panels["header"]._updateParams(True)
-        panels["graph"].stats["bandwidth"].resetStaticData()
         
         # if bandwidth graph is being shown then height might have changed
         if panels["graph"].currentDisplay == "bandwidth":
           panels["graph"].height = panels["graph"].stats["bandwidth"].height
         
+        # other panels that use torrc data
+        panels["conn"].resetOptions()
+        panels["graph"].stats["connections"].resetOptions(conn)
+        panels["graph"].stats["bandwidth"].resetOptions()
+        
         panels["torrc"].reset()
         sighupTracker.isReset = False
       
@@ -375,12 +404,11 @@
           panels[panelKey].recreate(stdscr, tmpStartY)
           tmpStartY += panels[panelKey].height
       
-      
       # if it's been at least ten seconds since the last BW event Tor's probably done
-      if not isUnresponsive and panels["log"].getHeartbeat() >= 10:
+      if not isUnresponsive and not panels["log"].controlPortClosed and panels["log"].getHeartbeat() >= 10:
         isUnresponsive = True
         panels["log"].monitor_event("NOTICE", "Relay unresponsive (last heartbeat: %s)" % time.ctime(panels["log"].lastHeartbeat))
-      elif isUnresponsive and panels["log"].getHeartbeat() < 10:
+      elif not panels["log"].controlPortClosed and (isUnresponsive and panels["log"].getHeartbeat() < 10):
         # shouldn't happen unless Tor freezes for a bit - BW events happen every second...
         isUnresponsive = False
         panels["log"].monitor_event("NOTICE", "Relay resumed")
@@ -429,6 +457,8 @@
         # joins on workers (prevents noisy termination)
         for worker in daemonThreads: worker.join()
         
+        conn.close() # joins on TorCtl event thread
+        
         break
     elif key == curses.KEY_LEFT or key == curses.KEY_RIGHT:
       # switch page
@@ -476,7 +506,12 @@
           popup.addstr(2, 41, "e: change logged events")
           
           regexLabel = "enabled" if panels["log"].regexFilter else "disabled"
-          popup.addfstr(3, 2, "r: log regex filter (<b>%s</b>)" % regexLabel)
+          popup.addfstr(3, 2, "f: log regex filter (<b>%s</b>)" % regexLabel)
+          
+          runlevelEventsLabel = "arm and tor"
+          if panels["log"].runlevelTypes == logPanel.RUNLEVEL_TOR_ONLY: runlevelEventsLabel = "tor only"
+          elif panels["log"].runlevelTypes == logPanel.RUNLEVEL_ARM_ONLY: runlevelEventsLabel = "arm only"
+          popup.addfstr(3, 41, "r: logged runlevels (<b>%s</b>)" % runlevelEventsLabel)
         if page == 1:
           popup.addstr(1, 2, "up arrow: scroll up a line")
           popup.addstr(1, 41, "down arrow: scroll down a line")
@@ -524,7 +559,9 @@
       # appends stats labels with first letters of each word capitalized
       initialSelection, i = -1, 1
       if not panels["graph"].currentDisplay: initialSelection = 0
-      for label in panels["graph"].stats.keys():
+      graphLabels = panels["graph"].stats.keys()
+      graphLabels.sort()
+      for label in graphLabels:
         if label == panels["graph"].currentDisplay: initialSelection = i
         words = label.split()
         options.append(" ".join(word[0].upper() + word[1:] for word in words))
@@ -630,7 +667,7 @@
         setPauseState(panels, isPaused, page)
       finally:
         cursesLock.release()
-    elif page == 0 and (key == ord('r') or key == ord('R')):
+    elif page == 0 and (key == ord('f') or key == ord('F')):
       # provides menu to pick previous regular expression filters or to add a new one
       # for syntax see: http://docs.python.org/library/re.html#regular-expression-syntax
       options = ["None"] + regexFilters + ["New..."]
@@ -698,6 +735,25 @@
       # reverts changes made for popup
       panels["graph"].showLabel = True
       setPauseState(panels, isPaused, page)
+    elif page == 0 and (key == ord('r') or key == ord('R')):
+      # provides menu to pick the type of runlevel events to log
+      options = ["tor only", "arm only", "arm and tor"]
+      initialSelection = panels["log"].runlevelTypes
+      
+      # hides top label of the graph panel and pauses panels
+      if panels["graph"].currentDisplay:
+        panels["graph"].showLabel = False
+        panels["graph"].redraw()
+      setPauseState(panels, isPaused, page, True)
+      
+      selection = showMenu(stdscr, panels["popup"], "Logged Runlevels:", options, initialSelection)
+      
+      # reverts changes made for popup
+      panels["graph"].showLabel = True
+      setPauseState(panels, isPaused, page)
+      
+      # applies new setting
+      if selection != -1: panels["log"].runlevelTypes = selection
     elif key == 27 and panels["conn"].listingType == connPanel.LIST_HOSTNAME and panels["control"].resolvingCounter != -1:
       # canceling hostname resolution (esc on any page)
       panels["conn"].listingType = connPanel.LIST_IP
@@ -783,7 +839,7 @@
             else:
               # ns lookup fails, can happen with localhost lookups if relay's having problems (orport not reachable)
               try: nsData = conn.get_network_status("id/%s" % fingerprint)
-              except (TorCtl.ErrorReply, TorCtl.TorCtlClosed): lookupErrored = True
+              except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed): lookupErrored = True
               
               if not lookupErrored:
                 if len(nsData) > 1:
@@ -796,7 +852,7 @@
                   descLookupCmd = "desc/id/%s" % fingerprint
                   descEntry = TorCtl.Router.build_from_desc(conn.get_info(descLookupCmd)[descLookupCmd].split("\n"), nsEntry)
                   relayLookupCache[selection] = (nsEntry, descEntry)
-                except TorCtl.ErrorReply: lookupErrored = True # desc lookup failed
+                except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed): lookupErrored = True # desc lookup failed
             
             if lookupErrored:
               popup.addstr(3, 2, "Unable to retrieve consensus data", format)
@@ -962,7 +1018,7 @@
       clientCircuits = None
       try:
         clientCircuits = conn.get_info("circuit-status")["circuit-status"].split("\n")
-      except (TorCtl.ErrorReply, TorCtl.TorCtlClosed, socket.error): pass
+      except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed): pass
       
       maxEntryLength = 0
       if clientCircuits:

Modified: arm/trunk/interface/descriptorPopup.py
===================================================================
--- arm/trunk/interface/descriptorPopup.py	2009-10-21 23:57:31 UTC (rev 20813)
+++ arm/trunk/interface/descriptorPopup.py	2009-10-22 03:03:12 UTC (rev 20814)
@@ -3,6 +3,7 @@
 # Released under the GPL v3 (http://www.gnu.org/licenses/gpl.html)
 
 import math
+import socket
 import curses
 from TorCtl import TorCtl
 
@@ -50,14 +51,14 @@
         nsCommand = "ns/id/%s" % fingerprint
         self.text.append(nsCommand)
         self.text = self.text + self.conn.get_info(nsCommand)[nsCommand].split("\n")
-      except (TorCtl.ErrorReply, TorCtl.TorCtlClosed):
+      except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed):
         self.text = self.text + [ERROR_MSG, ""]
       
       try:
         descCommand = "desc/id/%s" % fingerprint
         self.text.append(descCommand)
         self.text = self.text + self.conn.get_info(descCommand)[descCommand].split("\n")
-      except (TorCtl.ErrorReply, TorCtl.TorCtlClosed):
+      except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed):
         self.text = self.text + [ERROR_MSG]
   
   def handleKey(self, key, height):

Modified: arm/trunk/interface/headerPanel.py
===================================================================
--- arm/trunk/interface/headerPanel.py	2009-10-21 23:57:31 UTC (rev 20813)
+++ arm/trunk/interface/headerPanel.py	2009-10-22 03:03:12 UTC (rev 20814)
@@ -159,7 +159,7 @@
     
     infoFields = ["address", "fingerprint"] # keys for which get_info will be called
     if len(self.vals) <= 1 or forceReload:
-      isConnClosed = False
+      lookupFailed = False
       
       # first call (only contasns 'pid' mapping) - retrieve static params
       infoFields += ["version", "status/version/current"]
@@ -170,20 +170,19 @@
       self.vals["sys-os"] = unameVals[0]
       self.vals["sys-version"] = unameVals[2]
       
-      # parameters from the user's torrc
-      configFields = ["Nickname", "ORPort", "DirPort", "ControlPort", "ExitPolicy"]
-      try: self.vals.update(dict([(key, self.conn.get_option(key)[0][1]) for key in configFields]))
-      except TorCtl.TorCtlClosed: isConnClosed = True
-      
-      # simply keeps booleans for if authentication info is set
       try:
+        # parameters from the user's torrc
+        configFields = ["Nickname", "ORPort", "DirPort", "ControlPort", "ExitPolicy"]
+        self.vals.update(dict([(key, self.conn.get_option(key)[0][1]) for key in configFields]))
+        
+        # simply keeps booleans for if authentication info is set
         self.vals["IsPasswordAuthSet"] = not self.conn.get_option("HashedControlPassword")[0][1] == None
         self.vals["IsCookieAuthSet"] = self.conn.get_option("CookieAuthentication")[0][1] == "1"
         self.vals["IsAccountingEnabled"] = self.conn.get_info('accounting/enabled')['accounting/enabled'] == "1"
-      except TorCtl.TorCtlClosed: isConnClosed = True
+      except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed): lookupFailed = True
       
-      if isConnClosed:
-        # tor connection closed - keep old values if available, otherwise set to empty string / false
+      if lookupFailed:
+        # tor connection closed or gave error - keep old values if available, otherwise set to empty string / false
         for field in configFields:
           if field not in self.vals: self.vals[field] = ""
         
@@ -202,7 +201,7 @@
     self.vals["flags"] = []
     if self.vals["fingerprint"] != "Unknown":
       try: self.vals["flags"] = self.conn.get_network_status("id/%s" % self.vals["fingerprint"])[0].flags
-      except (TorCtl.TorCtlClosed, TorCtl.ErrorReply, socket.error): pass
+      except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed): pass
     
     psParams = ["%cpu", "rss", "%mem", "etime"]
     if self.vals["pid"]:

Modified: arm/trunk/interface/logPanel.py
===================================================================
--- arm/trunk/interface/logPanel.py	2009-10-21 23:57:31 UTC (rev 20813)
+++ arm/trunk/interface/logPanel.py	2009-10-22 03:03:12 UTC (rev 20814)
@@ -2,6 +2,7 @@
 # logPanel.py -- Resources related to Tor event monitoring.
 # Released under the GPL v3 (http://www.gnu.org/licenses/gpl.html)
 
+import os
 import time
 import curses
 from curses.ascii import isprint
@@ -10,8 +11,13 @@
 import util
 
 PRE_POPULATE_LOG = True               # attempts to retrieve events from log file if available
+
+# truncates to the last X log lines (needed to start in a decent time if the log's big)
+PRE_POPULATE_MIN_LIMIT = 1000             # limit in case of verbose logging
+PRE_POPULATE_MAX_LIMIT = 5000             # limit for NOTICE - ERR (since most lines are skipped)
 MAX_LOG_ENTRIES = 1000                # size of log buffer (max number of entries)
 RUNLEVEL_EVENT_COLOR = {"DEBUG": "magenta", "INFO": "blue", "NOTICE": "green", "WARN": "yellow", "ERR": "red"}
+RUNLEVEL_TOR_ONLY, RUNLEVEL_ARM_ONLY, RUNLEVEL_BOTH = range(3)
 
 EVENT_TYPES = {
   "d": "DEBUG",   "a": "ADDRMAP",       "l": "NEWDESC",     "v": "AUTHDIR_NEWDESCS",
@@ -30,6 +36,8 @@
         Aliases:    A All Events      X No Events       U Unknown Events
                     DINWE Runlevel and higher severity"""
 
+TOR_CTL_CLOSE_MSG = "Tor closed control connection. Exiting event thread."
+
 def expandEvents(eventAbbr):
   """
   Expands event abbreviations to their full names. Beside mappings privided in
@@ -83,11 +91,13 @@
     self.lastHeartbeat = time.time()      # time of last event
     self.regexFilter = None               # filter for presented log events (no filtering if None)
     self.eventTimeOverwrite = None        # replaces time for further events with this (uses time it occures if None)
+    self.runlevelTypes = RUNLEVEL_BOTH    # types of runlevels to show (arm, tor, or both)
+    self.controlPortClosed = False        # flag set if TorCtl provided notice that control port is closed
     
     # attempts to process events from log file
     if PRE_POPULATE_LOG:
       previousPauseState = self.isPaused
-      logFile = None
+      tailCall = None
       
       try:
         logFileLoc = None
@@ -103,20 +113,32 @@
           # prevents attempts to redraw while processing batch of events
           self.setPaused(True)
           
-          logFile = open(logFileLoc, "r")
-          for line in logFile:
+          # trims log to last entries to deal with logs when they're in the GB or TB range
+          # throws IOError if tail fails (falls to the catch-all later)
+          limit = PRE_POPULATE_MIN_LIMIT if ("DEBUG" in self.loggedEvents or "INFO" in self.loggedEvents) else PRE_POPULATE_MAX_LIMIT
+          tailCall = os.popen("tail -n %i %s 2> /dev/null" % (limit, logFileLoc))
+          
+          # truncates to entries for this tor instance
+          lines = tailCall.readlines()
+          instanceStart = 0
+          for i in range(len(lines) - 1, -1, -1):
+            if "opening log file" in lines[i]:
+              instanceStart = i
+              break
+          
+          for line in lines[instanceStart:]:
             lineComp = line.split()
             eventType = lineComp[3][1:-1].upper()
             
-            if eventType in loggedEvents:
+            if eventType in self.loggedEvents:
               timeComp = lineComp[2][:lineComp[2].find(".")].split(":")
               self.eventTimeOverwrite = (0, 0, 0, int(timeComp[0]), int(timeComp[1]), int(timeComp[2]))
               self.listen(TorCtl.LogEvent(eventType, " ".join(lineComp[4:])))
-      except Exception: pass # disreguard any issues that might arise in parsing
+      except Exception: pass # disreguard any issues that might arise
       finally:
         self.setPaused(previousPauseState)
         self.eventTimeOverwrite = None
-        if logFile: logFile.close()
+        if tailCall: tailCall.close()
   
   def handleKey(self, key):
     # scroll movement
@@ -171,6 +193,7 @@
     if "BW" in self.loggedEvents: self.registerEvent("BW", "READ: %i, WRITTEN: %i" % (event.read, event.written), "cyan")
   
   def msg_event(self, event):
+    if not self.runlevelTypes in (RUNLEVEL_TOR_ONLY, RUNLEVEL_BOTH): return
     self.registerEvent(event.level, event.msg, RUNLEVEL_EVENT_COLOR[event.level])
   
   def new_desc_event(self, event):
@@ -201,8 +224,29 @@
   
   def monitor_event(self, level, msg):
     # events provided by the arm monitor - types use the same as runlevel
+    if not self.runlevelTypes in (RUNLEVEL_ARM_ONLY, RUNLEVEL_BOTH): return
     if level in self.loggedEvents: self.registerEvent("ARM-%s" % level, msg, RUNLEVEL_EVENT_COLOR[level])
   
+  def write(self, msg):
+    """
+    Tracks TorCtl events. Ugly hack since TorCtl/TorUtil.py expects a file.
+    """
+    
+    timestampStart = msg.find("[")
+    timestampEnd = msg.find("]")
+    
+    level = msg[:timestampStart]
+    msg = msg[timestampEnd + 2:].strip()
+    
+    if TOR_CTL_CLOSE_MSG in msg:
+      # TorCtl providing notice that control port is closed
+      self.controlPortClosed = True
+      self.monitor_event("NOTICE", "Tor control port closed")
+    else:
+      self.monitor_event(level, "TorCtl: " + msg)
+  
+  def flush(self): pass
+  
   def registerEvent(self, type, msg, color):
     """
     Notes event and redraws log. If paused it's held in a temporary buffer. If 



More information about the tor-commits mailing list