[or-cvs] r20798: {arm} Resolving a few small issues that bugged me. change: using l (in arm/trunk: . interface)

atagar at seul.org atagar at seul.org
Sat Oct 17 03:19:19 UTC 2009


Author: atagar
Date: 2009-10-16 23:19:18 -0400 (Fri, 16 Oct 2009)
New Revision: 20798

Modified:
   arm/trunk/ChangeLog
   arm/trunk/README
   arm/trunk/TODO
   arm/trunk/arm.py
   arm/trunk/interface/connPanel.py
   arm/trunk/interface/controller.py
   arm/trunk/interface/logPanel.py
Log:
Resolving a few small issues that bugged me.
change: using log file to pre-populate events if available
change: asks for confirmation when quitting
change: provides warning when tor's descriptors won't be updated
change: event log now allows for multi-line messages
fix: occasional crashing error concerning connection cache when paused
fix: issue with tracking connection times when paused or not visible



Modified: arm/trunk/ChangeLog
===================================================================
--- arm/trunk/ChangeLog	2009-10-17 03:02:53 UTC (rev 20797)
+++ arm/trunk/ChangeLog	2009-10-17 03:19:18 UTC (rev 20798)
@@ -1,6 +1,16 @@
 CHANGE LOG
 
-9/28/09 - version 1.1.3
+10/16/09 - version 1.2.0
+Resolving a few small issues that bugged me.
+
+    * change: using log file to pre-populate events if available
+    * change: asks for confirmation when quitting
+    * change: provides warning when tor's descriptors won't be updated
+    * change: event log now allows for multi-line messages
+    * fix: occasional crashing error concerning connection cache when paused
+    * fix: issue with tracking connection times when paused or not visible
+
+9/28/09 - version 1.1.3 (r20678)
 More issues discussed on irc.
 
     * fix: made netstat lookups a best-effort service, separate from draw thread (caught by arma and StrangeCharm)

Modified: arm/trunk/README
===================================================================
--- arm/trunk/README	2009-10-17 03:02:53 UTC (rev 20797)
+++ arm/trunk/README	2009-10-17 03:19:18 UTC (rev 20798)
@@ -4,12 +4,12 @@
 Project page: www.atagar.com/arm
 
 Description:
-Command line application for monitoring Tor relays, providing real time status 
-information such as the current configuration, bandwidth usage, message log, 
-connections, etc. This uses a curses interface much like 'top' does for system 
-usage. The application is intended for command-line aficionados, ssh 
-connections, and anyone stuck with a tty terminal for checking their relay's 
-status. Releases should be stable so if you manage to make it crash (or have a 
+Command line application for monitoring Tor relays, providing real time status
+information such as the current configuration, bandwidth usage, message log,
+connections, etc. This uses a curses interface much like 'top' does for system
+usage. The application is intended for command-line aficionados, ssh
+connections, and anyone stuck with a tty terminal for checking their relay's
+status. Releases should be stable so if you manage to make it crash (or have a
 feature request) then please let me know!
 
 The project was originally proposed in 2008 by Jacob and Karsten:
@@ -31,54 +31,54 @@
 FAQ:
 > Why is it called 'arm'?
 
-Simple - because it makes the command short and memorable. Terminal 
-applications need to be easy to type (like 'top', 'ssh', etc), and anything 
-longer is just begging command-line aficionados to alias it down. I chose the 
+Simple - because it makes the command short and memorable. Terminal
+applications need to be easy to type (like 'top', 'ssh', etc), and anything
+longer is just begging command-line aficionados to alias it down. I chose the
 meaning of the acronym ('anonymizing relay monitor') afterward.
 
 > If you're listing connections then what about exit nodes? Won't this include 
 people's traffic?
 
-While arm isn't intended to be a sniffer it does provide real time connection 
-data which, for exit nodes, includes the endpoints being visited through you. 
-Unfortunately this is pretty unavoidable. The control port doesn't provide a 
-means of distinguishing those connections and trying to figure it out by 
+While arm isn't intended to be a sniffer it does provide real time connection
+data which, for exit nodes, includes the endpoints being visited through you.
+Unfortunately this is pretty unavoidable. The control port doesn't provide a
+means of distinguishing those connections and trying to figure it out by
 correlating against consensus data has proved pretty inaccurate.
 
-That said, this really isn't much of a concern. For Tor users the real threats 
-come from things like Wireshark and MITM attacks on their unencrypted traffic. 
-Simply seeing an unknown individual's endpoints is no great feat in itself. 
-Just attach netstat to a cron job and voilà! You've got a sniffer that's just 
+That said, this really isn't much of a concern. For Tor users the real threats
+come from things like Wireshark and MITM attacks on their unencrypted traffic.
+Simply seeing an unknown individual's endpoints is no great feat in itself.
+Just attach netstat to a cron job and voilà! You've got a sniffer that's just
 as mighty as arm.
 
 > Is it harmful to share the information provided by arm?
 
-Not really, but it's discouraged. The original plan for arm included a special 
-emphasis that it wouldn't log any data. The reason is that if a large number 
-of relay operators published the details of their connections then correlation 
-attacks could break Tor user's anonymity. Just show some moderation in what 
-you share and it should be fine.
+Not really, but it's discouraged. The original plan for arm included a special
+emphasis that it wouldn't log any data. The reason is that if a large number of
+relay operators published the details of their connections then correlation
+attacks could break Tor user's anonymity. Just show some moderation in what you
+share and it should be fine.
 
 > Is there any chance that arm will leak data?
 
-Yes - arm is a passive listener with one exception. The second page 
-(connections) provides the hostnames of Tor relays you're connected to. This 
-means reverse DNS lookups which, if monitored, could leak your current 
-connections to an eavesdropper. However, lookups are only made upon request 
-(when showing connection details or listing connections by hostname) and you 
-can disable lookups entirely with 'r' - see the page's help for the current 
+Yes - arm is a passive listener with one exception. The second page
+(connections) provides the hostnames of Tor relays you're connected to. This
+means reverse DNS lookups which, if monitored, could leak your current
+connections to an eavesdropper. However, lookups are only made upon request
+(when showing connection details or listing connections by hostname) and you
+can disable lookups entirely with 'r' - see the page's help for the current
 status.
 
-That said, this is not a terribly big whoop. ISPs and anyone sniffing your 
-connection already has this data - the only difference is that instead of 
-saying "I am talking to x" you're saying "I'm talking to x, who's x?", meaning 
+That said, this is not a terribly big whoop. ISPs and anyone sniffing your
+connection already has this data - the only difference is that instead of
+saying "I am talking to x" you're saying "I'm talking to x, who's x?", meaning
 the resolver's also aware of who they are.
 
 > When arm starts it gives "Unable to resolve tor pid, abandoning connection 
 listing"... why?
 
-If you're running multiple instances of tor then arm needs to figure out which 
-pid belongs to the open control port. If it's running as a different user 
-(such as being in a chroot jail) then it's probably failing due to permission 
-issues. Arm still runs, just no connection listing or ps stats.
+If you're running multiple instances of tor then arm needs to figure out which
+pid belongs to the open control port. If it's running as a different user (such
+as being in a chroot jail) then it's probably failing due to permission issues.
+Arm still runs, just no connection listing or ps stats.
 

Modified: arm/trunk/TODO
===================================================================
--- arm/trunk/TODO	2009-10-17 03:02:53 UTC (rev 20797)
+++ arm/trunk/TODO	2009-10-17 03:19:18 UTC (rev 20798)
@@ -4,6 +4,7 @@
 	* Mac OSX and BSD have issues with netstat options
 			Reported that they aren't cross platform. Possibly use lsof as a 
 			fallback if an issue's detected.
+			notify John Case <case at sdf.lonestar.org>
 			caught by Christopher Davis
 	* quitting can hang several seconds when there's hostnames left to resolve
 			Not sure how to address this - problem is that the calls to 'host' can 
@@ -11,10 +12,17 @@
 			Or forcefully terminate thread if it's taking too long (might be noisy)?
 	* version labels provided on Debian are longer than expected
 			caught by hexa
-	* new connections don't have uptime tracked when not visible
-			Previous fix attempted to resolve, but evidently didn't work.
 
 - Features / Site
+	* add page that allows raw control port access
+			Piggyback on the arm connection, providing something like an interactive
+			prompt. In addition, provide:
+				- irc like help (ex "/help GETINFO" could provide a summary of getinfo
+				commands, partly using the results from "GETINFO info/names")
+				- tab completion and up/down populates previous entries
+				- warn and get confirmation if command would disrupt arm (for instance
+				'SETEVENTS')
+				- 'guard' option that restricts to GETINFO only	(start with this)
 	* provide observed bandwidth
 			Newer relays have a 'w' entry that states the bandwidth and old versions
 			have client side measurements (third argument in 'Bandwidth' of
@@ -29,8 +37,15 @@
 	* add arm to listings of support programs
 			https://wiki.torproject.org/noreply/TheOnionRouter/SupportPrograms
 			https://www.torproject.org/projects/
+	* add svn / tarball fingerprint to site
 
 - Ideas (low priority)
+	* bundle script that dumps relay stats to stdout
+			Django has a small terminal coloring module that could be nice for
+			formatting. Could possibly include:
+				- desc / ns information for our relay
+				- ps / netstat stats like load, uptime, and connection counts, etc
+			derived from an idea by StrangeCharm
 	* provide performance ARM-DEBUG events
 			Might help with debugging bottlenecks. This requires that there's more
 			refined controls for selecting logged arm runlevel.

Modified: arm/trunk/arm.py
===================================================================
--- arm/trunk/arm.py	2009-10-17 03:02:53 UTC (rev 20797)
+++ arm/trunk/arm.py	2009-10-17 03:19:18 UTC (rev 20798)
@@ -19,8 +19,8 @@
 from interface import controller
 from interface import logPanel
 
-VERSION = "1.1.3"
-LAST_MODIFIED = "Sep 28, 2009"
+VERSION = "1.2.0"
+LAST_MODIFIED = "Oct 16, 2009"
 
 DEFAULT_CONTROL_ADDR = "127.0.0.1"
 DEFAULT_CONTROL_PORT = 9051

Modified: arm/trunk/interface/connPanel.py
===================================================================
--- arm/trunk/interface/connPanel.py	2009-10-17 03:02:53 UTC (rev 20797)
+++ arm/trunk/interface/connPanel.py	2009-10-17 03:19:18 UTC (rev 20798)
@@ -214,7 +214,7 @@
     connectionCountTmp = [0] * 5
     
     try:
-      if self.clientConnectionCache == None and not self.isPaused:
+      if self.clientConnectionCache == None:
         # client connection cache was invalidated
         self.clientConnectionCache = _getClientConnections(self.conn)
       
@@ -374,6 +374,7 @@
           if self.showingDetails:
             listingHeight -= 8
             isScrollBarVisible = len(self.connections) > self.maxY - 9
+            if self.maxX > 80: self.win.hline(8, 80, curses.ACS_HLINE, self.maxX - 81)
           else:
             isScrollBarVisible = len(self.connections) > self.maxY - 1
           xOffset = 3 if isScrollBarVisible else 0 # content offset for scroll bar

Modified: arm/trunk/interface/controller.py
===================================================================
--- arm/trunk/interface/controller.py	2009-10-17 03:02:53 UTC (rev 20797)
+++ arm/trunk/interface/controller.py	2009-10-17 03:19:18 UTC (rev 20798)
@@ -27,6 +27,7 @@
 import cpuMemMonitor
 import connCountMonitor
 
+CONFIRM_QUIT = True
 DISABLE_CONNECTIONS_PAGE = False
 REFRESH_RATE = 5        # seconds between redrawing screen
 cursesLock = RLock()    # global curses lock (curses isn't thread safe and
@@ -287,7 +288,7 @@
     "header": headerPanel.HeaderPanel(cursesLock, conn, torPid),
     "popup": util.Panel(cursesLock, 9),
     "graph": graphPanel.GraphPanel(cursesLock),
-    "log": logPanel.LogMonitor(cursesLock, loggedEvents),
+    "log": logPanel.LogMonitor(cursesLock, conn, loggedEvents),
     "torrc": confPanel.ConfPanel(cursesLock, confLocation)}
   
   # starts thread for processing netstat queries
@@ -322,6 +323,16 @@
   loggedEvents = setEventListening(loggedEvents, conn, panels["log"])
   panels["log"].loggedEvents = loggedEvents # strips any that couldn't be set
   
+  # warns if tor isn't updating descriptors
+  try:
+    if conn.get_option("FetchUselessDescriptors")[0][1] == "0" and conn.get_option("DirPort")[0][1] == "0":
+      warning = ["Descriptors won't be updated (causing some connection information to be stale) unless:", \
+                "  a. 'FetchUselessDescriptors 1' is set in your torrc", \
+                "  b. the directory service is provided ('DirPort' defined)", \
+                "  c. tor is used as a client"]
+      panels["log"].monitor_event("WARN", warning)
+  except (TorCtl.ErrorReply, TorCtl.TorCtlClosed, socket.error): pass
+  
   isUnresponsive = False    # true if it's been over ten seconds since the last BW event (probably due to Tor closing)
   isPaused = False          # if true updates are frozen
   page = 0
@@ -374,7 +385,7 @@
         isUnresponsive = False
         panels["log"].monitor_event("NOTICE", "Relay resumed")
       
-      if not panels["conn"].isPaused: panels["conn"].reset()
+      panels["conn"].reset()
       
       # I haven't the foggiest why, but doesn't work if redrawn out of order...
       for panelKey in (PAGE_S + PAGES[page]): panels[panelKey].redraw()
@@ -384,18 +395,41 @@
     
     key = stdscr.getch()
     if key == ord('q') or key == ord('Q'):
-      # quits arm
-      # very occasionally stderr gets "close failed: [Errno 11] Resource temporarily unavailable"
-      # this appears to be a python bug: http://bugs.python.org/issue3014
-      daemonThreads = panels["conn"].resolver.threadPool
+      quitConfirmed = not CONFIRM_QUIT
       
-      # sets halt flags for all worker daemon threads
-      for worker in daemonThreads: worker.halt = True
+      # provides prompt to confirm that arm should exit
+      if CONFIRM_QUIT:
+        cursesLock.acquire()
+        try:
+          setPauseState(panels, isPaused, page, True)
+          
+          # provides prompt
+          panels["control"].setMsg("Are you sure (q again to confirm)?", curses.A_BOLD)
+          panels["control"].redraw()
+          
+          curses.cbreak()
+          confirmationKey = stdscr.getch()
+          quitConfirmed = confirmationKey in (ord('q'), ord('Q'))
+          curses.halfdelay(REFRESH_RATE * 10)
+          
+          panels["control"].setMsg(CTL_PAUSED if isPaused else CTL_HELP)
+          setPauseState(panels, isPaused, page)
+        finally:
+          cursesLock.release()
       
-      # joins on workers (prevents noisy termination)
-      for worker in daemonThreads: worker.join()
-      
-      break
+      if quitConfirmed:
+        # quits arm
+        # very occasionally stderr gets "close failed: [Errno 11] Resource temporarily unavailable"
+        # this appears to be a python bug: http://bugs.python.org/issue3014
+        daemonThreads = panels["conn"].resolver.threadPool
+        
+        # sets halt flags for all worker daemon threads
+        for worker in daemonThreads: worker.halt = True
+        
+        # joins on workers (prevents noisy termination)
+        for worker in daemonThreads: worker.join()
+        
+        break
     elif key == curses.KEY_LEFT or key == curses.KEY_RIGHT:
       # switch page
       if key == curses.KEY_LEFT: page = (page - 1) % len(PAGES)
@@ -731,7 +765,7 @@
               popup.addstr(3, 2, "No consensus data found", format)
             else:
               # couldn't resolve due to multiple matches - list them all
-              popup.addstr(3, 2, "Muliple matching IPs, possible fingerprints are:", format)
+              popup.addstr(3, 2, "Muliple matches, possible fingerprints are:", format)
               matchings = panels["conn"].fingerprintMappings[selectedIp]
               
               line = 4

Modified: arm/trunk/interface/logPanel.py
===================================================================
--- arm/trunk/interface/logPanel.py	2009-10-17 03:02:53 UTC (rev 20797)
+++ arm/trunk/interface/logPanel.py	2009-10-17 03:19:18 UTC (rev 20798)
@@ -9,7 +9,8 @@
 
 import util
 
-MAX_LOG_ENTRIES = 1000               # size of log buffer (max number of entries)
+PRE_POPULATE_LOG = True               # attempts to retrieve events from log file if available
+MAX_LOG_ENTRIES = 1000                # size of log buffer (max number of entries)
 RUNLEVEL_EVENT_COLOR = {"DEBUG": "magenta", "INFO": "blue", "NOTICE": "green", "WARN": "yellow", "ERR": "red"}
 
 EVENT_TYPES = {
@@ -71,7 +72,7 @@
   Tor event listener, noting messages, the time, and their type in a panel.
   """
   
-  def __init__(self, lock, loggedEvents):
+  def __init__(self, lock, conn, loggedEvents):
     TorCtl.PostEventListener.__init__(self)
     util.Panel.__init__(self, lock, -1)
     self.scroll = 0
@@ -81,6 +82,41 @@
     self.loggedEvents = loggedEvents      # events we're listening to
     self.lastHeartbeat = time.time()      # time of last event
     self.regexFilter = None               # filter for presented log events (no filtering if None)
+    self.eventTimeOverwrite = None        # replaces time for further events with this (uses time it occures if None)
+    
+    # attempts to process events from log file
+    if PRE_POPULATE_LOG:
+      previousPauseState = self.isPaused
+      logFile = None
+      
+      try:
+        logFileLoc = None
+        loggingLocations = conn.get_option("Log")
+        
+        for entry in loggingLocations:
+          entryComp = entry[1].split()
+          if entryComp[1] == "file":
+            logFileLoc = entryComp[2]
+            break
+        
+        if logFileLoc:
+          # prevents attempts to redraw while processing batch of events
+          self.setPaused(True)
+          
+          logFile = open(logFileLoc, "r")
+          for line in logFile:
+            lineComp = line.split()
+            eventType = lineComp[3][1:-1].upper()
+            
+            if eventType in loggedEvents:
+              timeComp = lineComp[2][:lineComp[2].find(".")].split(":")
+              self.eventTimeOverwrite = (0, 0, 0, int(timeComp[0]), int(timeComp[1]), int(timeComp[2]))
+              self.listen(TorCtl.LogEvent(eventType, " ".join(lineComp[4:])))
+      except Exception: pass # disreguard any issues that might arise in parsing
+      finally:
+        self.setPaused(previousPauseState)
+        self.eventTimeOverwrite = None
+        if logFile: logFile.close()
   
   def handleKey(self, key):
     # scroll movement
@@ -169,22 +205,32 @@
   
   def registerEvent(self, type, msg, color):
     """
-    Notes event and redraws log. If paused it's held in a temporary buffer.
+    Notes event and redraws log. If paused it's held in a temporary buffer. If 
+    msg is a list then this is expanded to multiple lines.
     """
     
     if not type.startswith("ARM"): self.lastHeartbeat = time.time()
+    eventTime = self.eventTimeOverwrite if self.eventTimeOverwrite else time.localtime()
+    toAdd = []
     
-    # strips control characters to avoid screwing up the terminal
-    msg = "".join([char for char in msg if isprint(char)])
+    # wraps if a single line message
+    if isinstance(msg, str): msg = [msg]
     
-    eventTime = time.localtime()
-    msgLine = "%02i:%02i:%02i [%s] %s" % (eventTime[3], eventTime[4], eventTime[5], type, msg)
+    firstLine = True
+    for msgLine in msg:
+      # strips control characters to avoid screwing up the terminal
+      msgLine = "".join([char for char in msgLine if isprint(char)])
+      
+      header = "%02i:%02i:%02i %s" % (eventTime[3], eventTime[4], eventTime[5], "[%s]" % type) if firstLine else ""
+      toAdd.append("%s %s" % (header, msgLine))
+      firstLine = False
     
+    toAdd.reverse()
     if self.isPaused:
-      self.pauseBuffer.insert(0, (msgLine, color))
+      for msgLine in toAdd: self.pauseBuffer.insert(0, (msgLine, color))
       if len(self.pauseBuffer) > MAX_LOG_ENTRIES: del self.pauseBuffer[MAX_LOG_ENTRIES:]
     else:
-      self.msgLog.insert(0, (msgLine, color))
+      for msgLine in toAdd: self.msgLog.insert(0, (msgLine, color))
       if len(self.msgLog) > MAX_LOG_ENTRIES: del self.msgLog[MAX_LOG_ENTRIES:]
       self.redraw()
   



More information about the tor-commits mailing list