[or-cvs] r23660: {arm} Rewrite of the conf panel and several important bug fixes ch (in arm/trunk: . debian src src/interface src/interface/graphing src/util)
Damian Johnson
atagar1 at gmail.com
Fri Oct 22 05:21:30 UTC 2010
Author: atagar
Date: 2010-10-22 05:21:29 +0000 (Fri, 22 Oct 2010)
New Revision: 23660
Added:
arm/trunk/src/settings.cfg
arm/trunk/src/util/torrc.py
Removed:
arm/trunk/armrc.sample
arm/trunk/src/armrc.defaults
Modified:
arm/trunk/ChangeLog
arm/trunk/README
arm/trunk/TODO
arm/trunk/debian/MANIFEST
arm/trunk/setup.py
arm/trunk/src/interface/confPanel.py
arm/trunk/src/interface/controller.py
arm/trunk/src/interface/graphing/bandwidthStats.py
arm/trunk/src/interface/graphing/graphPanel.py
arm/trunk/src/interface/headerPanel.py
arm/trunk/src/interface/logPanel.py
arm/trunk/src/starter.py
arm/trunk/src/util/conf.py
arm/trunk/src/util/hostnames.py
arm/trunk/src/util/log.py
arm/trunk/src/util/panel.py
arm/trunk/src/util/torTools.py
arm/trunk/src/util/uiTools.py
Log:
Rewrite of the conf panel and several important bug fixes
change: full rewrite of the log panel, providing:
- change: simplified and expanded on config validation and display (performance improvements, friendly units for torrc corrections, etc)
- change: scrolling by displayed content rather than line numbers
- fix: unnecessary whitespace was being stripped
- fix: scrolling was buggy if comments were being stripped
- fix: log panel wasn't respecting the prepopulate* log level config options
- fix: torrc validation didn't recognize 'second' and 'byte' arguments
change: revised the arm config interface (simplified and expanded to include maps)
fix: not all worker threads were daemons, causing the process to persist in a broken state after exceptions and when quitting via ctrl+c
fix: custom armrcs resulted in the default parsing config options being unavailable
fix: rounding error in rendering the scrollbar, causing it to shrink a line when at the bottom
fix: off by one error when wrapping lines in the log panel
Modified: arm/trunk/ChangeLog
===================================================================
--- arm/trunk/ChangeLog 2010-10-21 06:37:31 UTC (rev 23659)
+++ arm/trunk/ChangeLog 2010-10-22 05:21:29 UTC (rev 23660)
@@ -1,6 +1,6 @@
CHANGE LOG
-10/6/10 - version 1.3.7
+10/6/10 - version 1.3.7 (r23439)
Numerous improvements, most notably being an expanded log panel, installer, and deb/rpm builds.
* added: installation/removal scripts and man page (thanks to kaner)
@@ -50,6 +50,7 @@
* fix: race condition between heartbeat detection and getting the first BW event
* fix: refreshing after popups to make the interface seem more responsive
* fix: crashing and minor display issues if orport was left unset
+ * fix (10/7/10, r23463): crashing from type issue in the graph panel (caught by tomb)
6/7/10 - version 1.3.6 (r22617)
Rewrite of the first third of the interface, providing vastly improved performance, maintainability, and a few very nice features. This improved the refresh rate (which is also related to system resource usage) from 30ms to 4ms (an 87% improvement).
Modified: arm/trunk/README
===================================================================
--- arm/trunk/README 2010-10-21 06:37:31 UTC (rev 23659)
+++ arm/trunk/README 2010-10-22 05:21:29 UTC (rev 23660)
@@ -102,10 +102,11 @@
src/
__init__.py
- starter.py - parses and validates commandline parameters
- prereq.py - checks python version and for required packages
- version.py - version and last modified information
- uninstall - removal script
+ starter.py - parses and validates commandline parameters
+ prereq.py - checks python version and for required packages
+ version.py - version and last modified information
+ settings.cfg - attributes loaded for parsing tor related data
+ uninstall - removal script
interface/
graphing/
Modified: arm/trunk/TODO
===================================================================
--- arm/trunk/TODO 2010-10-21 06:37:31 UTC (rev 23659)
+++ arm/trunk/TODO 2010-10-22 05:21:29 UTC (rev 23660)
@@ -10,33 +10,31 @@
bugs are being fixed while refactoring.
[ ] conf panel
- - move torrc validation into util
- - fetch text via getinfo rather than reading directly?
- conn.get_info("config-text")
- - improve parsing failure notice to give line number
- just giving "[ARM-WARN] Unable to validate torrc" isn't very
- helpful...
+ - display and validation needs to recognize config entries that span
+ multiple lines
+ - option to display the armrc configuration
+ - [validation] check if there's missing entries
+ might be able to use "GETINFO config-text" to determine entries that
+ differ from the defaults, then see if they're all in the torrc
[ ] conn panel
- expand client connections and note location in circuit (entry-exit)
- - for clients list all connections to detect what's going through tor
- and what isn't? If not then netstat calls are unnecessary.
- - check family connections to see if they're alive (VERSION cell
+ - for clients give an option to list all connections, to tell which are
+ going through tor and which might be leaking
+ - check family members to see if they're alive (VERSION cell
handshake?)
- fallback when pid or connection querying via pid is unavailable
List all connections listed both by netstat and the consensus
- note when connection times are estimates (color?), ie connection
was established before arm
- connection uptime to associate inbound/outbound connections?
- - Identify controller connections (if it's arm, vidalia, etc) with
+ - identify controller connections (if it's arm, vidalia, etc) with
special detail page for them
- - provide bridge / client country statistics
+ - provide bridge / client country / exiting port statistics
Include bridge related data via GETINFO option (feature request
by waltman and ioerror).
- pick apart applications like iftop and pktstat to see how they get
per-connection bandwidth usage. Forum thread discussing it:
https://bbs.archlinux.org/viewtopic.php?pid=715906
- - give usage stats for exit port usage (popup?)
- - country data for client connections (requested by ioerror)
[ ] attempt to clear controller password from memory
- http://www.codexon.com/posts/clearing-passwords-in-memory-with-python
* release prep
@@ -74,17 +72,6 @@
tor's uptime - blocked on implementation of the following proposal:
https://gitweb.torproject.org/tor.git/blob/HEAD:/doc/spec/proposals/173-getinfo-option-expansion.txt
- * conf panel:
- * torrc validation doesn't catch if parameters are missing
- * scrolling in the torrc isn't working properly when comments are stripped
- Current method of displaying torrc is pretty stupid (lots of repeated
- work in display loop). When rewritten fixing this bug should be
- trivial.
- * "ExitPolicy" entry in torrc (without path)
- Produces "May 26 22:11:03.484 [warn] The abbreviation 'ExitPolic' is
- deprecated. Please use 'ExitPolicy' instead". This is an error in the
- torrc parsing when only the key is provided.
-
* conn panel:
* *never* do reverse dns lookups for first hops (could be resolving via
tor and hence leaking to the exit)
@@ -145,13 +132,11 @@
fetching everything at each client
* possibly make these archives downloadable from peer relays (this is a
no-go for clients) via torrents or some dirport like scheme
- * look at vidalia for ideas
+ * look at Vidalia and TorK for ideas
* need to solicit for ideas on what would be most helpful to clients
- * general purpose method of erroring nicely
- Some errors cause portions of the display to die, but curses limps along
- and overwrites the stacktrace. This has been mostly solved, but all errors
- should result in a clean death, with the stacktrace saved and a nice
- message for the user.
+ * dialog with bridge statuses (idea by mikeperry)
+ https://trac.vidalia-project.net/ticket/570
+ https://trac.torproject.org/projects/tor/ticket/2068
* handle mutiple tor instances
* screen style (dialog for switching between instances)
* extra window with whatever stats can be aggregated over all instances
@@ -179,6 +164,8 @@
Plugin for distutils. Like most mac packaging, this can only run on a
mac. It also requires setuptools:
http://www.errorhelp.com/search/details/74034/importerror-no-module-named-setuptools
+ * look through vidalia's tickets for more ideas
+ https://trac.vidalia-project.net/
* look into additions to the used apis
* curses (python 2.6 extended?): http://docs.python.org/library/curses.html
* new control options (like "desc-annotations/id/<OR identity>")?
@@ -216,6 +203,9 @@
* implement control-spec proposals:
* https://gitweb.torproject.org/tor.git/blob/HEAD:/doc/spec/proposals/172-circ-getinfo-option.txt
* https://gitweb.torproject.org/tor.git/blob/HEAD:/doc/spec/proposals/173-getinfo-option-expansion.txt
+ * gui frontend (gtk?)
+ Look into if the arm utilities and codebase would fit nicely for a gui
+ controller like Vidalia and TorK.
* unit tests
Primarily for util, for instance 'addfstr' would be a good candidate.
* python 3 compatibility
Deleted: arm/trunk/armrc.sample
===================================================================
--- arm/trunk/armrc.sample 2010-10-21 06:37:31 UTC (rev 23659)
+++ arm/trunk/armrc.sample 2010-10-22 05:21:29 UTC (rev 23660)
@@ -1 +0,0 @@
-link src/armrc.defaults
\ No newline at end of file
Modified: arm/trunk/debian/MANIFEST
===================================================================
--- arm/trunk/debian/MANIFEST 2010-10-21 06:37:31 UTC (rev 23659)
+++ arm/trunk/debian/MANIFEST 2010-10-22 05:21:29 UTC (rev 23660)
@@ -2,12 +2,13 @@
setup.cfg
setup.py
arm
+armrc.sample
debian/arm.1.gz
src/__init__.py
src/prereq.py
src/starter.py
src/version.py
-src/armrc.defaults
+src/settings.cfg
src/TorCtl/GeoIPSupport.py
src/TorCtl/PathSupport.py
src/TorCtl/SQLSupport.py
Modified: arm/trunk/setup.py
===================================================================
--- arm/trunk/setup.py 2010-10-21 06:37:31 UTC (rev 23659)
+++ arm/trunk/setup.py 2010-10-22 05:21:29 UTC (rev 23660)
@@ -14,7 +14,7 @@
packages=['arm', 'arm.interface', 'arm.interface.graphing', 'arm.util', 'arm.TorCtl'],
package_dir={'arm': 'src'},
data_files=[("/usr/bin", ["arm"]),
- ("/usr/lib/arm", ["src/armrc.defaults"]),
+ ("/usr/lib/arm", ["src/settings.cfg"]),
("/usr/share/man/man1", ["debian/arm.1.gz"])],
)
Deleted: arm/trunk/src/armrc.defaults
===================================================================
--- arm/trunk/src/armrc.defaults 2010-10-21 06:37:31 UTC (rev 23659)
+++ arm/trunk/src/armrc.defaults 2010-10-22 05:21:29 UTC (rev 23660)
@@ -1,229 +0,0 @@
-# startup options
-startup.controlPassword
-startup.interface.ipAddress 127.0.0.1
-startup.interface.port 9051
-startup.blindModeEnabled false
-startup.events N3
-
-# Seconds between querying information
-queries.ps.rate 5
-queries.connections.minRate 5
-queries.refreshRate.rate 5
-
-# Renders the interface with color if set and the terminal supports it
-features.colorInterface true
-
-# Set this if you're running in a chroot jail or other environment where tor's
-# resources (log, state, etc) should have a prefix in their paths.
-features.pathPrefix
-
-# If set, arm appends any log messages it reports while running to the given
-# log file. This does not take filters into account or include prepopulated
-# events.
-features.logFile
-
-# Paremters for the log panel
-# ---------------------------
-# showDateDividers
-# show borders with dates for entries from previous days
-# showDuplicateEntries
-# shows all log entries if true, otherwise collapses similar entries with an
-# indicator for how much is being hidden
-# entryDuration
-# number of days log entries are kept before being dropped (if zero then
-# they're kept until cropped due to caching limits)
-# maxLinesPerEntry
-# max number of lines to display for a single log entry
-# prepopulate
-# attempts to read past events from the log file if true
-# prepopulateReadLimit
-# maximum entries read from the log file, used to prevent huge log files from
-# causing a slow startup time.
-# maxRefreshRate
-# rate limiting (in milliseconds) for drawing the log if updates are made
-# rapidly (for instance, when at the DEBUG runlevel)
-
-features.log.showDateDividers true
-features.log.showDuplicateEntries false
-features.log.entryDuration 7
-features.log.maxLinesPerEntry 4
-features.log.prepopulate true
-features.log.prepopulateReadLimit 5000
-features.log.maxRefreshRate 300
-
-# General graph parameters
-# ------------------------
-# height
-# height of graphed stats
-# maxWidth
-# maximum number of graphed entries
-# interval
-# 0 -> each second, 1 -> 5 seconds, 2 -> 30 seconds, 3 -> minutely,
-# 4 -> 15 minutes, 5 -> half hour, 6 -> hourly, 7 -> daily
-# bound
-# 0 -> global maxima, 1 -> local maxima, 2 -> tight
-# type
-# 0 -> None, 1 -> Bandwidth, 2 -> Connections, 3 -> System Resources
-# showIntermediateBounds
-# shows y-axis increments between the top/bottom bounds
-
-features.graph.height 7
-features.graph.maxWidth 150
-features.graph.interval 0
-features.graph.bound 1
-features.graph.type 1
-features.graph.showIntermediateBounds true
-
-# Parameters for graphing bandwidth stats
-# ---------------------------------------
-# prepopulate
-# attempts to use tor's state file to prepopulate the bandwidth graph at the
-# 15-minute interval (this requires the minimum of a day's worth of uptime)
-# transferInBystes
-# shows rate measurments in bytes if true, bits otherwise
-# accounting.show
-# provides accounting stats if AccountingMax was set
-# accounting.rate
-# seconds between querying accounting stats
-# accounting.isTimeLong
-# provides verbose measurements of time if true
-
-features.graph.bw.prepopulate true
-features.graph.bw.transferInBytes false
-features.graph.bw.accounting.show true
-features.graph.bw.accounting.rate 10
-features.graph.bw.accounting.isTimeLong false
-
-# Parameters for graphing ps stats
-# --------------------------------
-# primary/secondaryStat
-# any numeric field provided by the ps command
-# cachedOnly
-# determines if the graph should query ps or rely on cached results (this
-# lowers the call volume but limits the graph's granularity)
-
-features.graph.ps.primaryStat %cpu
-features.graph.ps.secondaryStat rss
-features.graph.ps.cachedOnly true
-
-# Thread pool size for hostname resolutions
-# Determines the maximum number of concurrent requests. Upping this to around
-# thirty or so seems to be problematic, causing intermittently seizing.
-
-queries.hostnames.poolSize 5
-
-# Method of resolving hostnames
-# If true, uses python's internal "socket.gethostbyaddr" to resolve addresses
-# rather than the host command. This is ignored if the system's unable to make
-# parallel requests. Resolving this way seems to be much slower than host calls
-# in practice.
-
-queries.hostnames.useSocketModule false
-
-# Caching parameters
-cache.sysCalls.size 600
-cache.hostnames.size 700000
-cache.hostnames.trimSize 200000
-cache.logPanel.size 1000
-cache.armLog.size 1000
-cache.armLog.trimSize 200
-
-# Runlevels at which arm logs its events
-log.refreshRate DEBUG
-log.configEntryNotFound NONE
-log.configEntryUndefined NOTICE
-log.configEntryTypeError NOTICE
-log.torCtlPortClosed NOTICE
-log.torGetInfo DEBUG
-log.torGetConf DEBUG
-log.torEventTypeUnrecognized NOTICE
-log.torPrefixPathInvalid NOTICE
-log.sysCallMade DEBUG
-log.sysCallCached NONE
-log.sysCallFailed INFO
-log.sysCallCacheGrowing INFO
-log.panelRecreated DEBUG
-log.graph.ps.invalidStat WARN
-log.graph.ps.abandon WARN
-log.graph.bw.prepopulateSuccess NOTICE
-log.graph.bw.prepopulateFailure NOTICE
-log.logPanel.prepopulateSuccess INFO
-log.logPanel.prepopulateFailed WARN
-log.logPanel.logFileOpened NOTICE
-log.logPanel.logFileWriteFailed ERR
-log.logPanel.forceDoubleRedraw DEBUG
-log.connLookupFailed INFO
-log.connLookupFailover NOTICE
-log.connLookupAbandon WARN
-log.connLookupRateGrowing NONE
-log.hostnameCacheTrimmed INFO
-log.cursesColorSupport INFO
-
-# Snippets from common log messages
-# These are static bits of log messages, used to determine when entries with
-# dynamic content (hostnames, numbers, etc) are the same. If this matches the
-# start of both messages then the entries are flagged as duplicates. If the
-# entry begins with an asterisk (*) then it checks if the substrings exist
-# anywhere in the messages.
-#
-# Examples for the complete messages:
-# [BW] READ: 0, WRITTEN: 0
-# [DEBUG] connection_handle_write(): After TLS write of 512: 0 read, 586 written
-# [DEBUG] flush_chunk_tls(): flushed 512 bytes, 0 ready to flush, 0 remain.
-# [DEBUG] conn_read_callback(): socket 7 wants to read.
-# [DEBUG] conn_write_callback(): socket 51 wants to write.
-# [DEBUG] connection_remove(): removing socket -1 (type OR), n_conns now 50
-# [DEBUG] connection_or_process_cells_from_inbuf(): 7: starting, inbuf_datalen 0 (0 pending in tls object).
-# [DEBUG] connection_read_to_buf(): 38: starting, inbuf_datalen 0 (0 pending in tls object). at_most 12800.
-# [DEBUG] connection_read_to_buf(): TLS connection closed on read. Closing. (Nickname moria1, address 128.31.0.34)
-# [INFO] run_connection_housekeeping(): Expiring non-open OR connection to fd 16 (79.193.61.171:443).
-# [INFO] rep_hist_downrate_old_runs(): Discounting all old stability info by a factor of 0.950000
-# [NOTICE] We stalled too much while trying to write 150 bytes to address
-# [scrubbed]. If this happens a lot, either something is wrong with
-# your network connection, or something is wrong with theirs. (fd 238,
-# type Directory, state 1, marked at main.c:702).
-# [NOTICE] I learned some more directory information, but not enough to build a
-# circuit: We have only 469/2027 usable descriptors.
-# [NOTICE] Attempt by %s to open a stream from unknown relay. Closing.
-# [WARN] You specified a server "Amunet8" by name, but this name is not
-# registered
-# [WARN] I have no descriptor for the router named "Amunet8" in my declared
-# family; I'll use the nickname as is, but this may confuse clients.
-# [WARN] Problem bootstrapping. Stuck at 80%: Connecting to the Tor network.
-# (Network is unreachable; NOROUTE; count 47; recommendation warn)
-# [WARN] 4 unknown, 1 missing key, 3 good, 0 bad, 1 no signature, 4 required
-# [ARM_DEBUG] refresh rate: 0.001 seconds
-# [ARM_DEBUG] system call: ps -p 2354 -o %cpu,rss,%mem,etime (runtime: 0.02)
-# [ARM_DEBUG] system call: netstat -npt | grep 2354/tor (runtime: 0.02)
-# [ARM_DEBUG] recreating panel 'graph' with the dimensions of 14/124
-# [ARM_DEBUG] redrawing the log panel with the corrected content height (estimat was off by 4)
-# [ARM_DEBUG] GETINFO accounting/bytes-left (runtime: 0.0006)
-
-msg.BW READ:
-msg.DEBUG connection_handle_write(): After TLS write of
-msg.DEBUG flush_chunk_tls(): flushed
-msg.DEBUG conn_read_callback(): socket
-msg.DEBUG conn_write_callback(): socket
-msg.DEBUG connection_remove(): removing socket
-msg.DEBUG connection_or_process_cells_from_inbuf():
-msg.DEBUG *pending in tls object). at_most
-msg.DEBUG connection_read_to_buf(): TLS connection closed on read. Closing.
-msg.INFO run_connection_housekeeping(): Expiring
-msg.INFO rep_hist_downrate_old_runs(): Discounting all old stability info by a factor of
-msg.NOTICE We stalled too much while trying to write
-msg.NOTICE I learned some more directory information, but not enough to build a circuit
-msg.NOTICE Attempt by
-msg.WARN You specified a server
-msg.WARN I have no descriptor for the router named
-msg.WARN Problem bootstrapping. Stuck at
-msg.WARN *missing key,
-msg.ARM_DEBUG refresh rate:
-msg.ARM_DEBUG system call: ps
-msg.ARM_DEBUG system call: netstat
-msg.ARM_DEBUG recreating panel '
-msg.ARM_DEBUG redrawing the log panel with the corrected content height (
-msg.ARM_DEBUG GETINFO accounting/bytes
-msg.ARM_DEBUG GETINFO accounting/bytes-left
-msg.ARM_DEBUG GETINFO accounting/interval-end
-msg.ARM_DEBUG GETINFO accounting/hibernating
-
Modified: arm/trunk/src/interface/confPanel.py
===================================================================
--- arm/trunk/src/interface/confPanel.py 2010-10-21 06:37:31 UTC (rev 23659)
+++ arm/trunk/src/interface/confPanel.py 2010-10-22 05:21:29 UTC (rev 23660)
@@ -1,292 +1,264 @@
-#!/usr/bin/env python
-# confPanel.py -- Presents torrc with syntax highlighting.
-# Released under the GPL v3 (http://www.gnu.org/licenses/gpl.html)
+"""
+Panel displaying the torrc and validation done against it.
+"""
import math
import curses
-import socket
+import threading
-import controller
-from TorCtl import TorCtl
-from util import log, panel, torTools, uiTools
+from util import log, panel, torrc, uiTools
-# torrc parameters that can be defined multiple times without overwriting
-# from src/or/config.c (entries with LINELIST or LINELIST_S)
-# last updated for tor version 0.2.1.19
-MULTI_LINE_PARAM = ["AlternateBridgeAuthority", "AlternateDirAuthority", "AlternateHSAuthority", "AuthDirBadDir", "AuthDirBadExit", "AuthDirInvalid", "AuthDirReject", "Bridge", "ControlListenAddress", "ControlSocket", "DirListenAddress", "DirPolicy", "DirServer", "DNSListenAddress", "ExitPolicy", "HashedControlPassword", "HiddenServiceDir", "HiddenServiceOptions", "HiddenServicePort", "HiddenServiceVersion", "HiddenServiceAuthorizeClient", "HidServAuth", "Log", "MapAddress", "NatdListenAddress", "NodeFamily", "ORListenAddress", "ReachableAddresses", "ReachableDirAddresses", "ReachableORAddresses", "RecommendedVersions", "RecommendedClientVersions", "RecommendedServerVersions", "SocksListenAddress", "SocksPolicy", "TransListenAddress", "__HashedControlSessionPassword"]
+DEFAULT_CONFIG = {"features.torrc.validate": True,
+ "features.config.showScrollbars": True,
+ "features.config.maxLinesPerEntry": 5,
+ "log.confPanel.torrcReadFailed": log.WARN,
+ "log.torrcValidation.duplicateEntries": log.NOTICE,
+ "log.torrcValidation.torStateDiffers": log.NOTICE}
-# hidden service options need to be fetched with HiddenServiceOptions
-HIDDEN_SERVICE_PARAM = ["HiddenServiceDir", "HiddenServiceOptions", "HiddenServicePort", "HiddenServiceVersion", "HiddenServiceAuthorizeClient"]
-HIDDEN_SERVICE_FETCH_PARAM = "HiddenServiceOptions"
-
-# size modifiers allowed by config.c
-LABEL_KB = ["kb", "kbyte", "kbytes", "kilobyte", "kilobytes"]
-LABEL_MB = ["m", "mb", "mbyte", "mbytes", "megabyte", "megabytes"]
-LABEL_GB = ["gb", "gbyte", "gbytes", "gigabyte", "gigabytes"]
-LABEL_TB = ["tb", "terabyte", "terabytes"]
-
-# GETCONF aliases (from the _option_abbrevs struct of src/or/config.c)
-# fix for: https://trac.torproject.org/projects/tor/ticket/1798
-# TODO: this has been fixed in tor- wait for a while then retest and remove
-# TODO: the following alias entry doesn't work on Tor 0.2.1.19:
-# "HashedControlPassword": "__HashedControlSessionPassword"
-CONF_ALIASES = {"l": "Log",
- "AllowUnverifiedNodes": "AllowInvalidNodes",
- "AutomapHostSuffixes": "AutomapHostsSuffixes",
- "AutomapHostOnResolve": "AutomapHostsOnResolve",
- "BandwidthRateBytes": "BandwidthRate",
- "BandwidthBurstBytes": "BandwidthBurst",
- "DirFetchPostPeriod": "StatusFetchPeriod",
- "MaxConn": "ConnLimit",
- "ORBindAddress": "ORListenAddress",
- "DirBindAddress": "DirListenAddress",
- "SocksBindAddress": "SocksListenAddress",
- "UseHelperNodes": "UseEntryGuards",
- "NumHelperNodes": "NumEntryGuards",
- "UseEntryNodes": "UseEntryGuards",
- "NumEntryNodes": "NumEntryGuards",
- "ResolvConf": "ServerDNSResolvConfFile",
- "SearchDomains": "ServerDNSSearchDomains",
- "ServerDNSAllowBrokenResolvConf": "ServerDNSAllowBrokenConfig",
- "PreferTunnelledDirConns": "PreferTunneledDirConns",
- "BridgeAuthoritativeDirectory": "BridgeAuthoritativeDir",
- "StrictEntryNodes": "StrictNodes",
- "StrictExitNodes": "StrictNodes"}
-
-
-# time modifiers allowed by config.c
-LABEL_MIN = ["minute", "minutes"]
-LABEL_HOUR = ["hour", "hours"]
-LABEL_DAY = ["day", "days"]
-LABEL_WEEK = ["week", "weeks"]
-
class ConfPanel(panel.Panel):
"""
- Presents torrc with syntax highlighting in a scroll-able area.
+ Presents torrc, armrc, or loaded settings with syntax highlighting in a
+ scrollable area.
"""
- def __init__(self, stdscr, confLocation, conn):
+ def __init__(self, stdscr, config=None):
panel.Panel.__init__(self, stdscr, "conf", 0)
- self.confLocation = confLocation
- self.showLineNum = True
- self.stripComments = False
- self.confContents = []
- self.scroll = 0
- # lines that don't matter due to duplicates
- self.irrelevantLines = []
+ self._config = dict(DEFAULT_CONFIG)
+ if config:
+ config.update(self._config, {"features.config.maxLinesPerEntry": 1})
- # used to check consistency with tor's actual values - corrections mapping
- # is of line numbers (one-indexed) to tor's actual values
+ self.valsLock = threading.RLock()
+ self.scroll = 0
+ self.showLineNum = True
+ self.stripComments = False
+ self.confLocation = ""
+ self.confContents = None # read torrc, None if it failed to load
self.corrections = {}
- self.conn = conn
+ # height of the content when last rendered (the cached value is invalid if
+ # _lastContentHeightArgs is None or differs from the current dimensions)
+ self._lastContentHeight = 1
+ self._lastContentHeightArgs = None
+
self.reset()
- def reset(self, logErrors=True):
+ def reset(self, logErrors = True):
"""
Reloads torrc contents and resets scroll height. Returns True if
successful, else false.
+
+ Arguments:
+ logErrors - logs if unable to read the torrc or issues are found during
+ validation
"""
+ self.valsLock.acquire()
+
try:
- resetSuccessful = True
-
- confFile = open(torTools.getPathPrefix() + self.confLocation, "r")
+ self.confLocation = torrc.getConfigLocation()
+ confFile = open(self.confLocation, "r")
self.confContents = confFile.readlines()
confFile.close()
+ self.scroll = 0
- # checks if torrc differs from get_option data
- self.irrelevantLines = []
- self.corrections = {}
- parsedCommands = {} # mapping of parsed commands to line numbers
+ # sets the content height to be something somewhat reasonable
+ self._lastContentHeight = len(self.confContents)
+ self._lastContentHeightArgs = None
+ except IOError, exc:
+ self.confContents = None
+ msg = "Unable to load torrc (%s)" % exc
+ if logErrors: log.log(self._config["log.confPanel.torrcReadFailed"], msg)
+ self.valsLock.release()
+ return False
+
+ if self._config["features.torrc.validate"]:
+ self.corrections = torrc.validate(self.confContents)
- for lineNumber in range(len(self.confContents)):
- lineText = self.confContents[lineNumber].strip()
+ if self.corrections and logErrors:
+ # logs issues found during validation
+ irrelevantLines, mismatchLines = [], []
+ for lineNum in self.corrections:
+ problem = self.corrections[lineNum][0]
+ if problem == torrc.VAL_DUPLICATE: irrelevantLines.append(lineNum)
+ elif problem == torrc.VAL_MISMATCH: mismatchLines.append(lineNum)
- if lineText and lineText[0] != "#":
- # relevant to tor (not blank nor comment)
- ctlEnd = lineText.find(" ") # end of command
- argEnd = lineText.find("#") # end of argument (start of comment or end of line)
- if argEnd == -1: argEnd = len(lineText)
- command, argument = lineText[:ctlEnd], lineText[ctlEnd:argEnd].strip()
+ if irrelevantLines:
+ irrelevantLines.sort()
- # replace aliases with the internal representation of the command
- if command in CONF_ALIASES: command = CONF_ALIASES[command]
-
- # tor appears to replace tabs with a space, for instance:
- # "accept\t*:563" is read back as "accept *:563"
- argument = argument.replace("\t", " ")
-
- # expands value if it's a size or time
- comp = argument.strip().lower().split(" ")
- if len(comp) > 1:
- size = 0
- if comp[1] in LABEL_KB: size = int(comp[0]) * 1024
- elif comp[1] in LABEL_MB: size = int(comp[0]) * 1048576
- elif comp[1] in LABEL_GB: size = int(comp[0]) * 1073741824
- elif comp[1] in LABEL_TB: size = int(comp[0]) * 1099511627776
- elif comp[1] in LABEL_MIN: size = int(comp[0]) * 60
- elif comp[1] in LABEL_HOUR: size = int(comp[0]) * 3600
- elif comp[1] in LABEL_DAY: size = int(comp[0]) * 86400
- elif comp[1] in LABEL_WEEK: size = int(comp[0]) * 604800
- if size != 0: argument = str(size)
-
- # most parameters are overwritten if defined multiple times, if so
- # it's erased from corrections and noted as duplicate instead
- if not command in MULTI_LINE_PARAM and command in parsedCommands.keys():
- previousLineNum = parsedCommands[command]
- self.irrelevantLines.append(previousLineNum)
- if previousLineNum in self.corrections.keys(): del self.corrections[previousLineNum]
-
- parsedCommands[command] = lineNumber + 1
-
- # check validity against tor's actual state
- try:
- actualValues = []
- if command in HIDDEN_SERVICE_PARAM:
- # hidden services are fetched via a special command
- hsInfo = self.conn.get_option(HIDDEN_SERVICE_FETCH_PARAM)
- for entry in hsInfo:
- if entry[0] == command:
- actualValues.append(entry[1])
- break
- else:
- # general case - fetch all valid values
- for key, val in self.conn.get_option(command):
- if val == None:
- # TODO: investigate situations where this might occure
- # (happens if trying to parse HIDDEN_SERVICE_PARAM)
- if logErrors: log.log(log.WARN, "BUG: Failed to find torrc value for %s" % key)
- continue
-
- # TODO: check for a better way of figuring out CSV parameters
- # (kinda doubt this is right... in config.c its listed as being
- # a 'LINELIST') - still, good enough for common cases
- if command in MULTI_LINE_PARAM: toAdd = val.split(",")
- else: toAdd = [val]
-
- for newVal in toAdd:
- newVal = newVal.strip()
- if newVal not in actualValues: actualValues.append(newVal)
-
- # there might be multiple values on a single line - if so, check each
- if command in MULTI_LINE_PARAM and "," in argument:
- arguments = []
- for entry in argument.split(","):
- arguments.append(entry.strip())
- else:
- arguments = [argument]
-
- for entry in arguments:
- if not entry in actualValues:
- self.corrections[lineNumber + 1] = ", ".join(actualValues)
- except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed):
- if logErrors: log.log(log.WARN, "Unable to validate line %i of the torrc: %s" % (lineNumber + 1, lineText))
-
- # logs issues that arose
- if self.irrelevantLines and logErrors:
- if len(self.irrelevantLines) > 1: first, second, third = "Entries", "are", ", including lines"
- else: first, second, third = "Entry", "is", " on line"
- baseMsg = "%s in your torrc %s ignored due to duplication%s" % (first, second, third)
+ if len(irrelevantLines) > 1: first, second, third = "Entries", "are", ", including lines"
+ else: first, second, third = "Entry", "is", " on line"
+ msgStart = "%s in your torrc %s ignored due to duplication%s" % (first, second, third)
+ msgLines = ", ".join([str(val + 1) for val in irrelevantLines])
+ msg = "%s: %s (highlighted in blue)" % (msgStart, msgLines)
+ log.log(self._config["log.torrcValidation.duplicateEntries"], msg)
- log.log(log.NOTICE, "%s: %s (highlighted in blue)" % (baseMsg, ", ".join([str(val) for val in self.irrelevantLines])))
-
- if self.corrections and logErrors:
- log.log(log.WARN, "Tor's state differs from loaded torrc")
- except IOError, exc:
- resetSuccessful = False
- self.confContents = ["### Unable to load torrc ###"]
- if logErrors: log.log(log.WARN, "Unable to load torrc (%s)" % str(exc))
+ if mismatchLines:
+ mismatchLines.sort()
+ msgStart = "Tor's state differs from loaded torrc on line%s" % ("s" if len(mismatchLines) > 1 else "")
+ msgLines = ", ".join([str(val + 1) for val in mismatchLines])
+ msg = "%s: %s" % (msgStart, msgLines)
+ log.log(self._config["log.torrcValidation.torStateDiffers"], msg)
- self.scroll = 0
- return resetSuccessful
+ if self.confContents:
+ # Restricts contents to be displayable characters:
+ # - Tabs print as three spaces. Keeping them as tabs is problematic for
+ # the layout since it's counted as a single character, but occupies
+ # several cells.
+ # - Strips control and unprintable characters.
+ for lineNum in range(len(self.confContents)):
+ lineText = self.confContents[lineNum]
+ lineText = lineText.replace("\t", " ")
+ lineText = "".join([char for char in lineText if curses.ascii.isprint(char)])
+ self.confContents[lineNum] = lineText
+
+ self.redraw(True)
+ self.valsLock.release()
+ return True
def handleKey(self, key):
- if uiTools.isScrollKey(key):
+ self.valsLock.acquire()
+ if uiTools.isScrollKey(key) and self.confContents != None:
pageHeight = self.getPreferredSize()[0] - 1
- contentHeight = len(self.confContents)
- self.scroll = uiTools.getScrollPosition(key, self.scroll, pageHeight, contentHeight)
- elif key == ord('n') or key == ord('N'): self.showLineNum = not self.showLineNum
+ newScroll = uiTools.getScrollPosition(key, self.scroll, pageHeight, self._lastContentHeight)
+
+ if self.scroll != newScroll:
+ self.scroll = newScroll
+ self.redraw(True)
+ elif key == ord('n') or key == ord('N'):
+ self.showLineNum = not self.showLineNum
+ self._lastContentHeightArgs = None
+ self.redraw(True)
elif key == ord('s') or key == ord('S'):
self.stripComments = not self.stripComments
self.scroll = 0
- self.redraw(True)
+ self._lastContentHeightArgs = None
+ self.redraw(True)
+
+ self.valsLock.release()
def draw(self, subwindow, width, height):
- self.addstr(0, 0, "Tor Config (%s):" % self.confLocation, curses.A_STANDOUT)
+ self.valsLock.acquire()
- pageHeight = height - 1
- if self.confContents: numFieldWidth = int(math.log10(len(self.confContents))) + 1
- else: numFieldWidth = 0 # torrc is blank
- lineNum, displayLineNum = self.scroll + 1, 1 # lineNum corresponds to torrc, displayLineNum concerns what's presented
+ # If true, we assume that the cached value in self._lastContentHeight is
+ # still accurate, and stop drawing when there's nothing more to display.
+ # Otherwise the self._lastContentHeight is suspect, and we'll process all
+ # the content to check if it's right (and redraw again with the corrected
+ # height if not).
+ trustLastContentHeight = self._lastContentHeightArgs == (width, height)
- # determine the ending line in the display (prevents us from going to the
- # effort of displaying lines that aren't visible - isn't really a
- # noticeable improvement unless the torrc is bazaarly long)
- if not self.stripComments:
- endingLine = min(len(self.confContents), self.scroll + pageHeight)
- else:
- # checks for the last line of displayable content (ie, non-comment)
- endingLine = self.scroll
- displayedLines = 0 # number of lines of content
- for i in range(self.scroll, len(self.confContents)):
- endingLine += 1
- lineText = self.confContents[i].strip()
-
- if lineText and lineText[0] != "#":
- displayedLines += 1
- if displayedLines == pageHeight: break
+ # draws the top label
+ locationLabel = " (%s)" % self.confLocation if self.confLocation else ""
+ self.addstr(0, 0, "Tor Config%s:" % locationLabel, curses.A_STANDOUT)
- for i in range(self.scroll, endingLine):
- lineText = self.confContents[i].strip()
- skipLine = False # true if we're not presenting line due to stripping
+ # restricts scroll location to valid bounds
+ self.scroll = max(0, min(self.scroll, self._lastContentHeight - height + 1))
+
+ renderedContents = self.confContents
+ if self.confContents == None:
+ renderedContents = ["### Unable to load torrc ###"]
+ elif self.stripComments:
+ renderedContents = torrc.stripComments(self.confContents)
+
+ # offset to make room for the line numbers
+ lineNumOffset = int(math.log10(len(renderedContents))) + 2 if self.showLineNum else 0
+
+ # draws left-hand scroll bar if content's longer than the height
+ scrollOffset = 0
+ if self._config["features.config.showScrollbars"] and self._lastContentHeight > height - 1:
+ scrollOffset = 3
+ self.addScrollBar(self.scroll, self.scroll + height - 1, self._lastContentHeight, 1)
+
+ displayLine = -self.scroll + 1 # line we're drawing on
+
+ for lineNumber in range(0, len(renderedContents)):
+ lineText = renderedContents[lineNumber]
+ lineText = lineText.rstrip() # remove ending whitespace
- command, argument, correction, comment = "", "", "", ""
- commandColor, argumentColor, correctionColor, commentColor = "green", "cyan", "cyan", "white"
+ # blank lines are hidden when stripping comments
+ hideLine = self.stripComments and not lineText
- if not lineText:
- # no text
- if self.stripComments: skipLine = True
- elif lineText[0] == "#":
- # whole line is commented out
- comment = lineText
- if self.stripComments: skipLine = True
+ # splits the line into its component (msg, format) tuples
+ lineComp = {"option": ["", curses.A_BOLD | uiTools.getColor("green")],
+ "argument": ["", curses.A_BOLD | uiTools.getColor("cyan")],
+ "correction": ["", curses.A_BOLD | uiTools.getColor("cyan")],
+ "comment": ["", uiTools.getColor("white")]}
+
+ # parses the comment
+ commentIndex = lineText.find("#")
+ if commentIndex != -1:
+ lineComp["comment"][0] = lineText[commentIndex:]
+ lineText = lineText[:commentIndex]
+
+ # splits the option and argument, preserving any whitespace around them
+ strippedLine = lineText.strip()
+ optionIndex = strippedLine.find(" ")
+ if optionIndex == -1:
+ lineComp["option"][0] = lineText # no argument provided
else:
- # parse out command, argument, and possible comment
- ctlEnd = lineText.find(" ") # end of command
- argEnd = lineText.find("#") # end of argument (start of comment or end of line)
- if argEnd == -1: argEnd = len(lineText)
+ optionText = strippedLine[:optionIndex]
+ optionEnd = lineText.find(optionText) + len(optionText)
+ lineComp["option"][0] = lineText[:optionEnd]
+ lineComp["argument"][0] = lineText[optionEnd:]
+
+ # gets the correction
+ if lineNumber in self.corrections:
+ lineIssue, lineIssueMsg = self.corrections[lineNumber]
- command, argument, comment = lineText[:ctlEnd], lineText[ctlEnd:argEnd], lineText[argEnd:]
- if self.stripComments: comment = ""
-
- # Tabs print as three spaces. Keeping them as tabs is problematic for
- # the layout since it's counted as a single character, but occupies
- # several cells.
- argument = argument.replace("\t", " ")
-
- # changes presentation if value's incorrect or irrelevant
- if lineNum in self.corrections.keys():
- argumentColor = "red"
- correction = " (%s)" % self.corrections[lineNum]
- elif lineNum in self.irrelevantLines:
- commandColor = "blue"
- argumentColor = "blue"
+ if lineIssue == torrc.VAL_DUPLICATE:
+ lineComp["option"][1] = curses.A_BOLD | uiTools.getColor("blue")
+ lineComp["argument"][1] = curses.A_BOLD | uiTools.getColor("blue")
+ elif lineIssue == torrc.VAL_MISMATCH:
+ lineComp["argument"][1] = curses.A_BOLD | uiTools.getColor("red")
+ lineComp["correction"][0] = " (%s)" % lineIssueMsg
- if not skipLine:
- numOffset = 0 # offset for line numbering
- if self.showLineNum:
- self.addstr(displayLineNum, 0, ("%%%ii" % numFieldWidth) % lineNum, curses.A_BOLD | uiTools.getColor("yellow"))
- numOffset = numFieldWidth + 1
+ # draws the line number
+ if self.showLineNum and not hideLine and displayLine < height and displayLine >= 1:
+ lineNumStr = ("%%%ii" % (lineNumOffset - 1)) % (lineNumber + 1)
+ self.addstr(displayLine, scrollOffset, lineNumStr, curses.A_BOLD | uiTools.getColor("yellow"))
+
+ # draws the rest of the components with line wrap
+ cursorLoc, lineOffset = lineNumOffset + scrollOffset, 0
+ maxLinesPerEntry = self._config["features.config.maxLinesPerEntry"]
+ displayQueue = [lineComp[entry] for entry in ("option", "argument", "correction", "comment")]
+
+ while displayQueue:
+ msg, format = displayQueue.pop(0)
+ if hideLine: break
- xLoc = 0
- displayLineNum, xLoc = controller.addstr_wrap(self, displayLineNum, xLoc, command, curses.A_BOLD | uiTools.getColor(commandColor), numOffset)
- displayLineNum, xLoc = controller.addstr_wrap(self, displayLineNum, xLoc, argument, curses.A_BOLD | uiTools.getColor(argumentColor), numOffset)
- displayLineNum, xLoc = controller.addstr_wrap(self, displayLineNum, xLoc, correction, curses.A_BOLD | uiTools.getColor(correctionColor), numOffset)
- displayLineNum, xLoc = controller.addstr_wrap(self, displayLineNum, xLoc, comment, uiTools.getColor(commentColor), numOffset)
+ maxMsgSize, includeBreak = width - cursorLoc, False
+ if len(msg) >= maxMsgSize:
+ # message is too long - break it up
+ includeBreak = True
+ if lineOffset == maxLinesPerEntry - 1:
+ msg = uiTools.cropStr(msg, maxMsgSize)
+ else:
+ msg, remainder = uiTools.cropStr(msg, maxMsgSize, 4, 4, uiTools.END_WITH_HYPHEN, True)
+ displayQueue.insert(0, (remainder.strip(), format))
- displayLineNum += 1
+ drawLine = displayLine + lineOffset
+ if msg and drawLine < height and drawLine >= 1:
+ self.addstr(drawLine, cursorLoc, msg, format)
+
+ cursorLoc += len(msg)
+ if includeBreak or not displayQueue:
+ lineOffset += 1
+ cursorLoc = lineNumOffset + scrollOffset
- lineNum += 1
+ displayLine += lineOffset
+
+ if trustLastContentHeight and displayLine >= height: break
+
+ if not trustLastContentHeight:
+ self._lastContentHeightArgs = (width, height)
+ newContentHeight = displayLine + self.scroll - 1
+
+ if self._lastContentHeight != newContentHeight:
+ self._lastContentHeight = newContentHeight
+ self.redraw(True)
+
+ self.valsLock.release()
+
+ def redraw(self, forceRedraw=False, block=False):
+ panel.Panel.redraw(self, forceRedraw, block)
Modified: arm/trunk/src/interface/controller.py
===================================================================
--- arm/trunk/src/interface/controller.py 2010-10-21 06:37:31 UTC (rev 23659)
+++ arm/trunk/src/interface/controller.py 2010-10-22 05:21:29 UTC (rev 23660)
@@ -344,16 +344,16 @@
# attempts to determine tor's current pid (left as None if unresolveable, logging an error later)
torPid = torTools.getConn().getMyPid()
- try:
- confLocation = conn.get_info("config-file")["config-file"]
- if confLocation[0] != "/":
- # relative path - attempt to add process pwd
- try:
- results = sysTools.call("pwdx %s" % torPid)
- if len(results) == 1 and len(results[0].split()) == 2: confLocation = "%s/%s" % (results[0].split()[1], confLocation)
- except IOError: pass # pwdx call failed
- except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed):
- confLocation = ""
+ #try:
+ # confLocation = conn.get_info("config-file")["config-file"]
+ # if confLocation[0] != "/":
+ # # relative path - attempt to add process pwd
+ # try:
+ # results = sysTools.call("pwdx %s" % torPid)
+ # if len(results) == 1 and len(results[0].split()) == 2: confLocation = "%s/%s" % (results[0].split()[1], confLocation)
+ # except IOError: pass # pwdx call failed
+ #except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed):
+ # confLocation = ""
# minor refinements for connection resolver
if not isBlindMode:
@@ -378,7 +378,7 @@
panels["conn"] = connPanel.ConnPanel(stdscr, conn, isBlindMode)
panels["control"] = ControlPanel(stdscr, isBlindMode)
- panels["torrc"] = confPanel.ConfPanel(stdscr, confLocation, conn)
+ panels["torrc"] = confPanel.ConfPanel(stdscr, config)
# provides error if pid coulnd't be determined (hopefully shouldn't happen...)
if not torPid: log.log(log.WARN, "Unable to resolve tor pid, abandoning connection listing")
@@ -448,7 +448,7 @@
# provides notice about any unused config keys
for key in config.getUnusedKeys():
- log.log(CONFIG["log.configEntryUndefined"], "unrecognized configuration entry: %s" % key)
+ log.log(CONFIG["log.configEntryUndefined"], "unused configuration entry: %s" % key)
lastPerformanceLog = 0 # ensures we don't do performance logging too frequently
redrawStartTime = time.time()
@@ -531,8 +531,8 @@
for panelKey in (PAGE_S + PAGES[page]):
# redrawing popup can result in display flicker when it should be hidden
if panelKey != "popup":
- if panelKey in ("header", "graph", "log"):
- # revised panel (handles its own content refreshing)
+ if panelKey in ("header", "graph", "log", "torrc"):
+ # revised panel (manages its own content refreshing)
panels[panelKey].redraw()
else:
panels[panelKey].redraw(True)
Modified: arm/trunk/src/interface/graphing/bandwidthStats.py
===================================================================
--- arm/trunk/src/interface/graphing/bandwidthStats.py 2010-10-21 06:37:31 UTC (rev 23659)
+++ arm/trunk/src/interface/graphing/bandwidthStats.py 2010-10-22 05:21:29 UTC (rev 23660)
@@ -37,8 +37,7 @@
self._config = dict(DEFAULT_CONFIG)
if config:
- config.update(self._config)
- self._config["features.graph.bw.accounting.rate"] = max(1, self._config["features.graph.bw.accounting.rate"])
+ config.update(self._config, {"features.graph.bw.accounting.rate": 1})
# accounting data (set by _updateAccountingInfo method)
self.accountingLastUpdated = 0
Modified: arm/trunk/src/interface/graphing/graphPanel.py
===================================================================
--- arm/trunk/src/interface/graphing/graphPanel.py 2010-10-21 06:37:31 UTC (rev 23659)
+++ arm/trunk/src/interface/graphing/graphPanel.py 2010-10-22 05:21:29 UTC (rev 23660)
@@ -48,11 +48,11 @@
"features.graph.showIntermediateBounds": True}
def loadConfig(config):
- config.update(CONFIG)
- CONFIG["features.graph.height"] = max(MIN_GRAPH_HEIGHT, CONFIG["features.graph.height"])
- CONFIG["features.graph.maxWidth"] = max(1, CONFIG["features.graph.maxWidth"])
- CONFIG["features.graph.interval"] = min(len(UPDATE_INTERVALS) - 1, max(0, CONFIG["features.graph.interval"]))
- CONFIG["features.graph.bound"] = min(2, max(0, CONFIG["features.graph.bound"]))
+ config.update(CONFIG, {
+ "features.graph.height": MIN_GRAPH_HEIGHT,
+ "features.graph.maxWidth": 1,
+ "features.graph.interval": (0, len(UPDATE_INTERVALS) - 1),
+ "features.graph.bound": (0, 2)})
class GraphStats(TorCtl.PostEventListener):
"""
Modified: arm/trunk/src/interface/headerPanel.py
===================================================================
--- arm/trunk/src/interface/headerPanel.py 2010-10-21 06:37:31 UTC (rev 23659)
+++ arm/trunk/src/interface/headerPanel.py 2010-10-22 05:21:29 UTC (rev 23660)
@@ -62,8 +62,7 @@
self._config = dict(DEFAULT_CONFIG)
if config:
- config.update(self._config)
- self._config["queries.ps.rate"] = max(self._config["queries.ps.rate"], 1)
+ config.update(self._config, {"queries.ps.rate": 1})
self.vals = {}
self.valsLock = threading.RLock()
Modified: arm/trunk/src/interface/logPanel.py
===================================================================
--- arm/trunk/src/interface/logPanel.py 2010-10-21 06:37:31 UTC (rev 23659)
+++ arm/trunk/src/interface/logPanel.py 2010-10-22 05:21:29 UTC (rev 23660)
@@ -168,10 +168,10 @@
for confKey in armConf.getKeys():
if confKey.startswith("msg."):
eventType = confKey[4:].upper()
- messages = armConf.get(confKey)
+ messages = armConf.get(confKey, [])
COMMON_LOG_MESSAGES[eventType] = messages
-def getLogFileEntries(runlevels, readLimit = None, addLimit = None):
+def getLogFileEntries(runlevels, readLimit = None, addLimit = None, config = None):
"""
Parses tor's log file for past events matching the given runlevels, providing
a list of log entries (ordered newest to oldest). Limiting the number of read
@@ -182,11 +182,15 @@
runlevels - event types (DEBUG - ERR) to be returned
readLimit - max lines of the log file that'll be read (unlimited if None)
addLimit - maximum entries to provide back (unlimited if None)
+ config - configuration parameters related to this panel, uses defaults
+ if left as None
"""
startTime = time.time()
if not runlevels: return []
+ if not config: config = DEFAULT_CONFIG
+
# checks tor's configuration for the log file's location (if any exists)
loggingTypes, loggingLocation = None, None
for loggingEntry in torTools.getConn().getOption("Log", [], True):
@@ -236,7 +240,7 @@
logFile.close()
except IOError:
msg = "Unable to read tor's log file: %s" % loggingLocation
- log.log(DEFAULT_CONFIG["log.logPanel.prepopulateFailed"], msg)
+ log.log(config["log.logPanel.prepopulateFailed"], msg)
if not lines: return []
@@ -278,7 +282,7 @@
if addLimit: loggedEvents = loggedEvents[:addLimit]
msg = "Read %i entries from tor's log file: %s (read limit: %i, runtime: %0.3f)" % (len(loggedEvents), loggingLocation, readLimit, time.time() - startTime)
- log.log(DEFAULT_CONFIG["log.logPanel.prepopulateSuccess"], msg)
+ log.log(config["log.logPanel.prepopulateSuccess"], msg)
return loggedEvents
def getDaybreaks(events, ignoreTimeForCache = False):
@@ -496,17 +500,16 @@
def __init__(self, stdscr, loggedEvents, config=None):
panel.Panel.__init__(self, stdscr, "log", 0)
threading.Thread.__init__(self)
+ self.setDaemon(True)
self._config = dict(DEFAULT_CONFIG)
if config:
- config.update(self._config)
-
- # ensures prepopulation and cache sizes are sane
- self._config["features.log.maxLinesPerEntry"] = max(self._config["features.log.maxLinesPerEntry"], 1)
- self._config["features.log.prepopulateReadLimit"] = max(self._config["features.log.prepopulateReadLimit"], 0)
- self._config["features.log.maxRefreshRate"] = max(self._config["features.log.maxRefreshRate"], 10)
- self._config["cache.logPanel.size"] = max(self._config["cache.logPanel.size"], 50)
+ config.update(self._config, {
+ "features.log.maxLinesPerEntry": 1,
+ "features.log.prepopulateReadLimit": 0,
+ "features.log.maxRefreshRate": 10,
+ "cache.logPanel.size": 50})
# collapses duplicate log entries if false, showing only the most recent
self.showDuplicates = self._config["features.log.showDuplicateEntries"]
@@ -543,7 +546,7 @@
setRunlevels = list(set.intersection(set(self.loggedEvents), set(RUNLEVELS)))
readLimit = self._config["features.log.prepopulateReadLimit"]
addLimit = self._config["cache.logPanel.size"]
- torEventBacklog = getLogFileEntries(setRunlevels, readLimit, addLimit)
+ torEventBacklog = getLogFileEntries(setRunlevels, readLimit, addLimit, self._config)
# adds arm listener and fetches past events
log.LOG_LOCK.acquire()
@@ -838,7 +841,7 @@
if lineOffset == maxEntriesPerLine: break
maxMsgSize = width - cursorLoc
- if len(msg) >= maxMsgSize:
+ if len(msg) > maxMsgSize:
# message is too long - break it up
if lineOffset == maxEntriesPerLine - 1:
msg = uiTools.cropStr(msg, maxMsgSize)
@@ -1047,6 +1050,8 @@
- grown beyond the cache limit
- outlived the configured log duration
+ Argument:
+ eventListing - listing of log entries
"""
cacheSize = self._config["cache.logPanel.size"]
Copied: arm/trunk/src/settings.cfg (from rev 23436, arm/trunk/src/armrc.defaults)
===================================================================
--- arm/trunk/src/settings.cfg (rev 0)
+++ arm/trunk/src/settings.cfg 2010-10-22 05:21:29 UTC (rev 23660)
@@ -0,0 +1,165 @@
+# Snippets from common log messages
+# These are static bits of log messages, used to determine when entries with
+# dynamic content (hostnames, numbers, etc) are the same. If this matches the
+# start of both messages then the entries are flagged as duplicates. If the
+# entry begins with an asterisk (*) then it checks if the substrings exist
+# anywhere in the messages.
+#
+# Examples for the complete messages:
+# [BW] READ: 0, WRITTEN: 0
+# [DEBUG] connection_handle_write(): After TLS write of 512: 0 read, 586 written
+# [DEBUG] flush_chunk_tls(): flushed 512 bytes, 0 ready to flush, 0 remain.
+# [DEBUG] conn_read_callback(): socket 7 wants to read.
+# [DEBUG] conn_write_callback(): socket 51 wants to write.
+# [DEBUG] connection_remove(): removing socket -1 (type OR), n_conns now 50
+# [DEBUG] connection_or_process_cells_from_inbuf(): 7: starting, inbuf_datalen
+# 0 (0 pending in tls object).
+# [DEBUG] connection_read_to_buf(): 38: starting, inbuf_datalen 0 (0 pending in
+# tls object). at_most 12800.
+# [DEBUG] connection_read_to_buf(): TLS connection closed on read. Closing.
+# (Nickname moria1, address 128.31.0.34)
+# [INFO] run_connection_housekeeping(): Expiring non-open OR connection to fd
+# 16 (79.193.61.171:443).
+# [INFO] rep_hist_downrate_old_runs(): Discounting all old stability info by a
+# factor of 0.950000
+# [NOTICE] We stalled too much while trying to write 150 bytes to address
+# [scrubbed]. If this happens a lot, either something is wrong with
+# your network connection, or something is wrong with theirs. (fd 238,
+# type Directory, state 1, marked at main.c:702).
+# [NOTICE] I learned some more directory information, but not enough to build a
+# circuit: We have only 469/2027 usable descriptors.
+# [NOTICE] Attempt by %s to open a stream from unknown relay. Closing.
+# [WARN] You specified a server "Amunet8" by name, but this name is not
+# registered
+# [WARN] I have no descriptor for the router named "Amunet8" in my declared
+# family; I'll use the nickname as is, but this may confuse clients.
+# [WARN] Problem bootstrapping. Stuck at 80%: Connecting to the Tor network.
+# (Network is unreachable; NOROUTE; count 47; recommendation warn)
+# [WARN] 4 unknown, 1 missing key, 3 good, 0 bad, 1 no signature, 4 required
+# [ARM_DEBUG] refresh rate: 0.001 seconds
+# [ARM_DEBUG] system call: ps -p 2354 -o %cpu,rss,%mem,etime (runtime: 0.02)
+# [ARM_DEBUG] system call: netstat -npt | grep 2354/tor (runtime: 0.02)
+# [ARM_DEBUG] recreating panel 'graph' with the dimensions of 14/124
+# [ARM_DEBUG] redrawing the log panel with the corrected content height (estimat was off by 4)
+# [ARM_DEBUG] GETINFO accounting/bytes-left (runtime: 0.0006)
+
+msg.BW READ:
+msg.DEBUG connection_handle_write(): After TLS write of
+msg.DEBUG flush_chunk_tls(): flushed
+msg.DEBUG conn_read_callback(): socket
+msg.DEBUG conn_write_callback(): socket
+msg.DEBUG connection_remove(): removing socket
+msg.DEBUG connection_or_process_cells_from_inbuf():
+msg.DEBUG *pending in tls object). at_most
+msg.DEBUG connection_read_to_buf(): TLS connection closed on read. Closing.
+msg.INFO run_connection_housekeeping(): Expiring
+msg.INFO rep_hist_downrate_old_runs(): Discounting all old stability info by a factor of
+msg.NOTICE We stalled too much while trying to write
+msg.NOTICE I learned some more directory information, but not enough to build a circuit
+msg.NOTICE Attempt by
+msg.WARN You specified a server
+msg.WARN I have no descriptor for the router named
+msg.WARN Problem bootstrapping. Stuck at
+msg.WARN *missing key,
+msg.ARM_DEBUG refresh rate:
+msg.ARM_DEBUG system call: ps
+msg.ARM_DEBUG system call: netstat
+msg.ARM_DEBUG recreating panel '
+msg.ARM_DEBUG redrawing the log panel with the corrected content height (
+msg.ARM_DEBUG GETINFO accounting/bytes
+msg.ARM_DEBUG GETINFO accounting/bytes-left
+msg.ARM_DEBUG GETINFO accounting/interval-end
+msg.ARM_DEBUG GETINFO accounting/hibernating
+
+# some config options are fetched via special values
+torrc.map HiddenServiceDir => HiddenServiceOptions
+torrc.map HiddenServicePort => HiddenServiceOptions
+torrc.map HiddenServiceVersion => HiddenServiceOptions
+torrc.map HiddenServiceAuthorizeClient => HiddenServiceOptions
+
+# torrc parameters that can be defined multiple times without overwriting
+# from src/or/config.c (entries with LINELIST or LINELIST_S)
+# last updated for tor version 0.2.1.19
+torrc.multiline AlternateBridgeAuthority
+torrc.multiline AlternateDirAuthority
+torrc.multiline AlternateHSAuthority
+torrc.multiline AuthDirBadDir
+torrc.multiline AuthDirBadExit
+torrc.multiline AuthDirInvalid
+torrc.multiline AuthDirReject
+torrc.multiline Bridge
+torrc.multiline ControlListenAddress
+torrc.multiline ControlSocket
+torrc.multiline DirListenAddress
+torrc.multiline DirPolicy
+torrc.multiline DirServer
+torrc.multiline DNSListenAddress
+torrc.multiline ExitPolicy
+torrc.multiline HashedControlPassword
+torrc.multiline HiddenServiceDir
+torrc.multiline HiddenServiceOptions
+torrc.multiline HiddenServicePort
+torrc.multiline HiddenServiceVersion
+torrc.multiline HiddenServiceAuthorizeClient
+torrc.multiline HidServAuth
+torrc.multiline Log
+torrc.multiline MapAddress
+torrc.multiline NatdListenAddress
+torrc.multiline NodeFamily
+torrc.multiline ORListenAddress
+torrc.multiline ReachableAddresses
+torrc.multiline ReachableDirAddresses
+torrc.multiline ReachableORAddresses
+torrc.multiline RecommendedVersions
+torrc.multiline RecommendedClientVersions
+torrc.multiline RecommendedServerVersions
+torrc.multiline SocksListenAddress
+torrc.multiline SocksPolicy
+torrc.multiline TransListenAddress
+torrc.multiline __HashedControlSessionPassword
+
+# valid torrc aliases from the _option_abbrevs struct of src/or/config.c
+# These couldn't be requested via GETCONF (in 0.2.1.19), but I think this has
+# been fixed. Discussion is in:
+# https://trac.torproject.org/projects/tor/ticket/1802
+#
+# TODO: This workaround should be dropped after a few releases.
+torrc.alias l => Log
+torrc.alias AllowUnverifiedNodes => AllowInvalidNodes
+torrc.alias AutomapHostSuffixes => AutomapHostsSuffixes
+torrc.alias AutomapHostOnResolve => AutomapHostsOnResolve
+torrc.alias BandwidthRateBytes => BandwidthRate
+torrc.alias BandwidthBurstBytes => BandwidthBurst
+torrc.alias DirFetchPostPeriod => StatusFetchPeriod
+torrc.alias MaxConn => ConnLimit
+torrc.alias ORBindAddress => ORListenAddress
+torrc.alias DirBindAddress => DirListenAddress
+torrc.alias SocksBindAddress => SocksListenAddress
+torrc.alias UseHelperNodes => UseEntryGuards
+torrc.alias NumHelperNodes => NumEntryGuards
+torrc.alias UseEntryNodes => UseEntryGuards
+torrc.alias NumEntryNodes => NumEntryGuards
+torrc.alias ResolvConf => ServerDNSResolvConfFile
+torrc.alias SearchDomains => ServerDNSSearchDomains
+torrc.alias ServerDNSAllowBrokenResolvConf => ServerDNSAllowBrokenConfig
+torrc.alias PreferTunnelledDirConns => PreferTunneledDirConns
+torrc.alias BridgeAuthoritativeDirectory => BridgeAuthoritativeDir
+torrc.alias StrictEntryNodes => StrictNodes
+torrc.alias StrictExitNodes => StrictNodes
+
+# using the following entry is problematic, despite being among the
+# __option_abbrevs mappings
+#torrc.alias HashedControlPassword => __HashedControlSessionPassword
+
+# size and time modifiers allowed by config.c
+torrc.label.size.b b, byte, bytes
+torrc.label.size.kb kb, kbyte, kbytes, kilobyte, kilobytes
+torrc.label.size.mb m, mb, mbyte, mbytes, megabyte, megabytes
+torrc.label.size.gb gb, gbyte, gbytes, gigabyte, gigabytes
+torrc.label.size.tb tb, terabyte, terabytes
+torrc.label.time.sec second, seconds
+torrc.label.time.min minute, minutes
+torrc.label.time.hour hour, hours
+torrc.label.time.day day, days
+torrc.label.time.week week, weeks
+
Modified: arm/trunk/src/starter.py
===================================================================
--- arm/trunk/src/starter.py 2010-10-21 06:37:31 UTC (rev 23659)
+++ arm/trunk/src/starter.py 2010-10-22 05:21:29 UTC (rev 23660)
@@ -19,6 +19,7 @@
import util.log
import util.panel
import util.sysTools
+import util.torrc
import util.torTools
import util.uiTools
import TorCtl.TorCtl
@@ -114,32 +115,32 @@
print HELP_MSG
sys.exit()
- # attempts to load user's custom configuration, using defaults if not found
- if not os.path.exists(configPath):
- msg = "No configuration found at '%s', using defaults" % configPath
- util.log.log(util.log.NOTICE, msg)
- configPath = "%s/armrc.defaults" % os.path.dirname(sys.argv[0])
-
config = util.conf.getConfig("arm")
- config.path = configPath
+ # attempts to fetch attributes for parsing tor's logs, configuration, etc
+ try: config.load("%s/settings.cfg" % os.path.dirname(sys.argv[0]))
+ except IOError, exc:
+ msg = "Failed to load the parsing configuration. This will be problematic for a few things like torrc validation and log duplication detection (%s)" % str(exc)
+ util.log.log(util.log.WARN, msg)
+
+ # loads user's personal armrc if available
if os.path.exists(configPath):
try:
- config.load()
+ config.load(configPath)
# revises defaults to match user's configuration
config.update(DEFAULTS)
# loads user preferences for utilities
- for utilModule in (util.conf, util.connections, util.hostnames, util.log, util.panel, util.sysTools, util.torTools, util.uiTools):
+ for utilModule in (util.conf, util.connections, util.hostnames, util.log, util.panel, util.sysTools, util.torrc, util.torTools, util.uiTools):
utilModule.loadConfig(config)
except IOError, exc:
msg = "Failed to load configuration (using defaults): \"%s\"" % str(exc)
util.log.log(util.log.WARN, msg)
else:
- # no local copy of the armrc defaults, so fall back to values in the source
- msg = "defaults file not found, falling back (log duplicate detection will be mostly nonfunctional)"
- util.log.log(util.log.WARN, msg)
+ # no armrc found, falling back to the defaults in the source
+ msg = "No configuration found at '%s', using defaults" % configPath
+ util.log.log(util.log.NOTICE, msg)
# overwrites undefined parameters with defaults
for key in param.keys():
Modified: arm/trunk/src/util/conf.py
===================================================================
--- arm/trunk/src/util/conf.py 2010-10-21 06:37:31 UTC (rev 23659)
+++ arm/trunk/src/util/conf.py 2010-10-22 05:21:29 UTC (rev 23660)
@@ -23,9 +23,6 @@
CONFIG = {"log.configEntryNotFound": None,
"log.configEntryTypeError": log.INFO}
-# key prefixes that can contain multiple values
-LIST_KEYS = ["msg."]
-
def loadConfig(config):
config.update(CONFIG)
@@ -42,21 +39,6 @@
if not handle in CONFS: CONFS[handle] = Config()
return CONFS[handle]
-def isListKey(configKey):
- """
- Provides true if the given configuration key can have multiple values (being
- a list), false otherwise.
-
- Arguments:
- configKey - configuration key to check
- """
-
- for listKeyPrefix in LIST_KEYS:
- if configKey.startswith(listKeyPrefix):
- return True
-
- return False
-
class Config():
"""
Handler for easily working with custom configurations, providing persistence
@@ -73,27 +55,28 @@
Creates a new configuration instance.
"""
- self.path = None # path to the associated configuration file
self.contents = {} # configuration key/value pairs
self.contentsLock = threading.RLock()
self.requestedKeys = set()
self.rawContents = [] # raw contents read from configuration file
- def getValue(self, key, default=None):
+ def getValue(self, key, default=None, multiple=False):
"""
- This provides the currently value associated with a given key, and a list
- of values if isListKey(key) is true. If no such key exists then this
- provides the default.
+ This provides the currently value associated with a given key. If no such
+ key exists then this provides the default.
Arguments:
- key - config setting to be fetched
- default - value provided if no such key exists
+ key - config setting to be fetched
+ default - value provided if no such key exists
+ multiple - provides back a list of all values if true, otherwise this
+ returns the last loaded configuration value
"""
self.contentsLock.acquire()
if key in self.contents:
val = self.contents[key]
+ if not multiple: val = val[-1]
self.requestedKeys.add(key)
else:
msg = "config entry '%s' not found, defaulting to '%s'" % (key, str(default))
@@ -104,31 +87,30 @@
return val
- def get(self, key, default=None, minValue=0, maxValue=None):
+ def get(self, key, default=None):
"""
Fetches the given configuration, using the key and default value to hint
the type it should be. Recognized types are:
+ - logging runlevel if key starts with "log."
- boolean if default is a boolean (valid values are 'true' and 'false',
anything else provides the default)
- integer or float if default is a number (provides default if fails to
cast)
- - logging runlevel if key starts with "log."
- - list if isListKey(key) is true
+ - list of all defined values default is a list
+ - mapping of all defined values (key/value split via "=>") if the default
+ is a dict
Arguments:
key - config setting to be fetched
default - value provided if no such key exists
- minValue - if set and default value is numeric then uses this constraint
- maxValue - if set and default value is numeric then uses this constraint
"""
callDefault = log.runlevelToStr(default) if key.startswith("log.") else default
- val = self.getValue(key, callDefault)
+ isMultivalue = isinstance(default, list) or isinstance(default, dict)
+ val = self.getValue(key, callDefault, isMultivalue)
if val == default: return val
- if isinstance(val, list):
- pass
- elif key.startswith("log."):
+ if key.startswith("log."):
if val.lower() in ("none", "debug", "info", "notice", "warn", "err"):
val = log.strToRunlevel(val)
else:
@@ -143,37 +125,53 @@
log.log(CONFIG["log.configEntryTypeError"], msg)
val = default
elif isinstance(default, int):
- try:
- val = int(val)
- if minValue: val = max(val, minValue)
- if maxValue: val = min(val, maxValue)
+ try: val = int(val)
except ValueError:
msg = "config entry '%s' is expected to be an integer, defaulting to '%i'" % (key, default)
log.log(CONFIG["log.configEntryTypeError"], msg)
val = default
elif isinstance(default, float):
- try:
- val = float(val)
- if minValue: val = max(val, minValue)
- if maxValue: val = min(val, maxValue)
+ try: val = float(val)
except ValueError:
msg = "config entry '%s' is expected to be a float, defaulting to '%f'" % (key, default)
log.log(CONFIG["log.configEntryTypeError"], msg)
val = default
+ elif isinstance(default, list):
+ pass # nothing special to do (already a list)
+ elif isinstance(default, dict):
+ valMap = {}
+ for entry in val:
+ if "=>" in entry:
+ entryKey, entryVal = entry.split("=>", 1)
+ valMap[entryKey.strip()] = entryVal.strip()
+ else:
+ msg = "ignoring invalid %s config entry (expected a mapping, but \"%s\" was missing \"=>\")" % (key, entry)
+ log.log(CONFIG["log.configEntryTypeError"], msg)
+ val = valMap
return val
- def update(self, confMappings):
+ def update(self, confMappings, limits = {}):
"""
Revises a set of key/value mappings to reflect the current configuration.
Undefined values are left with their current values.
Arguments:
confMappings - configuration key/value mappings to be revised
+ limits - mappings of limits on numeric values, expected to be of
+ the form "configKey -> min" or "configKey -> (min, max)"
"""
for entry in confMappings.keys():
- confMappings[entry] = self.get(entry, confMappings[entry])
+ val = self.get(entry, confMappings[entry])
+
+ if entry in limits and (isinstance(val, int) or isinstance(val, float)):
+ if isinstance(limits[entry], tuple):
+ val = max(val, limits[entry][0])
+ val = min(val, limits[entry][1])
+ else: val = max(val, limits[entry])
+
+ confMappings[entry] = val
def getKeys(self):
"""
@@ -211,45 +209,38 @@
self.contents.clear()
self.contentsLock.release()
- def load(self):
+ def load(self, path):
"""
- Reads in the contents of the currently set configuration file (appending
- any results to the current configuration). If the file's empty or doesn't
- exist then this doesn't do anything.
+ Reads in the contents of the given path, adding its configuration values
+ and overwriting any that already exist. If the file's empty then this
+ doesn't do anything. Other issues (like having insufficient permissions or
+ if the file doesn't exist) result in an IOError.
- Other issues (like having an unset path or insufficient permissions) result
- in an IOError.
+ Arguments:
+ path - file path to be loaded
"""
- if not self.path: raise IOError("unable to load (config path undefined)")
+ configFile = open(path, "r")
+ self.rawContents = configFile.readlines()
+ configFile.close()
- if os.path.exists(self.path):
- configFile = open(self.path, "r")
- self.rawContents = configFile.readlines()
- configFile.close()
+ self.contentsLock.acquire()
+
+ for line in self.rawContents:
+ # strips any commenting or excess whitespace
+ commentStart = line.find("#")
+ if commentStart != -1: line = line[:commentStart]
+ line = line.strip()
- self.contentsLock.acquire()
-
- for line in self.rawContents:
- # strips any commenting or excess whitespace
- commentStart = line.find("#")
- if commentStart != -1: line = line[:commentStart]
- line = line.strip()
+ # parse the key/value pair
+ if line and " " in line:
+ key, value = line.split(" ", 1)
+ value = value.strip()
- # parse the key/value pair
- if line:
- key, value = line, ""
-
- # gets the key/value pair (no value was given if there isn't a space)
- if " " in line: key, value = line.split(" ", 1)
-
- if isListKey(key):
- if key in self.contents: self.contents[key].append(value)
- else: self.contents[key] = [value]
- else:
- self.contents[key] = value
-
- self.contentsLock.release()
+ if key in self.contents: self.contents[key].append(value)
+ else: self.contents[key] = [value]
+
+ self.contentsLock.release()
def save(self, saveBackup=True):
"""
Modified: arm/trunk/src/util/hostnames.py
===================================================================
--- arm/trunk/src/util/hostnames.py 2010-10-21 06:37:31 UTC (rev 23659)
+++ arm/trunk/src/util/hostnames.py 2010-10-22 05:21:29 UTC (rev 23660)
@@ -47,12 +47,12 @@
"log.hostnameCacheTrimmed": log.INFO}
def loadConfig(config):
- config.update(CONFIG)
+ config.update(CONFIG, {
+ "queries.hostnames.poolSize": 1,
+ "cache.hostnames.size": 100,
+ "cache.hostnames.trimSize": 10})
- # ensures sane config values
- CONFIG["queries.hostnames.poolSize"] = max(1, CONFIG["queries.hostnames.poolSize"])
- CONFIG["cache.hostnames.size"] = max(100, CONFIG["cache.hostnames.size"])
- CONFIG["cache.hostnames.trimSize"] = max(10, min(CONFIG["cache.hostnames.trimSize"], CONFIG["cache.hostnames.size"] / 2))
+ CONFIG["cache.hostnames.trimSize"] = min(CONFIG["cache.hostnames.trimSize"], CONFIG["cache.hostnames.size"] / 2)
def start():
"""
Modified: arm/trunk/src/util/log.py
===================================================================
--- arm/trunk/src/util/log.py 2010-10-21 06:37:31 UTC (rev 23659)
+++ arm/trunk/src/util/log.py 2010-10-22 05:21:29 UTC (rev 23660)
@@ -28,11 +28,11 @@
"cache.armLog.trimSize": 200}
def loadConfig(config):
- config.update(CONFIG)
+ config.update(CONFIG, {
+ "cache.armLog.size": 10,
+ "cache.armLog.trimSize": 5})
- # ensures sane config values
- CONFIG["cache.armLog.size"] = max(10, CONFIG["cache.armLog.size"])
- CONFIG["cache.armLog.trimSize"] = max(5, min(CONFIG["cache.armLog.trimSize"], CONFIG["cache.armLog.size"] / 2))
+ CONFIG["cache.armLog.trimSize"] = min(CONFIG["cache.armLog.trimSize"], CONFIG["cache.armLog.size"] / 2)
def strToRunlevel(runlevelStr):
"""
Modified: arm/trunk/src/util/panel.py
===================================================================
--- arm/trunk/src/util/panel.py 2010-10-21 06:37:31 UTC (rev 23659)
+++ arm/trunk/src/util/panel.py 2010-10-22 05:21:29 UTC (rev 23660)
@@ -2,7 +2,6 @@
Wrapper for safely working with curses subwindows.
"""
-import sys
import traceback
import curses
from threading import RLock
@@ -224,15 +223,6 @@
self.win.erase() # clears any old contents
self.draw(self.win, self.maxX - 1, self.maxY)
self.win.refresh()
- except:
- # without terminating curses continues in a zombie state (requiring a
- # kill signal to quit, and screwing up the terminal)
- # TODO: provide a nicer, general purpose handler for unexpected exceptions
- try:
- tracebackFile = open("/tmp/armTraceback", "w")
- traceback.print_exc(file=tracebackFile)
- finally:
- sys.exit(1)
finally:
CURSES_LOCK.release()
@@ -375,6 +365,10 @@
if top > 0: sliderTop = max(sliderTop, 1)
if bottom != size: sliderTop = min(sliderTop, scrollbarHeight - sliderSize - 2)
+ # avoids a rounding error that causes the scrollbar to be too low when at
+ # the bottom
+ if bottom == size: sliderTop = scrollbarHeight - sliderSize - 1
+
# draws scrollbar slider
for i in range(scrollbarHeight):
if i >= sliderTop and i <= sliderTop + sliderSize:
Modified: arm/trunk/src/util/torTools.py
===================================================================
--- arm/trunk/src/util/torTools.py 2010-10-21 06:37:31 UTC (rev 23659)
+++ arm/trunk/src/util/torTools.py 2010-10-22 05:21:29 UTC (rev 23660)
@@ -319,9 +319,6 @@
if not suppressExc and raisedExc: raise raisedExc
else: return result
- # TODO: This could have client side caching if there were events to indicate
- # SETCONF events. See:
- # https://trac.torproject.org/projects/tor/ticket/1692
def getOption(self, param, default = None, multiple = False, suppressExc = True):
"""
Queries the control port for the given configuration option, providing the
@@ -332,24 +329,66 @@
Arguments:
param - configuration option to be queried
default - result if the query fails and exception's suppressed
- multiple - provides a list of results if true, otherwise this just
- returns the first value
+ multiple - provides a list with all returned values if true, otherwise
+ this just provides the first result
suppressExc - suppresses lookup errors (returning the default) if true,
otherwise this raises the original exception
"""
+ fetchType = "list" if multiple else "str"
+ return self._getOption(param, default, fetchType, suppressExc)
+
+ def getOptionMap(self, param, default = None, suppressExc = True):
+ """
+ Queries the control port for the given configuration option, providing back
+ a mapping of config options to a list of the values returned.
+
+ There's three use cases for GETCONF:
+ - a single value is provided
+ - multiple values are provided for the option queried
+ - a set of options that weren't necessarily requested are returned (for
+ instance querying HiddenServiceOptions gives HiddenServiceDir,
+ HiddenServicePort, etc)
+
+ The vast majority of the options fall into the first two catagories, in
+ which case calling getOption is sufficient. However, for the special
+ options that give a set of values this provides back the full response. As
+ of tor version 0.2.1.25 HiddenServiceOptions was the only option like this.
+
+ Arguments:
+ param - configuration option to be queried
+ default - result if the query fails and exception's suppressed
+ suppressExc - suppresses lookup errors (returning the default) if true,
+ otherwise this raises the original exception
+ """
+
+ return self._getOption(param, default, "map", suppressExc)
+
+ # TODO: This could have client side caching if there were events to indicate
+ # SETCONF events. See:
+ # https://trac.torproject.org/projects/tor/ticket/1692
+ def _getOption(self, param, default, fetchType, suppressExc):
+ if not fetchType in ("str", "list", "map"):
+ msg = "BUG: unrecognized fetchType in torTools._getOption (%s)" % fetchType
+ log.log(log.ERR, msg)
+ return default
+
self.connLock.acquire()
- startTime = time.time()
- result, raisedExc = [], None
+ startTime, raisedExc = time.time(), None
+ result = {} if fetchType == "map" else []
if self.isAlive():
try:
- if multiple:
- for key, value in self.conn.get_option(param):
- if value != None: result.append(value)
- else:
+ if fetchType == "str":
getConfVal = self.conn.get_option(param)[0][1]
if getConfVal != None: result = getConfVal
+ else:
+ for key, value in self.conn.get_option(param):
+ if value != None:
+ if fetchType == "list": result.append(value)
+ elif fetchType == "map":
+ if key in result: result.append(value)
+ else: result[key] = [value]
except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed), exc:
if type(exc) == TorCtl.TorCtlClosed: self.close()
result, raisedExc = default, exc
Added: arm/trunk/src/util/torrc.py
===================================================================
--- arm/trunk/src/util/torrc.py (rev 0)
+++ arm/trunk/src/util/torrc.py 2010-10-22 05:21:29 UTC (rev 23660)
@@ -0,0 +1,217 @@
+"""
+Helper functions for working with tor's configuration file.
+"""
+
+from util import sysTools, torTools, uiTools
+
+CONFIG = {"torrc.map": {},
+ "torrc.multiline": [],
+ "torrc.alias": {},
+ "torrc.label.size.b": [],
+ "torrc.label.size.kb": [],
+ "torrc.label.size.mb": [],
+ "torrc.label.size.gb": [],
+ "torrc.label.size.tb": [],
+ "torrc.label.time.sec": [],
+ "torrc.label.time.min": [],
+ "torrc.label.time.hour": [],
+ "torrc.label.time.day": [],
+ "torrc.label.time.week": []}
+
+# enums and values for numeric torrc entries
+UNRECOGNIZED, SIZE_VALUE, TIME_VALUE = range(1, 4)
+SIZE_MULT = {"b": 1, "kb": 1024, "mb": 1048576, "gb": 1073741824, "tb": 1099511627776}
+TIME_MULT = {"sec": 1, "min": 60, "hour": 3600, "day": 86400, "week": 604800}
+
+# enums for issues found during torrc validation:
+# VAL_DUPLICATE - entry is ignored due to being a duplicate
+# VAL_MISMATCH - the value doesn't match tor's current state
+VAL_DUPLICATE, VAL_MISMATCH = range(1, 3)
+
+# cached results for the stripComments function
+STRIP_COMMENTS_ARG, STRIP_COMMENTS_RESULT = None, None
+
+def loadConfig(config):
+ CONFIG["torrc.map"] = config.get("torrc.map", {})
+ CONFIG["torrc.multiline"] = config.get("torrc.multiline", [])
+ CONFIG["torrc.alias"] = config.get("torrc.alias", {})
+
+ # all the torrc.label.* values are comma separated lists
+ for configKey in CONFIG.keys():
+ if configKey.startswith("torrc.label."):
+ configValues = config.get(configKey, "").split(",")
+ if configValues: CONFIG[configKey] = [val.strip() for val in configValues]
+
+def getConfigLocation():
+ """
+ Provides the location of the torrc, raising an IOError with the reason if the
+ path can't be determined.
+ """
+
+ conn = torTools.getConn()
+ configLocation = conn.getInfo("config-file")
+ if not configLocation: raise IOError("unable to query the torrc location")
+
+ # checks if this is a relative path, needing the tor pwd to be appended
+ if configLocation[0] != "/":
+ torPid = conn.getMyPid()
+ failureMsg = "querying tor's pwd failed because %s"
+ if not torPid: raise IOError(failureMsg % "we couldn't get the pid")
+
+ try:
+ # pwdx results are of the form:
+ # 3799: /home/atagar
+ # 5839: No such process
+ results = sysTools.call("pwdx %s" % torPid)
+ if not results:
+ raise IOError(failureMsg % "pwdx didn't return any results")
+ elif results[0].endswith("No such process"):
+ raise IOError(failureMsg % ("pwdx reported no process for pid " + torPid))
+ elif len(results) != 1 or results.count(" ") != 1:
+ raise IOError(failureMsg % "we got unexpected output from pwdx")
+ else:
+ pwdPath = results[0][results[0].find(" ") + 1:]
+ configLocation = "%s/%s" % (pwdPath, configLocation)
+ except IOError, exc:
+ raise IOError(failureMsg % ("the pwdx call failed: " + str(exc)))
+
+ return torTools.getPathPrefix() + configLocation
+
+def stripComments(contents):
+ """
+ Provides the torrc contents back with comments and extra whitespace stripped.
+
+ Arguments:
+ contents - torrc contents
+ """
+
+ global STRIP_COMMENTS_ARG, STRIP_COMMENTS_RESULT
+
+ if contents == STRIP_COMMENTS_ARG:
+ return list(STRIP_COMMENTS_RESULT)
+
+ strippedContents = []
+ for line in contents:
+ # strips off comment
+ if line and "#" in line:
+ line = line[:line.find("#")]
+
+ strippedContents.append(line.strip())
+
+ STRIP_COMMENTS_ARG = list(contents)
+ STRIP_COMMENTS_RESULT = list(strippedContents)
+ return strippedContents
+
+def validate(contents):
+ """
+ Performs validation on the given torrc contents, providing back a mapping of
+ line numbers to tuples of the (issue, msg) found on them.
+
+ Arguments:
+ contents - torrc contents
+ """
+
+ conn = torTools.getConn()
+ contents = stripComments(contents)
+ issuesFound, seenOptions = {}, []
+ for lineNumber in range(len(contents) - 1, -1, -1):
+ lineText = contents[lineNumber]
+ if not lineText: continue
+
+ lineComp = lineText.split(None, 1)
+ if len(lineComp) == 2: option, value = lineComp
+ else: option, value = lineText, ""
+
+ # most parameters are overwritten if defined multiple times
+ if option in seenOptions and not option in CONFIG["torrc.multiline"]:
+ issuesFound[lineNumber] = (VAL_DUPLICATE, "")
+ continue
+ else: seenOptions.append(option)
+
+ # replace aliases with their recognized representation
+ if option in CONFIG["torrc.alias"]:
+ option = CONFIG["torrc.alias"][option]
+
+ # tor appears to replace tabs with a space, for instance:
+ # "accept\t*:563" is read back as "accept *:563"
+ value = value.replace("\t", " ")
+
+ # parse value if it's a size or time, expanding the units
+ value, valueType = _parseConfValue(value)
+
+ # issues GETCONF to get the values tor's currently configured to use
+ torValues = []
+ if option in CONFIG["torrc.map"]:
+ # special option that's fetched with special values
+ confMappings = conn.getOptionMap(CONFIG["torrc.map"][option], {})
+ if option in confMappings: torValues = confMappings[option]
+ else:
+ torValues = conn.getOption(option, [], True)
+
+ # multiline entries can be comma separated values (for both tor and conf)
+ valueList = [value]
+ if option in CONFIG["torrc.multiline"]:
+ valueList = [val.strip() for val in value.split(",")]
+
+ fetchedValues, torValues = torValues, []
+ for fetchedValue in fetchedValues:
+ for fetchedEntry in fetchedValue.split(","):
+ fetchedEntry = fetchedEntry.strip()
+ if not fetchedEntry in torValues:
+ torValues.append(fetchedEntry)
+
+ for val in valueList:
+ # checks if both the argument and tor's value are empty
+ isBlankMatch = not val and not torValues
+
+ if not isBlankMatch and not val in torValues:
+ # converts corrections to reader friedly size values
+ displayValues = torValues
+ if valueType == SIZE_VALUE:
+ displayValues = [uiTools.getSizeLabel(int(val)) for val in torValues]
+ elif valueType == TIME_VALUE:
+ displayValues = [uiTools.getTimeLabel(int(val)) for val in torValues]
+
+ issuesFound[lineNumber] = (VAL_MISMATCH, ", ".join(displayValues))
+
+ return issuesFound
+
+def _parseConfValue(confArg):
+ """
+ Converts size or time values to their lowest units (bytes or seconds) which
+ is what GETCONF calls provide. The returned is a tuple of the value and unit
+ type.
+
+ Arguments:
+ confArg - torrc argument
+ """
+
+ if confArg.count(" ") == 1:
+ val, unit = confArg.lower().split(" ", 1)
+ if not val.isdigit(): return confArg, UNRECOGNIZED
+ mult, multType = _getUnitType(unit)
+
+ if mult != None:
+ return str(int(val) * mult), multType
+
+ return confArg, UNRECOGNIZED
+
+def _getUnitType(unit):
+ """
+ Provides the type and multiplier for an argument's unit. The multiplier is
+ None if the unit isn't recognized.
+
+ Arguments:
+ unit - string representation of a unit
+ """
+
+ for label in SIZE_MULT:
+ if unit in CONFIG["torrc.label.size." + label]:
+ return SIZE_MULT[label], SIZE_VALUE
+
+ for label in TIME_MULT:
+ if unit in CONFIG["torrc.label.time." + label]:
+ return TIME_MULT[label], TIME_VALUE
+
+ return None, UNRECOGNIZED
+
Modified: arm/trunk/src/util/uiTools.py
===================================================================
--- arm/trunk/src/util/uiTools.py 2010-10-21 06:37:31 UTC (rev 23659)
+++ arm/trunk/src/util/uiTools.py 2010-10-22 05:21:29 UTC (rev 23660)
@@ -28,8 +28,8 @@
SIZE_UNITS_BYTES = [(1125899906842624.0, " PB", " Petabyte"), (1099511627776.0, " TB", " Terabyte"),
(1073741824.0, " GB", " Gigabyte"), (1048576.0, " MB", " Megabyte"),
(1024.0, " KB", " Kilobyte"), (1.0, " B", " Byte")]
-TIME_UNITS = [(86400.0, "d", " day"), (3600.0, "h", " hour"),
- (60.0, "m", " minute"), (1.0, "s", " second")]
+TIME_UNITS = [(86400.0, "d", " day"), (3600.0, "h", " hour"),
+ (60.0, "m", " minute"), (1.0, "s", " second")]
END_WITH_ELLIPSE, END_WITH_HYPHEN = range(1, 3)
SCROLL_KEYS = (curses.KEY_UP, curses.KEY_DOWN, curses.KEY_PPAGE, curses.KEY_NPAGE, curses.KEY_HOME, curses.KEY_END)
@@ -61,7 +61,9 @@
Provides the msg constrained to the given length, truncating on word breaks.
If the last words is long this truncates mid-word with an ellipse. If there
isn't room for even a truncated single word (or one word plus the ellipse if
- including those) then this provides an empty string. Examples:
+ including those) then this provides an empty string. If a cropped string ends
+ with a comma or period then it's stripped (unless we're providing the
+ remainder back). Examples:
cropStr("This is a looooong message", 17)
"This is a looo..."
@@ -86,26 +88,28 @@
cropped portion of the message
"""
- if minWordLen == None: minWordLen = sys.maxint
- minWordLen = max(0, minWordLen)
- minCrop = max(0, minCrop)
-
# checks if there's room for the whole message
if len(msg) <= size:
if getRemainder: return (msg, "")
else: return msg
+ # avoids negative input
+ size = max(0, size)
+ if minWordLen != None: minWordLen = max(0, minWordLen)
+ minCrop = max(0, minCrop)
+
# since we're cropping, the effective space available is less with an
# ellipse, and cropping words requires an extra space for hyphens
if endType == END_WITH_ELLIPSE: size -= 3
- elif endType == END_WITH_HYPHEN: minWordLen += 1
+ elif endType == END_WITH_HYPHEN and minWordLen != None: minWordLen += 1
# checks if there isn't the minimum space needed to include anything
- if size <= minWordLen:
+ lastWordbreak = msg.rfind(" ", 0, size + 1)
+ if (minWordLen != None and size < minWordLen) or (minWordLen == None and lastWordbreak < 1):
if getRemainder: return ("", msg)
else: return ""
- lastWordbreak = msg.rfind(" ", 0, size + 1)
+ if minWordLen == None: minWordLen = sys.maxint
includeCrop = size - lastWordbreak - 1 >= minWordLen
# if there's a max crop size then make sure we're cropping at least that many characters
@@ -122,7 +126,7 @@
else: returnMsg, remainder = msg[:lastWordbreak], msg[lastWordbreak:]
# if this is ending with a comma or period then strip it off
- if returnMsg[-1] in (",", "."): returnMsg = returnMsg[:-1]
+ if not getRemainder and returnMsg[-1] in (",", "."): returnMsg = returnMsg[:-1]
if endType == END_WITH_ELLIPSE: returnMsg += "..."
More information about the tor-commits
mailing list