[or-cvs] r24054: {arm} Arm release 1.4.1 (in arm: release release/src release/src/interface release/src/interface/graphing release/src/util resources resources/build/debian)
Damian Johnson
atagar1 at gmail.com
Sat Jan 8 06:35:08 UTC 2011
Author: atagar
Date: 2011-01-08 06:35:08 +0000 (Sat, 08 Jan 2011)
New Revision: 24054
Added:
arm/release/src/interface/graphing/resourceStats.py
arm/release/src/test.py
arm/release/src/torConfigDesc.txt
arm/release/src/util/procTools.py
Removed:
arm/release/src/interface/graphing/psStats.py
Modified:
arm/release/
arm/release/ChangeLog
arm/release/README
arm/release/TODO
arm/release/arm.1
arm/release/armrc.sample
arm/release/src/__init__.py
arm/release/src/interface/configPanel.py
arm/release/src/interface/connPanel.py
arm/release/src/interface/controller.py
arm/release/src/interface/graphing/__init__.py
arm/release/src/interface/graphing/bandwidthStats.py
arm/release/src/interface/headerPanel.py
arm/release/src/interface/logPanel.py
arm/release/src/settings.cfg
arm/release/src/starter.py
arm/release/src/uninstall
arm/release/src/util/__init__.py
arm/release/src/util/connections.py
arm/release/src/util/log.py
arm/release/src/util/panel.py
arm/release/src/util/sysTools.py
arm/release/src/util/torConfig.py
arm/release/src/util/torTools.py
arm/release/src/util/uiTools.py
arm/release/src/version.py
arm/resources/build/debian/changelog
arm/resources/deb-prep.sh
Log:
Arm release 1.4.1
Property changes on: arm/release
___________________________________________________________________
Modified: svn:mergeinfo
- /arm/trunk:22227-23872
+ /arm/trunk:22227-24053
Modified: arm/release/ChangeLog
===================================================================
--- arm/release/ChangeLog 2011-01-08 06:06:36 UTC (rev 24053)
+++ arm/release/ChangeLog 2011-01-08 06:35:08 UTC (rev 24054)
@@ -1,6 +1,48 @@
CHANGE LOG
-11/27/10 - version 1.4.0
+1/7/11 - version 1.4.1
+Platform specific enhancements including BSD compatibility and vastly improved performance on Linux.
+
+ * added: querying the proc contents when able for tor resource and connection samplings to *greatly* reduce arm's resource usage (many thanks to psutil)
+ * added: vastly improved BSD compatibility, thanks to patches by Fabian Keil
+ o pid resolution via pgrep (all platforms) and sockstat (bsd only)
+ o connection resolution via sockstat (all platforms) and procstat (bsd only)
+ o autodetecting the path prefix for FreeBSD jails
+ * added: displaying summaries of the options on the configuration panel (idea by Sebastian)
+ * added: arm cpu usage to the header panel and logs (with an estimate for system call usages)
+ * added: testing script for checking connection resolution performance, connection dumps, and the glyph demo
+ * added: option to dump arm debug logs (better failsafe option)
+ * change: incrementing the uptime field of the header panel each second
+ * change: centralizing arm resources in ~/.arm (suggested by Sebastian and also thanks to feedback from rransom)
+ * change: using exponential backoff of ps/proc resource resolutions when calls fail or tor isn't running
+ * change: reordered resolvers by order of performance
+ * change: when tor's man page is unavailable falling back to descriptions provided with arm (often the case with tbb)
+ * change: dropping support for graphing of custom ps attributes (feature was never used, kinda pointless, and incompatible with the proc enhancement)
+ * fix: providing proper cpu samplings rather than an average over the life of the process
+ * fix: expanding relative paths for the authentication cookie (mostly a problem for tbb instances)
+ * fix: crashing error when querying hidden service parameters (caught by StrangeCharm, fixed by katmagic and chiiph)
+ * fix: initially built diff differed from rebuilt version since it was missing the pycompat (caught by weasel)
+ * fix: startup script wasn't handling unusual paths, such as with spaces and dashes (caught by weasel)
+ * fix: startup script was forking the final process instead of calling exec (caught by weasel)
+ * fix: log entries weren't being bracketed by date dividers when no scroll bars were visible (caught thanks to twur)
+ * fix: workaround for config-text providing Log entries regardless of if it matches the default (caught thanks to Trystero)
+ * fix: config validation mistakenly thought that Tor stripped spaces from CSV getconf responses (caught by murble)
+ * fix: presenting a text input field with python 2.5 would crash (caught by murble)
+ * fix: reloading torrc contents when a sighup is issued (caught by StrangeCharm)
+ * fix: modifying lsof resolution to work with additional platforms (patch by Fabian Keil)
+ * fix: dropping the locale for internal connections (patch by Fabian Keil)
+ * fix: labeling connections to our socks port as being client connections (caught by Fabian Keil)
+ * fix: skipping internal -> external address translation when the external address is private (caught by Fabian Keil)
+ * fix: suppressing superfluous lsof warnings (patch by Hans Schnehl)
+ * fix: when the pid was unavailable some resolvers failed to work
+ * fix: including udp connection results (needed since exits proxy dns traffic)
+ * fix: crashing issue when nickname was undefined but displayed on the connection panel
+ * fix: concurrency issue caused the first connection resolution to often not have the pid
+ * fix: connection resolution wasn't finding results if tor was running under a different name
+ * fix: brought all Linux connection resolvers into parity (established tcp connections only)
+ * fix: commands with quoted pipes were being mis-parsed by the sysTools' call function
+
+11/27/10 - version 1.4.0 (r23873)
Introducing a new page for managing tor's configuration, along with several other improvements.
* added: editor for the tor configuration, providing:
@@ -42,6 +84,8 @@
* fix: making the interface more resilient to being resized while popups are visible
* fix: log panel wasn't respecting the prepopulate* log level config options
* fix: off by one error when wrapping lines in the log panel
+ * fix (11/30/10, r23882): install script was failing to make the temporary directory for the compressed man page
+ * fix (11/30/10, r23882): a torrc validation log entries had a typo
10/6/10 - version 1.3.7 (r23439)
Numerous improvements, most notably being an expanded log panel, installer, and deb/rpm builds.
Modified: arm/release/README
===================================================================
--- arm/release/README 2011-01-08 06:06:36 UTC (rev 24053)
+++ arm/release/README 2011-01-08 06:35:08 UTC (rev 24054)
@@ -26,11 +26,23 @@
... or including 'ControlPort <PORT>' in your torrc
For full functionality this also needs:
-Common *nix commands including: ps, pidof, tail, pwdx, host, ulimit, and a
- method of connection resolution (netstat, ss, or lsof)
-To be ran with the same user as tor to avoid permission issues with connection
- resolution and reading the torrc
+- To be ran with the same user as tor to avoid permission issues with
+ connection resolution and reading the torrc.
+- Common *nix commands including:
+ * ps
+ * a method of connection resolution (any of the following):
+ * sockstat
+ * netstat
+ * ss
+ * lsof
+ * procstat
+ * tail
+ * pwdx
+ * ulimit
+ * pgrep or pidof
+ * host (if dns lookups are enabled)
+
This is started via 'arm' (use the '--help' argument for usage).
-------------------------------------------------------------------------------
@@ -80,6 +92,12 @@
as being in a chroot jail) then it's probably failing due to permission issues.
Arm still runs, just no connection listing or ps stats.
+> The bandwidth graph showing up
+
+Some terminals, most notably screen sessions on Gentoo, appear to have a bug
+where highlighted spaces aren't rendered. A reported workaround is to set:
+ TERM="rxvt-unicode"
+
-------------------------------------------------------------------------------
Layout:
@@ -98,11 +116,13 @@
src/
__init__.py
- starter.py - parses and validates commandline parameters
- prereq.py - checks python version and for required packages
- version.py - version and last modified information
- settings.cfg - attributes loaded for parsing tor related data
- uninstall - removal script
+ starter.py - parses and validates commandline parameters
+ prereq.py - checks python version and for required packages
+ version.py - version and last modified information
+ test.py - method for starting tests and demos
+ settings.cfg - attributes loaded for parsing tor related data
+ torConfigDesc.txt - fallback descriptions of Tor's configuration options
+ uninstall - removal script
interface/
graphing/
@@ -132,6 +152,7 @@
hostnames.py - service providing nonblocking reverse dns lookups
log.py - aggregator for application events
panel.py - wrapper for safely working with curses subwindows
+ procTools.py - queries process & system information from /proc contents
sysTools.py - helper for system calls, providing client side caching
torConfig.py - functions for working with the torrc and config options
torTools.py - TorCtl wrapper, providing caching and derived information
Modified: arm/release/TODO
===================================================================
--- arm/release/TODO 2011-01-08 06:06:36 UTC (rev 24053)
+++ arm/release/TODO 2011-01-08 06:35:08 UTC (rev 24054)
@@ -1,6 +1,6 @@
TODO
-- Roadmap and completed work for next release (1.4.1)
+- Roadmap and completed work for next release (1.4.2)
[ ] refactor panels
Currently the interface is a bit of a rat's nest (especially the
controller). The goal is to use better modularization to both simplify
@@ -9,7 +9,7 @@
progress - /init and /util are done and /interface is partly done. Known
bugs are being fixed while refactoring.
- [ ] conn panel
+ * conn panel
- expand client connections and note location in circuit (entry-exit)
- for clients give an option to list all connections, to tell which are
going through tor and which might be leaking
@@ -25,27 +25,47 @@
- provide bridge / client country / exiting port statistics
Include bridge related data via GETINFO option (feature request
by waltman and ioerror).
+ - note the common port usage along with the exit statistics
+ - show the port used in scrubbed exit connections
- pick apart applications like iftop and pktstat to see how they get
per-connection bandwidth usage. Forum thread discussing it:
https://bbs.archlinux.org/viewtopic.php?pid=715906
- [ ] control port interpreter (interactive prompt)
- Panel and startup option (-t maybe?) for providing raw control port
- access along with usability improvements (piggybacking on the arm
- connection):
- - irc like help (ex "/help GETINFO" could provide a summary of
- getinfo commands, partly using the results from
- "GETINFO info/names")
- - tab completion and up/down for previous commands
- - warn and get confirmation if command would disrupt arm (for
- instance 'SETEVENTS')
- - 'safe' option that restricts to read-only access (start with this)
- - issue sighup reset
- [ ] low hanging fruit from the "client mode use cases" below
+ - include an option to show both the internal and external ips for the
+ local connection, ie:
+ myInternal --> myExternal --> foreign
+ idea and initial patch by Fabian Keil
+ * expand dump (--debug) information
+ - os and python versions
+ - arm and tor configs (scrubbing private attributes)
+ * classify config options as useful (defaultly shown), standard, and
+ deprecated (configured to be hidden by default)
+ * check tor source for deprecated options like 'group' (are they
+ ignored? idea is thanks to NightMonkey)
* release prep
* pylint --indent-string=" " --disable=C,R interface/foo.py | less
- * double check __init__.py and README for changes
+ * double check __init__.py and README for added or removed files
+ * wait a week, then bump package versions
+ * Debian
+ Contact: weasel (Peter Palfrader)
+ Initial Release: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=603056
+ Update Instructions:
+ * TBD
+
+ * Gentoo
+ Contact: NightMonkey (Jesse Adelman)
+ Initial Release: https://bugs.gentoo.org/show_bug.cgi?id=341731
+ Update Instructions:
+ * go to https://bugs.gentoo.org
+ * make a generic bug with "net-misc/arm-X.X.X version bump, please"
+
+ * ArchLinux
+ Contact: Spider.007
+ Initial Release: http://aur.archlinux.org/packages.php?ID=44172
+ Update Instructions:
+ * go to aur.archlinux.org
+ * select "Out-of-date" for the package
-- Roadmap for version 1.4.2
+- Roadmap for version 1.4.3
[ ] refactor panels
[ ] controller and popup panels
[ ] attempt to clear controller password from memory
@@ -59,13 +79,9 @@
[ ] controller and popup panels
- allow arm to resume after restarting tor
This requires a full move to the torTools controller.
- - provide measurements for startup time, and try to improve bottlenecks
- [ ] menus
- - http://gnosis.cx/publish/programming/charming_python_6.html ?
- - additional options:
- - make update rates configurable via the ui
- - dialog with flag descriptions and other help
- - menu with all torrc options (making them editable/toggleable)
+ - improve on performance bottlenecks for startup time and cpu usage
+ - intermittent concurrency bugs during shutdown, one possible source:
+ https://trac.torproject.org/projects/tor/ticket/2144
[ ] setup scripts for arm
[ ] updater (checks for a new tarball and installs it automatically)
- attempt to verify download signature, providing a warning if unable
@@ -77,14 +93,23 @@
- http://www.linuxjournal.com/article/5737
- Bugs
+ * The default resolver isn't configurable.
+ * The config write dialog (ie, the one for saving the config) has its a
+ misaligned border when it's smaller than the top detail section.
+ * The arm header panel doesn't properly reflect when the ip address
+ changes. This provides a notice event saying:
+ "Our IP Address has changed from X to Y; rebuilding descriptor (source Z)."
+ * The cpu usage spikes for scrollable content when the key's held. Try
+ coalescing the events.
+ * The manpage layout is system dependent, so the scraper needs to be more
+ resilient against being confused by whitespace. Another improvement is
+ including fallback results if the man page can't be parsed (suggested by
+ rransom, issue caught by NightMonkey).
* Log deduplication is currently an n^2 operation. Hence it can't handle
large logs (for instance, when at the DEBUG runlevel). Currently we're
timing out the function if it takes too long, but a more efficient method
for deduplication would be preferable.
* when in client mode and tor stops the header panel doesn't say so
- * util are assuming that tor is running under the default command name
- attempt to determine the command name at runtime (if the pid is available
- then ps can do the mapping)
* util/torTools.py: effective bandwidth rate/burst measurements don't take
SETCONF into consideration, blocked on:
https://trac.torproject.org/projects/tor/ticket/1692
@@ -120,6 +145,27 @@
* connection uptimes shouldn't show fractions of a second
* connections aren't cleared when control port closes
+- Packaging
+ * OpenWrt - OpenWrt uses the opkg packaging format which could make use of
+ arm's current deb packages. Packaging for this platform would help with
+ the Torouter project:
+ https://trac.torproject.org/projects/tor/wiki/TheOnionRouter/Torouter
+ * Mac - Couple of options include macport and dmg...
+ * macport (http://guide.macports.org/#development)
+ Build-from-source distribution method (like BSD portinstall). This has
+ been suggested by several people.
+
+ * dmg (http://en.wikipedia.org/wiki/Apple_Disk_Image)
+ Most conventional method of software distribution on mac. This is just
+ a container (no updating/removal support), but could contain an icon
+ for the dock that starts a terminal with arm. This might include a pkg
+ installer.
+
+ * mpkg (http://pypi.python.org/pypi/bdist_mpkg/)
+ Plugin for distutils. Like most mac packaging, this can only run on a
+ mac. It also requires setuptools:
+ http://www.errorhelp.com/search/details/74034/importerror-no-module-named-setuptools
+
- Future Features
* client mode use cases
* not sure what sort of information would be useful in the header (to
@@ -147,6 +193,27 @@
* dialog with bridge statuses (idea by mikeperry)
https://trac.vidalia-project.net/ticket/570
https://trac.torproject.org/projects/tor/ticket/2068
+ * menus
+ * http://gnosis.cx/publish/programming/charming_python_6.html ?
+ * additional options:
+ * make update rates configurable via the ui
+ * dialog with flag descriptions and other help
+ * menu with all torrc options (making them editable/toggleable)
+ * control port interpreter (interactive prompt)
+ Panel and startup option (-t maybe?) for providing raw control port
+ access along with usability improvements (piggybacking on the arm
+ connection):
+ * irc like help (ex "/help GETINFO" could provide a summary of
+ getinfo commands, partly using the results from
+ "GETINFO info/names")
+ * tab completion and up/down for previous commands
+ * warn and get confirmation if command would disrupt arm (for
+ instance 'SETEVENTS')
+ * 'safe' option that restricts to read-only access (start with this)
+ * issue sighup reset
+ * make use of the new process/* GETINFO options
+ They'll be available in the next tor release, as per:
+ https://trac.torproject.org/projects/tor/ticket/2291
* feature parity for arm's config values (armrc entries)
* editability
* parse descriptions from the man page? autogeneration of the man page from
@@ -162,22 +229,6 @@
* accounting and alerts for if the bandwidth drops to zero
* daily/weekly/etc alerts for basic status (log output, bandwidth history,
etc), borrowing from the consensus tracker for some of the formatting
- * mac installer
- * Couple of options include macport and dmg...
- * macport (http://guide.macports.org/#development)
- Build-from-source distribution method (like BSD portinstall). This has
- been suggested by several people.
-
- * dmg (http://en.wikipedia.org/wiki/Apple_Disk_Image)
- Most conventional method of software distribution on mac. This is just
- a container (no updating/removal support), but could contain an icon
- for the dock that starts a terminal with arm. This might include a pkg
- installer.
-
- * mpkg (http://pypi.python.org/pypi/bdist_mpkg/)
- Plugin for distutils. Like most mac packaging, this can only run on a
- mac. It also requires setuptools:
- http://www.errorhelp.com/search/details/74034/importerror-no-module-named-setuptools
* tab completion for input fields that expect a filesystem path
* look through vidalia's tickets for more ideas
https://trac.vidalia-project.net/
@@ -199,7 +250,8 @@
* setup wizard for new relays
Setting the password and such for torrc generation. Maybe a netinstaller
that fetches the right package for the plagform, verifies signatures, etc?
- (idea by ioerror)
+ Another alternative would be that when arm is started and tor isn't
+ running offer to start tor as a client, relay, or bridge. (idea by ioerror)
* audit what tor does
* Provide warnings if tor connections misbehaves, for instance:
* ensuring ExitPolicyRejectPrivate is being obeyed
Modified: arm/release/arm.1
===================================================================
--- arm/release/arm.1 2011-01-08 06:06:36 UTC (rev 24053)
+++ arm/release/arm.1 2011-01-08 06:35:08 UTC (rev 24054)
@@ -30,6 +30,10 @@
user provided configuration file (default is \fB~/.armrc\fR)
.TP
+\fB\-d\fR, \fB\-\-debug\fR
+writes all arm logs to /home/atagar/.arm/log
+
+.TP
\fB\-b\fR, \fB\-\-blind\fR
disable connection lookups (netstat, lsof, and ss), dropping the parts of the
interface that rely on this information
Modified: arm/release/armrc.sample
===================================================================
--- arm/release/armrc.sample 2011-01-08 06:06:36 UTC (rev 24053)
+++ arm/release/armrc.sample 2011-01-08 06:35:08 UTC (rev 24054)
@@ -1,15 +1,21 @@
-# startup options
+# Startup options
startup.controlPassword
startup.interface.ipAddress 127.0.0.1
startup.interface.port 9051
startup.blindModeEnabled false
startup.events N3
+startup.dataDirectory ~/.arm
# Seconds between querying information
-queries.ps.rate 5
+queries.resourceUsage.rate 5
queries.connections.minRate 5
queries.refreshRate.rate 5
+# Read the proc contents directly instead of calling ps, netstat, and other
+# resolvers. This provides very sizable performance benefits (around 90%
+# faster lookups) but this is only available on Linux.
+queries.useProc true
+
# Renders the interface with color if set and the terminal supports it
features.colorInterface true
@@ -148,18 +154,6 @@
features.graph.bw.accounting.rate 10
features.graph.bw.accounting.isTimeLong false
-# Parameters for graphing ps stats
-# --------------------------------
-# primary/secondaryStat
-# any numeric field provided by the ps command
-# cachedOnly
-# determines if the graph should query ps or rely on cached results (this
-# lowers the call volume but limits the graph's granularity)
-
-features.graph.ps.primaryStat %cpu
-features.graph.ps.secondaryStat rss
-features.graph.ps.cachedOnly true
-
# Thread pool size for hostname resolutions
# Determines the maximum number of concurrent requests. Upping this to around
# thirty or so seems to be problematic, causing intermittently seizing.
@@ -185,22 +179,23 @@
# Runlevels at which arm logs its events
log.startTime INFO
log.refreshRate DEBUG
+log.highCpuUsage WARN
log.configEntryNotFound NONE
log.configEntryUndefined NOTICE
log.configEntryTypeError NOTICE
log.torCtlPortClosed NOTICE
log.torGetInfo DEBUG
+log.torGetInfoCache NONE
log.torGetConf DEBUG
log.torSetConf INFO
log.torEventTypeUnrecognized NOTICE
log.torPrefixPathInvalid NOTICE
+log.procCallMade DEBUG
log.sysCallMade DEBUG
log.sysCallCached NONE
log.sysCallFailed INFO
log.sysCallCacheGrowing INFO
log.panelRecreated DEBUG
-log.graph.ps.invalidStat WARN
-log.graph.ps.abandon WARN
log.graph.bw.prepopulateSuccess NOTICE
log.graph.bw.prepopulateFailure NOTICE
log.logPanel.prepopulateSuccess INFO
@@ -210,18 +205,26 @@
log.logPanel.forceDoubleRedraw DEBUG
log.torrc.readFailed WARN
log.torrc.validation.torStateDiffers WARN
-log.torrc.validation.unnecessaryTorrcEntries WARN
+log.torrc.validation.unnecessaryTorrcEntries NOTICE
log.configDescriptions.readManPageSuccess INFO
-log.configDescriptions.readManPageFailed WARN
+log.configDescriptions.readManPageFailed NOTICE
log.configDescriptions.unrecognizedCategory NOTICE
+log.configDescriptions.internalLoadSuccess NOTICE
+log.configDescriptions.internalLoadFailed ERR
log.configDescriptions.persistance.loadSuccess INFO
log.configDescriptions.persistance.loadFailed INFO
log.configDescriptions.persistance.saveSuccess NOTICE
log.configDescriptions.persistance.saveFailed NOTICE
+log.connResolverOptions INFO
log.connLookupFailed INFO
log.connLookupFailover NOTICE
log.connLookupAbandon WARN
log.connLookupRateGrowing NONE
log.hostnameCacheTrimmed INFO
log.cursesColorSupport INFO
+log.bsdJailFound INFO
+log.unknownBsdJailId WARN
+log.stats.failedProcResolution DEBUG
+log.stats.procResolutionFailover INFO
+log.stats.failedPsResolution INFO
Modified: arm/release/src/__init__.py
===================================================================
--- arm/release/src/__init__.py 2011-01-08 06:06:36 UTC (rev 24053)
+++ arm/release/src/__init__.py 2011-01-08 06:35:08 UTC (rev 24054)
@@ -2,5 +2,5 @@
Scripts involved in validating user input, system state, and initializing arm.
"""
-__all__ = ["starter", "prereq", "version"]
+__all__ = ["starter", "prereq", "version", "test"]
Modified: arm/release/src/interface/configPanel.py
===================================================================
--- arm/release/src/interface/configPanel.py 2011-01-08 06:06:36 UTC (rev 24053)
+++ arm/release/src/interface/configPanel.py 2011-01-08 06:35:08 UTC (rev 24054)
@@ -30,13 +30,14 @@
torConfig.UNKNOWN: "white"}
# attributes of a ConfigEntry
-FIELD_CATEGORY, FIELD_OPTION, FIELD_VALUE, FIELD_TYPE, FIELD_ARG_USAGE, FIELD_DESCRIPTION, FIELD_MAN_ENTRY, FIELD_IS_DEFAULT = range(8)
+FIELD_CATEGORY, FIELD_OPTION, FIELD_VALUE, FIELD_TYPE, FIELD_ARG_USAGE, FIELD_SUMMARY, FIELD_DESCRIPTION, FIELD_MAN_ENTRY, FIELD_IS_DEFAULT = range(9)
DEFAULT_SORT_ORDER = (FIELD_CATEGORY, FIELD_MAN_ENTRY, FIELD_IS_DEFAULT)
FIELD_ATTR = {FIELD_CATEGORY: ("Category", "red"),
FIELD_OPTION: ("Option Name", "blue"),
FIELD_VALUE: ("Value", "cyan"),
FIELD_TYPE: ("Arg Type", "green"),
FIELD_ARG_USAGE: ("Arg Usage", "yellow"),
+ FIELD_SUMMARY: ("Summary", "green"),
FIELD_DESCRIPTION: ("Description", "white"),
FIELD_MAN_ENTRY: ("Man Page Entry", "blue"),
FIELD_IS_DEFAULT: ("Is Default", "magenta")}
@@ -46,7 +47,7 @@
Configuration option in the panel.
"""
- def __init__(self, option, type, isDefault, manEntry):
+ def __init__(self, option, type, isDefault, summary, manEntry):
self.fields = {}
self.fields[FIELD_OPTION] = option
self.fields[FIELD_TYPE] = type
@@ -62,6 +63,8 @@
self.fields[FIELD_CATEGORY] = torConfig.UNKNOWN
self.fields[FIELD_ARG_USAGE] = ""
self.fields[FIELD_DESCRIPTION] = ""
+
+ self.fields[FIELD_SUMMARY] = summary if summary != None else self.fields[FIELD_DESCRIPTION]
def get(self, field):
"""
@@ -144,8 +147,9 @@
elif not self._config["features.config.state.showVirtualOptions"] and confType == "Virtual":
continue
+ summary = torConfig.getConfigSummary(confOption)
manEntry = torConfig.getConfigDescription(confOption)
- self.confContents.append(ConfigEntry(confOption, confType, not confOption in customOptions, manEntry))
+ self.confContents.append(ConfigEntry(confOption, confType, not confOption in customOptions, summary, manEntry))
self.setSortOrder() # initial sorting of the contents
elif self.configType == ARM_STATE:
@@ -227,16 +231,14 @@
optionLabel = uiTools.cropStr(entry.get(FIELD_OPTION), optionWidth)
valueLabel = uiTools.cropStr(entry.get(FIELD_VALUE), valueWidth)
+ summaryLabel = uiTools.cropStr(entry.get(FIELD_SUMMARY), descriptionWidth, None)
- # ends description at the first newline
- descriptionLabel = uiTools.cropStr(entry.get(FIELD_DESCRIPTION).split("\n")[0], descriptionWidth, None)
-
lineFormat = curses.A_NORMAL if entry.get(FIELD_IS_DEFAULT) else curses.A_BOLD
if entry.get(FIELD_CATEGORY): lineFormat |= uiTools.getColor(CATEGORY_COLOR[entry.get(FIELD_CATEGORY)])
if entry == cursorSelection: lineFormat |= curses.A_STANDOUT
lineTextLayout = "%%-%is %%-%is %%-%is" % (optionWidth, valueWidth, descriptionWidth)
- lineText = lineTextLayout % (optionLabel, valueLabel, descriptionLabel)
+ lineText = lineTextLayout % (optionLabel, valueLabel, summaryLabel)
self.addstr(drawLine, scrollOffset, lineText, lineFormat)
if drawLine >= height: break
Modified: arm/release/src/interface/connPanel.py
===================================================================
--- arm/release/src/interface/connPanel.py 2011-01-08 06:06:36 UTC (rev 24053)
+++ arm/release/src/interface/connPanel.py 2011-01-08 06:35:08 UTC (rev 24054)
@@ -105,6 +105,20 @@
if sortLabel == label: return type
raise ValueError(sortLabel)
+def ipAddressIsPrivate(ipAddr):
+ """
+ Provides true if the IP address belongs on the local network or belongs to
+ loopback, false otherwise. These include:
+ Private ranges: 10.*, 172.16.* - 172.31.*, 192.168.*
+ Loopback: 127.*
+
+ Arguments:
+ ipAddr - IP address to be checked
+ """
+
+ # TODO: being lazy right now and just declaring all of 172.* as private
+ return ipAddr.startswith("10.") or ipAddr.startswith("192.168.") or ipAddr.startswith("172.") or ipAddr.startswith("127.")
+
class ConnPanel(TorCtl.PostEventListener, panel.Panel):
"""
Lists tor related connection data.
@@ -156,6 +170,7 @@
self.orPort = "0"
self.dirPort = "0"
self.controlPort = "0"
+ self.socksPort = "0"
self.family = [] # fingerpints of family entries
self.isBridge = False # true if BridgeRelay is set
self.exitPolicy = ""
@@ -180,6 +195,7 @@
try:
self.address = "" # fetched when needed if unset
self.nickname = self.conn.get_option("Nickname")[0][1]
+ if self.nickname == None: self.nickname = "Unnamed"
self.orPort = self.conn.get_option("ORPort")[0][1]
self.dirPort = self.conn.get_option("DirPort")[0][1]
@@ -191,6 +207,8 @@
self.listenPort = listenAddr[listenAddr.find(":") + 1:]
else: self.listenPort = self.orPort
+ self.socksPort = torTools.getConn().getOption("SocksPort", "0")
+
# entry is None if not set, otherwise of the format "$<fingerprint>,$<fingerprint>"
familyEntry = self.conn.get_option("MyFamily")[0][1]
if familyEntry: self.family = familyEntry.split(",")
@@ -215,6 +233,7 @@
self.orPort = "0"
self.dirPort = "0"
self.controlPort = "0"
+ self.socksPort = "0"
self.family = []
self.isBridge = False
self.exitPolicy = ""
@@ -324,6 +343,9 @@
type = "inbound"
connectionCountTmp[0] += 1
if SCRUB_PRIVATE_DATA and fIp not in self.fingerprintMappings.keys(): isPrivate = isGuard or self.isBridge
+ elif lPort == self.socksPort:
+ type = "client"
+ connectionCountTmp[2] += 1
elif lPort == self.controlPort:
type = "control"
connectionCountTmp[4] += 1
@@ -347,17 +369,22 @@
connectionCountTmp[1] += 1
if SCRUB_PRIVATE_DATA and fIp not in self.fingerprintMappings.keys(): isPrivate = isExitAllowed(fIp, fPort, self.exitPolicy, self.exitRejectPrivate)
- # replace nat address with external version if available
- if self.address and type != "control": lIp = self.address
+ # replace nat address with external version if available and the
+ # external address isn't a private IP
+ isPrivateIp = ipAddressIsPrivate(fIp)
+ if self.address and type != "control" and not isPrivateIp: lIp = self.address
- try:
- countryCodeQuery = "ip-to-country/%s" % fIp
- countryCode = self.conn.get_info(countryCodeQuery)[countryCodeQuery]
- except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed):
+ if isPrivateIp:
countryCode = "??"
- if not self.providedGeoipWarning:
- log.log(log.WARN, "Tor geoip database is unavailable.")
- self.providedGeoipWarning = True
+ else:
+ try:
+ countryCodeQuery = "ip-to-country/%s" % fIp
+ countryCode = self.conn.get_info(countryCodeQuery)[countryCodeQuery]
+ except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed):
+ countryCode = "??"
+ if not self.providedGeoipWarning:
+ log.log(log.WARN, "Tor geoip database is unavailable.")
+ self.providedGeoipWarning = True
if (fIp, fPort) in connTimes: connTime = connTimes[(fIp, fPort)]
else: connTime = time.time()
@@ -532,7 +559,9 @@
if self.listingType == LIST_IP:
# base data requires 73 characters
src = "%s:%s" % (entry[CONN_L_IP], entry[CONN_L_PORT])
- dst = "%s:%s %s" % (entry[CONN_F_IP], entry[CONN_F_PORT], "" if type == "control" else "(%s)" % entry[CONN_COUNTRY])
+ dst = "%s:%s" % (entry[CONN_F_IP], entry[CONN_F_PORT])
+ if not ipAddressIsPrivate(entry[CONN_F_IP]):
+ dst += " (%s)" % entry[CONN_COUNTRY]
if isPrivate: dst = "<scrubbed>"
@@ -565,9 +594,12 @@
foreignHostnameSpace -= 22
if isPrivate: ipEntry = "<scrubbed>"
- else: ipEntry = "%s %s" % (entry[CONN_F_IP], "" if type == "control" else "(%s)" % entry[CONN_COUNTRY])
+ else:
+ ipEntry = "%s:%s" % (entry[CONN_F_IP], entry[CONN_F_PORT])
+ if ipAddressIsPrivate(entry[CONN_F_IP]):
+ ipEntry += " (%s)" % entry[CONN_COUNTRY]
+
etc += "%-20s " % ipEntry
-
if width > 134 + xOffset:
# show fingerprint (column width: 42 characters)
foreignHostnameSpace -= 42
@@ -613,7 +645,11 @@
if width > 125 + xOffset:
# shows ip/port/locale (column width: 28 characters)
if isPrivate: ipEntry = "<scrubbed>"
- else: ipEntry = "%s:%s %s" % (entry[CONN_F_IP], entry[CONN_F_PORT], "" if type == "control" else "(%s)" % entry[CONN_COUNTRY])
+ else:
+ ipEntry = "%s:%s" % (entry[CONN_F_IP], entry[CONN_F_PORT])
+ if ipAddressIsPrivate(entry[CONN_F_IP]):
+ ipEntry += " (%s)" % entry[CONN_COUNTRY]
+
etc += "%-26s " % ipEntry
else:
# base data uses whatever extra room's available (using minimun of 50 characters)
@@ -635,7 +671,11 @@
foreignNicknameSpace -= 28
if isPrivate: ipEntry = "<scrubbed>"
- else: ipEntry = "%s:%s %s" % (entry[CONN_F_IP], entry[CONN_F_PORT], "" if type == "control" else "(%s)" % entry[CONN_COUNTRY])
+ else:
+ ipEntry = "%s:%s" % (entry[CONN_F_IP], entry[CONN_F_PORT])
+ if ipAddressIsPrivate(entry[CONN_F_IP]):
+ ipEntry += " (%s)" % entry[CONN_COUNTRY]
+
etc += "%-26s " % ipEntry
dst = ("%%-%is" % foreignNicknameSpace) % dst
Modified: arm/release/src/interface/controller.py
===================================================================
--- arm/release/src/interface/controller.py 2011-01-08 06:06:36 UTC (rev 24053)
+++ arm/release/src/interface/controller.py 2011-01-08 06:35:08 UTC (rev 24054)
@@ -14,7 +14,6 @@
import curses.textpad
import socket
from TorCtl import TorCtl
-from TorCtl import TorUtil
import headerPanel
import graphing.graphPanel
@@ -28,7 +27,7 @@
from util import conf, log, connections, hostnames, panel, sysTools, torConfig, torTools, uiTools
import graphing.bandwidthStats
import graphing.connStats
-import graphing.psStats
+import graphing.resourceStats
CONFIRM_QUIT = True
REFRESH_RATE = 5 # seconds between redrawing screen
@@ -54,9 +53,10 @@
"features.graph.bw.prepopulate": True,
"log.startTime": log.INFO,
"log.refreshRate": log.DEBUG,
+ "log.highCpuUsage": log.WARN,
"log.configEntryUndefined": log.NOTICE,
"log.torrc.validation.torStateDiffers": log.WARN,
- "log.torrc.validation.unnecessaryTorrcEntries": log.WARN}
+ "log.torrc.validation.unnecessaryTorrcEntries": log.NOTICE}
class ControlPanel(panel.Panel):
""" Draws single line label for interface controls. """
@@ -506,18 +506,18 @@
if mismatchLines:
if len(mismatchLines) > 1:
- msg += "\n- torrc values differ on line lines: "
+ msg += "\n- torrc values differ on lines: "
else:
- msg += "\n- torrc value differs on line line: "
+ msg += "\n- torrc value differs on line: "
mismatchLines.sort()
msg += ", ".join([str(val + 1) for val in mismatchLines])
if missingOptions:
if len(missingOptions) > 1:
- msg += "\n-configuration values are missing from the torrc: "
+ msg += "\n- configuration values are missing from the torrc: "
else:
- msg += "\n-configuration value is missing from the torrc: "
+ msg += "\n- configuration value is missing from the torrc: "
missingOptions.sort()
msg += ", ".join(missingOptions)
@@ -528,14 +528,18 @@
# minor refinements for connection resolver
if not isBlindMode:
- resolver = connections.getResolver("tor")
- if torPid: resolver.processPid = torPid # helps narrow connection results
+ if torPid:
+ # use the tor pid to help narrow connection results
+ torCmdName = sysTools.getProcessName(torPid, "tor")
+ resolver = connections.getResolver(torCmdName, torPid, "tor")
+ else:
+ resolver = connections.getResolver("tor")
# hack to display a better (arm specific) notice if all resolvers fail
connections.RESOLVER_FINAL_FAILURE_MSG += " (connection related portions of the monitor won't function)"
panels = {
- "header": headerPanel.HeaderPanel(stdscr, config),
+ "header": headerPanel.HeaderPanel(stdscr, startTime),
"popup": Popup(stdscr, 9),
"graph": graphing.graphPanel.GraphPanel(stdscr),
"log": logPanel.LogPanel(stdscr, loggedEvents, config)}
@@ -557,7 +561,7 @@
# statistical monitors for graph
panels["graph"].addStats("bandwidth", graphing.bandwidthStats.BandwidthStats(config))
- panels["graph"].addStats("system resources", graphing.psStats.PsStats(config))
+ panels["graph"].addStats("system resources", graphing.resourceStats.ResourceStats())
if not isBlindMode: panels["graph"].addStats("connections", graphing.connStats.ConnStats())
# sets graph based on config parameter
@@ -631,6 +635,12 @@
initTime = time.time() - startTime
log.log(CONFIG["log.startTime"], "arm started (initialization took %0.3f seconds)" % initTime)
+ # attributes to give a WARN level event if arm's resource usage is too high
+ isResourceWarningGiven = False
+ lastResourceCheck = startTime
+
+ lastSize = None
+
# TODO: come up with a nice, clean method for other threads to immediately
# terminate the draw loop and provide a stacktrace
while True:
@@ -657,6 +667,16 @@
# TODO: should redraw the torrcPanel
#panels["torrc"].loadConfig()
+
+ # reload the torrc if it's previously been loaded
+ if loadedTorrc.isLoaded():
+ try:
+ loadedTorrc.load()
+ if page == 3: panels["torrc"].redraw(True)
+ except IOError, exc:
+ msg = "Unable to load torrc (%s)" % sysTools.getFileErrorMsg(exc)
+ log.log(CONFIG["log.torrc.readFailed"], msg)
+
sighupTracker.isReset = False
# gives panels a chance to take advantage of the maximum bounds
@@ -707,9 +727,13 @@
for panelKey in (PAGE_S + PAGES[page]):
# redrawing popup can result in display flicker when it should be hidden
if panelKey != "popup":
+ newSize = stdscr.getmaxyx()
+ isResize = lastSize != newSize
+ lastSize = newSize
+
if panelKey in ("header", "graph", "log", "config", "torrc"):
# revised panel (manages its own content refreshing)
- panels[panelKey].redraw()
+ panels[panelKey].redraw(isResize)
else:
panels[panelKey].redraw(True)
@@ -717,8 +741,34 @@
currentTime = time.time()
if currentTime - lastPerformanceLog >= CONFIG["queries.refreshRate.rate"]:
- log.log(CONFIG["log.refreshRate"], "refresh rate: %0.3f seconds" % (currentTime - redrawStartTime))
+ cpuTotal = sum(os.times()[:3])
+ pythonCpuAvg = cpuTotal / (currentTime - startTime)
+ sysCallCpuAvg = sysTools.getSysCpuUsage()
+ totalCpuAvg = pythonCpuAvg + sysCallCpuAvg
+
+ if sysCallCpuAvg > 0.00001:
+ log.log(CONFIG["log.refreshRate"], "refresh rate: %0.3f seconds, average cpu usage: %0.3f%% (python), %0.3f%% (system calls), %0.3f%% (total)" % (currentTime - redrawStartTime, 100 * pythonCpuAvg, 100 * sysCallCpuAvg, 100 * totalCpuAvg))
+ else:
+ # with the proc enhancements the sysCallCpuAvg is usually zero
+ log.log(CONFIG["log.refreshRate"], "refresh rate: %0.3f seconds, average cpu usage: %0.3f%%" % (currentTime - redrawStartTime, 100 * totalCpuAvg))
+
lastPerformanceLog = currentTime
+
+ # once per minute check if the sustained cpu usage is above 5%, if so
+ # then give a warning (and if able, some advice for lowering it)
+ # TODO: disabling this for now (scrolling causes cpu spikes for quick
+ # redraws, ie this is usually triggered by user input)
+ if False and not isResourceWarningGiven and currentTime > (lastResourceCheck + 60):
+ if totalCpuAvg >= 0.05:
+ msg = "Arm's cpu usage is high (averaging %0.3f%%)." % (100 * totalCpuAvg)
+
+ if not isBlindMode:
+ msg += " You could lower it by dropping the connection data (running as \"arm -b\")."
+
+ log.log(CONFIG["log.highCpuUsage"], msg)
+ isResourceWarningGiven = True
+
+ lastResourceCheck = currentTime
finally:
panel.CURSES_LOCK.release()
@@ -758,20 +808,26 @@
# this appears to be a python bug: http://bugs.python.org/issue3014
# (haven't seen this is quite some time... mysteriously resolved?)
+ # stops panel daemons
+ panels["header"].stop()
+ panels["log"].stop()
+
+ panels["header"].join()
+ panels["log"].join()
+
# joins on utility daemon threads - this might take a moment since
# the internal threadpools being joined might be sleeping
+ conn = torTools.getConn()
+ myPid = conn.getMyPid()
+
+ resourceTracker = sysTools.getResourceTracker(myPid) if (myPid and sysTools.isTrackerAlive(myPid)) else None
resolver = connections.getResolver("tor") if connections.isResolverAlive("tor") else None
+ if resourceTracker: resourceTracker.stop()
if resolver: resolver.stop() # sets halt flag (returning immediately)
hostnames.stop() # halts and joins on hostname worker thread pool
+ if resourceTracker: resourceTracker.join()
if resolver: resolver.join() # joins on halted resolver
- # stops panel daemons
- panels["header"].stop()
- panels["log"].stop()
-
- panels["header"].join()
- panels["log"].join()
-
conn.close() # joins on TorCtl event thread
break
elif key == curses.KEY_LEFT or key == curses.KEY_RIGHT:
@@ -1401,7 +1457,7 @@
panels["conn"].sortConnections()
elif page == 1 and (key == ord('u') or key == ord('U')):
# provides menu to pick identification resolving utility
- optionTypes = [None, connections.CMD_NETSTAT, connections.CMD_SS, connections.CMD_LSOF]
+ optionTypes = [None, connections.CMD_PROC, connections.CMD_NETSTAT, connections.CMD_SOCKSTAT, connections.CMD_LSOF, connections.CMD_SS, connections.CMD_BSD_SOCKSTAT, connections.CMD_BSD_PROCSTAT]
options = ["auto"] + [connections.CMD_STR[util] for util in optionTypes[1:]]
initialSelection = connections.getResolver("tor").overwriteResolver # enums correspond to indices
Modified: arm/release/src/interface/graphing/__init__.py
===================================================================
--- arm/release/src/interface/graphing/__init__.py 2011-01-08 06:06:36 UTC (rev 24053)
+++ arm/release/src/interface/graphing/__init__.py 2011-01-08 06:35:08 UTC (rev 24054)
@@ -2,5 +2,5 @@
Panels, popups, and handlers comprising the arm user interface.
"""
-__all__ = ["graphPanel.py", "bandwidthStats", "connStats", "psStats"]
+__all__ = ["graphPanel.py", "bandwidthStats", "connStats", "resourceStats"]
Modified: arm/release/src/interface/graphing/bandwidthStats.py
===================================================================
--- arm/release/src/interface/graphing/bandwidthStats.py 2011-01-08 06:06:36 UTC (rev 24053)
+++ arm/release/src/interface/graphing/bandwidthStats.py 2011-01-08 06:35:08 UTC (rev 24054)
@@ -99,7 +99,7 @@
return False
# attempt to open the state file
- try: stateFile = open("%s%s/state" % (torTools.getPathPrefix(), dataDir), "r")
+ try: stateFile = open("%s%s/state" % (conn.getPathPrefix(), dataDir), "r")
except IOError:
msg = PREPOPULATE_FAILURE_MSG % "unable to read the state file"
log.log(self._config["log.graph.bw.prepopulateFailure"], msg)
Deleted: arm/release/src/interface/graphing/psStats.py
===================================================================
--- arm/release/src/interface/graphing/psStats.py 2011-01-08 06:06:36 UTC (rev 24053)
+++ arm/release/src/interface/graphing/psStats.py 2011-01-08 06:35:08 UTC (rev 24054)
@@ -1,141 +0,0 @@
-"""
-Tracks configured ps stats. If non-numeric then this fails, providing a blank
-graph. By default this provides the cpu and memory usage of the tor process.
-"""
-
-from interface.graphing import graphPanel
-from util import conf, log, sysTools, torTools, uiTools
-
-# number of subsequent failed queries before giving up
-FAILURE_THRESHOLD = 5
-
-# attempts to use cached results from the header panel's ps calls
-HEADER_PS_PARAM = ["%cpu", "rss", "%mem", "etime"]
-
-DEFAULT_CONFIG = {"features.graph.ps.primaryStat": "%cpu",
- "features.graph.ps.secondaryStat": "rss",
- "features.graph.ps.cachedOnly": True,
- "log.graph.ps.invalidStat": log.WARN,
- "log.graph.ps.abandon": log.WARN}
-
-class PsStats(graphPanel.GraphStats):
- """
- Tracks ps stats, defaulting to system resource usage (cpu and memory usage).
- """
-
- def __init__(self, config=None):
- graphPanel.GraphStats.__init__(self)
- self.failedCount = 0 # number of subsequent failed queries
-
- self._config = dict(DEFAULT_CONFIG)
- if config: config.update(self._config)
-
- self.queryPid = torTools.getConn().getMyPid()
- self.queryParam = [self._config["features.graph.ps.primaryStat"], self._config["features.graph.ps.secondaryStat"]]
-
- # If we're getting the same stats as the header panel then issues identical
- # queries to make use of cached results. If not, then disable cache usage.
- if self.queryParam[0] in HEADER_PS_PARAM and self.queryParam[1] in HEADER_PS_PARAM:
- self.queryParam = list(HEADER_PS_PARAM)
- else: self._config["features.graph.ps.cachedOnly"] = False
-
- # strips any empty entries
- while "" in self.queryParam: self.queryParam.remove("")
-
- self.cacheTime = 3600 if self._config["features.graph.ps.cachedOnly"] else 1
-
- def getTitle(self, width):
- return "System Resources:"
-
- def getRefreshRate(self):
- # provides the rate at which the panel has new stats to display
- if self._config["features.graph.ps.cachedOnly"]:
- return int(conf.getConfig("arm").get("queries.ps.rate", 5))
- else: return 1
-
- def getHeaderLabel(self, width, isPrimary):
- avg = (self.primaryTotal if isPrimary else self.secondaryTotal) / max(1, self.tick)
- lastAmount = self.lastPrimary if isPrimary else self.lastSecondary
-
- if isPrimary: statName = self._config["features.graph.ps.primaryStat"]
- else: statName = self._config["features.graph.ps.secondaryStat"]
-
- # provides nice labels for failures and common stats
- if not statName or self.failedCount >= FAILURE_THRESHOLD or not statName in self.queryParam:
- return ""
- elif statName == "%cpu":
- return "CPU (%s%%, avg: %0.1f%%):" % (lastAmount, avg)
- elif statName in ("rss", "size"):
- # memory sizes are converted from MB to B before generating labels
- statLabel = "Memory" if statName == "rss" else "Size"
- usageLabel = uiTools.getSizeLabel(lastAmount * 1048576, 1)
- avgLabel = uiTools.getSizeLabel(avg * 1048576, 1)
- return "%s (%s, avg: %s):" % (statLabel, usageLabel, avgLabel)
- else:
- # generic label (first letter of stat name is capitalized)
- statLabel = statName[0].upper() + statName[1:]
- return "%s (%s, avg: %s):" % (statLabel, lastAmount, avg)
-
- def isVisible(self):
- """
- Hides graph if unable to fetch stats.
- """
-
- if self.queryPid and self.queryParam and self.failedCount < FAILURE_THRESHOLD:
- return graphPanel.GraphStats.isVisible(self)
- else: return False
-
- def eventTick(self):
- """
- Processes a ps event.
- """
-
- psResults = {} # mapping of stat names to their results
- if self.queryPid and self.queryParam and self.failedCount < FAILURE_THRESHOLD:
- queryCmd = "ps -p %s -o %s" % (self.queryPid, ",".join(self.queryParam))
- psCall = sysTools.call(queryCmd, self.cacheTime, True)
-
- if psCall and len(psCall) == 2:
- # ps provided results (first line is headers, second is stats)
- stats = psCall[1].strip().split()
-
- if len(self.queryParam) == len(stats):
- # we have a result to match each stat - constructs mapping
- psResults = dict([(self.queryParam[i], stats[i]) for i in range(len(stats))])
- self.failedCount = 0 # had a successful call - reset failure count
-
- if not psResults:
- # ps call failed, if we fail too many times sequentially then abandon
- # listing (probably due to invalid ps parameters)
- self.failedCount += 1
-
- if self.failedCount == FAILURE_THRESHOLD:
- msg = "failed several attempts to query '%s', abandoning ps graph" % queryCmd
- log.log(self._config["log.graph.ps.abandon"], msg)
-
- # if something fails (no pid, ps call failed, etc) then uses last results
- primary, secondary = self.lastPrimary, self.lastSecondary
-
- for isPrimary in (True, False):
- if isPrimary: statName = self._config["features.graph.ps.primaryStat"]
- else: statName = self._config["features.graph.ps.secondaryStat"]
-
- if statName in psResults:
- try:
- result = float(psResults[statName])
-
- # The 'rss' and 'size' parameters provide memory usage in KB. This is
- # scaled up to MB so the graph's y-high is a reasonable value.
- if statName in ("rss", "size"): result /= 1024.0
-
- if isPrimary: primary = result
- else: secondary = result
- except ValueError:
- if self.queryParam != HEADER_PS_PARAM:
- # custom stat provides non-numeric results - give a warning and stop querying it
- msg = "unable to use non-numeric ps stat '%s' for graphing" % statName
- log.log(self._config["log.graph.ps.invalidStat"], msg)
- self.queryParam.remove(statName)
-
- self._processEvent(primary, secondary)
-
Copied: arm/release/src/interface/graphing/resourceStats.py (from rev 24053, arm/trunk/src/interface/graphing/resourceStats.py)
===================================================================
--- arm/release/src/interface/graphing/resourceStats.py (rev 0)
+++ arm/release/src/interface/graphing/resourceStats.py 2011-01-08 06:35:08 UTC (rev 24054)
@@ -0,0 +1,47 @@
+"""
+Tracks the system resource usage (cpu and memory) of the tor process.
+"""
+
+from interface.graphing import graphPanel
+from util import sysTools, torTools, uiTools
+
+class ResourceStats(graphPanel.GraphStats):
+ """
+ System resource usage tracker.
+ """
+
+ def __init__(self):
+ graphPanel.GraphStats.__init__(self)
+ self.queryPid = torTools.getConn().getMyPid()
+
+ def getTitle(self, width):
+ return "System Resources:"
+
+ def getHeaderLabel(self, width, isPrimary):
+ avg = (self.primaryTotal if isPrimary else self.secondaryTotal) / max(1, self.tick)
+ lastAmount = self.lastPrimary if isPrimary else self.lastSecondary
+
+ if isPrimary:
+ return "CPU (%0.1f%%, avg: %0.1f%%):" % (lastAmount, avg)
+ else:
+ # memory sizes are converted from MB to B before generating labels
+ usageLabel = uiTools.getSizeLabel(lastAmount * 1048576, 1)
+ avgLabel = uiTools.getSizeLabel(avg * 1048576, 1)
+ return "Memory (%s, avg: %s):" % (usageLabel, avgLabel)
+
+ def eventTick(self):
+ """
+ Fetch the cached measurement of resource usage from the ResourceTracker.
+ """
+
+ primary, secondary = 0, 0
+ if self.queryPid:
+ resourceTracker = sysTools.getResourceTracker(self.queryPid)
+
+ if not resourceTracker.lastQueryFailed():
+ primary, _, secondary, _ = resourceTracker.getResourceUsage()
+ primary *= 100 # decimal percentage to whole numbers
+ secondary /= 1048576 # translate size to MB so axis labels are short
+
+ self._processEvent(primary, secondary)
+
Modified: arm/release/src/interface/headerPanel.py
===================================================================
--- arm/release/src/interface/headerPanel.py 2011-01-08 06:06:36 UTC (rev 24053)
+++ arm/release/src/interface/headerPanel.py 2011-01-08 06:35:08 UTC (rev 24054)
@@ -32,38 +32,52 @@
VERSION_STATUS_COLORS = {"new": "blue", "new in series": "blue", "obsolete": "red", "recommended": "green",
"old": "red", "unrecommended": "red", "unknown": "cyan"}
-DEFAULT_CONFIG = {"queries.ps.rate": 5}
-
class HeaderPanel(panel.Panel, threading.Thread):
"""
Top area contenting tor settings and system information. Stats are stored in
the vals mapping, keys including:
- tor/ version, versionStatus, nickname, orPort, dirPort, controlPort,
- exitPolicy, isAuthPassword (bool), isAuthCookie (bool)
- *address, *fingerprint, *flags
- sys/ hostname, os, version
- ps/ *%cpu, *rss, *%mem, pid, *etime
+ tor/ version, versionStatus, nickname, orPort, dirPort, controlPort,
+ exitPolicy, isAuthPassword (bool), isAuthCookie (bool)
+ *address, *fingerprint, *flags, pid, startTime
+ sys/ hostname, os, version
+ stat/ *%torCpu, *%armCpu, *rss, *%mem
* volatile parameter that'll be reset on each update
"""
- def __init__(self, stdscr, config=None):
+ def __init__(self, stdscr, startTime):
panel.Panel.__init__(self, stdscr, "header", 0)
threading.Thread.__init__(self)
self.setDaemon(True)
self._isTorConnected = True
self._lastUpdate = -1 # time the content was last revised
- self._isLastDrawWide = False
- self._isChanged = False # new stats to be drawn if true
self._isPaused = False # prevents updates if true
self._halt = False # terminates thread if true
self._cond = threading.Condition() # used for pausing the thread
- self._config = dict(DEFAULT_CONFIG)
- if config:
- config.update(self._config, {"queries.ps.rate": 1})
+ # Time when the panel was paused or tor was stopped. This is used to
+ # freeze the uptime statistic (uptime increments normally when None).
+ self._haltTime = None
+ # The last arm cpu usage sampling taken. This is a tuple of the form:
+ # (total arm cpu time, sampling timestamp)
+ #
+ # The initial cpu total should be zero. However, at startup the cpu time
+ # in practice is often greater than the real time causing the initially
+ # reported cpu usage to be over 100% (which shouldn't be possible on
+ # single core systems).
+ #
+ # Setting the initial cpu total to the value at this panel's init tends to
+ # give smoother results (staying in the same ballpark as the second
+ # sampling) so fudging the numbers this way for now.
+
+ self._armCpuSampling = (sum(os.times()[:3]), startTime)
+
+ # Last sampling received from the ResourceTracker, used to detect when it
+ # changes.
+ self._lastResourceFetch = -1
+
self.vals = {}
self.valsLock = threading.RLock()
self._update(True)
@@ -138,14 +152,22 @@
# Line 3 / Line 1 Right (system usage info)
y, x = (0, leftWidth) if isWide else (2, 0)
- if self.vals["ps/rss"] != "0": memoryLabel = uiTools.getSizeLabel(int(self.vals["ps/rss"]) * 1024)
+ if self.vals["stat/rss"] != "0": memoryLabel = uiTools.getSizeLabel(int(self.vals["stat/rss"]))
else: memoryLabel = "0"
- sysFields = ((0, "cpu: %s%%" % self.vals["ps/%cpu"]),
- (13, "mem: %s (%s%%)" % (memoryLabel, self.vals["ps/%mem"])),
- (34, "pid: %s" % (self.vals["ps/pid"] if self._isTorConnected else "")),
- (47, "uptime: %s" % self.vals["ps/etime"]))
+ uptimeLabel = ""
+ if self.vals["tor/startTime"]:
+ if self._haltTime:
+ # freeze the uptime when paused or the tor process is stopped
+ uptimeLabel = uiTools.getShortTimeLabel(self._haltTime - self.vals["tor/startTime"])
+ else:
+ uptimeLabel = uiTools.getShortTimeLabel(time.time() - self.vals["tor/startTime"])
+ sysFields = ((0, "cpu: %s%% tor, %s%% arm" % (self.vals["stat/%torCpu"], self.vals["stat/%armCpu"])),
+ (27, "mem: %s (%s%%)" % (memoryLabel, self.vals["stat/%mem"])),
+ (47, "pid: %s" % (self.vals["tor/pid"] if self._isTorConnected else "")),
+ (59, "uptime: %s" % uptimeLabel))
+
for (start, label) in sysFields:
if start + len(label) <= rightWidth: self.addstr(y, x + start, label)
else: break
@@ -197,39 +219,55 @@
# TODO: not sure what information to provide here...
pass
- self._isLastDrawWide = isWide
- self._isChanged = False
self.valsLock.release()
- def redraw(self, forceRedraw=False, block=False):
- # determines if the content needs to be redrawn or not
- isWide = self.getParent().getmaxyx()[1] >= MIN_DUAL_COL_WIDTH
- panel.Panel.redraw(self, forceRedraw or self._isChanged or isWide != self._isLastDrawWide, block)
-
def setPaused(self, isPause):
"""
If true, prevents updates from being presented.
"""
- self._isPaused = isPause
+ if not self._isPaused == isPause:
+ self._isPaused = isPause
+ if self._isTorConnected:
+ if isPause: self._haltTime = time.time()
+ else: self._haltTime = None
+
+ # Redraw now so we'll be displaying the state right when paused
+ # (otherwise the uptime might be off by a second, and change when
+ # the panel's redrawn for other reasons).
+ self.redraw(True)
def run(self):
"""
Keeps stats updated, querying new information at a set rate.
"""
+ lastDraw = time.time() - 1
while not self._halt:
- timeSinceReset = time.time() - self._lastUpdate
- psRate = self._config["queries.ps.rate"]
+ currentTime = time.time()
- if self._isPaused or timeSinceReset < psRate or not self._isTorConnected:
- sleepTime = max(0.5, psRate - timeSinceReset)
+ if self._isPaused or currentTime - lastDraw < 1 or not self._isTorConnected:
self._cond.acquire()
- if not self._halt: self._cond.wait(sleepTime)
+ if not self._halt: self._cond.wait(0.2)
self._cond.release()
else:
- self._update()
- self.redraw()
+ # Update the volatile attributes (cpu, memory, flags, etc) if we have
+ # a new resource usage sampling (the most dynamic stat) or its been
+ # twenty seconds since last fetched (so we still refresh occasionally
+ # when resource fetches fail).
+ #
+ # Otherwise, just redraw the panel to change the uptime field.
+
+ isChanged = False
+ if self.vals["tor/pid"]:
+ resourceTracker = sysTools.getResourceTracker(self.vals["tor/pid"])
+ isChanged = self._lastResourceFetch != resourceTracker.getRunCount()
+
+ if isChanged or currentTime - self._lastUpdate >= 20:
+ self._update()
+
+ self.redraw(True)
+ lastDraw += 1
def stop(self):
"""
@@ -252,10 +290,14 @@
if eventType == torTools.TOR_INIT:
self._isTorConnected = True
+ if self._isPaused: self._haltTime = time.time()
+ else: self._haltTime = None
+
self._update(True)
- self.redraw()
+ self.redraw(True)
elif eventType == torTools.TOR_CLOSED:
self._isTorConnected = False
+ self._haltTime = time.time()
self._update()
self.redraw(True)
@@ -310,55 +352,54 @@
self.vals["sys/version"] = unameVals[2]
pid = conn.getMyPid()
- self.vals["ps/pid"] = pid if pid else ""
+ self.vals["tor/pid"] = pid if pid else ""
+ startTime = conn.getStartTime()
+ self.vals["tor/startTime"] = startTime if startTime else ""
+
# reverts volatile parameters to defaults
self.vals["tor/fingerprint"] = "Unknown"
self.vals["tor/flags"] = []
- self.vals["ps/%cpu"] = "0"
- self.vals["ps/rss"] = "0"
- self.vals["ps/%mem"] = "0"
- self.vals["ps/etime"] = ""
+ self.vals["stat/%torCpu"] = "0"
+ self.vals["stat/%armCpu"] = "0"
+ self.vals["stat/rss"] = "0"
+ self.vals["stat/%mem"] = "0"
# sets volatile parameters
- volatile = {}
-
# TODO: This can change, being reported by STATUS_SERVER -> EXTERNAL_ADDRESS
# events. Introduce caching via torTools?
if self.vals["tor/address"] == "Unknown":
- volatile["tor/address"] = conn.getInfo("address", self.vals["tor/address"])
+ self.vals["tor/address"] = conn.getInfo("address", self.vals["tor/address"])
- volatile["tor/fingerprint"] = conn.getInfo("fingerprint", self.vals["tor/fingerprint"])
- volatile["tor/flags"] = conn.getMyFlags(self.vals["tor/flags"])
+ self.vals["tor/fingerprint"] = conn.getInfo("fingerprint", self.vals["tor/fingerprint"])
+ self.vals["tor/flags"] = conn.getMyFlags(self.vals["tor/flags"])
- # ps derived stats
- psParams = ["%cpu", "rss", "%mem", "etime"]
- if self.vals["ps/pid"]:
- # if call fails then everything except etime are zeroed out (most likely
- # tor's no longer running)
- volatile["ps/%cpu"] = "0"
- volatile["ps/rss"] = "0"
- volatile["ps/%mem"] = "0"
+ # ps or proc derived resource usage stats
+ if self.vals["tor/pid"]:
+ resourceTracker = sysTools.getResourceTracker(self.vals["tor/pid"])
- # the ps call formats results as:
- # %CPU RSS %MEM ELAPSED
- # 0.3 14096 1.3 29:51
- psRate = self._config["queries.ps.rate"]
- psCall = sysTools.call("ps -p %s -o %s" % (self.vals["ps/pid"], ",".join(psParams)), psRate, True)
-
- if psCall and len(psCall) >= 2:
- stats = psCall[1].strip().split()
-
- if len(stats) == len(psParams):
- for i in range(len(psParams)):
- volatile["ps/" + psParams[i]] = stats[i]
+ if resourceTracker.lastQueryFailed():
+ self.vals["stat/%torCpu"] = "0"
+ self.vals["stat/rss"] = "0"
+ self.vals["stat/%mem"] = "0"
+ else:
+ cpuUsage, _, memUsage, memUsagePercent = resourceTracker.getResourceUsage()
+ self._lastResourceFetch = resourceTracker.getRunCount()
+ self.vals["stat/%torCpu"] = "%0.1f" % (100 * cpuUsage)
+ self.vals["stat/rss"] = str(memUsage)
+ self.vals["stat/%mem"] = "%0.1f" % (100 * memUsagePercent)
- # checks if any changes have been made and merges volatile into vals
- self._isChanged |= setStatic
- for key, val in volatile.items():
- self._isChanged |= self.vals[key] != val
- self.vals[key] = val
+ # determines the cpu time for the arm process (including user and system
+ # time of both the primary and child processes)
- self._lastUpdate = time.time()
+ totalArmCpuTime, currentTime = sum(os.times()[:3]), time.time()
+ armCpuDelta = totalArmCpuTime - self._armCpuSampling[0]
+ armTimeDelta = currentTime - self._armCpuSampling[1]
+ pythonCpuTime = armCpuDelta / armTimeDelta
+ sysCallCpuTime = sysTools.getSysCpuUsage()
+ self.vals["stat/%armCpu"] = "%0.1f" % (100 * (pythonCpuTime + sysCallCpuTime))
+ self._armCpuSampling = (totalArmCpuTime, currentTime)
+
+ self._lastUpdate = currentTime
self.valsLock.release()
Modified: arm/release/src/interface/logPanel.py
===================================================================
--- arm/release/src/interface/logPanel.py 2011-01-08 06:06:36 UTC (rev 24053)
+++ arm/release/src/interface/logPanel.py 2011-01-08 06:35:08 UTC (rev 24054)
@@ -206,7 +206,7 @@
if not loggingLocation: return []
# includes the prefix for tor paths
- loggingLocation = torTools.getPathPrefix() + loggingLocation
+ loggingLocation = torTools.getConn().getPathPrefix() + loggingLocation
# if the runlevels argument is a superset of the log file then we can
# limit the read contents to the addLimit
@@ -794,7 +794,7 @@
self.scroll = max(0, min(self.scroll, self.lastContentHeight - height + 1))
# draws left-hand scroll bar if content's longer than the height
- msgIndent, dividerIndent = 0, 0 # offsets for scroll bar
+ msgIndent, dividerIndent = 1, 0 # offsets for scroll bar
isScrollBarVisible = self.lastContentHeight > height - 1
if isScrollBarVisible:
msgIndent, dividerIndent = 3, 2
Modified: arm/release/src/settings.cfg
===================================================================
--- arm/release/src/settings.cfg 2011-01-08 06:06:36 UTC (rev 24053)
+++ arm/release/src/settings.cfg 2011-01-08 06:35:08 UTC (rev 24054)
@@ -1,3 +1,116 @@
+# Summary descriptions for Tor configuration options
+# General Config Options
+config.summary.BandwidthRate Average bandwidth usage limit
+config.summary.BandwidthBurst Maximum bandwidth usage limit
+config.summary.MaxAdvertisedBandwidth Limit for the bandwidth we advertise as being available for relaying
+config.summary.RelayBandwidthRate Average bandwidth usage limit for relaying
+config.summary.RelayBandwidthBurst Maximum bandwidth usage limit for relaying
+config.summary.PerConnBWRate Average relayed bandwidth limit per connection
+config.summary.PerConnBWBurst Maximum relayed bandwidth limit per connection
+config.summary.ConnLimit Minimum number of file descriptors for Tor to start
+config.summary.ConstrainedSockets Shrinks sockets to ConstrainedSockSize
+config.summary.ConstrainedSockSize Limit for the received and transmit buffers of sockets
+config.summary.ControlPort Port providing access to tor controllers (arm, vidalia, etc)
+config.summary.ControlListenAddress Address providing controller access
+config.summary.ControlSocket Socket providing controller access
+config.summary.HashedControlPassword Hash of the password for authenticating to the control port
+config.summary.CookieAuthentication If set, authenticates controllers via a cookie
+config.summary.CookieAuthFile Location of the authentication cookie
+config.summary.CookieAuthFileGroupReadable Group read permissions for the authentication cookie
+config.summary.DataDirectory Location for storing runtime data (state, keys, etc)
+config.summary.DirServer Alternative directory authorities
+config.summary.AlternateDirAuthority Alternative directory authorities (consensus only)
+config.summary.AlternateHSAuthority Alternative directory authorities (hidden services only)
+config.summary.AlternateBridgeAuthority Alternative directory authorities (bridges only)
+config.summary.DisableAllSwap Locks all allocated memory so they can't be paged out
+config.summary.FetchDirInfoEarly Keeps consensus information up to date, even if unnecessary
+config.summary.FetchDirInfoExtraEarly Updates consensus information when it's first available
+config.summary.FetchHidServDescriptors Toggles if hidden service descriptors are fetched automatically or not
+config.summary.FetchServerDescriptors Toggles if the consensus is fetched automatically or not
+config.summary.FetchUselessDescriptors Toggles if relay descriptors are fetched when they aren't strictly necessary
+config.summary.Group GID for the process when started
+config.summary.HttpProxy HTTP proxy for connecting to tor
+config.summary.HttpProxyAuthenticator Authentication credentials for HttpProxy
+config.summary.HttpsProxy SSL proxy for connecting to tor
+config.summary.HttpsProxyAuthenticator Authentication credentials for HttpsProxy
+config.summary.Socks4Proxy SOCKS 4 proxy for connecting to tor
+config.summary.Socks5Proxy SOCKS 5 for connecting to tor
+config.summary.Socks5ProxyUsername Username for connecting to the Socks5Proxy
+config.summary.Socks5ProxyPassword Password for connecting to the Socks5Proxy
+config.summary.KeepalivePeriod Rate at which to send keepalive packets
+config.summary.Log Runlevels and location for tor logging
+config.summary.OutboundBindAddress Sets the IP used for connecting to tor
+config.summary.PidFile Path for a file tor writes containing its process id
+config.summary.ProtocolWarnings Toggles if protocol errors give warnings or not
+config.summary.RunAsDaemon Toggles if tor runs as a daemon process
+config.summary.SafeLogging Toggles if logs are scrubbed of sensitive information
+config.summary.User UID for the process when started
+config.summary.HardwareAccel Toggles if tor attempts to use hardware acceleration
+config.summary.AccelName OpenSSL engine name for crypto acceleration
+config.summary.AccelDir Crypto acceleration library path
+config.summary.AvoidDiskWrites Toggles if tor avoids frequently writing to disk
+config.summary.TunnelDirConns Toggles if directory requests can be made over the ORPort
+config.summary.PreferTunneledDirConns Avoids directory requests that can't be made over the ORPort if set
+config.summary.CircuitPriorityHalflife Overwrite method for prioritizing traffic among relayed connections
+
+# Client Config Options
+config.summary.AllowInvalidNodes Permits use of relays flagged as invalid by authorities
+config.summary.ExcludeSingleHopRelays Permits use of relays that allow single hop connections
+config.summary.Bridge Available bridges
+config.summary.LearnCircuitBuildTimeout Toggles adaptive timeouts for circuit creation
+config.summary.CircuitBuildTimeout Initial timeout for circuit creation
+config.summary.CircuitIdleTimeout Timeout for closing circuits that have never been used
+config.summary.CircuitStreamTimeout Timeout for shifting streams among circuits
+config.summary.ClientOnly Ensures that we aren't used as a relay or directory mirror
+config.summary.ExcludeNodes Relays or locales never to be used in circuits
+config.summary.ExcludeExitNodes Relays or locales never to be used for exits
+config.summary.EntryNodes Preferred first hops for circuits
+config.summary.ExitNodes Preferred final hop for circuits
+config.summary.StrictNodes Never uses notes outside of Entry/ExitNodes
+config.summary.FascistFirewall Only make outbound connections on FirewallPorts
+config.summary.FirewallPorts Ports used by FascistFirewall
+config.summary.HidServAuth Authentication credentials for connecting to a hidden service
+config.summary.ReachableAddresses Rules for bypassing the local firewall
+config.summary.ReachableDirAddresses Rules for bypassing the local firewall (directory fetches)
+config.summary.ReachableORAddresses Rules for bypassing the local firewall (OR connections)
+config.summary.LongLivedPorts Ports requiring highly reliable relays
+config.summary.MapAddress Alias mappings for address requests
+config.summary.NewCircuitPeriod Period for considering the creation of new circuits
+config.summary.MaxCircuitDirtiness Duration for reusing constructed circuits
+config.summary.NodeFamily Define relays as belonging to a family
+config.summary.EnforceDistinctSubnets Prevent use of multiple relays from the same subnet on a circuit
+config.summary.SocksPort Port for using tor as a Socks proxy
+config.summary.SocksListenAddress Address from which Socks connections can be made
+config.summary.SocksPolicy Access policy for the pocks port
+config.summary.SocksTimeout Time until idle or unestablished socks connections are closed
+config.summary.TrackHostExits Maintains use of the same exit whenever connecting to this destination
+config.summary.TrackHostExitsExpire Time until use of an exit for tracking expires
+config.summary.UpdateBridgesFromAuthority Toggles fetching bridge descriptors from the authorities
+config.summary.UseBridges Make use of configured bridges
+config.summary.UseEntryGuards Use guard relays for first hop
+config.summary.NumEntryGuards Pool size of guard relays we'll select from
+config.summary.SafeSocks Toggles rejecting unsafe variants of the socks protocol
+config.summary.TestSocks Provide notices for if socks connections are of the safe or unsafe variants
+config.summary.WarnUnsafeSocks Toggle warning of unsafe socks connection
+config.summary.VirtualAddrNetwork Address range used with MAPADDRESS
+config.summary.AllowNonRFC953Hostnames Toggles blocking invalid characters in hostname resolution
+config.summary.AllowDotExit Toggles allowing exit notation in addresses
+config.summary.FastFirstHopPK Toggle public key usage for the first hop
+config.summary.TransPort Port for transparent proxying if the OS supports it
+config.summary.TransListenAddress Address from which transparent proxy connections can be made
+config.summary.NATDPort Port for forwarding ipfw NATD connections
+config.summary.NATDListenAddress Address from which NATD forwarded connections can be made
+config.summary.AutomapHostsOnResolve Map addresses ending with special suffixes to virtual addresses
+config.summary.AutomapHostsSuffixes Address suffixes recognized by AutomapHostsOnResolve
+config.summary.DNSPort Port from which DNS responses are fetched instead of tor
+config.summary.DNSListenAddress Address for performing DNS resolution
+config.summary.ClientDNSRejectInternalAddresses Ignores DNS responses for internal addresses
+config.summary.DownloadExtraInfo Toggles fetching of extra information about relays
+config.summary.FallbackNetworkstatusFile Path for a fallback cache of the consensus
+config.summary.WarnPlaintextPorts Toggles warnings for using risky ports
+config.summary.RejectPlaintextPorts Prevents connections on risky ports
+config.summary.AllowSingleHopCircuits Makes use of single hop exits if able
+
# Snippets from common log messages
# These are static bits of log messages, used to determine when entries with
# dynamic content (hostnames, numbers, etc) are the same. If this matches the
@@ -39,12 +152,14 @@
# (Network is unreachable; NOROUTE; count 47; recommendation warn)
# [WARN] 4 unknown, 1 missing key, 3 good, 0 bad, 1 no signature, 4 required
# [ARM_DEBUG] refresh rate: 0.001 seconds
+# [ARM_DEBUG] proc call (process connections): /proc/net/[tcp|udp] (runtime: 0.0018)
# [ARM_DEBUG] system call: ps -p 2354 -o %cpu,rss,%mem,etime (runtime: 0.02)
# [ARM_DEBUG] system call: netstat -npt | grep 2354/tor (runtime: 0.02)
# [ARM_DEBUG] recreating panel 'graph' with the dimensions of 14/124
# [ARM_DEBUG] redrawing the log panel with the corrected content height (estimat was off by 4)
# [ARM_DEBUG] GETINFO accounting/bytes-left (runtime: 0.0006)
# [ARM_DEBUG] GETCONF MyFamily (runtime: 0.0007)
+# [ARM_DEBUG] Unable to query process resource usage from ps, waiting 6.25 seconds (unrecognized output from ps: ...)
msg.BW READ:
msg.DEBUG connection_handle_write(): After TLS write of
@@ -66,6 +181,13 @@
msg.WARN Problem bootstrapping. Stuck at
msg.WARN *missing key,
msg.ARM_DEBUG refresh rate:
+msg.ARM_DEBUG proc call (cwd):
+msg.ARM_DEBUG proc call (memory usage):
+msg.ARM_DEBUG proc call (process command
+msg.ARM_DEBUG proc call (process utime
+msg.ARM_DEBUG proc call (process stime
+msg.ARM_DEBUG proc call (process start time
+msg.ARM_DEBUG proc call (process connections):
msg.ARM_DEBUG system call: ps
msg.ARM_DEBUG system call: netstat
msg.ARM_DEBUG recreating panel '
@@ -75,6 +197,7 @@
msg.ARM_DEBUG GETINFO accounting/interval-end
msg.ARM_DEBUG GETINFO accounting/hibernating
msg.ARM_DEBUG GETCONF
+msg.ARM_DEBUG Unable to query process resource usage from ps
# some config options are fetched via special values
torrc.map HiddenServiceDir => HiddenServiceOptions
Modified: arm/release/src/starter.py
===================================================================
--- arm/release/src/starter.py 2011-01-08 06:06:36 UTC (rev 24053)
+++ arm/release/src/starter.py 2011-01-08 06:35:08 UTC (rev 24054)
@@ -10,6 +10,7 @@
import sys
import time
import getopt
+import socket
import version
import interface.controller
@@ -19,6 +20,7 @@
import util.hostnames
import util.log
import util.panel
+import util.procTools
import util.sysTools
import util.torConfig
import util.torTools
@@ -26,29 +28,33 @@
import TorCtl.TorCtl
import TorCtl.TorUtil
-DEFAULT_CONFIG = os.path.expanduser("~/.armrc")
+LOG_DUMP_PATH = os.path.expanduser("~/.arm/log")
+DEFAULT_CONFIG = os.path.expanduser("~/.arm/armrc")
CONFIG = {"startup.controlPassword": None,
"startup.interface.ipAddress": "127.0.0.1",
"startup.interface.port": 9051,
"startup.blindModeEnabled": False,
"startup.events": "N3",
+ "data.cache.path": "~/.arm/cache",
"features.config.descriptions.enabled": True,
- "features.config.descriptions.persistPath": "/tmp/arm/torConfigDescriptions.txt",
"log.configDescriptions.readManPageSuccess": util.log.INFO,
- "log.configDescriptions.readManPageFailed": util.log.WARN,
+ "log.configDescriptions.readManPageFailed": util.log.NOTICE,
+ "log.configDescriptions.internalLoadSuccess": util.log.NOTICE,
+ "log.configDescriptions.internalLoadFailed": util.log.ERR,
"log.configDescriptions.persistance.loadSuccess": util.log.INFO,
"log.configDescriptions.persistance.loadFailed": util.log.INFO,
"log.configDescriptions.persistance.saveSuccess": util.log.INFO,
"log.configDescriptions.persistance.saveFailed": util.log.NOTICE}
-OPT = "i:c:be:vh"
-OPT_EXPANDED = ["interface=", "config=", "blind", "event=", "version", "help"]
+OPT = "i:c:dbe:vh"
+OPT_EXPANDED = ["interface=", "config=", "debug", "blind", "event=", "version", "help"]
HELP_MSG = """Usage arm [OPTION]
Terminal status monitor for Tor relays.
-i, --interface [ADDRESS:]PORT change control interface from %s:%i
-c, --config CONFIG_PATH loaded configuration options, CONFIG_PATH
defaults to: %s
+ -d, --debug writes all arm logs to %s
-b, --blind disable connection lookups
-e, --event EVENT_FLAGS event types in message log (default: %s)
%s
@@ -58,13 +64,18 @@
Example:
arm -b -i 1643 hide connection data, attaching to control port 1643
arm -e we -c /tmp/cfg use this configuration file with 'WARN'/'ERR' events
-""" % (CONFIG["startup.interface.ipAddress"], CONFIG["startup.interface.port"], DEFAULT_CONFIG, CONFIG["startup.events"], interface.logPanel.EVENT_LISTING)
+""" % (CONFIG["startup.interface.ipAddress"], CONFIG["startup.interface.port"], DEFAULT_CONFIG, LOG_DUMP_PATH, CONFIG["startup.events"], interface.logPanel.EVENT_LISTING)
+# filename used for cached tor config descriptions
+CONFIG_DESC_FILENAME = "torConfigDesc.txt"
+
# messages related to loading the tor configuration descriptions
DESC_LOAD_SUCCESS_MSG = "Loaded configuration descriptions from '%s' (runtime: %0.3f)"
DESC_LOAD_FAILED_MSG = "Unable to load configuration descriptions (%s)"
+DESC_INTERNAL_LOAD_SUCCESS_MSG = "Falling back to descriptions for Tor %s"
+DESC_INTERNAL_LOAD_FAILED_MSG = "Unable to load fallback descriptions. Categories and help for Tor's configuration options won't be available. (%s)"
DESC_READ_MAN_SUCCESS_MSG = "Read descriptions for tor's configuration options from its man page (runtime %0.3f)"
-DESC_READ_MAN_FAILED_MSG = "Unable to read descriptions for tor's configuration options from its man page (%s)"
+DESC_READ_MAN_FAILED_MSG = "Unable to get the descriptions of Tor's configuration options from its man page (%s)"
DESC_SAVE_SUCCESS_MSG = "Saved configuration descriptions to '%s' (runtime: %0.3f)"
DESC_SAVE_FAILED_MSG = "Unable to save configuration descriptions (%s)"
@@ -95,7 +106,7 @@
return True
-def _loadConfigurationDescriptions():
+def _loadConfigurationDescriptions(pathPrefix):
"""
Attempts to load descriptions for tor's configuration options, fetching them
from the man page and persisting them to a file to speed future startups.
@@ -107,9 +118,16 @@
if CONFIG["features.config.descriptions.enabled"]:
isConfigDescriptionsLoaded = False
- descriptorPath = CONFIG["features.config.descriptions.persistPath"]
- # attempts to load persisted configuration descriptions
+ # determines the path where cached descriptions should be persisted (left
+ # undefined of arm caching is disabled)
+ cachePath, descriptorPath = CONFIG["data.cache.path"], None
+
+ if cachePath:
+ if not cachePath.endswith("/"): cachePath += "/"
+ descriptorPath = os.path.expanduser(cachePath) + CONFIG_DESC_FILENAME
+
+ # attempts to load configuration descriptions cached in the data directory
if descriptorPath:
try:
loadStartTime = time.time()
@@ -122,9 +140,9 @@
msg = DESC_LOAD_FAILED_MSG % util.sysTools.getFileErrorMsg(exc)
util.log.log(CONFIG["log.configDescriptions.persistance.loadFailed"], msg)
+ # fetches configuration options from the man page
if not isConfigDescriptionsLoaded:
try:
- # fetches configuration options from the man page
loadStartTime = time.time()
util.torConfig.loadOptionDescriptions()
isConfigDescriptionsLoaded = True
@@ -146,10 +164,25 @@
except IOError, exc:
msg = DESC_SAVE_FAILED_MSG % util.sysTools.getFileErrorMsg(exc)
util.log.log(CONFIG["log.configDescriptions.persistance.saveFailed"], msg)
+
+ # finally fall back to the cached descriptors provided with arm (this is
+ # often the case for tbb and manual builds)
+ if not isConfigDescriptionsLoaded:
+ try:
+ loadStartTime = time.time()
+ loadedVersion = util.torConfig.loadOptionDescriptions(pathPrefix + CONFIG_DESC_FILENAME, False)
+ isConfigDescriptionsLoaded = True
+
+ msg = DESC_INTERNAL_LOAD_SUCCESS_MSG % loadedVersion
+ util.log.log(CONFIG["log.configDescriptions.internalLoadSuccess"], msg)
+ except IOError, exc:
+ msg = DESC_INTERNAL_LOAD_FAILED_MSG % util.sysTools.getFileErrorMsg(exc)
+ util.log.log(CONFIG["log.configDescriptions.internalLoadFailed"], msg)
if __name__ == '__main__':
startTime = time.time()
param = dict([(key, None) for key in CONFIG.keys()])
+ isDebugMode = False
configPath = DEFAULT_CONFIG # path used for customized configuration
# parses user input, noting any issues
@@ -178,6 +211,7 @@
param["startup.interface.ipAddress"] = controlAddr
param["startup.interface.port"] = controlPort
elif opt in ("-c", "--config"): configPath = arg # sets path of user's config
+ elif opt in ("-d", "--debug"): isDebugMode = True # dumps all logs
elif opt in ("-b", "--blind"):
param["startup.blindModeEnabled"] = True # prevents connection lookups
elif opt in ("-e", "--event"):
@@ -189,14 +223,27 @@
print HELP_MSG
sys.exit()
+ if isDebugMode:
+ try:
+ util.log.setDumpFile(LOG_DUMP_PATH)
+
+ currentTime = time.localtime()
+ timeLabel = time.strftime("%H:%M:%S %m/%d/%Y (%Z)", currentTime)
+ initMsg = "Arm %s Debug Dump, %s" % (version.VERSION, timeLabel)
+
+ util.log.DUMP_FILE.write("%s\n%s\n" % (initMsg, "-" * len(initMsg)))
+ util.log.DUMP_FILE.flush()
+ except (OSError, IOError), exc:
+ print "Unable to write to debug log file: %s" % util.sysTools.getFileErrorMsg(exc)
+
config = util.conf.getConfig("arm")
# attempts to fetch attributes for parsing tor's logs, configuration, etc
+ pathPrefix = os.path.dirname(sys.argv[0])
+ if pathPrefix and not pathPrefix.endswith("/"):
+ pathPrefix = pathPrefix + "/"
+
try:
- pathPrefix = os.path.dirname(sys.argv[0])
- if pathPrefix and not pathPrefix.endswith("/"):
- pathPrefix = pathPrefix + "/"
-
config.load("%ssettings.cfg" % pathPrefix)
except IOError, exc:
msg = NO_INTERNAL_CFG_MSG % util.sysTools.getFileErrorMsg(exc)
@@ -218,7 +265,7 @@
config.update(CONFIG)
# loads user preferences for utilities
- for utilModule in (util.conf, util.connections, util.hostnames, util.log, util.panel, util.sysTools, util.torConfig, util.torTools, util.uiTools):
+ for utilModule in (util.conf, util.connections, util.hostnames, util.log, util.panel, util.procTools, util.sysTools, util.torConfig, util.torTools, util.uiTools):
utilModule.loadConfig(config)
# overwrites undefined parameters with defaults
@@ -252,7 +299,36 @@
TorCtl.INCORRECT_PASSWORD_MSG = "Controller password found in '%s' was incorrect" % configPath
authPassword = config.get("startup.controlPassword", CONFIG["startup.controlPassword"])
conn = TorCtl.TorCtl.connect(controlAddr, controlPort, authPassword)
- if conn == None: sys.exit(1)
+ if conn == None:
+ # Connecting to the control port will probably fail if it's using cookie
+ # authentication and the cookie path is relative (unfortunately this is
+ # the case for TBB). This is discussed in:
+ # https://trac.torproject.org/projects/tor/ticket/1101
+ #
+ # Until this is fixed including a hack to expand the relative path in
+ # these cases, setting conn to the established connection if successful
+ # and leaving it undefined otherwise. Even if successful this prints the
+ # error message saying that the auth cookie couldn't be found
+ # (unfortunately this is unavoidable without either changing TorCtl or
+ # making this a much bigger hack).
+
+ try:
+ s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+ s.connect((controlAddr, controlPort))
+ tmpConn = TorCtl.TorCtl.Connection(s)
+
+ if tmpConn.get_auth_type() == TorCtl.TorCtl.AUTH_TYPE.COOKIE:
+ cookiePath = tmpConn.get_auth_cookie_path()
+ torPid = util.torTools.getPid(controlPort)
+
+ if torPid and cookiePath[0] != "/":
+ # previous attempt to connect failed due to having a relative path - fix it
+ tmpConn._cookiePath = util.sysTools.expandRelativePath(cookiePath, torPid)
+ tmpConn.authenticate(cookiePath)
+ conn = tmpConn # success!
+ except: pass
+
+ if conn == None: sys.exit(1)
# removing references to the controller password so the memory can be freed
# (unfortunately python does allow for direct access to the memory so this
@@ -277,7 +353,7 @@
controller.init(conn)
# fetches descriptions for tor's configuration options
- _loadConfigurationDescriptions()
+ _loadConfigurationDescriptions(pathPrefix)
interface.controller.startTorMonitor(time.time() - initTime, expandedEvents, param["startup.blindModeEnabled"])
conn.close()
Copied: arm/release/src/test.py (from rev 24053, arm/trunk/src/test.py)
===================================================================
--- arm/release/src/test.py (rev 0)
+++ arm/release/src/test.py 2011-01-08 06:35:08 UTC (rev 24054)
@@ -0,0 +1,106 @@
+#!/usr/bin/env python
+
+"""
+Handler for arm tests and demos.
+"""
+
+import time
+from util import connections, torTools, uiTools
+
+MENU = """Arm Test Options:
+ 1. Resolver Performance Test
+ 2. Resolver Dump
+ 3. Glyph Demo
+ q. Quit
+
+Selection: """
+
+def printDivider():
+ print("\n" + "-" * 40 + "\n")
+
+conn = None
+while True:
+ userInput = raw_input(MENU)
+
+ # initiate the TorCtl connection if the test needs it
+ if userInput in ("1", "2") and not conn:
+ conn = torTools.getConn()
+ conn.init()
+
+ # prefetch pid so extra system calls don't effect the timing of tests
+ conn.getMyPid()
+
+ if userInput == "q":
+ break # quit test script
+ elif userInput == "1":
+ systemResolvers = connections.getSystemResolvers()
+ printDivider()
+
+ allConnectionResults = []
+ for resolver in systemResolvers:
+ startTime = time.time()
+ connectionResults = connections.getConnections(resolver, "tor", conn.getMyPid())
+ connectionResults.sort()
+ allConnectionResults.append(connectionResults)
+
+ resolverLabel = "%-10s" % connections.CMD_STR[resolver]
+ countLabel = "%4i results" % len(connectionResults)
+ timeLabel = "%0.4f seconds" % (time.time() - startTime)
+ print "%s %s %s" % (resolverLabel, countLabel, timeLabel)
+
+ allResolversMatch = True
+ firstResult = allConnectionResults.pop()
+ while allConnectionResults:
+ if allConnectionResults.pop() != firstResult:
+ allResolversMatch = False
+ break
+
+ if allResolversMatch:
+ print("\nThe results of all the connection resolvers match")
+ else:
+ print("\nWarning: Connection resolver results differ")
+
+ printDivider()
+ elif userInput == "2":
+ # use the given resolver to fetch tor's connections
+ while True:
+ # provide the selection options
+ printDivider()
+ print("Select a resolver:")
+ for i in range(1, 8):
+ print(" %i. %s" % (i, connections.CMD_STR[i]))
+ print(" q. Go back to the main menu")
+
+ userSelection = raw_input("\nSelection: ")
+ if userSelection == "q":
+ printDivider()
+ break
+
+ if userSelection.isdigit() and int(userSelection) in range(1, 8):
+ try:
+ resolver = int(userSelection)
+ startTime = time.time()
+
+ print(connections.getResolverCommand(resolver, "tor", conn.getMyPid()))
+ connectionResults = connections.getConnections(resolver, "tor", conn.getMyPid())
+ connectionResults.sort()
+
+ # prints results
+ printDivider()
+ for lIp, lPort, fIp, fPort in connectionResults:
+ print(" %s:%s -> %s:%s" % (lIp, lPort, fIp, fPort))
+
+ print("\n Runtime: %0.4f seconds" % (time.time() - startTime))
+ except IOError, exc:
+ print exc
+ else:
+ print("'%s' isn't a valid selection\n" % userSelection)
+ elif userInput == "3":
+ uiTools.demoGlyphs()
+
+ # Switching to a curses context and back repeatedly seems to screw up the
+ # terminal. Just to be safe this ends the process after the demo.
+ break
+ else:
+ print("'%s' isn't a valid selection\n" % userInput)
+
Copied: arm/release/src/torConfigDesc.txt (from rev 24053, arm/trunk/src/torConfigDesc.txt)
===================================================================
--- arm/release/src/torConfigDesc.txt (rev 0)
+++ arm/release/src/torConfigDesc.txt 2011-01-08 06:35:08 UTC (rev 24054)
@@ -0,0 +1,1123 @@
+Tor Version 0.2.2.13-alpha
+General
+index: 46
+acceldir
+DIR
+Specify this option if using dynamic hardware acceleration and the engine implementation library resides somewhere other than the OpenSSL default.
+--------------------------------------------------------------------------------
+General
+index: 45
+accelname
+NAME
+When using OpenSSL hardware crypto acceleration attempt to load the dynamic engine of this name. This must be used for any dynamic hardware engine. Names can be verified with the openssl engine command.
+--------------------------------------------------------------------------------
+Relay
+index: 119
+accountingmax
+N bytes|KB|MB|GB|TB
+Never send more than the specified number of bytes in a given accounting period, or receive more than that number in the period. For example, with AccountingMax set to 1 GB, a server could send 900 MB and receive 800 MB and continue running. It will only hibernate once one of the two reaches 1 GB. When the number of bytes is exhausted, Tor will hibernate until some time in the next accounting period. To prevent all servers from waking at the same time, Tor will also wait until a random point in each period before waking up. If you have bandwidth cost issues, enabling hibernation is preferable to setting a low bandwidth, since it provides users with a collection of fast servers that are up some of the time, which is more useful than a set of slow servers that are always "available".
+--------------------------------------------------------------------------------
+Relay
+index: 120
+accountingstart
+day|week|month [day] HH:MM
+Specify how long accounting periods last. If month is given, each accounting period runs from the time HH:MM on the dayth day of one month to the same day and time of the next. (The day must be between 1 and 28.) If week is given, each accounting period runs from the time HH:MM of the dayth day of one week to the same day and time of the next week, with Monday as day 1 and Sunday as day 7. If day is given, each accounting period runs from the time HH:MM each day to the same time on the next day. All times are local, and given in 24-hour time. (Defaults to "month 1 0:00".)
+--------------------------------------------------------------------------------
+Relay
+index: 104
+address
+address
+The IP address or fully qualified domain name of this server (e.g. moria.mit.edu). You can leave this unset, and Tor will guess your IP address.
+--------------------------------------------------------------------------------
+Client
+index: 89
+allowdotexit
+0|1
+If enabled, we convert "www.google.com.foo.exit" addresses on the SocksPort/TransPort/NatdPort into "www.google.com" addresses that exit from the node "foo". Disabled by default since attacking websites and exit relays can use it to manipulate your path selection. (Default: 0)
+--------------------------------------------------------------------------------
+Client
+index: 51
+allowinvalidnodes
+entry|exit|middle|introduction|rendezvous|...
+If some Tor servers are obviously not working right, the directory authorities can manually mark them as invalid, meaning that it's not recommended you use them for entry or exit positions in your circuits. You can opt to use them in some circuit positions, though. The default is "middle,rendezvous", and other choices are not advised.
+--------------------------------------------------------------------------------
+Client
+index: 88
+allownonrfc953hostnames
+0|1
+When this option is disabled, Tor blocks hostnames containing illegal characters (like @ and :) rather than sending them to an exit node to be resolved. This helps trap accidental attempts to resolve URLs and so on. (Default: 0)
+--------------------------------------------------------------------------------
+Relay
+index: 105
+allowsinglehopexits
+0|1
+This option controls whether clients can use this server as a single hop proxy. If set to 1, clients can use this server as an exit even if it is the only hop in the circuit. (Default: 0)
+--------------------------------------------------------------------------------
+General
+index: 20
+alternatebridgeauthority
+[nickname] [flags] address:port fingerprint
+As DirServer, but replaces less of the default directory authorities. Using AlternateDirAuthority replaces the default Tor directory authorities, but leaves the hidden service authorities and bridge authorities in place. Similarly, Using AlternateHSAuthority replaces the default hidden service authorities, but not the directory or bridge authorities.
+--------------------------------------------------------------------------------
+General
+index: 18
+alternatedirauthority
+[nickname] [flags] address:port fingerprint
+
+--------------------------------------------------------------------------------
+General
+index: 19
+alternatehsauthority
+[nickname] [flags] address:port fingerprint
+
+--------------------------------------------------------------------------------
+Relay
+index: 106
+assumereachable
+0|1
+This option is used when bootstrapping a new Tor network. If set to 1, don't do self-reachability testing; just upload your server descriptor immediately. If AuthoritativeDirectory is also set, this option instructs the dirserver to bypass remote reachability testing too and list all connected servers as running.
+--------------------------------------------------------------------------------
+Authority
+index: 154
+authdirbaddir
+AddressPattern...
+Authoritative directories only. A set of address patterns for servers that will be listed as bad directories in any network status document this authority publishes, if AuthDirListBadDirs is set.
+--------------------------------------------------------------------------------
+Authority
+index: 155
+authdirbadexit
+AddressPattern...
+Authoritative directories only. A set of address patterns for servers that will be listed as bad exits in any network status document this authority publishes, if AuthDirListBadExits is set.
+--------------------------------------------------------------------------------
+Authority
+index: 156
+authdirinvalid
+AddressPattern...
+Authoritative directories only. A set of address patterns for servers that will never be listed as "valid" in any network status document that this authority publishes.
+--------------------------------------------------------------------------------
+Authority
+index: 158
+authdirlistbaddirs
+0|1
+Authoritative directories only. If set to 1, this directory has some opinion about which nodes are unsuitable as directory caches. (Do not set this to 1 unless you plan to list non-functioning directories as bad; otherwise, you are effectively voting in favor of every declared directory.)
+--------------------------------------------------------------------------------
+Authority
+index: 159
+authdirlistbadexits
+0|1
+Authoritative directories only. If set to 1, this directory has some opinion about which nodes are unsuitable as exit nodes. (Do not set this to 1 unless you plan to list non-functioning exits as bad; otherwise, you are effectively voting in favor of every declared exit as an exit.)
+--------------------------------------------------------------------------------
+Authority
+index: 161
+authdirmaxserversperaddr
+NUM
+Authoritative directories only. The maximum number of servers that we will list as acceptable on a single IP address. Set this to "0" for "no limit". (Default: 2)
+--------------------------------------------------------------------------------
+Authority
+index: 162
+authdirmaxserversperauthaddr
+NUM
+Authoritative directories only. Like AuthDirMaxServersPerAddr, but applies to addresses shared with directory authorities. (Default: 5)
+--------------------------------------------------------------------------------
+Authority
+index: 157
+authdirreject
+AddressPattern...
+Authoritative directories only. A set of address patterns for servers that will never be listed at all in any network status document that this authority publishes, or accepted as an OR address in any descriptor submitted for publication by this authority.
+--------------------------------------------------------------------------------
+Authority
+index: 160
+authdirrejectunlisted
+0|1
+Authoritative directories only. If set to 1, the directory server rejects all uploaded server descriptors that aren't explicitly listed in the fingerprints file. This acts as a "panic button" if we get hit with a Sybil attack. (Default: 0)
+--------------------------------------------------------------------------------
+Directory
+index: 135
+authoritativedirectory
+0|1
+When this option is set to 1, Tor operates as an authoritative directory server. Instead of caching the directory, it generates its own list of good servers, signs it, and sends that to the clients. Unless the clients already have you listed as a trusted directory, you probably do not want to set this option. Please coordinate with the other admins at tor-ops at torproject.org if you think you should be a directory.
+--------------------------------------------------------------------------------
+Client
+index: 95
+automaphostsonresolve
+0|1
+When this option is enabled, and we get a request to resolve an address that ends with one of the suffixes in AutomapHostsSuffixes, we map an unused virtual address to that address, and return the new virtual address. This is handy for making ".onion" addresses work with applications that resolve an address and then connect to it. (Default: 0).
+--------------------------------------------------------------------------------
+Client
+index: 96
+automaphostssuffixes
+SUFFIX,SUFFIX,...
+A comma-separated list of suffixes to use with AutomapHostsOnResolve. The "." suffix is equivalent to "all addresses." (Default: .exit,.onion).
+--------------------------------------------------------------------------------
+General
+index: 47
+avoiddiskwrites
+0|1
+If non-zero, try to write to disk less frequently than we would otherwise. This is useful when running on flash memory or other media that support only a limited number of writes. (Default: 0)
+--------------------------------------------------------------------------------
+General
+index: 1
+bandwidthburst
+N bytes|KB|MB|GB
+Limit the maximum token bucket size (also known as the burst) to the given number of bytes in each direction. (Default: 10 MB)
+--------------------------------------------------------------------------------
+General
+index: 0
+bandwidthrate
+N bytes|KB|MB|GB
+A token bucket limits the average incoming bandwidth usage on this node to the specified number of bytes per second, and the average outgoing bandwidth usage to that same value. (Default: 5 MB)
+--------------------------------------------------------------------------------
+Client
+index: 53
+bridge
+IP:ORPort [fingerprint]
+When set along with UseBridges, instructs Tor to use the relay at "IP:ORPort" as a "bridge" relaying into the Tor network. If "fingerprint" is provided (using the same format as for DirServer), we will verify that the relay running at that location has the right fingerprint. We also use fingerprint to look up the bridge descriptor at the bridge authority, if it's provided and if UpdateBridgesFromAuthority is set too.
+--------------------------------------------------------------------------------
+Directory
+index: 144
+bridgeauthoritativedir
+0|1
+When this option is set in addition to AuthoritativeDirectory, Tor accepts and serves router descriptors, but it caches and serves the main networkstatus documents rather than generating its own. (Default: 0)
+--------------------------------------------------------------------------------
+Relay
+index: 127
+bridgerecordusagebycountry
+0|1
+When this option is enabled and BridgeRelay is also enabled, and we have GeoIP data, Tor keeps a keep a per-country count of how many client addresses have contacted it so that it can help the bridge authority guess which countries have blocked access to it. (Default: 1)
+--------------------------------------------------------------------------------
+Relay
+index: 107
+bridgerelay
+0|1
+Sets the relay to act as a "bridge" with respect to relaying connections from bridge users to the Tor network. Mainly it influences how the relay will cache and serve directory information. Usually used in combination with PublishServerDescriptor.
+--------------------------------------------------------------------------------
+Relay
+index: 130
+cellstatistics
+0|1
+When this option is enabled, Tor writes statistics on the mean time that cells spend in circuit queues to disk every 24 hours. Cannot be changed while Tor is running. (Default: 0)
+--------------------------------------------------------------------------------
+Client
+index: 54
+circuitbuildtimeout
+NUM
+Try for at most NUM seconds when building circuits. If the circuit isn't open in that time, give up on it. (Default: 1 minute.)
+--------------------------------------------------------------------------------
+Client
+index: 55
+circuitidletimeout
+NUM
+If we have kept a clean (never used) circuit around for NUM seconds, then close it. This way when the Tor client is entirely idle, it can expire all of its circuits, and then expire its TLS connections. Also, if we end up making a circuit that is not useful for exiting any of the requests we're receiving, it won't forever take up a slot in the circuit list. (Default: 1 hour.)
+--------------------------------------------------------------------------------
+General
+index: 50
+circuitpriorityhalflife
+NUM1
+If this value is set, we override the default algorithm for choosing which circuit's cell to deliver or relay next. When the value is 0, we round-robin between the active circuits on a connection, delivering one cell from each in turn. When the value is positive, we prefer delivering cells from whichever connection has the lowest weighted cell count, where cells are weighted exponentially according to the supplied CircuitPriorityHalflife value (in seconds). If this option is not set at all, we use the behavior recommended in the current consensus networkstatus. This is an advanced option; you generally shouldn't have to mess with it. (Default: not set.)
+--------------------------------------------------------------------------------
+Client
+index: 56
+circuitstreamtimeout
+NUM
+If non-zero, this option overrides our internal timeout schedule for how many seconds until we detach a stream from a circuit and try a new circuit. If your network is particularly slow, you might want to set this to a number like 60. (Default: 0)
+--------------------------------------------------------------------------------
+Client
+index: 99
+clientdnsrejectinternaladdresses
+0|1
+If true, Tor does not believe any anonymously retrieved DNS answer that tells it that an address resolves to an internal address (like 127.0.0.1 or 192.168.0.1). This option prevents certain browser-based attacks; don't turn it off unless you know what you're doing. (Default: 1).
+--------------------------------------------------------------------------------
+Client
+index: 57
+clientonly
+0|1
+If set to 1, Tor will under no circumstances run as a server or serve directory requests. The default is to run as a client unless ORPort is configured. (Usually, you don't need to set this; Tor is pretty smart at figuring out whether you are reliable and high-bandwidth enough to be a useful server.) (Default: 0)
+--------------------------------------------------------------------------------
+Authority
+index: 152
+consensusparams
+STRING
+STRING is a space-separated list of key=value pairs that Tor will include in the "params" line of its networkstatus vote.
+--------------------------------------------------------------------------------
+General
+index: 7
+constrainedsockets
+0|1
+If set, Tor will tell the kernel to attempt to shrink the buffers for all sockets to the size specified in ConstrainedSockSize. This is useful for virtual servers and other environments where system level TCP buffers may be limited. If you're on a virtual server, and you encounter the "Error creating network socket: No buffer space available" message, you are likely experiencing this problem.
+
+The preferred solution is to have the admin increase the buffer pool for the host itself via /proc/sys/net/ipv4/tcp_mem or equivalent facility; this configuration option is a second-resort.
+
+The DirPort option should also not be used if TCP buffers are scarce. The cached directory requests consume additional sockets which exacerbates the problem.
+
+You should not enable this feature unless you encounter the "no buffer space available" issue. Reducing the TCP buffers affects window size for the TCP stream and will reduce throughput in proportion to round trip time on long paths. (Default: 0.)
+--------------------------------------------------------------------------------
+General
+index: 8
+constrainedsocksize
+N bytes|KB
+When ConstrainedSockets is enabled the receive and transmit buffers for all sockets will be set to this limit. Must be a value between 2048 and 262144, in 1024 byte increments. Default of 8192 is recommended.
+--------------------------------------------------------------------------------
+Relay
+index: 108
+contactinfo
+email_address
+Administrative contact information for server. This line might get picked up by spam harvesters, so you may want to obscure the fact that it's an email address.
+--------------------------------------------------------------------------------
+General
+index: 10
+controllistenaddress
+IP[:PORT]
+Bind the controller listener to this address. If you specify a port, bind to this port rather than the one specified in ControlPort. We strongly recommend that you leave this alone unless you know what you're doing, since giving attackers access to your control listener is really dangerous. (Default: 127.0.0.1) This directive can be specified multiple times to bind to multiple addresses/ports.
+--------------------------------------------------------------------------------
+General
+index: 9
+controlport
+Port
+If set, Tor will accept connections on this port and allow those connections to control the Tor process using the Tor Control Protocol (described in control-spec.txt). Note: unless you also specify one of HashedControlPassword or CookieAuthentication, setting this option will cause Tor to allow any process on the local host to control it. This option is required for many Tor controllers; most use the value of 9051.
+--------------------------------------------------------------------------------
+General
+index: 11
+controlsocket
+Path
+Like ControlPort, but listens on a Unix domain socket, rather than a TCP socket. (Unix and Unix-like systems only.)
+--------------------------------------------------------------------------------
+General
+index: 13
+cookieauthentication
+0|1
+If this option is set to 1, don't allow any connections on the control port except when the connecting process knows the contents of a file named "control_auth_cookie", which Tor will create in its data directory. This authentication method should only be used on systems with good filesystem security. (Default: 0)
+--------------------------------------------------------------------------------
+General
+index: 14
+cookieauthfile
+Path
+If set, this option overrides the default location and file name for Tor's cookie file. (See CookieAuthentication above.)
+--------------------------------------------------------------------------------
+General
+index: 15
+cookieauthfilegroupreadable
+0|1|Groupname
+If this option is set to 0, don't allow the filesystem group to read the cookie file. If the option is set to 1, make the cookie file readable by the default GID. [Making the file readable by other groups is not yet implemented; let us know if you need this for some reason.] (Default: 0).
+--------------------------------------------------------------------------------
+General
+index: 16
+datadirectory
+DIR
+Store working data in DIR (Default: /usr/local/var/lib/tor)
+--------------------------------------------------------------------------------
+Authority
+index: 153
+dirallowprivateaddresses
+0|1
+If set to 1, Tor will accept router descriptors with arbitrary "Address" elements. Otherwise, if the address is not an IP address or is a private IP address, it will reject the router descriptor. Defaults to 0.
+--------------------------------------------------------------------------------
+Directory
+index: 147
+dirlistenaddress
+IP[:PORT]
+Bind the directory service to this address. If you specify a port, bind to this port rather than the one specified in DirPort. (Default: 0.0.0.0) This directive can be specified multiple times to bind to multiple addresses/ports.
+--------------------------------------------------------------------------------
+Directory
+index: 148
+dirpolicy
+policy,policy,...
+Set an entrance policy for this server, to limit who can connect to the directory ports. The policies have the same form as exit policies above.
+--------------------------------------------------------------------------------
+Directory
+index: 146
+dirport
+PORT
+Advertise the directory service on this port.
+--------------------------------------------------------------------------------
+Directory
+index: 136
+dirportfrontpage
+FILENAME
+When this option is set, it takes an HTML file and publishes it as "/" on the DirPort. Now relay operators can provide a disclaimer without needing to set up a separate webserver. There's a sample disclaimer in contrib/tor-exit-notice.html.
+--------------------------------------------------------------------------------
+Relay
+index: 131
+dirreqstatistics
+0|1
+When this option is enabled, Tor writes statistics on the number and response time of network status requests to disk every 24 hours. Cannot be changed while Tor is running. (Default: 0)
+--------------------------------------------------------------------------------
+General
+index: 17
+dirserver
+[nickname] [flags] address:port fingerprint
+Use a nonstandard authoritative directory server at the provided address and port, with the specified key fingerprint. This option can be repeated many times, for multiple authoritative directory servers. Flags are separated by spaces, and determine what kind of an authority this directory is. By default, every authority is authoritative for current ("v2")-style directories, unless the "no-v2" flag is given. If the "v1" flags is provided, Tor will use this server as an authority for old-style (v1) directories as well. (Only directory mirrors care about this.) Tor will use this server as an authority for hidden service information if the "hs" flag is set, or if the "v1" flag is set and the "no-hs" flag is not set. Tor will use this authority as a bridge authoritative directory if the "bridge" flag is set. If a flag "orport=port" is given, Tor will use the given port when opening encrypted tunnels to the dirserver. Lastly, if a flag "v3ident=fp" is given, the dirserver is a v3
directory authority whose v3 long-term signing key has the fingerprint fp.
+
+If no dirserver line is given, Tor will use the default directory servers. NOTE: this option is intended for setting up a private Tor network with its own directory authorities. If you use it, you will be distinguishable from other users, because you won't believe the same authorities they do.
+--------------------------------------------------------------------------------
+General
+index: 21
+disableallswap
+0|1
+If set to 1, Tor will attempt to lock all current and future memory pages, so that memory cannot be paged out. Windows, OS X and Solaris are currently not supported. We believe that this feature works on modern Gnu/Linux distributions, and that it should work on *BSD systems (untested). This option requires that you start your Tor as root, and you should use the User option to properly reduce Tor's privileges. (Default: 0)
+--------------------------------------------------------------------------------
+Client
+index: 98
+dnslistenaddress
+IP[:PORT]
+Bind to this address to listen for DNS connections. (Default: 127.0.0.1).
+--------------------------------------------------------------------------------
+Client
+index: 97
+dnsport
+PORT
+If non-zero, Tor listens for UDP DNS requests on this port and resolves them anonymously. (Default: 0).
+--------------------------------------------------------------------------------
+Client
+index: 100
+downloadextrainfo
+0|1
+If true, Tor downloads and caches "extra-info" documents. These documents contain information about servers other than the information in their regular router descriptors. Tor does not use this information for anything itself; to save bandwidth, leave this option turned off. (Default: 0).
+--------------------------------------------------------------------------------
+Client
+index: 74
+enforcedistinctsubnets
+0|1
+If 1, Tor will not put two servers whose IP addresses are "too close" on the same circuit. Currently, two addresses are "too close" if they lie in the same /16 range. (Default: 1)
+--------------------------------------------------------------------------------
+Client
+index: 60
+entrynodes
+node,node,...
+A list of identity fingerprints, nicknames, country codes and address patterns of nodes to use for the first hop in normal circuits. These are treated only as preferences unless StrictNodes (see below) is also set.
+--------------------------------------------------------------------------------
+Relay
+index: 132
+entrystatistics
+0|1
+When this option is enabled, Tor writes statistics on the number of directly connecting clients to disk every 24 hours. Cannot be changed while Tor is running. (Default: 0)
+--------------------------------------------------------------------------------
+Client
+index: 59
+excludeexitnodes
+node,node,...
+A list of identity fingerprints, nicknames, country codes and address patterns of nodes to never use when picking an exit node. Note that any node listed in ExcludeNodes is automatically considered to be part of this list.
+--------------------------------------------------------------------------------
+Client
+index: 58
+excludenodes
+node,node,...
+A list of identity fingerprints, nicknames, country codes and address patterns of nodes to never use when building a circuit. (Example: ExcludeNodes SlowServer, $ EFFFFFFFFFFFFFFF, {cc}, 255.254.0.0/8)
+--------------------------------------------------------------------------------
+Client
+index: 52
+excludesinglehoprelays
+0|1
+This option controls whether circuits built by Tor will include relays with the AllowSingleHopExits flag set to true. If ExcludeSingleHopRelays is set to 0, these relays will be included. Note that these relays might be at higher risk of being seized or observed, so they are not normally included. (Default: 1)
+--------------------------------------------------------------------------------
+Client
+index: 61
+exitnodes
+node,node,...
+A list of identity fingerprints, nicknames, country codes and address patterns of nodes to use for the last hop in normal exit circuits. These are treated only as preferences unless StrictNodes (see below) is also set.
+--------------------------------------------------------------------------------
+Relay
+index: 109
+exitpolicy
+policy,policy,...
+Set an exit policy for this server. Each policy is of the form "accept|reject ADDR[/MASK][:PORT]". If /MASK is omitted then this policy just applies to the host given. Instead of giving a host or network you can also use "*" to denote the universe (0.0.0.0/0). PORT can be a single port number, an interval of ports "FROM_PORT-TO_PORT", or "*". If PORT is omitted, that means "*".
+
+For example, "accept 18.7.22.69:*,reject 18.0.0.0/8:*,accept *:*" would reject any traffic destined for MIT except for web.mit.edu, and accept anything else.
+
+To specify all internal and link-local networks (including 0.0.0.0/8, 169.254.0.0/16, 127.0.0.0/8, 192.168.0.0/16, 10.0.0.0/8, and 172.16.0.0/12), you can use the "private" alias instead of an address. These addresses are rejected by default (at the beginning of your exit policy), along with your public IP address, unless you set the ExitPolicyRejectPrivate config option to 0. For example, once you've done that, you could allow HTTP to 127.0.0.1 and block all other connections to internal networks with "accept 127.0.0.1:80,reject private:*", though that may also allow connections to your own computer that are addressed to its public (external) IP address. See RFC 1918 and RFC 3330 for more details about internal and reserved IP address space.
+
+This directive can be specified multiple times so you don't have to put it all on one line.
+
+Policies are considered first to last, and the first match wins. If you want to _replace_ the default exit policy, end your exit policy with either a reject *:* or an accept *:*. Otherwise, you're _augmenting_ (prepending to) the default exit policy. The default exit policy is:
+
+ reject *:25
+ reject *:119
+ reject *:135-139
+ reject *:445
+ reject *:563
+ reject *:1214
+ reject *:4661-4666
+ reject *:6346-6429
+ reject *:6699
+ reject *:6881-6999
+ accept *:*
+--------------------------------------------------------------------------------
+Relay
+index: 110
+exitpolicyrejectprivate
+0|1
+Reject all private (local) networks, along with your own public IP address, at the beginning of your exit policy. See above entry on ExitPolicy. (Default: 1)
+--------------------------------------------------------------------------------
+Relay
+index: 133
+exitportstatistics
+0|1
+When this option is enabled, Tor writes statistics on the number of relayed bytes and opened stream per exit port to disk every 24 hours. Cannot be changed while Tor is running. (Default: 0)
+--------------------------------------------------------------------------------
+Relay
+index: 134
+extrainfostatistics
+0|1
+When this option is enabled, Tor includes previously gathered statistics in its extra-info documents that it uploads to the directory authorities. (Default: 0)
+--------------------------------------------------------------------------------
+Client
+index: 101
+fallbacknetworkstatusfile
+FILENAME
+If Tor doesn't have a cached networkstatus file, it starts out using this one instead. Even if this file is out of date, Tor can still use it to learn about directory mirrors, so it doesn't need to put load on the authorities. (Default: None).
+--------------------------------------------------------------------------------
+Client
+index: 63
+fascistfirewall
+0|1
+If 1, Tor will only create outgoing connections to ORs running on ports that your firewall allows (defaults to 80 and 443; see FirewallPorts). This will allow you to run Tor as a client behind a firewall with restrictive policies, but will not allow you to run as a server behind such a firewall. If you prefer more fine-grained control, use ReachableAddresses instead.
+--------------------------------------------------------------------------------
+Client
+index: 90
+fastfirsthoppk
+0|1
+When this option is disabled, Tor uses the public key step for the first hop of creating circuits. Skipping it is generally safe since we have already used TLS to authenticate the relay and to establish forward-secure keys. Turning this option off makes circuit building slower.
+
+Note that Tor will always use the public key step for the first hop if it's operating as a relay, and it will never use the public key step if it doesn't yet know the onion key of the first hop. (Default: 1)
+--------------------------------------------------------------------------------
+General
+index: 22
+fetchdirinfoearly
+0|1
+If set to 1, Tor will always fetch directory information like other directory caches, even if you don't meet the normal criteria for fetching early. Normal users should leave it off. (Default: 0)
+--------------------------------------------------------------------------------
+General
+index: 23
+fetchdirinfoextraearly
+0|1
+If set to 1, Tor will fetch directory information before other directory caches. It will attempt to download directory information closer to the start of the consensus period. Normal users should leave it off. (Default: 0)
+--------------------------------------------------------------------------------
+General
+index: 24
+fetchhidservdescriptors
+0|1
+If set to 0, Tor will never fetch any hidden service descriptors from the rendezvous directories. This option is only useful if you're using a Tor controller that handles hidden service fetches for you. (Default: 1)
+--------------------------------------------------------------------------------
+General
+index: 25
+fetchserverdescriptors
+0|1
+If set to 0, Tor will never fetch any network status summaries or server descriptors from the directory servers. This option is only useful if you're using a Tor controller that handles directory fetches for you. (Default: 1)
+--------------------------------------------------------------------------------
+General
+index: 26
+fetchuselessdescriptors
+0|1
+If set to 1, Tor will fetch every non-obsolete descriptor from the authorities that it hears about. Otherwise, it will avoid fetching useless descriptors, for example for routers that are not running. This option is useful if you're using the contributed "exitlist" script to enumerate Tor nodes that exit to certain addresses. (Default: 0)
+--------------------------------------------------------------------------------
+Client
+index: 64
+firewallports
+PORTS
+A list of ports that your firewall allows you to connect to. Only used when FascistFirewall is set. This option is deprecated; use ReachableAddresses instead. (Default: 80, 443)
+--------------------------------------------------------------------------------
+Relay
+index: 129
+geoipfile
+filename
+A filename containing GeoIP data, for use with BridgeRecordUsageByCountry.
+--------------------------------------------------------------------------------
+General
+index: 44
+hardwareaccel
+0|1
+If non-zero, try to use built-in (static) crypto hardware acceleration when available. (Default: 0)
+--------------------------------------------------------------------------------
+General
+index: 12
+hashedcontrolpassword
+hashed_password
+Don't allow any connections on the control port except when the other process knows the password whose one-way hash is hashed_password. You can compute the hash of a password by running "tor --hash-password password". You can provide several acceptable passwords by using more than one HashedControlPassword line.
+--------------------------------------------------------------------------------
+Hidden Service
+index: 171
+hiddenserviceauthorizeclient
+auth-type client-name,client-name,...
+If configured, the hidden service is accessible for authorized clients only. The auth-type can either be 'basic' for a general-purpose authorization protocol or 'stealth' for a less scalable protocol that also hides service activity from unauthorized clients. Only clients that are listed here are authorized to access the hidden service. Valid client names are 1 to 19 characters long and only use characters in A-Za-z0-9+-_ (no spaces). If this option is set, the hidden service is not accessible for clients without authorization any more. Generated authorization data can be found in the hostname file. Clients need to put this authorization data in their configuration file using HidServAuth.
+--------------------------------------------------------------------------------
+Hidden Service
+index: 167
+hiddenservicedir
+DIRECTORY
+Store data files for a hidden service in DIRECTORY. Every hidden service must have a separate directory. You may use this option multiple times to specify multiple services.
+--------------------------------------------------------------------------------
+Hidden Service
+index: 168
+hiddenserviceport
+VIRTPORT [TARGET]
+Configure a virtual port VIRTPORT for a hidden service. You may use this option multiple times; each time applies to the service using the most recent hiddenservicedir. By default, this option maps the virtual port to the same port on 127.0.0.1. You may override the target port, address, or both by specifying a target of addr, port, or addr:port. You may also have multiple lines with the same VIRTPORT: when a user connects to that VIRTPORT, one of the TARGETs from those lines will be chosen at random.
+--------------------------------------------------------------------------------
+Hidden Service
+index: 170
+hiddenserviceversion
+version,version,...
+A list of rendezvous service descriptor versions to publish for the hidden service. Currently, only version 2 is supported. (Default: 2)
+--------------------------------------------------------------------------------
+Client
+index: 65
+hidservauth
+onion-address auth-cookie [service-name]
+Client authorization for a hidden service. Valid onion addresses contain 16 characters in a-z2-7 plus ".onion", and valid auth cookies contain 22 characters in A-Za-z0-9+/. The service name is only used for internal purposes, e.g., for Tor controllers. This option may be used multiple times for different hidden services. If a hidden service uses authorization and this option is not set, the hidden service is not accessible. Hidden services can be configured to require authorization using the HiddenServiceAuthorizeClient option.
+--------------------------------------------------------------------------------
+Directory
+index: 143
+hidservdirectoryv2
+0|1
+When this option is set, Tor accepts and serves v2 hidden service descriptors. Setting DirPort is not required for this, because clients connect via the ORPort by default. (Default: 1)
+--------------------------------------------------------------------------------
+Directory
+index: 142
+hsauthoritativedir
+0|1
+When this option is set in addition to AuthoritativeDirectory, Tor also accepts and serves hidden service descriptors. (Default: 0)
+--------------------------------------------------------------------------------
+General
+index: 27
+httpproxy
+host[:port]
+Tor will make all its directory requests through this host:port (or host:80 if port is not specified), rather than connecting directly to any directory servers.
+--------------------------------------------------------------------------------
+General
+index: 28
+httpproxyauthenticator
+username:password
+If defined, Tor will use this username:password for Basic HTTP proxy authentication, as in RFC 2617. This is currently the only form of HTTP proxy authentication that Tor supports; feel free to submit a patch if you want it to support others.
+--------------------------------------------------------------------------------
+General
+index: 29
+httpsproxy
+host[:port]
+Tor will make all its OR (SSL) connections through this host:port (or host:443 if port is not specified), via HTTP CONNECT rather than connecting directly to servers. You may want to set FascistFirewall to restrict the set of ports you might try to connect to, if your HTTPS proxy only allows connecting to certain ports.
+--------------------------------------------------------------------------------
+General
+index: 30
+httpsproxyauthenticator
+username:password
+If defined, Tor will use this username:password for Basic HTTPS proxy authentication, as in RFC 2617. This is currently the only form of HTTPS proxy authentication that Tor supports; feel free to submit a patch if you want it to support others.
+--------------------------------------------------------------------------------
+General
+index: 35
+keepaliveperiod
+NUM
+To keep firewalls from expiring connections, send a padding keepalive cell every NUM seconds on open connections that are in use. If the connection has no open circuits, it will instead be closed after NUM seconds of idleness. (Default: 5 minutes)
+--------------------------------------------------------------------------------
+General
+index: 37
+log
+minSeverity[-maxSeverity] file FILENAME
+As above, but send log messages to the listed filename. The "Log" option may appear more than once in a configuration file. Messages are sent to all the logs that match their severity level.
+--------------------------------------------------------------------------------
+Client
+index: 69
+longlivedports
+PORTS
+A list of ports for services that tend to have long-running connections (e.g. chat and interactive shells). Circuits for streams that use these ports will contain only high-uptime nodes, to reduce the chance that a node will go down before the stream is finished. (Default: 21, 22, 706, 1863, 5050, 5190, 5222, 5223, 6667, 6697, 8300)
+--------------------------------------------------------------------------------
+Client
+index: 70
+mapaddress
+address newaddress
+When a request for address arrives to Tor, it will rewrite it to newaddress before processing it. For example, if you always want connections to www.indymedia.org to exit via torserver (where torserver is the nickname of the server), use "MapAddress www.indymedia.org www.indymedia.org.torserver.exit".
+--------------------------------------------------------------------------------
+General
+index: 2
+maxadvertisedbandwidth
+N bytes|KB|MB|GB
+If set, we will not advertise more than this amount of bandwidth for our BandwidthRate. Server operators who want to reduce the number of clients who ask to build circuits through them (since this is proportional to advertised bandwidth rate) can thus reduce the CPU demands on their server without impacting network performance.
+--------------------------------------------------------------------------------
+Client
+index: 72
+maxcircuitdirtiness
+NUM
+Feel free to reuse a circuit that was first used at most NUM seconds ago, but never attach a new stream to a circuit that is too old. (Default: 10 minutes)
+--------------------------------------------------------------------------------
+Relay
+index: 111
+maxonionspending
+NUM
+If you have more than this number of onionskins queued for decrypt, reject new ones. (Default: 100)
+--------------------------------------------------------------------------------
+Directory
+index: 145
+minuptimehidservdirectoryv2
+N seconds|minutes|hours|days|weeks
+Minimum uptime of a v2 hidden service directory to be accepted as such by authoritative directories. (Default: 24 hours)
+--------------------------------------------------------------------------------
+Relay
+index: 112
+myfamily
+node,node,...
+Declare that this Tor server is controlled or administered by a group or organization identical or similar to that of the other servers, defined by their identity fingerprints or nicknames. When two servers both declare that they are in the same 'family', Tor clients will not use them in the same circuit. (Each server only needs to list the other servers in its family; it doesn't need to list itself, but it won't hurt.)
+--------------------------------------------------------------------------------
+Directory
+index: 141
+namingauthoritativedirectory
+0|1
+When this option is set to 1, then the server advertises that it has opinions about nickname-to-fingerprint bindings. It will include these opinions in its published network-status pages, by listing servers with the flag "Named" if a correct binding between that nickname and fingerprint has been registered with the dirserver. Naming dirservers will refuse to accept or publish descriptors that contradict a registered binding. See approved-routers in the FILES section below.
+--------------------------------------------------------------------------------
+Client
+index: 94
+natdlistenaddress
+IP[:PORT]
+Bind to this address to listen for NATD connections. (Default: 127.0.0.1).
+--------------------------------------------------------------------------------
+Client
+index: 93
+natdport
+PORT
+Allow old versions of ipfw (as included in old versions of FreeBSD, etc.) to send connections through Tor using the NATD protocol. This option is only for people who cannot use TransPort.
+--------------------------------------------------------------------------------
+Client
+index: 71
+newcircuitperiod
+NUM
+Every NUM seconds consider whether to build a new circuit. (Default: 30 seconds)
+--------------------------------------------------------------------------------
+Relay
+index: 113
+nickname
+name
+Set the server's nickname to 'name'. Nicknames must be between 1 and 19 characters inclusive, and must contain only the characters [a-zA-Z0-9].
+--------------------------------------------------------------------------------
+Client
+index: 73
+nodefamily
+node,node,...
+The Tor servers, defined by their identity fingerprints or nicknames, constitute a "family" of similar or co-administered servers, so never use any two of them in the same circuit. Defining a NodeFamily is only needed when a server doesn't list the family itself (with MyFamily). This option can be used multiple times.
+--------------------------------------------------------------------------------
+Relay
+index: 114
+numcpus
+num
+How many processes to use at once for decrypting onionskins. (Default: 1)
+--------------------------------------------------------------------------------
+Client
+index: 84
+numentryguards
+NUM
+If UseEntryGuards is set to 1, we will try to pick a total of NUM routers as long-term entries for our circuits. (Defaults to 3.)
+--------------------------------------------------------------------------------
+Relay
+index: 116
+orlistenaddress
+IP[:PORT]
+Bind to this IP address to listen for connections from Tor clients and servers. If you specify a port, bind to this port rather than the one specified in ORPort. (Default: 0.0.0.0) This directive can be specified multiple times to bind to multiple addresses/ports.
+--------------------------------------------------------------------------------
+Relay
+index: 115
+orport
+PORT
+Advertise this port to listen for connections from Tor clients and servers.
+--------------------------------------------------------------------------------
+General
+index: 38
+outboundbindaddress
+IP
+Make all outbound connections originate from the IP address specified. This is only useful when you have multiple network interfaces, and you want all of Tor's outgoing connections to use a single one. This setting will be ignored for connections to the loopback addresses (127.0.0.0/8 and ::1).
+--------------------------------------------------------------------------------
+General
+index: 6
+perconnbwburst
+N bytes|KB|MB|GB
+If set, do separate rate limiting for each connection from a non-relay. You should never need to change this value, since a network-wide value is published in the consensus and your relay will use that value. (Default: 0)
+--------------------------------------------------------------------------------
+General
+index: 5
+perconnbwrate
+N bytes|KB|MB|GB
+If set, do separate rate limiting for each connection from a non-relay. You should never need to change this value, since a network-wide value is published in the consensus and your relay will use that value. (Default: 0)
+--------------------------------------------------------------------------------
+General
+index: 39
+pidfile
+FILE
+On startup, write our PID to FILE. On clean shutdown, remove FILE.
+--------------------------------------------------------------------------------
+General
+index: 49
+prefertunneleddirconns
+0|1
+If non-zero, we will avoid directory servers that don't support tunneled directory connections, when possible. (Default: 1)
+--------------------------------------------------------------------------------
+General
+index: 40
+protocolwarnings
+0|1
+If 1, Tor will log with severity 'warn' various cases of other parties not following the Tor specification. Otherwise, they are logged with severity 'info'. (Default: 0)
+--------------------------------------------------------------------------------
+Hidden Service
+index: 169
+publishhidservdescriptors
+0|1
+If set to 0, Tor will run any hidden services you configure, but it won't advertise them to the rendezvous directory. This option is only useful if you're using a Tor controller that handles hidserv publishing for you. (Default: 1)
+--------------------------------------------------------------------------------
+Relay
+index: 117
+publishserverdescriptor
+0|1|v1|v2|v3|bridge|hidserv,...
+This option is only considered if you have an ORPort defined. You can choose multiple arguments, separated by commas. If set to 0, Tor will act as a server but it will not publish its descriptor to the directory authorities. (This is useful if you're testing out your server, or if you're using a Tor controller that handles directory publishing for you.) Otherwise, Tor will publish its descriptor to all directory authorities of the type(s) specified. The value "1" is the default, which means "publish to the appropriate authorities".
+--------------------------------------------------------------------------------
+Client
+index: 66
+reachableaddresses
+ADDR[/MASK][:PORT]...
+A comma-separated list of IP addresses and ports that your firewall allows you to connect to. The format is as for the addresses in ExitPolicy, except that "accept" is understood unless "reject" is explicitly provided. For example, 'ReachableAddresses 99.0.0.0/8, reject 18.0.0.0/8:80, accept *:80' means that your firewall allows connections to everything inside net 99, rejects port 80 connections to net 18, and accepts connections to port 80 otherwise. (Default: 'accept *:*'.)
+--------------------------------------------------------------------------------
+Client
+index: 67
+reachablediraddresses
+ADDR[/MASK][:PORT]...
+Like ReachableAddresses, a list of addresses and ports. Tor will obey these restrictions when fetching directory information, using standard HTTP GET requests. If not set explicitly then the value of ReachableAddresses is used. If HTTPProxy is set then these connections will go through that proxy.
+--------------------------------------------------------------------------------
+Client
+index: 68
+reachableoraddresses
+ADDR[/MASK][:PORT]...
+Like ReachableAddresses, a list of addresses and ports. Tor will obey these restrictions when connecting to Onion Routers, using TLS/SSL. If not set explicitly then the value of ReachableAddresses is used. If HTTPSProxy is set then these connections will go through that proxy.
+
+The separation between ReachableORAddresses and ReachableDirAddresses is only interesting when you are connecting through proxies (see HTTPProxy and HTTPSProxy). Most proxies limit TLS connections (which Tor uses to connect to Onion Routers) to port 443, and some limit HTTP GET requests (which Tor uses for fetching directory information) to port 80.
+--------------------------------------------------------------------------------
+Authority
+index: 150
+recommendedclientversions
+STRING
+STRING is a comma-separated list of Tor versions currently believed to be safe for clients to use. This information is included in version 2 directories. If this is not set then the value of RecommendedVersions is used. When this is set then VersioningAuthoritativeDirectory should be set too.
+--------------------------------------------------------------------------------
+Authority
+index: 151
+recommendedserverversions
+STRING
+STRING is a comma-separated list of Tor versions currently believed to be safe for servers to use. This information is included in version 2 directories. If this is not set then the value of RecommendedVersions is used. When this is set then VersioningAuthoritativeDirectory should be set too.
+--------------------------------------------------------------------------------
+Authority
+index: 149
+recommendedversions
+STRING
+STRING is a comma-separated list of Tor versions currently believed to be safe. The list is included in each directory, and nodes which pull down the directory learn whether they need to upgrade. This option can appear multiple times: the values from multiple lines are spliced together. When this is set then VersioningAuthoritativeDirectory should be set too.
+--------------------------------------------------------------------------------
+Client
+index: 103
+rejectplaintextports
+port,port,...
+Like WarnPlaintextPorts, but instead of warning about risky port uses, Tor will instead refuse to make the connection. (Default: None).
+--------------------------------------------------------------------------------
+General
+index: 4
+relaybandwidthburst
+N bytes|KB|MB|GB
+Limit the maximum token bucket size (also known as the burst) for _relayed traffic_ to the given number of bytes in each direction. (Default: 0)
+--------------------------------------------------------------------------------
+General
+index: 3
+relaybandwidthrate
+N bytes|KB|MB|GB
+If defined, a separate token bucket limits the average incoming bandwidth usage for _relayed traffic_ on this node to the specified number of bytes per second, and the average outgoing bandwidth usage to that same value. Relayed traffic currently is calculated to include answers to directory requests, but that may change in future versions. (Default: 0)
+--------------------------------------------------------------------------------
+Hidden Service
+index: 172
+rendpostperiod
+N seconds|minutes|hours|days|weeks
+Every time the specified period elapses, Tor uploads any rendezvous service descriptors to the directory servers. This information is also uploaded whenever it changes. (Default: 20 minutes)
+--------------------------------------------------------------------------------
+General
+index: 41
+runasdaemon
+0|1
+If 1, Tor forks and daemonizes to the background. This option has no effect on Windows; instead you should use the --service command-line option. (Default: 0)
+--------------------------------------------------------------------------------
+General
+index: 42
+safelogging
+0|1|relay
+Tor can scrub potentially sensitive strings from log messages (e.g. addresses) by replacing them with the string [scrubbed]. This way logs can still be useful, but they don't leave behind personally identifying information about what sites a user might have visited.
+
+If this option is set to 0, Tor will not perform any scrubbing, if it is set to 1, all potentially sensitive strings are replaced. If it is set to relay, all log messages generated when acting as a relay are sanitized, but all messages generated when acting as a client are not. (Default: 1)
+--------------------------------------------------------------------------------
+Client
+index: 85
+safesocks
+0|1
+When this option is enabled, Tor will reject application connections that use unsafe variants of the socks protocol ones that only provide an IP address, meaning the application is doing a DNS resolve first. Specifically, these are socks4 and socks5 when not doing remote DNS. (Defaults to 0.)
+--------------------------------------------------------------------------------
+Relay
+index: 122
+serverdnsallowbrokenconfig
+0|1
+If this option is false, Tor exits immediately if there are problems parsing the system DNS configuration or connecting to nameservers. Otherwise, Tor continues to periodically retry the system nameservers until it eventually succeeds. (Defaults to "1".)
+--------------------------------------------------------------------------------
+Relay
+index: 126
+serverdnsallownonrfc953hostnames
+0|1
+When this option is disabled, Tor does not try to resolve hostnames containing illegal characters (like @ and :) rather than sending them to an exit node to be resolved. This helps trap accidental attempts to resolve URLs and so on. This option only affects name lookups that your server does on behalf of clients. (Default: 0)
+--------------------------------------------------------------------------------
+Relay
+index: 124
+serverdnsdetecthijacking
+0|1
+When this option is set to 1, we will test periodically to determine whether our local nameservers have been configured to hijack failing DNS requests (usually to an advertising site). If they are, we will attempt to correct this. This option only affects name lookups that your server does on behalf of clients. (Defaults to "1".)
+--------------------------------------------------------------------------------
+Relay
+index: 128
+serverdnsrandomizecase
+0|1
+When this option is set, Tor sets the case of each character randomly in outgoing DNS requests, and makes sure that the case matches in DNS replies. This so-called "0x20 hack" helps resist some types of DNS poisoning attack. For more information, see "Increased DNS Forgery Resistance through 0x20-Bit Encoding". This option only affects name lookups that your server does on behalf of clients. (Default: 1)
+--------------------------------------------------------------------------------
+Relay
+index: 121
+serverdnsresolvconffile
+filename
+Overrides the default DNS configuration with the configuration in filename. The file format is the same as the standard Unix "resolv.conf" file (7). This option, like all other ServerDNS options, only affects name lookups that your server does on behalf of clients. (Defaults to use the system DNS configuration.)
+--------------------------------------------------------------------------------
+Relay
+index: 123
+serverdnssearchdomains
+0|1
+If set to 1, then we will search for addresses in the local search domain. For example, if this system is configured to believe it is in "example.com", and a client tries to connect to "www", the client will be connected to "www.example.com". This option only affects name lookups that your server does on behalf of clients. (Defaults to "0".)
+--------------------------------------------------------------------------------
+Relay
+index: 125
+serverdnstestaddresses
+address,address,...
+When we're detecting DNS hijacking, make sure that these valid addresses aren't getting redirected. If they are, then our DNS is completely useless, and we'll reset our exit policy to "reject :". This option only affects name lookups that your server does on behalf of clients. (Defaults to "www.google.com, www.mit.edu, www.yahoo.com, www.slashdot.org".)
+--------------------------------------------------------------------------------
+Relay
+index: 118
+shutdownwaitlength
+NUM
+When we get a SIGINT and we're a server, we begin shutting down: we close listeners and start refusing new circuits. After NUM seconds, we exit. If we get a second SIGINT, we exit immedi- ately. (Default: 30 seconds)
+--------------------------------------------------------------------------------
+General
+index: 31
+socks4proxy
+host[:port]
+Tor will make all OR connections through the SOCKS 4 proxy at host:port (or host:1080 if port is not specified).
+--------------------------------------------------------------------------------
+General
+index: 32
+socks5proxy
+host[:port]
+Tor will make all OR connections through the SOCKS 5 proxy at host:port (or host:1080 if port is not specified).
+--------------------------------------------------------------------------------
+General
+index: 34
+socks5proxypassword
+password
+If defined, authenticate to the SOCKS 5 server using username and password in accordance to RFC 1929. Both username and password must be between 1 and 255 characters.
+--------------------------------------------------------------------------------
+General
+index: 33
+socks5proxyusername
+username
+
+--------------------------------------------------------------------------------
+Client
+index: 76
+sockslistenaddress
+IP[:PORT]
+Bind to this address to listen for connections from Socks-speaking applications. (Default: 127.0.0.1) You can also specify a port (e.g. 192.168.0.1:9100). This directive can be specified multiple times to bind to multiple addresses/ports.
+--------------------------------------------------------------------------------
+Client
+index: 77
+sockspolicy
+policy,policy,...
+Set an entrance policy for this server, to limit who can connect to the SocksPort and DNSPort ports. The policies have the same form as exit policies below.
+--------------------------------------------------------------------------------
+Client
+index: 75
+socksport
+PORT
+Advertise this port to listen for connections from Socks-speaking applications. Set this to 0 if you don't want to allow application connections. (Default: 9050)
+--------------------------------------------------------------------------------
+Client
+index: 78
+sockstimeout
+NUM
+Let a socks connection wait NUM seconds handshaking, and NUM seconds unattached waiting for an appropriate circuit, before we fail it. (Default: 2 minutes.)
+--------------------------------------------------------------------------------
+Client
+index: 62
+strictnodes
+0|1
+If 1 and EntryNodes config option is set, Tor will never use any nodes besides those listed in EntryNodes for the first hop of a normal circuit. If 1 and ExitNodes config option is set, Tor will never use any nodes besides those listed in ExitNodes for the last hop of a normal exit circuit. Note that Tor might still use these nodes for non-exit circuits such as one-hop directory fetches or hidden service support circuits.
+--------------------------------------------------------------------------------
+Testing
+index: 177
+testingauthdirtimetolearnreachability
+N minutes|hours
+After starting as an authority, do not make claims about whether routers are Running until this much time has passed. Changing this requires that TestingTorNetwork is set. (Default: 30 minutes)
+--------------------------------------------------------------------------------
+Testing
+index: 178
+testingestimateddescriptorpropagationtime
+N minutes|hours
+Clients try downloading router descriptors from directory caches after this time. Changing this requires that TestingTorNetwork is set. (Default: 10 minutes)
+
+SIGNALS
+--------------------------------------------------------------------------------
+Testing
+index: 173
+testingtornetwork
+0|1
+If set to 1, Tor adjusts default values of the configuration options below, so that it is easier to set up a testing Tor network. May only be set if non-default set of DirServers is set. Cannot be unset while Tor is running. (Default: 0)
+
+ ServerDNSAllowBrokenConfig 1
+ DirAllowPrivateAddresses 1
+ EnforceDistinctSubnets 0
+ AssumeReachable 1
+ AuthDirMaxServersPerAddr 0
+ AuthDirMaxServersPerAuthAddr 0
+ ClientDNSRejectInternalAddresses 0
+ ExitPolicyRejectPrivate 0
+ V3AuthVotingInterval 5 minutes
+ V3AuthVoteDelay 20 seconds
+ V3AuthDistDelay 20 seconds
+ TestingV3AuthInitialVotingInterval 5 minutes
+ TestingV3AuthInitialVoteDelay 20 seconds
+ TestingV3AuthInitialDistDelay 20 seconds
+ TestingAuthDirTimeToLearnReachability 0 minutes
+ TestingEstimatedDescriptorPropagationTime 0 minutes
+--------------------------------------------------------------------------------
+Testing
+index: 176
+testingv3authinitialdistdelay
+N minutes|hours
+Like TestingV3AuthInitialDistDelay, but for initial voting interval before the first consensus has been created. Changing this requires that TestingTorNetwork is set. (Default: 5 minutes)
+--------------------------------------------------------------------------------
+Testing
+index: 175
+testingv3authinitialvotedelay
+N minutes|hours
+Like TestingV3AuthInitialVoteDelay, but for initial voting interval before the first consensus has been created. Changing this requires that TestingTorNetwork is set. (Default: 5 minutes)
+--------------------------------------------------------------------------------
+Testing
+index: 174
+testingv3authinitialvotinginterval
+N minutes|hours
+Like V3AuthVotingInterval, but for initial voting interval before the first consensus has been created. Changing this requires that TestingTorNetwork is set. (Default: 30 minutes)
+--------------------------------------------------------------------------------
+Client
+index: 86
+testsocks
+0|1
+When this option is enabled, Tor will make a notice-level log entry for each connection to the Socks port indicating whether the request used a safe socks protocol or an unsafe one (see above entry on SafeSocks). This helps to determine whether an application using Tor is possibly leaking DNS requests. (Default: 0)
+--------------------------------------------------------------------------------
+Client
+index: 79
+trackhostexits
+host,.domain,...
+For each value in the comma separated list, Tor will track recent connections to hosts that match this value and attempt to reuse the same exit node for each. If the value is prepended with a '.', it is treated as matching an entire domain. If one of the values is just a '.', it means match everything. This option is useful if you frequently connect to sites that will expire all your authentication cookies (i.e. log you out) if your IP address changes. Note that this option does have the disadvantage of making it more clear that a given history is associated with a single user. However, most people who would wish to observe this will observe it through cookies or other protocol-specific means anyhow.
+--------------------------------------------------------------------------------
+Client
+index: 80
+trackhostexitsexpire
+NUM
+Since exit servers go up and down, it is desirable to expire the association between host and exit server after NUM seconds. The default is 1800 seconds (30 minutes).
+--------------------------------------------------------------------------------
+Client
+index: 92
+translistenaddress
+IP[:PORT]
+Bind to this address to listen for transparent proxy connections. (Default: 127.0.0.1). This is useful for exporting a transparent proxy server to an entire network.
+--------------------------------------------------------------------------------
+Client
+index: 91
+transport
+PORT
+If non-zero, enables transparent proxy support on PORT (by convention, 9040). Requires OS support for transparent proxies, such as BSDs' pf or Linux's IPTables. If you're planning to use Tor as a transparent proxy for a network, you'll want to examine and change VirtualAddrNetwork from the default setting. You'll also want to set the TransListenAddress option for the network you'd like to proxy. (Default: 0).
+--------------------------------------------------------------------------------
+General
+index: 48
+tunneldirconns
+0|1
+If non-zero, when a directory server we contact supports it, we will build a one-hop circuit and make an encrypted connection via its ORPort. (Default: 1)
+--------------------------------------------------------------------------------
+Client
+index: 81
+updatebridgesfromauthority
+0|1
+When set (along with UseBridges), Tor will try to fetch bridge descriptors from the configured bridge authorities when feasible. It will fall back to a direct request if the authority responds with a 404. (Default: 0)
+--------------------------------------------------------------------------------
+Client
+index: 82
+usebridges
+0|1
+When set, Tor will fetch descriptors for each bridge listed in the "Bridge" config lines, and use these relays as both entry guards and directory guards. (Default: 0)
+--------------------------------------------------------------------------------
+Client
+index: 83
+useentryguards
+0|1
+If this option is set to 1, we pick a few long-term entry servers, and try to stick with them. This is desirable because constantly changing servers increases the odds that an adversary who owns some servers will observe a fraction of your paths. (Defaults to 1.)
+--------------------------------------------------------------------------------
+General
+index: 43
+user
+UID
+On startup, setuid to this user and setgid to their primary group.
+--------------------------------------------------------------------------------
+Directory
+index: 137
+v1authoritativedirectory
+0|1
+When this option is set in addition to AuthoritativeDirectory, Tor generates version 1 directory and running-routers documents (for legacy Tor clients up to 0.1.0.x).
+--------------------------------------------------------------------------------
+Directory
+index: 138
+v2authoritativedirectory
+0|1
+When this option is set in addition to AuthoritativeDirectory, Tor generates version 2 network statuses and serves descriptors, etc as described in doc/spec/dir-spec-v2.txt (for Tor clients and servers running 0.1.1.x and 0.1.2.x).
+--------------------------------------------------------------------------------
+Authority
+index: 165
+v3authdistdelay
+N minutes|hours
+V3 authoritative directories only. Configures the server's preferred delay between publishing its consensus and signature and assuming it has all the signatures from all the other authorities. Note that the actual time used is not the server's preferred time, but the consensus of all preferences. (Default: 5 minutes.)
+--------------------------------------------------------------------------------
+Authority
+index: 166
+v3authnintervalsvalid
+NUM
+V3 authoritative directories only. Configures the number of VotingIntervals for which each consensus should be valid for. Choosing high numbers increases network partitioning risks; choosing low numbers increases directory traffic. Note that the actual number of intervals used is not the server's preferred number, but the consensus of all preferences. Must be at least 2. (Default: 3.)
+--------------------------------------------------------------------------------
+Directory
+index: 139
+v3authoritativedirectory
+0|1
+When this option is set in addition to AuthoritativeDirectory, Tor generates version 3 network statuses and serves descriptors, etc as described in doc/spec/dir-spec.txt (for Tor clients and servers running at least 0.2.0.x).
+--------------------------------------------------------------------------------
+Authority
+index: 164
+v3authvotedelay
+N minutes|hours
+V3 authoritative directories only. Configures the server's preferred delay between publishing its vote and assuming it has all the votes from all the other authorities. Note that the actual time used is not the server's preferred time, but the consensus of all preferences. (Default: 5 minutes.)
+--------------------------------------------------------------------------------
+Authority
+index: 163
+v3authvotinginterval
+N minutes|hours
+V3 authoritative directories only. Configures the server's preferred voting interval. Note that voting will actually happen at an interval chosen by consensus from all the authorities' preferred intervals. This time SHOULD divide evenly into a day. (Default: 1 hour)
+--------------------------------------------------------------------------------
+Directory
+index: 140
+versioningauthoritativedirectory
+0|1
+When this option is set to 1, Tor adds information on which versions of Tor are still believed safe for use to the published directory. Each version 1 authority is automatically a versioning authority; version 2 authorities provide this service optionally. See RecommendedVersions, RecommendedClientVersions, and RecommendedServerVersions.
+--------------------------------------------------------------------------------
+Client
+index: 87
+virtualaddrnetwork
+Address/bits
+When a controller asks for a virtual (unused) address with the MAPADDRESS command, Tor picks an unassigned address from this range. (Default: 127.192.0.0/10)
+
+When providing proxy server service to a network of computers using a tool like dns-proxy-tor, change this address to "10.192.0.0/10" or "172.16.0.0/12". The default VirtualAddrNetwork address range on a properly configured machine will route to the loopback interface. For local use, no change to the default VirtualAddrNetwork setting is needed.
+--------------------------------------------------------------------------------
+Client
+index: 102
+warnplaintextports
+port,port,...
+Tells Tor to issue a warnings whenever the user tries to make an anonymous connection to one of these ports. This option is designed to alert users to services that risk sending passwords in the clear. (Default: 23,109,110,143).
Modified: arm/release/src/uninstall
===================================================================
--- arm/release/src/uninstall 2011-01-08 06:06:36 UTC (rev 24053)
+++ arm/release/src/uninstall 2011-01-08 06:35:08 UTC (rev 24054)
@@ -1,7 +1,7 @@
#!/bin/sh
files="/usr/bin/arm /usr/share/man/man1/arm.1.gz /usr/share/arm"
-for i in $files
+for i in $files
do
if [ -f $i -o -d $i ]; then
rm -rf $i
Modified: arm/release/src/util/__init__.py
===================================================================
--- arm/release/src/util/__init__.py 2011-01-08 06:06:36 UTC (rev 24053)
+++ arm/release/src/util/__init__.py 2011-01-08 06:35:08 UTC (rev 24054)
@@ -4,5 +4,5 @@
and safely working with curses (hiding some of the gory details).
"""
-__all__ = ["conf", "connections", "hostnames", "log", "panel", "sysTools", "torConfig", "torTools", "uiTools"]
+__all__ = ["conf", "connections", "hostnames", "log", "panel", "procTools", "sysTools", "torConfig", "torTools", "uiTools"]
Modified: arm/release/src/util/connections.py
===================================================================
--- arm/release/src/util/connections.py 2011-01-08 06:06:36 UTC (rev 24053)
+++ arm/release/src/util/connections.py 2011-01-08 06:35:08 UTC (rev 24054)
@@ -2,24 +2,36 @@
Fetches connection data (IP addresses and ports) associated with a given
process. This sort of data can be retrieved via a variety of common *nix
utilities:
-- netstat netstat -npt | grep <pid>/<process>
-- ss ss -p | grep "\"<process>\",<pid>"
-- lsof lsof -nPi | grep "<process>\s*<pid>.*(ESTABLISHED)"
+- netstat netstat -np | grep "ESTABLISHED <pid>/<process>"
+- sockstat sockstat | egrep "<process> *<pid>.*ESTABLISHED"
+- lsof lsof -wnPi | egrep "^<process> *<pid>.*((UDP.*)|(\(ESTABLISHED\)))"
+- ss ss -nptu | grep "ESTAB.*\"<process>\",<pid>"
-all queries dump its stderr (directing it to /dev/null). Unfortunately FreeBSD
-lacks support for the needed netstat flags and has a completely different
-program for 'ss', so this is quite likely to fail there.
+all queries dump its stderr (directing it to /dev/null). Results include UDP
+and established TCP connections.
+
+FreeBSD lacks support for the needed netstat flags and has a completely
+different program for 'ss'. However, lsof works and there's a couple other
+options that perform even better (thanks to Fabian Keil and Hans Schnehl):
+- sockstat sockstat -4c | grep '<process> *<pid>'
+- procstat procstat -f <pid> | grep TCP | grep -v 0.0.0.0:0
"""
-import sys
+import os
import time
import threading
-from util import log, sysTools
+from util import log, procTools, sysTools
# enums for connection resolution utilities
-CMD_NETSTAT, CMD_SS, CMD_LSOF = range(1, 4)
-CMD_STR = {CMD_NETSTAT: "netstat", CMD_SS: "ss", CMD_LSOF: "lsof"}
+CMD_PROC, CMD_NETSTAT, CMD_SOCKSTAT, CMD_LSOF, CMD_SS, CMD_BSD_SOCKSTAT, CMD_BSD_PROCSTAT = range(1, 8)
+CMD_STR = {CMD_PROC: "proc",
+ CMD_NETSTAT: "netstat",
+ CMD_SS: "ss",
+ CMD_LSOF: "lsof",
+ CMD_SOCKSTAT: "sockstat",
+ CMD_BSD_SOCKSTAT: "sockstat (bsd)",
+ CMD_BSD_PROCSTAT: "procstat (bsd)"}
# If true this provides new instantiations for resolvers if the old one has
# been stopped. This can make it difficult ensure all threads are terminated
@@ -28,29 +40,43 @@
# formatted strings for the commands to be executed with the various resolvers
# options are:
-# n = prevents dns lookups, p = include process, t = tcp only
+# n = prevents dns lookups, p = include process
# output:
# tcp 0 0 127.0.0.1:9051 127.0.0.1:53308 ESTABLISHED 9912/tor
# *note: bsd uses a different variant ('-t' => '-p tcp', but worse an
# equivilant -p doesn't exist so this can't function)
-RUN_NETSTAT = "netstat -npt | grep %s/%s"
+RUN_NETSTAT = "netstat -np | grep \"ESTABLISHED %s/%s\""
-# n = numeric ports, p = include process
+# n = numeric ports, p = include process, t = tcp sockets, u = udp sockets
# output:
# ESTAB 0 0 127.0.0.1:9051 127.0.0.1:53308 users:(("tor",9912,20))
# *note: under freebsd this command belongs to a spreadsheet program
-RUN_SS = "ss -np | grep \"\\\"%s\\\",%s\""
+RUN_SS = "ss -nptu | grep \"ESTAB.*\\\"%s\\\",%s\""
-# n = prevent dns lookups, P = show port numbers (not names), i = ip only
+# n = prevent dns lookups, P = show port numbers (not names), i = ip only,
+# -w = no warnings
# output:
-# tor 9912 atagar 20u IPv4 33453 TCP 127.0.0.1:9051->127.0.0.1:53308
-RUN_LSOF = "lsof -nPi | grep \"%s\s*%s.*(ESTABLISHED)\""
+# tor 3873 atagar 45u IPv4 40994 0t0 TCP 10.243.55.20:45724->194.154.227.109:9001 (ESTABLISHED)
+#
+# oddly, using the -p flag via:
+# lsof lsof -nPi -p <pid> | grep "^<process>.*(ESTABLISHED)"
+# is much slower (11-28% in tests I ran)
+RUN_LSOF = "lsof -wnPi | egrep \"^%s *%s.*((UDP.*)|(\\(ESTABLISHED\\)))\""
+# output:
+# atagar tor 3475 tcp4 127.0.0.1:9051 127.0.0.1:38942 ESTABLISHED
+# *note: this isn't available by default under ubuntu
+RUN_SOCKSTAT = "sockstat | egrep \"%s *%s.*ESTABLISHED\""
+
+RUN_BSD_SOCKSTAT = "sockstat -4c | grep '%s *%s'"
+RUN_BSD_PROCSTAT = "procstat -f %s | grep TCP | grep -v 0.0.0.0:0"
+
RESOLVERS = [] # connection resolvers available via the singleton constructor
RESOLVER_FAILURE_TOLERANCE = 3 # number of subsequent failures before moving on to another resolver
RESOLVER_SERIAL_FAILURE_MSG = "Querying connections with %s failed, trying %s"
RESOLVER_FINAL_FAILURE_MSG = "All connection resolvers failed"
CONFIG = {"queries.connections.minRate": 5,
+ "log.connResolverOptions": log.INFO,
"log.connLookupFailed": log.INFO,
"log.connLookupFailover": log.NOTICE,
"log.connLookupAbandon": log.WARN,
@@ -59,6 +85,35 @@
def loadConfig(config):
config.update(CONFIG)
+def getResolverCommand(resolutionCmd, processName, processPid = ""):
+ """
+ Provides the command that would be processed for the given resolver type.
+ This raises a ValueError if either the resolutionCmd isn't recognized or a
+ pid was requited but not provided.
+
+ Arguments:
+ resolutionCmd - command to use in resolving the address
+ processName - name of the process for which connections are fetched
+ processPid - process ID (this helps improve accuracy)
+ """
+
+ if not processPid:
+ # the pid is required for procstat resolution
+ if resolutionCmd == CMD_BSD_PROCSTAT:
+ raise ValueError("procstat resolution requires a pid")
+
+ # if the pid was undefined then match any in that field
+ processPid = "[0-9]*"
+
+ if resolutionCmd == CMD_PROC: return ""
+ elif resolutionCmd == CMD_NETSTAT: return RUN_NETSTAT % (processPid, processName)
+ elif resolutionCmd == CMD_SS: return RUN_SS % (processName, processPid)
+ elif resolutionCmd == CMD_LSOF: return RUN_LSOF % (processName, processPid)
+ elif resolutionCmd == CMD_SOCKSTAT: return RUN_SOCKSTAT % (processName, processPid)
+ elif resolutionCmd == CMD_BSD_SOCKSTAT: return RUN_BSD_SOCKSTAT % (processName, processPid)
+ elif resolutionCmd == CMD_BSD_PROCSTAT: return RUN_BSD_PROCSTAT % processPid
+ else: raise ValueError("Unrecognized resolution type: %s" % resolutionCmd)
+
def getConnections(resolutionCmd, processName, processPid = ""):
"""
Retrieves a list of the current connections for a given process, providing a
@@ -76,31 +131,56 @@
processPid - process ID (this helps improve accuracy)
"""
- if resolutionCmd == CMD_NETSTAT: cmd = RUN_NETSTAT % (processPid, processName)
- elif resolutionCmd == CMD_SS: cmd = RUN_SS % (processName, processPid)
- else: cmd = RUN_LSOF % (processName, processPid)
-
- # raises an IOError if the command fails or isn't available
- results = sysTools.call(cmd)
-
- if not results: raise IOError("No results found using: %s" % cmd)
-
- # parses results for the resolution command
- conn = []
- for line in results:
- comp = line.split()
+ if resolutionCmd == CMD_PROC:
+ # Attempts resolution via checking the proc contents.
+ if not processPid:
+ raise ValueError("proc resolution requires a pid")
- if resolutionCmd == CMD_NETSTAT or resolutionCmd == CMD_SS:
- localIp, localPort = comp[3].split(":")
- foreignIp, foreignPort = comp[4].split(":")
- else:
- local, foreign = comp[8].split("->")
- localIp, localPort = local.split(":")
- foreignIp, foreignPort = foreign.split(":")
+ try:
+ return procTools.getConnections(processPid)
+ except Exception, exc:
+ raise IOError(str(exc))
+ else:
+ # Queries a resolution utility (netstat, lsof, etc). This raises an
+ # IOError if the command fails or isn't available.
+ cmd = getResolverCommand(resolutionCmd, processName, processPid)
+ results = sysTools.call(cmd)
- conn.append((localIp, localPort, foreignIp, foreignPort))
-
- return conn
+ if not results: raise IOError("No results found using: %s" % cmd)
+
+ # parses results for the resolution command
+ conn = []
+ for line in results:
+ if resolutionCmd == CMD_LSOF:
+ # Different versions of lsof have different numbers of columns, so
+ # stripping off the optional 'established' entry so we can just use
+ # the last one.
+ comp = line.replace("(ESTABLISHED)", "").strip().split()
+ else: comp = line.split()
+
+ if resolutionCmd == CMD_NETSTAT:
+ localIp, localPort = comp[3].split(":")
+ foreignIp, foreignPort = comp[4].split(":")
+ elif resolutionCmd == CMD_SS:
+ localIp, localPort = comp[4].split(":")
+ foreignIp, foreignPort = comp[5].split(":")
+ elif resolutionCmd == CMD_LSOF:
+ local, foreign = comp[-1].split("->")
+ localIp, localPort = local.split(":")
+ foreignIp, foreignPort = foreign.split(":")
+ elif resolutionCmd == CMD_SOCKSTAT:
+ localIp, localPort = comp[4].split(":")
+ foreignIp, foreignPort = comp[5].split(":")
+ elif resolutionCmd == CMD_BSD_SOCKSTAT:
+ localIp, localPort = comp[5].split(":")
+ foreignIp, foreignPort = comp[6].split(":")
+ elif resolutionCmd == CMD_BSD_PROCSTAT:
+ localIp, localPort = comp[9].split(":")
+ foreignIp, foreignPort = comp[10].split(":")
+
+ conn.append((localIp, localPort, foreignIp, foreignPort))
+
+ return conn
def isResolverAlive(processName, processPid = ""):
"""
@@ -119,7 +199,7 @@
return False
-def getResolver(processName, processPid = ""):
+def getResolver(processName, processPid = "", alias=None):
"""
Singleton constructor for resolver instances. If a resolver already exists
for the process then it's returned. Otherwise one is created and started.
@@ -128,18 +208,20 @@
processName - name of the process being resolved
processPid - pid of the process being resolved, if undefined this matches
against any resolver with the process name
+ alias - alternative handle under which the resolver can be requested
"""
# check if one's already been created
+ requestHandle = alias if alias else processName
haltedIndex = -1 # old instance of this resolver with the _halt flag set
for i in range(len(RESOLVERS)):
resolver = RESOLVERS[i]
- if resolver.processName == processName and (not processPid or resolver.processPid == processPid):
+ if resolver.handle == requestHandle and (not processPid or resolver.processPid == processPid):
if resolver._halt and RECREATE_HALTED_RESOLVERS: haltedIndex = i
else: return resolver
# make a new resolver
- r = ConnectionResolver(processName, processPid)
+ r = ConnectionResolver(processName, processPid, handle = requestHandle)
r.start()
# overwrites halted instance of this resolver if it exists, otherwise append
@@ -147,37 +229,27 @@
else: RESOLVERS[haltedIndex] = r
return r
-def test():
- # quick method for testing connection resolution
- userInput = raw_input("Enter query (<ss, netstat, lsof> PROCESS_NAME [PID]): ").split()
+def getSystemResolvers(osType = None):
+ """
+ Provides the types of connection resolvers available on this operating
+ system.
- # checks if there's enough arguments
- if len(userInput) == 0: sys.exit(0)
- elif len(userInput) == 1:
- print "no process name provided"
- sys.exit(1)
+ Arguments:
+ osType - operating system type, fetched from the os module if undefined
+ """
- # translates resolver string to enum
- userInput[0] = userInput[0].lower()
- if userInput[0] == "ss": userInput[0] = CMD_SS
- elif userInput[0] == "netstat": userInput[0] = CMD_NETSTAT
- elif userInput[0] == "lsof": userInput[0] = CMD_LSOF
+ if osType == None: osType = os.uname()[0]
+
+ if osType == "FreeBSD":
+ resolvers = [CMD_BSD_SOCKSTAT, CMD_BSD_PROCSTAT, CMD_LSOF]
else:
- print "unrecognized type of resolver: %s" % userInput[2]
- sys.exit(1)
+ resolvers = [CMD_NETSTAT, CMD_SOCKSTAT, CMD_LSOF, CMD_SS]
- # resolves connections
- try:
- if len(userInput) == 2: connections = getConnections(userInput[0], userInput[1])
- else: connections = getConnections(userInput[0], userInput[1], userInput[2])
- except IOError, exc:
- print exc
- sys.exit(1)
+ # proc resolution, by far, outperforms the others so defaults to this is able
+ if procTools.isProcAvailable():
+ resolvers = [CMD_PROC] + resolvers
- # prints results
- print "-" * 40
- for lIp, lPort, fIp, fPort in connections:
- print "%s:%s -> %s:%s" % (lIp, lPort, fIp, fPort)
+ return resolvers
class ConnectionResolver(threading.Thread):
"""
@@ -216,11 +288,12 @@
overwriteResolver - method of resolution (uses default if None)
* defaultResolver - resolver used by default (None if all resolution
methods have been exhausted)
+ resolverOptions - resolvers to be cycled through (differ by os)
* read-only
"""
- def __init__(self, processName, processPid = "", resolveRate = None):
+ def __init__(self, processName, processPid = "", resolveRate = None, handle = None):
"""
Initializes a new resolver daemon. When no longer needed it's suggested
that this is stopped.
@@ -230,6 +303,8 @@
processPid - pid of the process being resolved
resolveRate - time between resolving connections (in seconds, None if
chosen dynamically)
+ handle - name used to query this resolver, this is the processName
+ if undefined
"""
threading.Thread.__init__(self)
@@ -238,15 +313,22 @@
self.processName = processName
self.processPid = processPid
self.resolveRate = resolveRate
+ self.handle = handle if handle else processName
self.defaultRate = CONFIG["queries.connections.minRate"]
self.lastLookup = -1
self.overwriteResolver = None
- self.defaultResolver = CMD_NETSTAT
+ self.defaultResolver = CMD_PROC
+ osType = os.uname()[0]
+ self.resolverOptions = getSystemResolvers(osType)
+
+ resolverLabels = ", ".join([CMD_STR[option] for option in self.resolverOptions])
+ log.log(CONFIG["log.connResolverOptions"], "Operating System: %s, Connection Resolvers: %s" % (osType, resolverLabels))
+
# sets the default resolver to be the first found in the system's PATH
# (left as netstat if none are found)
- for resolver in [CMD_NETSTAT, CMD_SS, CMD_LSOF]:
- if sysTools.isAvailable(CMD_STR[resolver]):
+ for resolver in self.resolverOptions:
+ if resolver == CMD_PROC or sysTools.isAvailable(CMD_STR[resolver]):
self.defaultResolver = resolver
break
@@ -320,7 +402,7 @@
# pick another (non-blacklisted) resolver
newResolver = None
- for r in [CMD_NETSTAT, CMD_SS, CMD_LSOF]:
+ for r in self.resolverOptions:
if not r in self._resolverBlacklist:
newResolver = r
break
Modified: arm/release/src/util/log.py
===================================================================
--- arm/release/src/util/log.py 2011-01-08 06:06:36 UTC (rev 24053)
+++ arm/release/src/util/log.py 2011-01-08 06:35:08 UTC (rev 24054)
@@ -6,6 +6,7 @@
safe.
"""
+import os
import time
from sys import maxint
from threading import RLock
@@ -27,6 +28,8 @@
CONFIG = {"cache.armLog.size": 1000,
"cache.armLog.trimSize": 200}
+DUMP_FILE = None
+
def loadConfig(config):
config.update(CONFIG, {
"cache.armLog.size": 10,
@@ -34,6 +37,24 @@
CONFIG["cache.armLog.trimSize"] = min(CONFIG["cache.armLog.trimSize"], CONFIG["cache.armLog.size"] / 2)
+def setDumpFile(logPath):
+ """
+ Logs all future logged events to the given path. This raises an IOError if
+ the file fails to be opened. If the file already exists then this overwrites
+ it.
+
+ Arguments:
+ logPath - path where to persist logs
+ """
+
+ global DUMP_FILE
+
+ # make sure that the parent directory exists
+ baseDir = os.path.dirname(logPath)
+ if not os.path.exists(baseDir): os.makedirs(baseDir)
+
+ DUMP_FILE = open(logPath, "w")
+
def strToRunlevel(runlevelStr):
"""
Converts runlevel strings ("DEBUG", "INFO", "NOTICE", etc) to their
@@ -75,6 +96,7 @@
eventTime - unix time at which the event occurred, current time if undefined
"""
+ global DUMP_FILE
if not level: return
if eventTime == None: eventTime = time.time()
@@ -101,6 +123,18 @@
toDelete = len(eventBacklog) - CONFIG["cache.armLog.size"]
if toDelete >= 0: del eventBacklog[: toDelete + CONFIG["cache.armLog.trimSize"]]
+ # persists the event if a debug file's been set
+ if DUMP_FILE:
+ try:
+ entryTime = time.localtime(eventTime)
+ timeLabel = "%i/%i/%i %02i:%02i:%02i" % (entryTime[1], entryTime[2], entryTime[0], entryTime[3], entryTime[4], entryTime[5])
+ logEntry = "%s [%s] %s\n" % (timeLabel, runlevelToStr(level), msg)
+ DUMP_FILE.write(logEntry)
+ DUMP_FILE.flush()
+ except IOError, exc:
+ DUMP_FILE = None
+ log(ERR, "Failed to write to the debug file - %s" % exc)
+
# notifies listeners
for callback in _listeners[level]:
callback(RUNLEVEL_STR[level], msg, eventTime)
Modified: arm/release/src/util/panel.py
===================================================================
--- arm/release/src/util/panel.py 2011-01-08 06:06:36 UTC (rev 24053)
+++ arm/release/src/util/panel.py 2011-01-08 06:35:08 UTC (rev 24054)
@@ -2,6 +2,7 @@
Wrapper for safely working with curses subwindows.
"""
+import sys
import traceback
import curses
from threading import RLock
@@ -358,8 +359,16 @@
# Displays the text field, blocking until the user's done. This closes the
# text panel and returns userInput to the initial text if the user presses
- # escape.
- textbox = curses.textpad.Textbox(inputSubwindow, True)
+ # escape. Insert mode is available in Python 2.6+, before that the
+ # constructor only accepted a subwindow argument as per:
+ # https://trac.torproject.org/projects/tor/ticket/2354
+
+ majorVersion, minorVersion = sys.version_info[:2]
+ if majorVersion == 2 and minorVersion >= 6:
+ textbox = curses.textpad.Textbox(inputSubwindow, True)
+ else:
+ textbox = curses.textpad.Textbox(inputSubwindow)
+
userInput = textbox.edit(lambda key: _textboxValidate(textbox, key)).strip()
if textbox.lastcmd == curses.ascii.BEL: userInput = None
Copied: arm/release/src/util/procTools.py (from rev 24053, arm/trunk/src/util/procTools.py)
===================================================================
--- arm/release/src/util/procTools.py (rev 0)
+++ arm/release/src/util/procTools.py 2011-01-08 06:35:08 UTC (rev 24054)
@@ -0,0 +1,273 @@
+"""
+Helper functions for querying process and system information from the /proc
+contents. Fetching information this way provides huge performance benefits
+over lookups via system utilities (ps, netstat, etc). For instance, resolving
+connections this way cuts the runtime by around 90% verses the alternatives.
+These functions may not work on all platforms (only Linux?).
+
+All functions raise IOErrors if unable to read their respective proc files.
+
+The method for reading these files (and some of the code) are borrowed from
+psutil:
+https://code.google.com/p/psutil/
+which was written by Jay Loden, Dave Daeschler, Giampaolo Rodola' and is under
+the BSD license.
+"""
+
+import os
+import sys
+import time
+import socket
+import base64
+
+from util import log
+
+# cached system values
+SYS_START_TIME, SYS_PHYSICAL_MEMORY = None, None
+CLOCK_TICKS = os.sysconf(os.sysconf_names["SC_CLK_TCK"])
+STAT_COMMAND, STAT_CPU_UTIME, STAT_CPU_STIME, STAT_START_TIME = range(4)
+
+CONFIG = {"queries.useProc": True,
+ "log.procCallMade": log.DEBUG}
+
+def loadConfig(config):
+ config.update(CONFIG)
+
+def isProcAvailable():
+ """
+ Provides true if configured to use proc resolution and it's available on the
+ platform, false otherwise.
+ """
+
+ return CONFIG["queries.useProc"] and os.uname()[0] == "Linux"
+
+def getSystemStartTime():
+ """
+ Provides the unix time (seconds since epoch) when the system started.
+ """
+
+ global SYS_START_TIME
+ if not SYS_START_TIME:
+ startTime = time.time()
+ statFile = open('/proc/stat')
+ statLines = statFile.readlines()
+ statFile.close()
+
+ for line in statLines:
+ if line.startswith('btime'):
+ SYS_START_TIME = float(line.strip().split()[1])
+ break
+
+ _logProcRuntime("system start time", "/proc/stat[btime]", startTime)
+
+ return SYS_START_TIME
+
+def getPhysicalMemory():
+ """
+ Provides the total physical memory on the system in bytes.
+ """
+
+ global SYS_PHYSICAL_MEMORY
+ if not SYS_PHYSICAL_MEMORY:
+ startTime = time.time()
+ memFile = open('/proc/meminfo')
+ memLines = memFile.readlines()
+ memFile.close()
+
+ for line in memLines:
+ if line.startswith('MemTotal:'):
+ SYS_PHYSICAL_MEMORY = int(line.split()[1]) * 1024
+
+ _logProcRuntime("system physical memory", "/proc/meminfo[MemTotal]", startTime)
+
+ return SYS_PHYSICAL_MEMORY
+
+def getPwd(pid):
+ """
+ Provides the current working directory for the given process.
+
+ Arguments:
+ pid - queried process
+ """
+
+ startTime = time.time()
+ if pid == 0: cwd = ""
+ else: cwd = os.readlink("/proc/%s/cwd" % pid)
+ _logProcRuntime("cwd", "/proc/%s/cwd" % pid, startTime)
+ return cwd
+
+def getMemoryUsage(pid):
+ """
+ Provides the memory usage in bytes for the given process of the form:
+ (residentSize, virtualSize)
+
+ Arguments:
+ pid - queried process
+ """
+
+ # checks if this is the kernel process
+ if pid == 0: return (0, 0)
+
+ startTime = time.time()
+ statusFile = open("/proc/%s/status" % pid)
+ statusFileLines = statusFile.readlines()
+ statusFile.close()
+
+ residentSize, virtualSize = None, None
+ for line in statusFileLines:
+ if line.startswith("VmRSS"):
+ residentSize = int(line.split()[1]) * 1024
+ if virtualSize != None: break
+ elif line.startswith("VmSize:"):
+ virtualSize = int(line.split()[1]) * 1024
+ if residentSize != None: break
+
+ _logProcRuntime("memory usage", "/proc/%s/status[VmRSS|VmSize]" % pid, startTime)
+ return (residentSize, virtualSize)
+
+def getStats(pid, *statTypes):
+ """
+ Provides process specific information. Options are:
+ STAT_COMMAND command name under which the process is running
+ STAT_CPU_UTIME total user time spent on the process
+ STAT_CPU_STIME total system time spent on the process
+ STAT_START_TIME when this process began, in unix time
+
+ Arguments:
+ pid - queried process
+ statTypes - information to be provided back
+ """
+
+ startTime = time.time()
+ statFilePath = "/proc/%s/stat" % pid
+ statFile = open(statFilePath)
+ statContents = statFile.read().strip()
+ statFile.close()
+
+ # contents are of the form:
+ # 8438 (tor) S 8407 8438 8407 34818 8438 4202496...
+ statComp = []
+ cmdStart, cmdEnd = statContents.find("("), statContents.find(")")
+
+ if cmdStart != -1 and cmdEnd != -1:
+ statComp.append(statContents[:cmdStart])
+ statComp.append(statContents[cmdStart + 1:cmdEnd])
+ statComp += statContents[cmdEnd + 1:].split()
+
+ if len(statComp) != 44:
+ raise IOError("stat file had an unexpected format: %s" % statFilePath)
+
+ results, queriedStats = [], []
+ for statType in statTypes:
+ if statType == STAT_COMMAND:
+ queriedStats.append("command")
+ if pid == 0: results.append("sched")
+ else: results.append(statComp[1])
+ elif statType == STAT_CPU_UTIME:
+ queriedStats.append("utime")
+ if pid == 0: results.append("0")
+ else: results.append(str(float(statComp[13]) / CLOCK_TICKS))
+ elif statType == STAT_CPU_STIME:
+ queriedStats.append("stime")
+ if pid == 0: results.append("0")
+ else: results.append(str(float(statComp[14]) / CLOCK_TICKS))
+ elif statType == STAT_START_TIME:
+ queriedStats.append("start time")
+ if pid == 0: return getSystemStartTime()
+ else:
+ # According to documentation, starttime is in field 21 and the unit is
+ # jiffies (clock ticks). We divide it for clock ticks, then add the
+ # uptime to get the seconds since the epoch.
+ pStartTime = float(statComp[21]) / CLOCK_TICKS
+ results.append(str(pStartTime + getSystemStartTime()))
+
+ _logProcRuntime("process %s" % ", ".join(queriedStats), "/proc/%s/stat" % pid, startTime)
+ return results
+
+def getConnections(pid):
+ """
+ Provides a listing of connection tuples of the form:
+ [(local_ipAddr1, local_port1, foreign_ipAddr1, foreign_port1), ...]
+
+ If the information about a connection can't be queried (often due to
+ permission issues) then it's excluded from the listing.
+
+ Arguments:
+ pid - ID of the process to be resolved
+ """
+
+ if pid == "0": return []
+
+ # fetches the inode numbers for socket file descriptors
+ startTime = time.time()
+ inodes = []
+ for fd in os.listdir("/proc/%s/fd" % pid):
+ try:
+ # File descriptor link, such as 'socket:[30899]'
+ fdName = os.readlink("/proc/%s/fd/%s" % (pid, fd))
+
+ if fdName.startswith('socket:['):
+ inodes.append(fdName[8:-1])
+ except OSError:
+ pass # most likely couldn't be read due to permissions
+
+ if not inodes:
+ # unable to fetch any connections for this process
+ return []
+
+ # check for the connection information from the /proc/net contents
+ conn = []
+ for procFilePath in ("/proc/net/tcp", "/proc/net/udp"):
+ procFile = open(procFilePath)
+ procFile.readline() # skip the first line
+
+ for line in procFile:
+ _, lAddr, fAddr, status, _, _, _, _, _, inode = line.split()[:10]
+
+ if inode in inodes:
+ # if a tcp connection, skip if it isn't yet established
+ if procFilePath.endswith("/tcp") and status != "01":
+ continue
+
+ localIp, localPort = _decodeProcAddressEncoding(lAddr)
+ foreignIp, foreignPort = _decodeProcAddressEncoding(fAddr)
+ conn.append((localIp, localPort, foreignIp, foreignPort))
+
+ procFile.close()
+
+ _logProcRuntime("process connections", "/proc/net/[tcp|udp]", startTime)
+
+ return conn
+
+def _decodeProcAddressEncoding(addr):
+ """
+ Translates an address entry in the /proc/net/* contents to a human readable
+ form, for instance:
+ "0500000A:0016" -> ("10.0.0.5", "22")
+
+ Reference:
+ http://linuxdevcenter.com/pub/a/linux/2000/11/16/LinuxAdmin.html
+
+ Arguments:
+ addr - proc address entry to be decoded
+ """
+
+ ip, port = addr.split(':')
+
+ # the port is represented as a two-byte hexadecimal number
+ port = str(int(port, 16))
+
+ if sys.version_info >= (3,):
+ ip = ip.encode('ascii')
+
+ # The IPv4 address portion is a little-endian four-byte hexadecimal number.
+ # That is, the least significant byte is listed first, so we need to reverse
+ # the order of the bytes to convert it to an IP address.
+ ip = socket.inet_ntop(socket.AF_INET, base64.b16decode(ip)[::-1])
+
+ return (ip, port)
+
+def _logProcRuntime(parameter, procLocation, startTime):
+ msg = "proc call (%s): %s (runtime: %0.4f)" % (parameter, procLocation, time.time() - startTime)
+ log.log(CONFIG["log.procCallMade"], msg)
+
Modified: arm/release/src/util/sysTools.py
===================================================================
--- arm/release/src/util/sysTools.py 2011-01-08 06:06:36 UTC (rev 24053)
+++ arm/release/src/util/sysTools.py 2011-01-08 06:35:08 UTC (rev 24054)
@@ -6,7 +6,7 @@
import time
import threading
-from util import log
+from util import log, procTools, uiTools
# mapping of commands to if they're available or not
CMD_AVAILABLE_CACHE = {}
@@ -16,15 +16,46 @@
IS_FAILURES_CACHED = True # caches both successful and failed results if true
CALL_CACHE_LOCK = threading.RLock() # governs concurrent modifications of CALL_CACHE
-CONFIG = {"cache.sysCalls.size": 600,
+PROCESS_NAME_CACHE = {} # mapping of pids to their process names
+PWD_CACHE = {} # mapping of pids to their present working directory
+RESOURCE_TRACKERS = {} # mapping of pids to their resource tracker instances
+
+# Runtimes for system calls, used to estimate cpu usage. Entries are tuples of
+# the form:
+# (time called, runtime)
+RUNTIMES = []
+SAMPLING_PERIOD = 5 # time of the sampling period
+
+CONFIG = {"queries.resourceUsage.rate": 5,
+ "cache.sysCalls.size": 600,
"log.sysCallMade": log.DEBUG,
"log.sysCallCached": None,
"log.sysCallFailed": log.INFO,
- "log.sysCallCacheGrowing": log.INFO}
+ "log.sysCallCacheGrowing": log.INFO,
+ "log.stats.failedProcResolution": log.DEBUG,
+ "log.stats.procResolutionFailover": log.INFO,
+ "log.stats.failedPsResolution": log.INFO}
def loadConfig(config):
config.update(CONFIG)
+def getSysCpuUsage():
+ """
+ Provides an estimate of the cpu usage for system calls made through this
+ module, based on a sampling period of five seconds. The os.times() function,
+ unfortunately, doesn't seem to take popen calls into account. This returns a
+ float representing the percentage used.
+ """
+
+ currentTime = time.time()
+
+ # removes any runtimes outside of our sampling period
+ while RUNTIMES and currentTime - RUNTIMES[0][0] > SAMPLING_PERIOD:
+ RUNTIMES.pop(0)
+
+ runtimeSum = sum([entry[1] for entry in RUNTIMES])
+ return runtimeSum / SAMPLING_PERIOD
+
def isAvailable(command, cached=True):
"""
Checks the current PATH to see if a command is available or not. If a full
@@ -74,6 +105,109 @@
return excStr
+def getProcessName(pid, default = None, cacheFailure = True):
+ """
+ Provides the name associated with the given process id. This isn't available
+ on all platforms.
+
+ Arguments:
+ pid - process id for the process being returned
+ default - result if the process name can't be retrieved (raises an
+ IOError on failure instead if undefined)
+ cacheFailure - if the lookup fails and there's a default then caches the
+ default value to prevent further lookups
+ """
+
+ if pid in PROCESS_NAME_CACHE:
+ return PROCESS_NAME_CACHE[pid]
+
+ processName, raisedExc = "", None
+
+ # fetch it from proc contents if available
+ if procTools.isProcAvailable():
+ try:
+ processName = procTools.getStats(pid, procTools.STAT_COMMAND)[0]
+ except IOError, exc:
+ raisedExc = exc
+
+ # fall back to querying via ps
+ if not processName:
+ # the ps call formats results as:
+ # COMMAND
+ # tor
+ psCall = call("ps -p %s -o command" % pid)
+
+ if psCall and len(psCall) >= 2 and not " " in psCall[1]:
+ processName, raisedExc = psCall[1].strip(), None
+ else:
+ raisedExc = ValueError("Unexpected output from ps: %s" % psCall)
+
+ if raisedExc:
+ if default == None: raise raisedExc
+ else:
+ if cacheFailure:
+ PROCESS_NAME_CACHE[pid] = default
+
+ return default
+ else:
+ PROCESS_NAME_CACHE[pid] = processName
+ return processName
+
+def getPwd(pid):
+ """
+ Provices the working directory of the given process. This raises an IOError
+ if it can't be determined.
+
+ Arguments:
+ pid - pid of the process
+ """
+
+ if not pid: raise IOError("we couldn't get the pid")
+ elif pid in PWD_CACHE: return PWD_CACHE[pid]
+
+ # try fetching via the proc contents if available
+ if procTools.isProcAvailable():
+ try:
+ pwd = procTools.getPwd(pid)
+ PWD_CACHE[pid] = pwd
+ return pwd
+ except IOError: pass # fall back to pwdx
+
+ try:
+ # pwdx results are of the form:
+ # 3799: /home/atagar
+ # 5839: No such process
+ results = call("pwdx %s" % pid)
+ if not results:
+ raise IOError("pwdx didn't return any results")
+ elif results[0].endswith("No such process"):
+ raise IOError("pwdx reported no process for pid " + pid)
+ elif len(results) != 1 or results[0].count(" ") != 1:
+ raise IOError("we got unexpected output from pwdx")
+ else:
+ pwd = results[0][results[0].find(" ") + 1:].strip()
+ PWD_CACHE[pid] = pwd
+ return pwd
+ except IOError, exc:
+ raise IOError("the pwdx call failed: " + str(exc))
+
+def expandRelativePath(path, ownerPid):
+ """
+ Expands relative paths to be an absolute path with reference to a given
+ process. This raises an IOError if the process pwd is required and can't be
+ resolved.
+
+ Arguments:
+ path - path to be expanded
+ ownerPid - pid of the process to which the path belongs
+ """
+
+ if not path or path[0] == "/": return path
+ else:
+ if path.startswith("./"): path = path[2:]
+ processPwd = getPwd(ownerPid)
+ return "%s/%s" % (processPwd, path)
+
def call(command, cacheAge=0, suppressExc=False, quiet=True):
"""
Convenience function for performing system calls, providing:
@@ -149,9 +283,19 @@
return cachedResults
startTime = time.time()
- commandComp = command.split("|")
commandCall, results, errorExc = None, None, None
+ # Gets all the commands involved, taking piping into consideration. If the
+ # pipe is quoted (ie, echo "an | example") then it's ignored.
+
+ commandComp = []
+ for component in command.split("|"):
+ if not commandComp or component.count("\"") % 2 == 0:
+ commandComp.append(component)
+ else:
+ # pipe is within quotes
+ commandComp[-1] += "|" + component
+
# preprocessing for the commands to prevent anything going to stdout
for i in range(len(commandComp)):
subcommand = commandComp[i].strip()
@@ -184,9 +328,16 @@
else: raise errorExc
else:
# log call information and if we're caching then save the results
- msg = "system call: %s (runtime: %0.2f)" % (command, time.time() - startTime)
+ currentTime = time.time()
+ runtime = currentTime - startTime
+ msg = "system call: %s (runtime: %0.2f)" % (command, runtime)
log.log(CONFIG["log.sysCallMade"], msg)
+ # append the runtime, and remove any outside of the sampling period
+ RUNTIMES.append((currentTime, runtime))
+ while RUNTIMES and currentTime - RUNTIMES[0][0] > SAMPLING_PERIOD:
+ RUNTIMES.pop(0)
+
if cacheAge > 0:
CALL_CACHE_LOCK.acquire()
CALL_CACHE[command] = (time.time(), results)
@@ -194,3 +345,218 @@
return results
+def isTrackerAlive(pid):
+ """
+ Provides true if a running, singleton instance exists for the given pid,
+ false otherwise.
+
+ Arguments:
+ pid - pid of the process being tracked
+ """
+
+ if pid in RESOURCE_TRACKERS:
+ if RESOURCE_TRACKERS[pid].isAlive(): return True
+ else: del RESOURCE_TRACKERS[pid]
+
+ return False
+
+def getResourceTracker(pid):
+ """
+ Provides a running singleton ResourceTracker instance for the given pid.
+
+ Arguments:
+ pid - pid of the process being tracked
+ """
+
+ if pid in RESOURCE_TRACKERS:
+ tracker = RESOURCE_TRACKERS[pid]
+ if tracker.isAlive(): return tracker
+ else: del RESOURCE_TRACKERS[pid]
+
+ tracker = ResourceTracker(pid, CONFIG["queries.resourceUsage.rate"])
+ RESOURCE_TRACKERS[pid] = tracker
+ tracker.start()
+ return tracker
+
+class ResourceTracker(threading.Thread):
+ """
+ Periodically fetches the resource usage (cpu and memory usage) for a given
+ process.
+ """
+
+ def __init__(self, processPid, resolveRate):
+ """
+ Initializes a new resolver daemon. When no longer needed it's suggested
+ that this is stopped.
+
+ Arguments:
+ processPid - pid of the process being tracked
+ resolveRate - time between resolving resource usage
+ """
+
+ threading.Thread.__init__(self)
+ self.setDaemon(True)
+
+ self.processPid = processPid
+ self.resolveRate = resolveRate
+
+ self.cpuSampling = 0.0 # latest cpu usage sampling
+ self.cpuAvg = 0.0 # total average cpu usage
+ self.memUsage = 0 # last sampled memory usage in bytes
+ self.memUsagePercentage = 0.0 # percentage cpu usage
+
+ # resolves usage via proc results if true, ps otherwise
+ self._useProc = procTools.isProcAvailable()
+
+ # used to get the deltas when querying cpu time
+ self._lastCpuTotal = 0
+
+ self.lastLookup = -1
+ self._halt = False # terminates thread if true
+ self._valLock = threading.RLock()
+ self._cond = threading.Condition() # used for pausing the thread
+
+ # number of successful calls we've made
+ self._runCount = 0
+
+ # sequential times we've failed with this method of resolution
+ self._failureCount = 0
+
+ def getResourceUsage(self):
+ """
+ Provides the last cached resource usage as a tuple of the form:
+ (cpuUsage_sampling, cpuUsage_avg, memUsage_bytes, memUsage_percent)
+ """
+
+ self._valLock.acquire()
+ results = (self.cpuSampling, self.cpuAvg, self.memUsage, self.memUsagePercentage)
+ self._valLock.release()
+
+ return results
+
+ def getRunCount(self):
+ """
+ Provides the number of times we've successfully fetched the resource
+ usages.
+ """
+
+ return self._runCount
+
+ def lastQueryFailed(self):
+ """
+ Provides true if, since we fetched the currently cached results, we've
+ failed to get new results. False otherwise.
+ """
+
+ return self._failureCount != 0
+
+ def run(self):
+ while not self._halt:
+ timeSinceReset = time.time() - self.lastLookup
+
+ if timeSinceReset < self.resolveRate:
+ sleepTime = max(0.2, self.resolveRate - timeSinceReset)
+
+ self._cond.acquire()
+ if not self._halt: self._cond.wait(sleepTime)
+ self._cond.release()
+
+ continue # done waiting, try again
+
+ newValues = {}
+ try:
+ if self._useProc:
+ utime, stime, startTime = procTools.getStats(self.processPid, procTools.STAT_CPU_UTIME, procTools.STAT_CPU_STIME, procTools.STAT_START_TIME)
+ totalCpuTime = float(utime) + float(stime)
+ cpuDelta = totalCpuTime - self._lastCpuTotal
+ newValues["cpuSampling"] = cpuDelta / timeSinceReset
+ newValues["cpuAvg"] = totalCpuTime / (time.time() - float(startTime))
+ newValues["_lastCpuTotal"] = totalCpuTime
+
+ memUsage = int(procTools.getMemoryUsage(self.processPid)[0])
+ totalMemory = procTools.getPhysicalMemory()
+ newValues["memUsage"] = memUsage
+ newValues["memUsagePercentage"] = float(memUsage) / totalMemory
+ else:
+ # the ps call formats results as:
+ # %CPU RSS %MEM ELAPSED
+ # 0.3 14096 1.3 29:51
+ psCall = call("ps -p %s -o cputime,etime,rss,%%mem" % self.processPid)
+
+ isSuccessful = False
+ if psCall and len(psCall) >= 2:
+ stats = psCall[1].strip().split()
+
+ if len(stats) == 4:
+ try:
+ totalCpuTime = uiTools.parseShortTimeLabel(stats[0])
+ uptime = uiTools.parseShortTimeLabel(stats[1])
+ cpuDelta = totalCpuTime - self._lastCpuTotal
+ newValues["cpuSampling"] = cpuDelta / timeSinceReset
+ newValues["cpuAvg"] = totalCpuTime / uptime
+ newValues["_lastCpuTotal"] = totalCpuTime
+
+ newValues["memUsage"] = int(stats[2]) * 1024 # ps size is in kb
+ newValues["memUsagePercentage"] = float(stats[3]) / 100.0
+ isSuccessful = True
+ except ValueError, exc: pass
+
+ if not isSuccessful:
+ raise IOError("unrecognized output from ps: %s" % psCall)
+ except IOError, exc:
+ self._failureCount += 1
+
+ if self._useProc:
+ if self._failureCount >= 3:
+ # We've failed three times resolving via proc. Warn, and fall back
+ # to ps resolutions.
+ msg = "Failed three attempts to get process resource usage from proc, falling back to ps (%s)" % exc
+ log.log(CONFIG["log.stats.procResolutionFailover"], msg)
+
+ self._useProc = False
+ self._failureCount = 1 # prevents lastQueryFailed() from thinking that we succeeded
+ else:
+ # wait a bit and try again
+ msg = "Unable to query process resource usage from proc (%s)" % exc
+ log.log(CONFIG["log.stats.failedProcResolution"], msg)
+ self._cond.acquire()
+ if not self._halt: self._cond.wait(0.5)
+ self._cond.release()
+ else:
+ # exponential backoff on making failed ps calls
+ sleepTime = 0.01 * (2 ** self._failureCount) + self._failureCount
+ msg = "Unable to query process resource usage from ps, waiting %0.2f seconds (%s)" % (sleepTime, exc)
+ log.log(CONFIG["log.stats.failedProcResolution"], msg)
+ self._cond.acquire()
+ if not self._halt: self._cond.wait(sleepTime)
+ self._cond.release()
+
+ # If this is the first run then the cpuSampling stat is meaningless
+ # (there isn't a previous tick to sample from so it's zero at this
+ # point). Setting it to the average, which is a fairer estimate.
+ if self.lastLookup == -1:
+ newValues["cpuSampling"] = newValues["cpuAvg"]
+
+ # sets the new values
+ if newValues:
+ self._valLock.acquire()
+ self.cpuSampling = newValues["cpuSampling"]
+ self.cpuAvg = newValues["cpuAvg"]
+ self.memUsage = newValues["memUsage"]
+ self.memUsagePercentage = newValues["memUsagePercentage"]
+ self._lastCpuTotal = newValues["_lastCpuTotal"]
+ self.lastLookup = time.time()
+ self._runCount += 1
+ self._failureCount = 0
+ self._valLock.release()
+
+ def stop(self):
+ """
+ Halts further resolutions and terminates the thread.
+ """
+
+ self._cond.acquire()
+ self._halt = True
+ self._cond.notifyAll()
+ self._cond.release()
+
Modified: arm/release/src/util/torConfig.py
===================================================================
--- arm/release/src/util/torConfig.py 2011-01-08 06:06:36 UTC (rev 24053)
+++ arm/release/src/util/torConfig.py 2011-01-08 06:35:08 UTC (rev 24054)
@@ -53,6 +53,11 @@
def loadConfig(config):
CONFIG["torrc.alias"] = config.get("torrc.alias", {})
+ # fetches any config.summary.* values
+ for configKey in config.getKeys():
+ if configKey.startswith("config.summary."):
+ CONFIG[configKey.lower()] = config.get(configKey)
+
# all the torrc.label.* values are comma separated lists
for configKey in CONFIG.keys():
if configKey.startswith("torrc.label."):
@@ -80,25 +85,29 @@
if TORRC == None: TORRC = Torrc()
return TORRC
-def loadOptionDescriptions(loadPath = None):
+def loadOptionDescriptions(loadPath = None, checkVersion = True):
"""
Fetches and parses descriptions for tor's configuration options from its man
page. This can be a somewhat lengthy call, and raises an IOError if issues
- occure.
+ occure. When successful loading from a file this returns the version for the
+ contents loaded.
If available, this can load the configuration descriptions from a file where
they were previously persisted to cut down on the load time (latency for this
is around 200ms).
Arguments:
- loadPath - if set, this attempts to fetch the configuration descriptions
- from the given path instead of the man page
+ loadPath - if set, this attempts to fetch the configuration
+ descriptions from the given path instead of the man page
+ checkVersion - discards the results if true and tor's version doens't
+ match the cached descriptors, otherwise accepts anyway
"""
CONFIG_DESCRIPTIONS_LOCK.acquire()
CONFIG_DESCRIPTIONS.clear()
raisedExc = None
+ loadedVersion = ""
try:
if loadPath:
# Input file is expected to be of the form:
@@ -118,8 +127,10 @@
if versionLine.startswith("Tor Version "):
fileVersion = versionLine[12:]
+ loadedVersion = fileVersion
torVersion = torTools.getConn().getInfo("version", "")
- if fileVersion != torVersion:
+
+ if checkVersion and fileVersion != torVersion:
msg = "wrong version, tor is %s but the file's from %s" % (torVersion, fileVersion)
raise IOError(msg)
else:
@@ -161,6 +172,9 @@
else:
manCallResults = sysTools.call("man tor")
+ if not manCallResults:
+ raise IOError("man page not found")
+
# Fetches all options available with this tor instance. This isn't
# vital, and the validOptions are left empty if the call fails.
conn, validOptions = torTools.getConn(), []
@@ -230,6 +244,7 @@
CONFIG_DESCRIPTIONS_LOCK.release()
if raisedExc: raise raisedExc
+ else: return loadedVersion
def saveOptionDescriptions(path):
"""
@@ -260,6 +275,17 @@
outputFile.close()
CONFIG_DESCRIPTIONS_LOCK.release()
+def getConfigSummary(option):
+ """
+ Provides a short summary description of th configuration option. If none is
+ known then this proivdes None.
+
+ Arguments:
+ option - tor config option
+ """
+
+ return CONFIG.get("config.summary.%s" % option.lower())
+
def getConfigDescription(option):
"""
Provides ManPageEntry instances populated with information fetched from the
@@ -288,32 +314,13 @@
conn = torTools.getConn()
configLocation = conn.getInfo("config-file")
+ torPid, torPrefix = conn.getMyPid(), conn.getPathPrefix()
if not configLocation: raise IOError("unable to query the torrc location")
- # checks if this is a relative path, needing the tor pwd to be appended
- if configLocation[0] != "/":
- torPid = conn.getMyPid()
- failureMsg = "querying tor's pwd failed because %s"
- if not torPid: raise IOError(failureMsg % "we couldn't get the pid")
-
- try:
- # pwdx results are of the form:
- # 3799: /home/atagar
- # 5839: No such process
- results = sysTools.call("pwdx %s" % torPid)
- if not results:
- raise IOError(failureMsg % "pwdx didn't return any results")
- elif results[0].endswith("No such process"):
- raise IOError(failureMsg % ("pwdx reported no process for pid " + torPid))
- elif len(results) != 1 or results.count(" ") != 1:
- raise IOError(failureMsg % "we got unexpected output from pwdx")
- else:
- pwdPath = results[0][results[0].find(" ") + 1:]
- configLocation = "%s/%s" % (pwdPath, configLocation)
- except IOError, exc:
- raise IOError(failureMsg % ("the pwdx call failed: " + str(exc)))
-
- return torTools.getPathPrefix() + configLocation
+ try:
+ return torPrefix + sysTools.expandRelativePath(configLocation, torPid)
+ except IOError, exc:
+ raise IOError("querying tor's pwd failed because %s" % exc)
def getMultilineParameters():
"""
@@ -345,7 +352,11 @@
customOptions, conn = set(), torTools.getConn()
configTextQuery = conn.getInfo("config-text", "").strip().split("\n")
- for entry in configTextQuery: customOptions.add(entry[:entry.find(" ")])
+
+ for entry in configTextQuery:
+ # tor provides a Log entry even if it matches the default
+ if entry != "Log notice stdout":
+ customOptions.add(entry[:entry.find(" ")])
return customOptions
def validate(contents = None):
@@ -414,14 +425,6 @@
# issues GETCONF to get the values tor's currently configured to use
torValues = conn.getOption(option, [], True)
- # Some singleline entries are lists, in which case tor provides csv values
- # without spaces, such as:
- # lolcat1,lolcat2,cutebunny,extracutebunny,birthdaynode
- # so we need to strip spaces in comma separated values.
-
- if "," in value:
- value = ",".join([val.strip() for val in value.split(",")])
-
# multiline entries can be comma separated values (for both tor and conf)
valueList = [value]
if option in getMultilineParameters():
Modified: arm/release/src/util/torTools.py
===================================================================
--- arm/release/src/util/torTools.py 2011-01-08 06:06:36 UTC (rev 24053)
+++ arm/release/src/util/torTools.py 2011-01-08 06:35:08 UTC (rev 24054)
@@ -17,7 +17,7 @@
from TorCtl import TorCtl, TorUtil
-from util import log, sysTools
+from util import log, procTools, sysTools, uiTools
# enums for tor's controller state:
# TOR_INIT - attached to a new controller or restart/sighup signal received
@@ -42,7 +42,7 @@
CACHE_ARGS = ("version", "config-file", "exit-policy/default", "fingerprint",
"config/names", "info/names", "features/names", "events/names",
"nsEntry", "descEntry", "bwRate", "bwBurst", "bwObserved",
- "bwMeasured", "flags", "pid")
+ "bwMeasured", "flags", "pid", "pathPrefix", "startTime")
TOR_CTL_CLOSE_MSG = "Tor closed control connection. Exiting event thread."
UNKNOWN = "UNKNOWN" # value used by cached information if undefined
@@ -50,9 +50,12 @@
"features.pathPrefix": "",
"log.torCtlPortClosed": log.NOTICE,
"log.torGetInfo": log.DEBUG,
+ "log.torGetInfoCache": None,
"log.torGetConf": log.DEBUG,
"log.torSetConf": log.INFO,
- "log.torPrefixPathInvalid": log.NOTICE}
+ "log.torPrefixPathInvalid": log.NOTICE,
+ "log.bsdJailFound": log.INFO,
+ "log.unknownBsdJailId": log.WARN}
# events used for controller functionality:
# NOTICE - used to detect when tor is shut down
@@ -67,35 +70,17 @@
def loadConfig(config):
config.update(CONFIG)
-
- # make sure the path prefix is valid and exists (providing a notice if not)
- prefixPath = CONFIG["features.pathPrefix"].strip()
-
- if prefixPath:
- if prefixPath.endswith("/"): prefixPath = prefixPath[:-1]
-
- if prefixPath and not os.path.exists(prefixPath):
- msg = "The prefix path set in your config (%s) doesn't exist." % prefixPath
- log.log(CONFIG["log.torPrefixPathInvalid"], msg)
- prefixPath = ""
-
- CONFIG["features.pathPrefix"] = prefixPath
-def getPathPrefix():
- """
- Provides the path prefix that should be used for fetching tor resources.
- """
-
- return CONFIG["features.pathPrefix"]
-
def getPid(controlPort=9051, pidFilePath=None):
"""
Attempts to determine the process id for a running tor process, using the
following:
1. GETCONF PidFile
- 2. "pidof tor"
- 3. "netstat -npl | grep 127.0.0.1:%s" % <tor control port>
- 4. "ps -o pid -C tor"
+ 2. "pgrep -x tor"
+ 3. "pidof tor"
+ 4. "netstat -npl | grep 127.0.0.1:%s" % <tor control port>
+ 5. "ps -o pid -C tor"
+ 6. "sockstat -4l -P tcp -p %i | grep tor" % <tor control port>
If pidof or ps provide multiple tor instances then their results are
discarded (since only netstat can differentiate using the control port). This
@@ -119,6 +104,16 @@
if pidEntry.isdigit(): return pidEntry
except: pass
+ # attempts to resolve using pgrep, failing if:
+ # - tor is running under a different name
+ # - there are multiple instances of tor
+ try:
+ results = sysTools.call("pgrep -x tor")
+ if len(results) == 1 and len(results[0].split()) == 1:
+ pid = results[0].strip()
+ if pid.isdigit(): return pid
+ except IOError: pass
+
# attempts to resolve using pidof, failing if:
# - tor's running under a different name
# - there's multiple instances of tor
@@ -150,8 +145,46 @@
if pid.isdigit(): return pid
except IOError: pass
+ # attempts to resolve using sockstat, failing if:
+ # - sockstat doesn't accept the -4 flag (BSD only)
+ # - tor is running under a different name
+ # - there are multiple instances of Tor, using the
+ # same control port on different addresses.
+ #
+ # TODO: the later two issues could be solved by filtering for the control
+ # port IP address instead of the process name.
+ try:
+ results = sysTools.call("sockstat -4l -P tcp -p %i | grep tor" % controlPort)
+ if len(results) == 1 and len(results[0].split()) == 7:
+ pid = results[0].split()[2]
+ if pid.isdigit(): return pid
+ except IOError: pass
+
return None
+def getBsdJailId():
+ """
+ Get the FreeBSD jail id for the monitored Tor process.
+ """
+
+ # Output when called from a FreeBSD jail or when Tor isn't jailed:
+ # JID
+ # 0
+ #
+ # Otherwise it's something like:
+ # JID
+ # 1
+
+ torPid = getConn().getMyPid()
+ psOutput = sysTools.call("ps -p %s -o jid" % torPid)
+
+ if len(psOutput) == 2 and len(psOutput[1].split()) == 1:
+ jid = psOutput[1].strip()
+ if jid.isdigit(): return int(jid)
+
+ log.log(CONFIG["log.unknownBsdJailId"], "Failed to figure out the FreeBSD jail id. Assuming 0.")
+ return 0
+
def getConn():
"""
Singleton constructor for a Controller. Be aware that this starts as being
@@ -182,6 +215,11 @@
self._statusTime = 0 # unix time-stamp for the duration of the status
self.lastHeartbeat = 0 # time of the last tor event
+ # Logs issues and notices when fetching the path prefix if true. This is
+ # only done once for the duration of the application to avoid pointless
+ # messages.
+ self._pathPrefixLogging = True
+
# cached GETINFO parameters (None if unset or possibly changed)
self._cachedParam = dict([(arg, "") for arg in CACHE_ARGS])
@@ -316,9 +354,12 @@
if not isFromCache and result and param in CACHE_ARGS:
self._cachedParam[param] = result
- runtimeLabel = "cache fetch" if isFromCache else "runtime: %0.4f" % (time.time() - startTime)
- msg = "GETINFO %s (%s)" % (param, runtimeLabel)
- log.log(CONFIG["log.torGetInfo"], msg)
+ if isFromCache:
+ msg = "GETINFO %s (cache fetch)" % param
+ log.log(CONFIG["log.torGetInfoCache"], msg)
+ else:
+ msg = "GETINFO %s (runtime: %0.4f)" % (param, time.time() - startTime)
+ log.log(CONFIG["log.torGetInfo"], msg)
self.connLock.release()
@@ -411,7 +452,7 @@
if value != None:
if fetchType == "list": result.append(value)
elif fetchType == "map":
- if key in result: result.append(value)
+ if key in result: result[key].append(value)
else: result[key] = [value]
except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed), exc:
if type(exc) == TorCtl.TorCtlClosed: self.close()
@@ -582,6 +623,29 @@
return self._getRelayAttr("pid", None)
+ def getPathPrefix(self):
+ """
+ Provides the path prefix that should be used for fetching tor resources.
+ If undefined and Tor is inside a jail under FreeBsd then this provides the
+ jail's path.
+ """
+
+ result = self._getRelayAttr("pathPrefix", "")
+
+ if result == UNKNOWN: return ""
+ else: return result
+
+ def getStartTime(self):
+ """
+ Provides the unix time for when the tor process first started. If this
+ can't be determined then this provides None.
+ """
+
+ result = self._getRelayAttr("startTime", None)
+
+ if result == UNKNOWN: return None
+ else: return result
+
def getStatus(self):
"""
Provides a tuple consisting of the control port's current status and unix
@@ -981,6 +1045,53 @@
break
elif key == "pid":
result = getPid(int(self.getOption("ControlPort", 9051)), self.getOption("PidFile"))
+ elif key == "pathPrefix":
+ # make sure the path prefix is valid and exists (providing a notice if not)
+ prefixPath = CONFIG["features.pathPrefix"].strip()
+
+ # adjusts the prefix path to account for jails under FreeBSD (many
+ # thanks to Fabian Keil!)
+ if not prefixPath and os.uname()[0] == "FreeBSD":
+ jid = getBsdJailId()
+ if jid != 0:
+ # Output should be something like:
+ # JID IP Address Hostname Path
+ # 1 10.0.0.2 tor-jail /usr/jails/tor-jail
+ jlsOutput = sysTools.call("jls -j %s" % jid)
+
+ if len(jlsOutput) == 2 and len(jlsOutput[1].split()) == 4:
+ prefixPath = jlsOutput[1].split()[3]
+
+ if self._pathPrefixLogging:
+ msg = "Adjusting paths to account for Tor running in a jail at: %s" % prefixPath
+ log.log(CONFIG["log.bsdJailFound"], msg)
+
+ if prefixPath:
+ # strips off ending slash from the path
+ if prefixPath.endswith("/"): prefixPath = prefixPath[:-1]
+
+ # avoid using paths that don't exist
+ if self._pathPrefixLogging and prefixPath and not os.path.exists(prefixPath):
+ msg = "The prefix path set in your config (%s) doesn't exist." % prefixPath
+ log.log(CONFIG["log.torPrefixPathInvalid"], msg)
+ prefixPath = ""
+
+ self._pathPrefixLogging = False # prevents logging if fetched again
+ result = prefixPath
+ elif key == "startTime":
+ myPid = self.getMyPid()
+
+ if myPid:
+ try:
+ if procTools.isProcAvailable():
+ result = float(procTools.getStats(myPid, procTools.STAT_START_TIME)[0])
+ else:
+ psCall = sysTools.call("ps -p %s -o etime" % myPid)
+
+ if psCall and len(psCall) >= 2:
+ etimeEntry = psCall[1].strip()
+ result = time.time() - uiTools.parseShortTimeLabel(etimeEntry)
+ except: pass
# cache value
if result: self._cachedParam[key] = result
Modified: arm/release/src/util/uiTools.py
===================================================================
--- arm/release/src/util/uiTools.py 2011-01-08 06:06:36 UTC (rev 24053)
+++ arm/release/src/util/uiTools.py 2011-01-08 06:35:08 UTC (rev 24054)
@@ -298,13 +298,74 @@
timeLabels = []
- for countPerUnit, shortLabel, longLabel in TIME_UNITS:
+ for countPerUnit, _, _ in TIME_UNITS:
if seconds >= countPerUnit:
timeLabels.append(_getLabel(TIME_UNITS, seconds, 0, isLong))
seconds %= countPerUnit
return timeLabels
+def getShortTimeLabel(seconds):
+ """
+ Provides a time in the following format:
+ [[dd-]hh:]mm:ss
+
+ Arguments:
+ seconds - source number of seconds for conversion
+ """
+
+ timeComp = {}
+
+ for amount, _, label in TIME_UNITS:
+ count = int(seconds / amount)
+ seconds %= amount
+ timeComp[label.strip()] = count
+
+ labelPrefix = ""
+ if timeComp["day"]:
+ labelPrefix = "%i-%02i:" % (timeComp["day"], timeComp["hour"])
+ elif timeComp["hour"]:
+ labelPrefix = "%02i:" % timeComp["hour"]
+
+ return "%s%02i:%02i" % (labelPrefix, timeComp["minute"], timeComp["second"])
+
+def parseShortTimeLabel(timeEntry):
+ """
+ Provides the number of seconds corresponding to the formatting used for the
+ cputime and etime fields of ps:
+ [[dd-]hh:]mm:ss
+
+ If the input entry is malformed then this raises a ValueError.
+
+ Arguments:
+ timeEntry - formatting ps time entry
+ """
+
+ days, hours, minutes, seconds = 0, 0, 0, 0
+ errorMsg = "invalidly formatted ps time entry: %s" % timeEntry
+
+ dateDivider = timeEntry.find("-")
+ if dateDivider != -1:
+ days = int(timeEntry[:dateDivider])
+ timeEntry = timeEntry[dateDivider+1:]
+
+ timeComp = timeEntry.split(":")
+ if len(timeComp) == 3:
+ hours, minutes, seconds = timeComp
+ elif len(timeComp) == 2:
+ minutes, seconds = timeComp
+ else:
+ raise ValueError(errorMsg)
+
+ try:
+ timeSum = int(seconds)
+ timeSum += int(minutes) * 60
+ timeSum += int(hours) * 3600
+ timeSum += int(days) * 86400
+ return timeSum
+ except ValueError:
+ raise ValueError(errorMsg)
+
class Scroller:
"""
Tracks the scrolling position when there might be a visible cursor. This
Modified: arm/release/src/version.py
===================================================================
--- arm/release/src/version.py 2011-01-08 06:06:36 UTC (rev 24053)
+++ arm/release/src/version.py 2011-01-08 06:35:08 UTC (rev 24054)
@@ -2,6 +2,6 @@
Provides arm's version and release date.
"""
-VERSION = '1.4.0.2'
-LAST_MODIFIED = "November 30, 2010"
+VERSION = '1.4.1'
+LAST_MODIFIED = "January 7, 2011"
Modified: arm/resources/build/debian/changelog
===================================================================
--- arm/resources/build/debian/changelog 2011-01-08 06:06:36 UTC (rev 24053)
+++ arm/resources/build/debian/changelog 2011-01-08 06:35:08 UTC (rev 24054)
@@ -1,3 +1,15 @@
+tor-arm (1.4.1.0-1) unstable; urgency=low
+
+ * Upstream release 1.4.1.0.
+
+ -- Damian Johnson <atagar at torproject.org> Fri, 7 Jan 2011 23:30:43 -0700
+
+tor-arm (1.4.0.1-2) unstable; urgency=low
+
+ * Fixing lintian issue with spacing in the rules file (discussed in #607281).
+
+ -- Damian Johnson <atagar at torproject.org> Tue, 16 Dec 2010 23:30:43 -0700
+
tor-arm (1.4.0.1-1) unstable; urgency=low
* Upstream release 1.4.0.1.
Modified: arm/resources/deb-prep.sh
===================================================================
--- arm/resources/deb-prep.sh 2011-01-08 06:06:36 UTC (rev 24053)
+++ arm/resources/deb-prep.sh 2011-01-08 06:35:08 UTC (rev 24054)
@@ -19,6 +19,6 @@
# /usr/share/doc/arm/armrc.sample -> /usr/share/doc/tor-arm/armrc.sample.gz
sed -i 's/\/usr\/share\/doc\/arm\/armrc.sample/\/usr\/share\/doc\/tor-arm\/armrc.sample.gz/g' release_deb/arm.1
-tar czf tor-arm_1.4.0.1.orig.tar.gz release_deb
+tar czf tor-arm_1.4.1.0.orig.tar.gz release_deb
svn export resources/build/debian release_deb/debian
More information about the tor-commits
mailing list