[or-cvs] r22469: {arm} Revised todo with all roadmap for next version and lots of o (arm/trunk)
Damian Johnson
atagar1 at gmail.com
Fri Jun 4 16:29:58 UTC 2010
Author: atagar
Date: 2010-06-04 16:29:58 +0000 (Fri, 04 Jun 2010)
New Revision: 22469
Modified:
arm/trunk/TODO
Log:
Revised todo with all roadmap for next version and lots of other updates.
Modified: arm/trunk/TODO
===================================================================
--- arm/trunk/TODO 2010-06-03 21:59:41 UTC (rev 22468)
+++ arm/trunk/TODO 2010-06-04 16:29:58 UTC (rev 22469)
@@ -1,124 +1,195 @@
TODO
+- Tasks for next release (1.3.6)
+ * refactor panels
+ Currently the interface is a bit of a rat's nest (especially the
+ controller). The goal is to use better modularization to both simplify
+ the codebase and make it possible to use smarter caching to improve
+ performance (far too much is done in the ui logic). This work is in
+ progress - /init and /util are done and /interface is in progress. Known
+ bugs are being fixed while refactoring.
+ * <done> header panel
+ - show advertised bandwidth
+ If there's extra room available show 'MaxAdvertisedBandwidth'?
+ - check if tor's been shut down or merely control port disconnect
+ (overwrite flags with tor status)?
+ * graph panel
+ - graph for arm cpu/mem usage
+ Trivial to implement but not sure if this would be useful.
+ * log panel
+ - option to clear log
+ - allow home/end keys to jump to start/end
+ also do this for the conn panel and conf panel (request by dun)
+ - make log parsing script stand alone, with syntax hilighting, regex,
+ sorting, etc
+ * conf panel
+ - move torrc validation into util
+ - condense tor/arm log listing types if they're the same
+ Ie, make default "TOR/ARM NOTICE - ERR"
+ * conn panel (for version 1.3.7)
+ - check family connections to see if they're alive (VERSION cell
+ handshake?)
+ - fallback when pid or connection querying via pid is unavailable
+ List all connections listed both by netstat and the consensus
+ - note when connection times are estimates (color?), ie connection
+ was established before arm
+ - connection uptime to associate inbound/outbound connections?
+ - Identify controller connections (if it's arm, vidalia, etc) with
+ special detail page for them
+ * controller (for version 1.3.7)
+ * provide performance ARM-DEBUG events
+ Help with diagnosing performance bottlenecks. This is pending the
+ codebase revisions to figure out the low hanging fruit for caching.
+ * wrapper for user config settings
+ * <done> user customizable armrc
+ * make various interface update rates setable via the config
+ * log messages about loading armrc (info level indicating if armrc was
+ found, etc)
+ * tor util
+ * <done> wrapper for accessing torctl
+ * allow arm to resume after restarting tor (attaching to a new torctl
+ instance)
+ * setup scripts for arm
+ * setup scrpt to add to /usr/bin/arm (requested by ioerror)
+ * provide Debian repository
+ Look into debian packaging, note system call dependencies, and mail
+ submit at bugs.debian.org with subject "RFP: arm" and starting with a line
+ "Package: wnpp". Also add to 'deb.torprojec.org'. (requested by helmut)
+ * look into CAPs to get around permission issues for connection listing
+ sudo wrapper for arm to help arm run as the same user as tor?
+ irc suggestions:
+ - man capabilities
+ - http://www.linuxjournal.com/article/5737
+ * release prep
+ * check performance of this version vs last version (general screen refresh
+ times)
+ * pylint --indent-string=" " --disable-msg-cat=CR interface/foo.py | less
+ * double check __init__.py and README for changes
+
- Bugs
- * Mac OSX and BSD have issues with netstat options
- Reported that they aren't cross platform. Possibly use lsof as a
- fallback if an issue's detected.
- notify John Case <case at sdf.lonestar.org>
- caught by Christopher Davis
- * torrc validation doesn't catch if parameters are missing
- * revise multikey sort of connections
- Currently using a pretty ugly hack. Look at:
- http://www.velocityreviews.com/forums/
- t356461-sorting-a-list-of-objects-by-multiple-attributes.html
- and check for performance difference.
- * header panel isn't properly detecting catch-all exit policies
- Missing edge cases
- * avoid hostname lookups of private connections
- Stripped most of them but suspect there might be others (have assertions
- check for this in a debug mode?)
- * exit policy checks aren't handling all inputs
- Still need to handle masks, private keyword, and prepended policy,
- currently erroring on the side of caution.
- * not catching events unexpected by arm
- Future tor and TorCtl revisions could provide new events - these should
- be given the "UNKNOWN" type.
- * regex fails for multiline log entries
- * when logging no events still showing brackets
- The current code for dynamically sizing the events label is kinda
- tricky. Putting this off until I've made a utility to handle this
- uglyness.
- * scrolling in the torrc isn't working properly when comments are stripped
- Current method of displaying torrc is pretty stupid (lots of repeated
- work in display loop). When rewritten fixing this bug should be trivial.
- * quitting can hang several seconds when there's hostnames left to resolve
- Not sure how to address this - problem is that the calls to 'host' can
- take a while to time out. Might need another thread to kill the calls?
- Or forcefully terminate thread if it's taking too long (might be noisy)?
+ * log panel:
+ * not catching events unexpected by arm
+ Future tor and TorCtl revisions could provide new events - these should
+ be given the "UNKNOWN" type.
+ * regex fails for multiline log entries
+ * test that torctl events are being caught (not spotting them...)
+ * torctl events have their own configurable runlevels (provide options for
+ this)
+ * when logging no events still showing brackets
+ The current code for dynamically sizing the events label is kinda
+ tricky. Putting this off until revising this section.
+
+ * conf panel:
+ * torrc validation doesn't catch if parameters are missing
+ * scrolling in the torrc isn't working properly when comments are stripped
+ Current method of displaying torrc is pretty stupid (lots of repeated
+ work in display loop). When rewritten fixing this bug should be
+ trivial.
+ * "ExitPolicy" entry in torrc (without path)
+ Produces "May 26 22:11:03.484 [warn] The abbreviation 'ExitPolic' is
+ deprecated. Please use 'ExitPolicy' instead". This is an error in the
+ torrc parsing when only the key is provided.
+
+ * conn panel:
+ * revise multikey sort of connections
+ Currently using a pretty ugly hack. Look at:
+ http://www.velocityreviews.com/forums/
+ t356461-sorting-a-list-of-objects-by-multiple-attributes.html
+ and check for performance difference.
+ * replace checks against exit policy with Mike's torctl version
+ My version still isn't handling all inputs anyway (still need to handle
+ masks, private keyword, and prepended policy). Parse it from the rest
+ of the router if too heavy ("TorCtl.Router.will_exit_to instead").
+ * prevent connection lookups when tor is shut down (currently presents
+ failover messages)
+ * avoid hostname lookups of private connections
+ Stripped most of them but suspect there might be others (have assertions
+ check for this in a debug mode?)
+ * connection uptimes shouldn't show fractions of a second
+ * connections aren't cleared when control port closes
- Features / Site
- * rewrite codebase
- Currently the interface is a bit of a rat's nest (especially the
- controller). The goal is to use better modularization to both simplify
- the codebase and make it possible to use smarter caching to improve
- performance (far too much is done in the ui logic). This work is in
- progress, having started with the initialization (/init) and now
- concerning the utilities (/util). Migrating the following to util:
- - os calls (to provide transparent platform independence)
- - torrc validation
- - wrapper for tor connection, state, and data parsing (abstracting
- TorCtl connection should allow for arm to be resumed if tor restarts)
- * provide bridge statistics
- Include bridge related data via GETINFO option (feature request by
- waltman).
- * provide performance ARM-DEBUG events
- Help with diagnosing performance bottlenecks. This is pending the
- codebase revisions to figure out the low hanging fruit for caching.
- * condense tor/arm log listing types if they're the same
- Ie, make default "TOR/ARM NOTICE - ERR"
- * graph for arm cpu/mem usage
- Trivial to implement but not sure if this would be helpful.
- * startup option to restrict resource usage or set refresh rate
- * audit tor connections
- Provide warnings if tor misbehaves, checks possibly including:
- - ensuring ExitPolicyRejectPrivate is being obeyed
- - check that ExitPolicy violations don't occure (not possible yet since
- not all relays aren't identified)
- - check that all connections are properly related to a circuit, for
- instance no outbound connections without a corresponding inbound (not
- possible yet due to being unable to correlate connections to circuts)
- * add page that allows raw control port access
- Piggyback on the arm connection, providing something like an interactive
- prompt. In addition, provide:
- - irc like help (ex "/help GETINFO" could provide a summary of getinfo
- commands, partly using the results from "GETINFO info/names")
- - tab completion and up/down populates previous entries
- - warn and get confirmation if command would disrupt arm (for instance
- 'SETEVENTS')
- - 'guard' option that restricts to GETINFO only (start with this)
- - issue sighup reset
- * provide observed bandwidth
- Newer relays have a 'w' entry that states the bandwidth and old versions
- have client side measurements (third argument in 'Bandwidth' of
- descriptor, note that it's in KB/s). Label the former (server side) as
- 'Measured' and later (client side) as 'Observed' to differentiate.
- requested by arma
- * show advertised bandwidth
- if set and there's extra room available show 'MaxAdvertisedBandwidth'
- * check family connections to see if they're alive (VERSION cell handshake?)
- * look into providing UPnP support
- This might be provided by tor itself so wait and see...
- * unit tests
- Primarily for util, for instance 'addfstr' woudl be a good candidate.
+ * attempt to clear controller password from memory
+ http://www.codexon.com/posts/clearing-passwords-in-memory-with-python
+ * try/catch check when starting for curses support?
+ * excaping function for uiTools' formatted strings
+ * provide bridge / client country statistics
+ - Include bridge related data via GETINFO option (feature request by
+ waltman).
+ - Country data for client connections (requested by ioerror)
+ * make update rates configurable via the ui
+ Also provide option for saving these settings to the config
+ * config option to restrict resource usage
+ * dialog with flag descriptions and other help
+ * switch check of ip address validity to regex?
+ match = re.match("(\d*)\.(\d*)\.(\d*)\.(\d*)", ip)
+ http://wang.yuxuan.org/blog/2009/4/2/python_script_to_convert_from_ip_range_to_ip_mask
+ * audit tor connections
+ Provide warnings if tor misbehaves, checks possibly including:
+ - ensuring ExitPolicyRejectPrivate is being obeyed
+ - check that ExitPolicy violations don't occure (not possible yet since
+ not all relays aren't identified)
+ - check that all connections are properly related to a circuit, for
+ instance no outbound connections without a corresponding inbound (not
+ possible yet due to being unable to correlate connections to circuts)
+ * check file descriptors being accessed by tor to see if they're outside the
+ known pattern
+ * add page that allows raw control port access
+ Piggyback on the arm connection, providing something like an interactive
+ prompt. In addition, provide:
+ - irc like help (ex "/help GETINFO" could provide a summary of getinfo
+ commands, partly using the results from "GETINFO info/names")
+ - tab completion and up/down populates previous entries
+ - warn and get confirmation if command would disrupt arm (for instance
+ 'SETEVENTS')
+ - 'guard' option that restricts to GETINFO only (start with this)
+ - issue sighup reset
+ * provide observed bandwidth
+ Newer relays have a 'w' entry that states the bandwidth and old versions
+ have client side measurements (third argument in 'Bandwidth' of
+ descriptor, note that it's in KB/s). Label the former (server side) as
+ 'Measured' and later (client side) as 'Observed' to differentiate.
+ requested by arma
+ * menu with all torrc options (making them editable/toggleable)
+ * Setup wizard for new relays
+ Setting the password and such for torrc generation (idea by ioerror)
+ * menus?
+ http://gnosis.cx/publish/programming/charming_python_6.html
+ * look into better supporting hidden services (what could be useful here?)
+ * look into providing UPnP support
+ This might be provided by tor itself so wait and see...
+ * unit tests
+ Primarily for util, for instance 'addfstr' woudl be a good candidate.
+ * Investigations of other possible tools:
+ * look into additions to the used apis
+ - curses (python 2.6 extended?): http://docs.python.org/library/curses.html
+ - new control options (like "desc-annotations/id/<OR identity>")?
+ - look deeper into TorCtl functions (has a resolve function? hu?)
+ * whois lookup for relays? ISP listing?
+ * look into what sort of information tcpdump and iptraf provides (probably
+ can't use for privacy reasons)
+ * vnstat, nload, mrtg, and traceroute
- Ideas (low priority)
- * python 3 compatability
- Currently blocked on TorCtl support.
- * bundle script that dumps relay stats to stdout
- Django has a small terminal coloring module that could be nice for
- formatting. Could possibly include:
- - desc / ns information for our relay
- - ps / netstat stats like load, uptime, and connection counts, etc
- derived from an idea by StrangeCharm
- * show qos stats
- Take a look at 'linux-tor-prio.sh' to see if any of the stats are
- available and interesting.
- * localization
- Abstract strings from code and provide on translation portal. Thus far
- there hasn't been any requests for this.
- * provide option for a consensus page
- Shows full consensus with an interface similar to the connection panel.
- For this Mike's ConsensusTracker would be helpful (though boost the
- startup time by several seconds)
- * provide Debian repository for arm
- Look into debian packaging, note system call dependencies, and mail
- submit at bugs.debian.org with subject "RFP: arm" and starting with a line
- "Package: wnpp".
- requested by helmut
- * follow up on control-spec proposal
- Proposal and related information is available at:
- http://www.atagar.com/arm/controlSpecProposal.txt
-
- Unfortunatley this doesn't seem to be going anywhere so mothballed for
- now.
+ * python 3 compatability
+ Currently blocked on TorCtl support.
+ * bundle script that dumps relay stats to stdout
+ Django has a small terminal coloring module that could be nice for
+ formatting. Could possibly include:
+ - desc / ns information for our relay
+ - ps / netstat stats like load, uptime, and connection counts, etc
+ derived from an idea by StrangeCharm
+ * show qos stats
+ Take a look at 'linux-tor-prio.sh' to see if any of the stats are
+ available and interesting.
+ * localization
+ Abstract strings from code and provide on translation portal. Thus far
+ there hasn't been any requests for this.
+ * provide option for a consensus page
+ Shows full consensus with an interface similar to the connection panel.
+ For this Mike's ConsensusTracker would be helpful (though boost the
+ startup time by several seconds)
+ * follow up on control-spec proposals
+ Proposal and related information is available at:
+ http://archives.seul.org/or/dev/Jun-2010/msg00008.html
More information about the tor-commits
mailing list