Author: atagar Date: 2011-02-27 21:37:03 +0000 (Sun, 27 Feb 2011) New Revision: 24274 Removed: arm/trunk/TODO Log: Removing internal TODO document (it's being replaced by its wiki page: https://trac.torproject.org/projects/tor/wiki/projects/arm) Deleted: arm/trunk/TODO =================================================================== --- arm/trunk/TODO 2011-02-27 21:36:19 UTC (rev 24273) +++ arm/trunk/TODO 2011-02-27 21:37:03 UTC (rev 24274) @@ -1,360 +0,0 @@ -Contents: - 1. Easy Bugs / Improvements - 2. Known Bugs - 3. Features - 3a. Improvements - 3b. Projects - 4. Ideas (low priority or not yet really) - 4. Investigational - - - - - - - - - -- Roadmap and completed work for next release (1.4.2) - [ ] refactor panels - Currently the interface is a bit of a rat's nest (especially the - controller). The goal is to use better modularization to both simplify - the codebase and make it possible to use smarter caching to improve - performance (far too much is done in the ui logic). This work is in - progress - /init and /util are done and /interface is partly done. Known - bugs are being fixed while refactoring. - - * conn panel - - expand client connections and note location in circuit (entry-exit) - - for clients give an option to list all connections, to tell which are - going through tor and which might be leaking - - check family members to see if they're alive (VERSION cell - handshake?) - - fallback when pid or connection querying via pid is unavailable - List all connections listed both by netstat and the consensus - - note when connection times are estimates (color?), ie connection - was established before arm - - connection uptime to associate inbound/outbound connections? - - identify controller connections (if it's arm, vidalia, etc) with - special detail page for them - - provide bridge / client country / exiting port statistics - Include bridge related data via GETINFO option (feature request - by waltman and ioerror). - - note the common port usage along with the exit statistics - - show the port used in scrubbed exit connections - - pick apart applications like iftop and pktstat to see how they get - per-connection bandwidth usage. Forum thread discussing it: - https://bbs.archlinux.org/viewtopic.php?pid=715906 - - include an option to show both the internal and external ips for the - local connection, ie: - myInternal --> myExternal --> foreign - idea and initial patch by Fabian Keil - - give a warning if family relays don't name us - - use SAVECONF instead when overwriting the current torrc - * classify config options as useful (defaultly shown), standard, and - deprecated (configured to be hidden by default) - * check tor source for deprecated options like 'group' (are they - ignored? idea is thanks to NightMonkey) - * elaborate on the password prompt (suggestion by weasel) - -- Roadmap for version 1.4.3 - [ ] refactor panels - [ ] controller and popup panels - - allow arm to resume after restarting tor - This requires a full move to the torTools controller. - - improve on performance bottlenecks for startup time and cpu usage - - intermittent concurrency bugs during shutdown, one possible source: - https://trac.torproject.org/projects/tor/ticket/2144 - [ ] setup scripts for arm - [ ] updater (checks for a new tarball and installs it automatically) - - attempt to verify download signature, providing a warning if unable - to do so - [ ] look into CAPs to get around permission issues for connection - listing sudo wrapper for arm to help arm run as the same user as - tor? Irc suggestions: - - man capabilities - - http://www.linuxjournal.com/article/5737 - -- Bugs - * Config options with long names (ex. UpdateBridgesFromAuthority) are causing - the layout on the config panel to be shifted. - * The default resolver isn't configurable. - * Default config value checks don't work with older versions of tor. We need - a version check for the feature. From Sjon: - > Okay, I found the problem with all values being tagged as default, - > 'GETINFO config-text' has been introduced in 0.2.2.7, while I am - > running 0.2.1.28. Maybe a check in arm for the version-number for that - > feature to be enabled might be an idea? - also: https://trac.torproject.org/projects/tor/ticket/2501 - * When saving the config the Log entry should be filtered out if unnecessary. - * The config write dialog (ie, the one for saving the config) has its a - misaligned border when it's smaller than the top detail section. - * The cpu usage spikes for scrollable content when the key's held. Try - coalescing the events. - * The manpage layout is system dependent, so the scraper needs to be more - resilient against being confused by whitespace. Another improvement is - including fallback results if the man page can't be parsed (suggested by - rransom, issue caught by NightMonkey). - * Log deduplication is currently an n^2 operation. Hence it can't handle - large logs (for instance, when at the DEBUG runlevel). Currently we're - timing out the function if it takes too long, but a more efficient method - for deduplication would be preferable. - * when in client mode and tor stops the header panel doesn't say so - * util/torTools.py: effective bandwidth rate/burst measurements don't take - SETCONF into consideration, blocked on: - https://trac.torproject.org/projects/tor/ticket/1692 - * log prepopulation fails to limit entries to the current tor instance if - the file isn't logged to at the NOTICE level. A fix is to use the - timestamps to see if it belongs to this tor instance. This requires - tor's uptime - blocked on implementation of the following proposal: - https://gitweb.torproject.org/tor.git/blob/HEAD:/doc/spec/proposals/173-geti... - * the STATUS_SERVER event may not be supported - 18:52 < mikeperry> atagar: I believe there is no event parsing for STATUS_SERVER - 18:53 < mikeperry> atagar: see TorCtl.EventSink and classes that inherit from it - 18:54 < mikeperry> specifically, TorCtl.EventHandler._decode1, _handle1, and _map1 - - * conn panel: - * *never* do reverse dns lookups for first hops (could be resolving via - tor and hence leaking to the exit) - * If there's duplicate family entries (and harder case: both nickname and - fingerprint entries for the same relay) then the duplicate should be - removed. This is also causing a bad scrolling bug where the cursor can't - get past the pair of duplicate entries. - * revise multikey sort of connections - Currently using a pretty ugly hack. Look at: - http://www.velocityreviews.com/forums/ - t356461-sorting-a-list-of-objects-by-multiple-attributes.html - and check for performance difference. - * replace checks against exit policy with Mike's torctl version - My version still isn't handling all inputs anyway (still need to handle - masks, private keyword, and prepended policy). Parse it from the rest - of the router if too heavy ("TorCtl.Router.will_exit_to instead"). - * avoid hostname lookups of private connections - Stripped most of them but suspect there might be others (have assertions - check for this in a debug mode?) - * connection uptimes shouldn't show fractions of a second - * connections aren't cleared when control port closes - -- Packaging - * OpenWrt - OpenWrt uses the opkg packaging format which could make use of - arm's current deb packages. Packaging for this platform would help with - the Torouter project: - https://trac.torproject.org/projects/tor/wiki/TheOnionRouter/Torouter - * Mac - Couple of options include macport and dmg... - * macport (http://guide.macports.org/#development) - Build-from-source distribution method (like BSD portinstall). This has - been suggested by several people. - - * dmg (http://en.wikipedia.org/wiki/Apple_Disk_Image) - Most conventional method of software distribution on mac. This is just - a container (no updating/removal support), but could contain an icon - for the dock that starts a terminal with arm. This might include a pkg - installer. - - * mpkg (http://pypi.python.org/pypi/bdist_mpkg/) - Plugin for distutils. Like most mac packaging, this can only run on a - mac. It also requires setuptools: - http://www.errorhelp.com/search/details/74034/importerror-no-module-named-se... - -- Future Features - * Option for the graph to move right to left (easy - feature request by - ioerror and rdegraaf) - * client mode use cases - * not sure what sort of information would be useful in the header (to - replace the orport, fingerprint, flags, etc) - * one idea by velope: - "whether you configured a dnsport, transport, etc. and whether they - were successfully opened. might be nice to know this after the log - messages might be gone." - [notice] Opening Socks listener on 127.0.0.1:9050 - [notice] Opening Transparent pf/netfilter listener on 127.0.0.1:9040 - [notice] Opening DNS listener on 127.0.0.1:53 - * rdns and whois lookups (to find ISP, country, and jurisdiction, etc) - To avoid disclosing connection data to third parties this needs to be - an all-or-nothing operation (ie, needs to fetch information on all - relays or none of them). Plan is something like: - * add resolving/caching capabilities to fetch information on all relays - and distil whois entries to just what we care about (hosting provider - or ISP), by default updating the cache on a daily basis - * construct tarball and make this available for download rather than - fetching everything at each client - * possibly make these archives downloadable from peer relays (this is a - no-go for clients) via torrents or some dirport like scheme - * look at Vidalia and TorK for ideas - * need to solicit for ideas on what would be most helpful to clients - * dialog with bridge statuses (idea by mikeperry) - https://trac.vidalia-project.net/ticket/570 - https://trac.torproject.org/projects/tor/ticket/2068 - * menus - * http://gnosis.cx/publish/programming/charming_python_6.html ? - * additional options: - * make update rates configurable via the ui - * dialog with flag descriptions and other help - * menu with all torrc options (making them editable/toggleable) - * control port interpreter (interactive prompt) - Panel and startup option (-t maybe?) for providing raw control port - access along with usability improvements (piggybacking on the arm - connection): - * irc like help (ex "/help GETINFO" could provide a summary of - getinfo commands, partly using the results from - "GETINFO info/names") - * tab completion and up/down for previous commands - * warn and get confirmation if command would disrupt arm (for - instance 'SETEVENTS') - * 'safe' option that restricts to read-only access (start with this) - * issue sighup reset - * options for more deailed control of when bandwidth's contributed - https://trac.torproject.org/projects/tor/ticket/1730 - * make use of the new process/* GETINFO options - They'll be available in the next tor release, as per: - https://trac.torproject.org/projects/tor/ticket/2291 - * feature parity for arm's config values (armrc entries) - * editability - * parse descriptions from the man page? autogeneration of the man page from - something storing the descriptions - * handle mutiple tor instances - * screen style (dialog for switching between instances) - * extra window with whatever stats can be aggregated over all instances, - or a config option to aggregate stats for bw, resource usage, etc - * option to save the current settings to the config - * provide warning at startup if the armrc doesn't exist, with instructions - for generating it - * email alerts for changes to the relay's status, similar to tor-weather - * simple alert if tor shuts down - * accounting and alerts for if the bandwidth drops to zero - * daily/weekly/etc status reports - * log output, bandwidth history, etc - * graphic of the graphed bandwidth usage - * aggregated port usage for exits and country of origin for clients - * merge in and improve the consensus tracker - * tab completion for input fields that expect a filesystem path - * look through vidalia's tickets for more ideas - https://trac.vidalia-project.net/ - * look into additions to the used apis - * curses (python 2.6 extended?): http://docs.python.org/library/curses.html - * new control options (like "desc-annotations/id/<OR identity>")? - * look into better supporting hidden services (what could be useful here?) - * provide option for a consensus page - Shows full consensus with an interface similar to the connection panel. - For this Mike's ConsensusTracker would be helpful (though boost the - startup time by several seconds) - * splash screen when starting up to indicate that arm is loading, which - would be especially helpful when arm's taking a while to parse the man - page (idea by Sjon) - * show qos stats - Take a look at 'linux-tor-prio.sh' to see if any of the stats are - available and interesting. - * escaping function for uiTools' formatted strings - * provide an option for showing cpu usage on a per-core basis - This would be similar to top when you press 1, ie cpu0 X%, cpu1 Y%, etc. - This would help relay operators in checking system load (feature request - by Jordan) - * setup wizard for new relays - Setting the password and such for torrc generation. Maybe a netinstaller - that fetches the right package for the plagform, verifies signatures, etc? - Another alternative would be that when arm is started and tor isn't - running offer to start tor as a client, relay, or bridge. (idea by ioerror) - * audit what tor does - * Provide warnings if tor connections misbehaves, for instance: - * ensuring ExitPolicyRejectPrivate is being obeyed - * check that ExitPolicy violations don't occur (not possible yet since - not all relays aren't identified) - * check that all connections are properly related to a circuit, for - instance no outbound connections without a corresponding inbound (not - possible yet due to being unable to correlate connections to circuits) - * check file descriptors being accessed by tor to see if they're outside a - known pattern - * script that dumps relay stats to stdout - Derived from an idea by StrangeCharm. Django has a small terminal coloring - module that could be nice for formatting. Could possibly include: - * desc / ns information for our relay - * ps / netstat stats like load, uptime, and connection counts, etc - * implement control-spec proposals: - * https://gitweb.torproject.org/tor.git/blob/HEAD:/doc/spec/proposals/172-circ... - * https://gitweb.torproject.org/tor.git/blob/HEAD:/doc/spec/proposals/173-geti... - * urwid improvements - Urwid (http://excess.org/urwid/) is an open source python library for - terminal applications. It provides a nicer interface, ideal for terminal - forms and some common UI elements. - - As I recall it isn't compatable with using curses directly, but checking - the library for interesting capabilities and code snippets could be very - helpful. For instance, the screenshots exemplify color support beyond the - 8 colors (RGB + CMYK) available with curses... *very* curious how they do - this and how widely higher color support is available! - * gui frontend (gtk?) - Look into if the arm utilities and codebase would fit nicely for a gui - controller like Vidalia and TorK. - * unit tests - Primarily for util, for instance 'addfstr' would be a good candidate. - * python 3 compatibility - Currently blocked on TorCtl support. - -- Release Checklist - 1. Run a sanity check over major changes: - - pylint --indent-string=" " --disable=C,R interface/foo.py | less - - double check __init__.py and README for added or removed files - 2. Revise the TODO (adding plans for the next release) - 3. Update release notes - - Spellcheck notes and copy them to the site (atagar.com/arm/log.php) - - Copy rendered release notes into the ChangeLog - 4. Check into release and upload the signed tarball - - svn co https://svn.torproject.org/svn/arm/ - - svn merge release trunk - - bump the version - - release/src/version.py - - resources/deb-prep.sh - - resources/build/debian/changelog - - svn commit -m "Arm release 1.4.1" - - note the commit number in the site log and ChangeLog - - svn export release arm - - copy the export and perform a final sanity check that it works - - tar cjf arm-1.4.1.0.tar.bz2 arm - - gpg --detach-sig --armor arm-1.4.1.0.tar.bz2 - - copy tarball and sig to the webserver (both to resources and static) - - update links and bump the release in download.php and index.php - - double check that there aren't any broken links - 5. Release notifications - - Log into https://blog.torproject.org/user/ - - Write a new blog post, example: - https://blog.torproject.org/blog/arm-release-141 - - Send an update to or-talk, example: - http://archives.seul.org/or/talk/Jan-2011/msg00096.html - 6. After a week bump package versions: - - Debian - Contact: weasel (Peter Palfrader) - Initial Release: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=603056 - Update Instructions: - - update resources/build/debian/changelog - - ./resources/deb-prep.sh - - cd release_deb - - ./debian/make-deb - - copy the results to the webserver, example: - http://www.atagar.com/transfer/tmp/armBuild_12-7-10/ - - send the dsc link to weasel - - sign deb and copy that and sig for the download page - - - Red Hat - Contact: None - Update Instructions: - - update resources/build/redHat/MANIFEST - - ./resources/rpm-prep.sh - - cd release_rpm - - ./debian/make-rpm - - copy the results to the webserver - - sign rpm and copy that and sig for the download page - - - Gentoo - Contact: NightMonkey (Jesse Adelman) - Initial Release: https://bugs.gentoo.org/show_bug.cgi?id=341731 - Update Instructions: - - go to https://bugs.gentoo.org - - make a generic bug with "net-misc/arm-X.X.X version bump, please" - - - ArchLinux - Contact: Spider.007 - Initial Release: http://aur.archlinux.org/packages.php?ID=44172 - Update Instructions: - - go to aur.archlinux.org - - select "Out-of-date" for the package -