January 2012 - tor-commits - lists.torproject.org

r25334: {website} 0.2.3.11-alpha tarball is up (website/trunk/include)
by Roger Dingledine 23 Jan '12

23 Jan '12

Author: arma Date: 2012-01-23 07:23:57 +0000 (Mon, 23 Jan 2012) New Revision: 25334 Modified: website/trunk/include/versions.wmi Log: 0.2.3.11-alpha tarball is up Modified: website/trunk/include/versions.wmi =================================================================== --- website/trunk/include/versions.wmi 2012-01-23 02:10:55 UTC (rev 25333) +++ website/trunk/include/versions.wmi 2012-01-23 07:23:57 UTC (rev 25334) @@ -1,5 +1,5 @@ <define-tag version-stable whitespace=delete… [View More]

1 0

[stem/master] Synchronizing runner config
by atagar＠torproject.org 23 Jan '12

23 Jan '12

commit 0a5ff78074309be31ce6f71fde2b0eca86a8c3e6 Author: Damian Johnson <atagar(a)torproject.org> Date: Sun Jan 22 22:15:52 2012 -0800 Synchronizing runner config Using the shiny new sync() method for the runner config to make the world a better place. --- test/runner.py | 20 +++++++++----------- 1 files changed, 9 insertions(+), 11 deletions(-) diff --git a/test/runner.py b/test/runner.py index 2ca380b..c9e3c3e 100644 --- a/test/runner.py +++ b/test/runner.py @@ -… [View More]40,13 +40,15 @@ import stem.util.conf import stem.util.enum import stem.util.term as term -DEFAULT_CONFIG = { +CONFIG = { "test.integ.test_directory": "./test/data", "test.integ.log": "./test/data/log", "test.target.online": False, "test.target.relative_data_dir": False, } +stem.util.conf.get_config("test").sync(CONFIG) + STATUS_ATTR = (term.Color.BLUE, term.Attr.BOLD) SUBSTATUS_ATTR = (term.Color.BLUE, ) ERROR_ATTR = (term.Color.RED, term.Attr.BOLD) @@ -110,7 +112,6 @@ def get_runner(): class Runner: def __init__(self): - self._config = dict(DEFAULT_CONFIG) self._runner_lock = threading.RLock() # runtime attributes, set by the start method @@ -137,9 +138,6 @@ class Runner: self._runner_lock.acquire() - test_config = stem.util.conf.get_config("test") - test_config.update(self._config) - # if we're holding on to a tor process (running or not) then clean up after # it so we can start a fresh instance if self._tor_process: self.stop(quiet) @@ -149,7 +147,7 @@ class Runner: # if 'test_directory' is unset then we make a new data directory in /tmp # and clean it up when we're done - config_test_dir = self._config["test.integ.test_directory"] + config_test_dir = CONFIG["test.integ.test_directory"] if config_test_dir: self._test_dir = stem.util.system.expand_path(config_test_dir, STEM_BASE) @@ -158,7 +156,7 @@ class Runner: original_cwd, data_dir_path = os.getcwd(), self._test_dir - if self._config["test.target.relative_data_dir"]: + if CONFIG["test.target.relative_data_dir"]: tor_cwd = os.path.dirname(self._test_dir) if not os.path.exists(tor_cwd): os.makedirs(tor_cwd) @@ -177,7 +175,7 @@ class Runner: self._start_tor(tor_cmd, quiet) # revert our cwd back to normal - if self._config["test.target.relative_data_dir"]: + if CONFIG["test.target.relative_data_dir"]: os.chdir(original_cwd) except OSError, exc: self.stop(quiet) @@ -201,7 +199,7 @@ class Runner: self._tor_process.communicate() # blocks until the process is done # if we've made a temporary data directory then clean it up - if self._test_dir and self._config["test.integ.test_directory"] == "": + if self._test_dir and CONFIG["test.integ.test_directory"] == "": shutil.rmtree(self._test_dir, ignore_errors = True) self._test_dir = "" @@ -472,7 +470,7 @@ class Runner: raise exc # configures logging - logging_path = self._config["test.integ.log"] + logging_path = CONFIG["test.integ.log"] if logging_path: logging_path = stem.util.system.expand_path(logging_path, STEM_BASE) @@ -532,7 +530,7 @@ class Runner: try: # wait to fully complete if we're running tests with network activity, # otherwise finish after local bootstraping - complete_percent = 100 if self._config["test.target.online"] else 5 + complete_percent = 100 if CONFIG["test.target.online"] else 5 # prints output from tor's stdout while it starts up print_init_line = lambda line: _print_status(" %s\n" % line, SUBSTATUS_ATTR, quiet) [View Less]

1 0

[stem/master] Implementing testing --no-color arg
by atagar＠torproject.org 23 Jan '12

23 Jan '12

commit cfb2c383fc487f4a129345c5724b6a73d96c712d Author: Damian Johnson <atagar(a)torproject.org> Date: Sun Jan 22 23:03:41 2012 -0800 Implementing testing --no-color arg An option for suppressing color escape sequences is important for when test output is piped to a file (the escape characters then just look like gibberish). Still... output is sooooo ugly without color. :( Routing most of the testing stdout writes through the output module, and either … [View More]respecting or discarding formatting there based on if we had a '--no-color' argument. --- run_tests.py | 16 ++++++---- test/output.py | 43 ++++++++++++++++++--------- test/runner.py | 87 +++++++++++++++++++++---------------------------------- 3 files changed, 70 insertions(+), 76 deletions(-) diff --git a/run_tests.py b/run_tests.py index b344111..c7dd91e 100755 --- a/run_tests.py +++ b/run_tests.py @@ -147,6 +147,8 @@ def load_user_configuration(test_config): arg_overrides["argument.log"] = arg.upper() elif opt in ("--tor"): arg_overrides["argument.tor"] = arg + elif opt in ("--no-color"): + arg_overrides["argument.no_color"] = "true" elif opt in ("-h", "--help"): # Prints usage information and quits. This includes a listing of the # valid integration targets. @@ -202,7 +204,7 @@ if __name__ == '__main__': load_user_configuration(test_config) if not CONFIG["argument.unit"] and not CONFIG["argument.integ"]: - print "Nothing to run (for usage provide --help)\n" + test.output.print_line("Nothing to run (for usage provide --help)\n") sys.exit() # if we have verbose logging then provide the testing config @@ -289,12 +291,12 @@ if __name__ == '__main__': if opt in test.runner.Torrc.keys(): torrc_opts.append(test.runner.Torrc[opt]) else: - print "'%s' isn't a test.runner.Torrc enumeration" % opt + test.output.print_line("'%s' isn't a test.runner.Torrc enumeration" % opt) sys.exit(1) integ_runner.start(CONFIG["argument.tor"], extra_torrc_opts = torrc_opts) - print term.format("Running tests...", term.Color.BLUE, term.Attr.BOLD) + test.output.print_line("Running tests...", term.Color.BLUE, term.Attr.BOLD) print for test_class in INTEG_TESTS: @@ -317,7 +319,7 @@ if __name__ == '__main__': for target in skip_targets: req_version = stem.version.Requirement[CONFIG["target.prereq"][target]] - print term.format("Unable to run target %s, this requires tor version %s" % (target, req_version), term.Color.RED, term.Attr.BOLD) + test.output.print_line("Unable to run target %s, this requires tor version %s" % (target, req_version), term.Color.RED, term.Attr.BOLD) print @@ -326,11 +328,11 @@ if __name__ == '__main__': runtime_label = "(%i seconds)" % (time.time() - start_time) if error_tracker.has_error_occured(): - print term.format("TESTING FAILED %s" % runtime_label, term.Color.RED, term.Attr.BOLD) + test.output.print_line("TESTING FAILED %s" % runtime_label, term.Color.RED, term.Attr.BOLD) for line in error_tracker: - print term.format(" %s" % line, term.Color.RED, term.Attr.BOLD) + test.output.print_line(" %s" % line, term.Color.RED, term.Attr.BOLD) else: - print term.format("TESTING PASSED %s" % runtime_label, term.Color.GREEN, term.Attr.BOLD) + test.output.print_line("TESTING PASSED %s" % runtime_label, term.Color.GREEN, term.Attr.BOLD) print diff --git a/test/output.py b/test/output.py index 327482a..5ba9ca4 100644 --- a/test/output.py +++ b/test/output.py @@ -4,11 +4,19 @@ together for improved readability. """ import re +import sys import logging +import stem.util.conf import stem.util.enum import stem.util.term as term +CONFIG = { + "argument.no_color": False, +} + +stem.util.conf.get_config("test").sync(CONFIG) + DIVIDER = "=" * 70 HEADER_ATTR = (term.Color.CYAN, term.Attr.BOLD) CATEGORY_ATTR = (term.Color.GREEN, term.Attr.BOLD) @@ -30,33 +38,37 @@ LINE_ATTR = { LineType.CONTENT: (term.Color.CYAN,), } +def print_line(msg, *attr): + if CONFIG["argument.no_color"]: print msg + else: print term.format(msg, *attr) + +def print_noline(msg, *attr): + if CONFIG["argument.no_color"]: sys.stdout.write(msg) + else: sys.stdout.write(term.format(msg, *attr)) + def print_divider(msg, is_header = False): attr = HEADER_ATTR if is_header else CATEGORY_ATTR - print term.format("%s\n%s\n%s\n" % (DIVIDER, msg.center(70), DIVIDER), *attr) + print_line("%s\n%s\n%s\n" % (DIVIDER, msg.center(70), DIVIDER), *attr) def print_logging(logging_buffer): if not logging_buffer.is_empty(): for entry in logging_buffer: - print term.format(entry.replace("\n", "\n "), term.Color.MAGENTA) + print_line(entry.replace("\n", "\n "), term.Color.MAGENTA) print def print_config(test_config): print_divider("TESTING CONFIG", True) + print_line("Test configuration... ", term.Color.BLUE, term.Attr.BOLD) - try: - print term.format("Test configuration... ", term.Color.BLUE, term.Attr.BOLD) + for config_key in test_config.keys(): + key_entry = " %s => " % config_key - for config_key in test_config.keys(): - key_entry = " %s => " % config_key - - # if there's multiple values then list them on separate lines - value_div = ",\n" + (" " * len(key_entry)) - value_entry = value_div.join(test_config.get_value(config_key, multiple = True)) - - print term.format(key_entry + value_entry, term.Color.BLUE) - except IOError, exc: - sys.stdout.write(term.format("failed (%s)\n" % exc, term.Color.RED, term.Attr.BOLD)) + # if there's multiple values then list them on separate lines + value_div = ",\n" + (" " * len(key_entry)) + value_entry = value_div.join(test_config.get_value(config_key, multiple = True)) + + print_line(key_entry + value_entry, term.Color.BLUE) print @@ -102,7 +114,8 @@ def colorize(line_type, line_content): Applies escape sequences so each line is colored according to its type. """ - return term.format(line_content, *LINE_ATTR[line_type]) + if CONFIG["argument.no_color"]: return line_content + else: return term.format(line_content, *LINE_ATTR[line_type]) def strip_module(line_type, line_content): """ diff --git a/test/runner.py b/test/runner.py index f7a78dc..405f96c 100644 --- a/test/runner.py +++ b/test/runner.py @@ -39,6 +39,7 @@ import stem.version import stem.util.conf import stem.util.enum import stem.util.term as term +import test.output CONFIG = { "integ.test_directory": "./test/data", @@ -121,7 +122,7 @@ class Runner: self._custom_opts = None self._tor_process = None - def start(self, tor_cmd, extra_torrc_opts, quiet = False): + def start(self, tor_cmd, extra_torrc_opts): """ Makes temporary testing resources and starts tor, blocking until it completes. @@ -129,8 +130,6 @@ class Runner: Arguments: tor_cmd (str) - command to start tor with extra_torrc_opts (list) - additional torrc options for our test instance - quiet (bool) - if False then this prints status information as we start - up to stdout Raises: OSError if unable to run test preparations or start tor @@ -140,9 +139,9 @@ class Runner: # if we're holding on to a tor process (running or not) then clean up after # it so we can start a fresh instance - if self._tor_process: self.stop(quiet) + if self._tor_process: self.stop() - _print_status("Setting up a test instance...\n", STATUS_ATTR, quiet) + test.output.print_line("Setting up a test instance...", *STATUS_ATTR) # if 'test_directory' is unset then we make a new data directory in /tmp # and clean it up when we're done @@ -171,28 +170,25 @@ class Runner: try: self._tor_cwd = os.getcwd() - self._run_setup(quiet) - self._start_tor(tor_cmd, quiet) + self._run_setup() + self._start_tor(tor_cmd) # revert our cwd back to normal if CONFIG["test.target.relative_data_dir"]: os.chdir(original_cwd) except OSError, exc: - self.stop(quiet) + self.stop() raise exc finally: self._runner_lock.release() - def stop(self, quiet = False): + def stop(self): """ Stops our tor test instance and cleans up any temporary resources. - - Argument: - quiet (bool) - prints status information to stdout if False """ self._runner_lock.acquire() - _print_status("Shutting down tor... ", STATUS_ATTR, quiet) + test.output.print_noline("Shutting down tor... ", *STATUS_ATTR) if self._tor_process: self._tor_process.kill() @@ -208,7 +204,7 @@ class Runner: self._custom_opts = None self._tor_process = None - _print_status("done\n", STATUS_ATTR, quiet) + test.output.print_line("done", *STATUS_ATTR) self._runner_lock.release() def is_running(self): @@ -423,28 +419,25 @@ class Runner: finally: self._runner_lock.release() - def _run_setup(self, quiet): + def _run_setup(self): """ Makes a temporary runtime resources of our integration test instance. - Arguments: - quiet (bool) - prints status information to stdout if False - Raises: OSError if unsuccessful """ # makes a temporary data directory if needed try: - _print_status(" making test directory (%s)... " % self._test_dir, STATUS_ATTR, quiet) + test.output.print_noline(" making test directory (%s)... " % self._test_dir, *STATUS_ATTR) if os.path.exists(self._test_dir): - _print_status("skipped\n", STATUS_ATTR, quiet) + test.output.print_line("skipped", *STATUS_ATTR) else: os.makedirs(self._test_dir) - _print_status("done\n", STATUS_ATTR, quiet) + test.output.print_line("done", *STATUS_ATTR) except OSError, exc: - _print_status("failed (%s)\n" % exc, ERROR_ATTR, quiet) + test.output.print_line("failed (%s)" % exc, *ERROR_ATTR) raise exc # Makes a directory for the control socket if needed. As of, at least, Tor @@ -455,18 +448,18 @@ class Runner: if Torrc.SOCKET in self._custom_opts: try: socket_dir = os.path.dirname(CONTROL_SOCKET_PATH) - _print_status(" making control socket directory (%s)... " % socket_dir, STATUS_ATTR, quiet) + test.output.print_noline(" making control socket directory (%s)... " % socket_dir, *STATUS_ATTR) if os.path.exists(socket_dir) and stat.S_IMODE(os.stat(socket_dir).st_mode) == 0700: - _print_status("skipped\n", STATUS_ATTR, quiet) + test.output.print_line("skipped", *STATUS_ATTR) else: if not os.path.exists(socket_dir): os.makedirs(socket_dir) os.chmod(socket_dir, 0700) - _print_status("done\n", STATUS_ATTR, quiet) + test.output.print_line("done", *STATUS_ATTR) except OSError, exc: - _print_status("failed (%s)\n" % exc, ERROR_ATTR, quiet) + test.output.print_line("failed (%s)" % exc, *ERROR_ATTR) raise exc # configures logging @@ -474,7 +467,7 @@ class Runner: if logging_path: logging_path = stem.util.system.expand_path(logging_path, STEM_BASE) - _print_status(" configuring logger (%s)... " % logging_path, STATUS_ATTR, quiet) + test.output.print_noline(" configuring logger (%s)... " % logging_path, *STATUS_ATTR) # delete the old log if os.path.exists(logging_path): @@ -487,44 +480,43 @@ class Runner: datefmt = '%D %H:%M:%S', ) - _print_status("done\n", STATUS_ATTR, quiet) + test.output.print_line("done", *STATUS_ATTR) else: - _print_status(" configuring logger... skipped\n", STATUS_ATTR, quiet) + test.output.print_line(" configuring logger... skipped", *STATUS_ATTR) # writes our testing torrc torrc_dst = os.path.join(self._test_dir, "torrc") try: - _print_status(" writing torrc (%s)... " % torrc_dst, STATUS_ATTR, quiet) + test.output.print_noline(" writing torrc (%s)... " % torrc_dst, *STATUS_ATTR) torrc_file = open(torrc_dst, "w") torrc_file.write(self._torrc_contents) torrc_file.close() - _print_status("done\n", STATUS_ATTR, quiet) + test.output.print_line("done", *STATUS_ATTR) for line in self._torrc_contents.strip().splitlines(): - _print_status(" %s\n" % line.strip(), SUBSTATUS_ATTR, quiet) + test.output.print_line(" %s" % line.strip(), *SUBSTATUS_ATTR) - _print_status("\n", (), quiet) + print except Exception, exc: - _print_status("failed (%s)\n\n" % exc, ERROR_ATTR, quiet) + test.output.print_line("failed (%s)\n" % exc, *ERROR_ATTR) raise OSError(exc) - def _start_tor(self, tor_cmd, quiet): + def _start_tor(self, tor_cmd): """ Initializes a tor process. This blocks until initialization completes or we error out. Arguments: tor_cmd (str) - command to start tor with - quiet (bool) - prints status information to stdout if False Raises: OSError if we either fail to create the tor process or reached a timeout without success """ - _print_status("Starting tor...\n", STATUS_ATTR, quiet) + test.output.print_line("Starting tor...\n", *STATUS_ATTR) start_time = time.time() try: @@ -533,30 +525,17 @@ class Runner: complete_percent = 100 if CONFIG["test.target.online"] else 5 # prints output from tor's stdout while it starts up - print_init_line = lambda line: _print_status(" %s\n" % line, SUBSTATUS_ATTR, quiet) + print_init_line = lambda line: test.output.print_line(" %s" % line, *SUBSTATUS_ATTR) torrc_dst = os.path.join(self._test_dir, "torrc") self._tor_process = stem.process.launch_tor(tor_cmd, torrc_dst, complete_percent, print_init_line) runtime = time.time() - start_time - _print_status(" done (%i seconds)\n\n" % runtime, STATUS_ATTR, quiet) + test.output.print_line(" done (%i seconds)\n" % runtime, *STATUS_ATTR) except KeyboardInterrupt: - _print_status(" aborted starting tor: keyboard interrupt\n\n", ERROR_ATTR, quiet) + test.output.print_line(" aborted starting tor: keyboard interrupt\n", *ERROR_ATTR) raise OSError("keyboard interrupt") except OSError, exc: - _print_status(" failed to start tor: %s\n\n" % exc, ERROR_ATTR, quiet) + test.output.print_line(" failed to start tor: %s\n" % exc, *ERROR_ATTR) raise exc -def _print_status(msg, attr = (), quiet = False): - """ - Short alias for printing status messages. - - Arguments: - msg (str) - text to be printed - attr (tuple) - list of term attributes to be applied to the text - quiet (bool) - no-op if true, prints otherwise - """ - - if not quiet: - sys.stdout.write(term.format(msg, *attr)) - [View Less]

1 0

[stem/master] Eliminating the 'test.' config prefix
by atagar＠torproject.org 23 Jan '12

23 Jan '12

commit 92f97544455aee94a229daee20f099f14f92829e Author: Damian Johnson <atagar(a)torproject.org> Date: Sun Jan 22 22:24:53 2012 -0800 Eliminating the 'test.' config prefix It's redundant for testing configurations to have a 'test.' prefix. Configurations are already segregated by their handle so they would be only accessed via the 'test' config anyway. My heart isn't set on these config hierarchies but this will do until someone proposes something … [View More]better. --- run_tests.py | 38 +++++++++++++------------- test/runner.py | 10 +++--- test/settings.cfg | 22 ++++++++-------- test/testrc.sample | 72 ++++++++++++++++++++++++++-------------------------- 4 files changed, 71 insertions(+), 71 deletions(-) diff --git a/run_tests.py b/run_tests.py index 364d411..b344111 100755 --- a/run_tests.py +++ b/run_tests.py @@ -37,11 +37,11 @@ OPT_EXPANDED = ["unit", "integ", "config=", "targets=", "log=", "tor=", "no-colo DIVIDER = "=" * 70 CONFIG = { - "test.arg.unit": False, - "test.arg.integ": False, - "test.arg.log": None, - "test.arg.tor": "tor", - "test.arg.no_color": False, + "argument.unit": False, + "argument.integ": False, + "argument.log": None, + "argument.tor": "tor", + "argument.no_color": False, "target.config": {}, "target.description": {}, "target.prereq": {}, @@ -123,9 +123,9 @@ def load_user_configuration(test_config): for opt, arg in opts: if opt in ("-u", "--unit"): - arg_overrides["test.arg.unit"] = "true" + arg_overrides["argument.unit"] = "true" elif opt in ("-i", "--integ"): - arg_overrides["test.arg.integ"] = "true" + arg_overrides["argument.integ"] = "true" elif opt in ("-c", "--config"): config_path = os.path.abspath(arg) elif opt in ("-t", "--targets"): @@ -144,9 +144,9 @@ def load_user_configuration(test_config): target_config = test_config.get("target.config", {}).get(target) if target_config: arg_overrides[target_config] = "true" elif opt in ("-l", "--log"): - arg_overrides["test.arg.log"] = arg.upper() + arg_overrides["argument.log"] = arg.upper() elif opt in ("--tor"): - arg_overrides["test.arg.tor"] = arg + arg_overrides["argument.tor"] = arg elif opt in ("-h", "--help"): # Prints usage information and quits. This includes a listing of the # valid integration targets. @@ -178,13 +178,13 @@ def load_user_configuration(test_config): # basic validation on user input - log_config = CONFIG["test.arg.log"] + log_config = CONFIG["argument.log"] if log_config and not log_config in log.LOG_VALUES: print "'%s' isn't a logging runlevel, use one of the following instead:" % log_config print " TRACE, DEBUG, INFO, NOTICE, WARN, ERROR" sys.exit(1) - tor_config = CONFIG["test.arg.tor"] + tor_config = CONFIG["argument.tor"] if not os.path.exists(tor_config) and not stem.util.system.is_available(tor_config): print "Unable to start tor, '%s' does not exists." % tor_config sys.exit(1) @@ -201,12 +201,12 @@ if __name__ == '__main__': load_user_configuration(test_config) - if not CONFIG["test.arg.unit"] and not CONFIG["test.arg.integ"]: + if not CONFIG["argument.unit"] and not CONFIG["argument.integ"]: print "Nothing to run (for usage provide --help)\n" sys.exit() # if we have verbose logging then provide the testing config - our_level = stem.util.log.logging_level(CONFIG["test.arg.log"]) + our_level = stem.util.log.logging_level(CONFIG["argument.log"]) info_level = stem.util.log.logging_level(stem.util.log.INFO) if our_level <= info_level: test.output.print_config(test_config) @@ -220,10 +220,10 @@ if __name__ == '__main__': ) stem_logger = log.get_logger() - logging_buffer = log.LogBuffer(CONFIG["test.arg.log"]) + logging_buffer = log.LogBuffer(CONFIG["argument.log"]) stem_logger.addHandler(logging_buffer) - if CONFIG["test.arg.unit"]: + if CONFIG["argument.unit"]: test.output.print_divider("UNIT TESTS", True) for test_class in UNIT_TESTS: @@ -239,7 +239,7 @@ if __name__ == '__main__': print - if CONFIG["test.arg.integ"]: + if CONFIG["argument.integ"]: test.output.print_divider("INTEGRATION TESTS", True) integ_runner = test.runner.get_runner() @@ -248,7 +248,7 @@ if __name__ == '__main__': integ_run_targets = [] all_run_targets = [t for t in TARGETS if CONFIG["target.torrc"].get(t)] - if test_config.get("test.target.run.all", False): + if test_config.get("integ.target.run.all", False): # test against everything with torrc options integ_run_targets = all_run_targets else: @@ -273,7 +273,7 @@ if __name__ == '__main__': if target_prereq: # lazy loaded to skip system call if we don't have any prereqs if not our_version: - our_version = stem.version.get_system_tor_version(CONFIG["test.arg.tor"]) + our_version = stem.version.get_system_tor_version(CONFIG["argument.tor"]) if our_version < stem.version.Requirement[target_prereq]: skip_targets.append(target) @@ -292,7 +292,7 @@ if __name__ == '__main__': print "'%s' isn't a test.runner.Torrc enumeration" % opt sys.exit(1) - integ_runner.start(CONFIG["test.arg.tor"], extra_torrc_opts = torrc_opts) + integ_runner.start(CONFIG["argument.tor"], extra_torrc_opts = torrc_opts) print term.format("Running tests...", term.Color.BLUE, term.Attr.BOLD) print diff --git a/test/runner.py b/test/runner.py index c9e3c3e..f7a78dc 100644 --- a/test/runner.py +++ b/test/runner.py @@ -41,8 +41,8 @@ import stem.util.enum import stem.util.term as term CONFIG = { - "test.integ.test_directory": "./test/data", - "test.integ.log": "./test/data/log", + "integ.test_directory": "./test/data", + "integ.log": "./test/data/log", "test.target.online": False, "test.target.relative_data_dir": False, } @@ -147,7 +147,7 @@ class Runner: # if 'test_directory' is unset then we make a new data directory in /tmp # and clean it up when we're done - config_test_dir = CONFIG["test.integ.test_directory"] + config_test_dir = CONFIG["integ.test_directory"] if config_test_dir: self._test_dir = stem.util.system.expand_path(config_test_dir, STEM_BASE) @@ -199,7 +199,7 @@ class Runner: self._tor_process.communicate() # blocks until the process is done # if we've made a temporary data directory then clean it up - if self._test_dir and CONFIG["test.integ.test_directory"] == "": + if self._test_dir and CONFIG["integ.test_directory"] == "": shutil.rmtree(self._test_dir, ignore_errors = True) self._test_dir = "" @@ -470,7 +470,7 @@ class Runner: raise exc # configures logging - logging_path = CONFIG["test.integ.log"] + logging_path = CONFIG["integ.log"] if logging_path: logging_path = stem.util.system.expand_path(logging_path, STEM_BASE) diff --git a/test/settings.cfg b/test/settings.cfg index bf90ee0..19f5187 100644 --- a/test/settings.cfg +++ b/test/settings.cfg @@ -13,17 +13,17 @@ # Configuration option with which the target is synced. If an option is set via # both the config and '--target' argument then the argument takes precedence. -target.config ONLINE => test.target.online -target.config RELATIVE => test.target.relative_data_dir -target.config RUN_NONE => test.target.run.none -target.config RUN_OPEN => test.target.run.open -target.config RUN_PASSWORD => test.target.run.password -target.config RUN_COOKIE => test.target.run.cookie -target.config RUN_MULTIPLE => test.target.run.multiple -target.config RUN_SOCKET => test.target.run.socket -target.config RUN_SCOOKIE => test.target.run.scookie -target.config RUN_PTRACE => test.target.run.ptrace -target.config RUN_ALL => test.target.run.all +target.config ONLINE => integ.target.online +target.config RELATIVE => integ.target.relative_data_dir +target.config RUN_NONE => integ.target.run.none +target.config RUN_OPEN => integ.target.run.open +target.config RUN_PASSWORD => integ.target.run.password +target.config RUN_COOKIE => integ.target.run.cookie +target.config RUN_MULTIPLE => integ.target.run.multiple +target.config RUN_SOCKET => integ.target.run.socket +target.config RUN_SCOOKIE => integ.target.run.scookie +target.config RUN_PTRACE => integ.target.run.ptrace +target.config RUN_ALL => integ.target.run.all # The '--help' description of the target. diff --git a/test/testrc.sample b/test/testrc.sample index 25ca9aa..a1cd5a6 100644 --- a/test/testrc.sample +++ b/test/testrc.sample @@ -1,13 +1,13 @@ # Integration Test Settings # -# test.arg.unit -# test.arg.integ -# test.arg.log -# test.arg.tor -# test.arg.no_color +# argument.unit +# argument.integ +# argument.log +# argument.tor +# argument.no_color # Default values for runner arguments. # -# test.integ.test_directory +# integ.test_directory # Path used for our data directory and any temporary test resources. Relative # paths are expanded in reference to the location of 'run_tests.py'. # @@ -15,48 +15,48 @@ # have a faster startup and lower load on authorities). If set to an empty # value then this makes a fresh data directory for each test run. # -# test.integ.log +# integ.log # Path runtime logs are placed. Relative paths are expanded in reference to # 'run_tests.py'. Logging is disabled if set ot an empty value. # -# test.target.online +# integ.target.online # Runs tests with network activity. If set then we'll wait for tor to fully # bootstrap when starting, which won't happen without a network connection. # -# test.target.relative_data_dir +# integ.target.relative_data_dir # Uses a relative path for the tor data directory if set. # -# test.target.run.none -# test.target.run.open -# test.target.run.password -# test.target.run.cookie -# test.target.run.multiple -# test.target.run.socket -# test.target.run.scookie -# test.target.run.ptrace -# test.target.run.all +# integ.target.run.none +# integ.target.run.open +# integ.target.run.password +# integ.target.run.cookie +# integ.target.run.multiple +# integ.target.run.socket +# integ.target.run.scookie +# integ.target.run.ptrace +# integ.target.run.all # Runs the integration test suite for all of the given connection and # authentication configurations. If the 'all' option is set then the other # flags are ignored. -test.arg.unit false -test.arg.integ false -test.arg.log -test.arg.tor tor -test.arg.no_color false +argument.unit false +argument.integ false +argument.log +argument.tor tor +argument.no_color false -test.integ.test_directory ./test/data -test.integ.log ./test/data/log +integ.test_directory ./test/data +integ.log ./test/data/log -test.target.online false -test.target.relative_data_dir false -test.target.run.none false -test.target.run.open true -test.target.run.password false -test.target.run.cookie false -test.target.run.muiltipe false -test.target.run.socket false -test.target.run.scookie false -test.target.run.ptrace false -test.target.run.all false +integ.target.online false +integ.target.relative_data_dir false +integ.target.run.none false +integ.target.run.open true +integ.target.run.password false +integ.target.run.cookie false +integ.target.run.muiltipe false +integ.target.run.socket false +integ.target.run.scookie false +integ.target.run.ptrace false +integ.target.run.all false [View Less]

1 0

[stem/master] Expanding Config class' set() method
by atagar＠torproject.org 23 Jan '12

23 Jan '12

commit a0be1932e96e9e52a12a4bbc49e72a059c005f78 Author: Damian Johnson <atagar(a)torproject.org> Date: Sun Jan 22 13:16:01 2012 -0800 Expanding Config class' set() method Users of the set method would usually expect it to overwrite our current configuration value rather than append to it (thanks to Sathyanarayanan for the catch). Also expanding it to handle list or tuple values. --- stem/util/conf.py | 21 +++++++++++++++------ 1 files changed, 15 insertions(+), … [View More]

1 0

[stem/master] Renaming CONN_* targets to RUN_*
by atagar＠torproject.org 23 Jan '12

23 Jan '12

commit d465a9162c58915cbf41c544844cd3520329e3e3 Author: Damian Johnson <atagar(a)torproject.org> Date: Sun Jan 22 11:53:35 2012 -0800 Renaming CONN_* targets to RUN_* The former CONN_* targets each represent a run of the integration tests. These were originally called CONN_* because they defined torrc parameters for connecting to tor, but as time went on they became more general. RUN_* (for instance RUN_ALL) is far more intuitive to users. Also … [View More]dropping the 'integ' prefix from target configurations. It's accurate, but pointless to have. Target options are used frequently enough that we shouldn't make them more verbose without a reason. --- run_tests.py | 6 ++-- test/integ/connection/authentication.py | 2 +- test/integ/connection/connect.py | 2 +- test/integ/connection/protocolinfo.py | 2 +- test/runner.py | 10 +++--- test/settings.cfg | 58 +++++++++++++++--------------- test/testrc.sample | 44 ++++++++++++------------ 7 files changed, 62 insertions(+), 62 deletions(-) diff --git a/run_tests.py b/run_tests.py index be0080b..df9c538 100755 --- a/run_tests.py +++ b/run_tests.py @@ -62,7 +62,7 @@ INTEG_TESTS = ( ) # Integration tests above the basic suite. -TARGETS = stem.util.enum.Enum(*[(v, v) for v in ("ONLINE", "RELATIVE", "CONN_NONE", "CONN_OPEN", "CONN_PASSWORD", "CONN_COOKIE", "CONN_MULTIPLE", "CONN_SOCKET", "CONN_SCOOKIE", "CONN_PTRACE", "CONN_ALL")]) +TARGETS = stem.util.enum.Enum(*[(v, v) for v in ("ONLINE", "RELATIVE", "RUN_NONE", "RUN_OPEN", "RUN_PASSWORD", "RUN_COOKIE", "RUN_MULTIPLE", "RUN_SOCKET", "RUN_SCOOKIE", "RUN_PTRACE", "RUN_ALL")]) CONFIG = { "target.config": {}, @@ -71,7 +71,7 @@ CONFIG = { "target.torrc": {}, } -DEFAULT_RUN_TARGET = TARGETS.CONN_OPEN +DEFAULT_RUN_TARGET = TARGETS.RUN_OPEN HELP_MSG = """Usage runTests.py [OPTION] Runs tests for the stem library. @@ -263,7 +263,7 @@ if __name__ == '__main__': integ_run_targets = [] all_run_targets = [t for t in TARGETS if CONFIG["target.torrc"].get(t)] - if test_config.get("test.integ.target.connection.all", False): + if test_config.get("test.target.run.all", False): # test against everything with torrc options integ_run_targets = all_run_targets else: diff --git a/test/integ/connection/authentication.py b/test/integ/connection/authentication.py index e84542f..886fbb8 100644 --- a/test/integ/connection/authentication.py +++ b/test/integ/connection/authentication.py @@ -25,7 +25,7 @@ INCORRECT_PASSWORD_FAIL = "Authentication failed: Password did not match HashedC class TestAuthenticate(unittest.TestCase): """ - Tests the authentication methods. This should be run with the 'CONN_ALL' + Tests the authentication methods. This should be run with the 'RUN_ALL' integ target to exercise the widest range of use cases. """ diff --git a/test/integ/connection/connect.py b/test/integ/connection/connect.py index d67f702..fb13d20 100644 --- a/test/integ/connection/connect.py +++ b/test/integ/connection/connect.py @@ -11,7 +11,7 @@ import test.runner class TestConnect(unittest.TestCase): """ - Tests the connection methods. This should be run with the 'CONN_ALL' integ + Tests the connection methods. This should be run with the 'RUN_ALL' integ target to exercise the widest range of use cases. """ diff --git a/test/integ/connection/protocolinfo.py b/test/integ/connection/protocolinfo.py index 76ef38e..51076a2 100644 --- a/test/integ/connection/protocolinfo.py +++ b/test/integ/connection/protocolinfo.py @@ -12,7 +12,7 @@ import stem.util.system class TestProtocolInfo(unittest.TestCase): """ - Queries and parses PROTOCOLINFO. This should be run with the 'CONN_ALL' + Queries and parses PROTOCOLINFO. This should be run with the 'RUN_ALL' integ target to exercise the widest range of use cases. """ diff --git a/test/runner.py b/test/runner.py index f78bbc9..2ca380b 100644 --- a/test/runner.py +++ b/test/runner.py @@ -43,8 +43,8 @@ import stem.util.term as term DEFAULT_CONFIG = { "test.integ.test_directory": "./test/data", "test.integ.log": "./test/data/log", - "test.integ.target.online": False, - "test.integ.target.relative_data_dir": False, + "test.target.online": False, + "test.target.relative_data_dir": False, } STATUS_ATTR = (term.Color.BLUE, term.Attr.BOLD) @@ -158,7 +158,7 @@ class Runner: original_cwd, data_dir_path = os.getcwd(), self._test_dir - if self._config["test.integ.target.relative_data_dir"]: + if self._config["test.target.relative_data_dir"]: tor_cwd = os.path.dirname(self._test_dir) if not os.path.exists(tor_cwd): os.makedirs(tor_cwd) @@ -177,7 +177,7 @@ class Runner: self._start_tor(tor_cmd, quiet) # revert our cwd back to normal - if self._config["test.integ.target.relative_data_dir"]: + if self._config["test.target.relative_data_dir"]: os.chdir(original_cwd) except OSError, exc: self.stop(quiet) @@ -532,7 +532,7 @@ class Runner: try: # wait to fully complete if we're running tests with network activity, # otherwise finish after local bootstraping - complete_percent = 100 if self._config["test.integ.target.online"] else 5 + complete_percent = 100 if self._config["test.target.online"] else 5 # prints output from tor's stdout while it starts up print_init_line = lambda line: _print_status(" %s\n" % line, SUBSTATUS_ATTR, quiet) diff --git a/test/settings.cfg b/test/settings.cfg index 75977d6..3766dd4 100644 --- a/test/settings.cfg +++ b/test/settings.cfg @@ -13,46 +13,46 @@ # Configuration option with which the target is synced. If an option is set via # both the config and '--target' argument then the argument takes precedence. -target.config ONLINE => test.integ.target.online -target.config RELATIVE => test.integ.target.relative_data_dir -target.config CONN_NONE => test.integ.target.connection.none -target.config CONN_OPEN => test.integ.target.connection.open -target.config CONN_PASSWORD => test.integ.target.connection.password -target.config CONN_COOKIE => test.integ.target.connection.cookie -target.config CONN_MULTIPLE => test.integ.target.connection.multiple -target.config CONN_SOCKET => test.integ.target.connection.socket -target.config CONN_SCOOKIE => test.integ.target.connection.scookie -target.config CONN_PTRACE => test.integ.target.connection.ptrace -target.config CONN_ALL => test.integ.target.connection.all +target.config ONLINE => test.target.online +target.config RELATIVE => test.target.relative_data_dir +target.config RUN_NONE => test.target.run.none +target.config RUN_OPEN => test.target.run.open +target.config RUN_PASSWORD => test.target.run.password +target.config RUN_COOKIE => test.target.run.cookie +target.config RUN_MULTIPLE => test.target.run.multiple +target.config RUN_SOCKET => test.target.run.socket +target.config RUN_SCOOKIE => test.target.run.scookie +target.config RUN_PTRACE => test.target.run.ptrace +target.config RUN_ALL => test.target.run.all # The '--help' description of the target. target.description ONLINE => Includes tests that require network activity. target.description RELATIVE => Uses a relative path for tor's data directory. -target.description CONN_NONE => Configuration without a way for controllers to connect. -target.description CONN_OPEN => Configuration with an open control port (default). -target.description CONN_PASSWORD => Configuration with password authentication. -target.description CONN_COOKIE => Configuration with an authentication cookie. -target.description CONN_MULTIPLE => Configuration with both password and cookie authentication. -target.description CONN_SOCKET => Configuration with a control socket. -target.description CONN_SCOOKIE => Configuration with a control socket and authentication cookie. -target.description CONN_PTRACE => Configuration with an open control port and 'DisableDebuggerAttachment 0' -target.description CONN_ALL => Runs integration tests for all connection configurations. +target.description RUN_NONE => Configuration without a way for controllers to connect. +target.description RUN_OPEN => Configuration with an open control port (default). +target.description RUN_PASSWORD => Configuration with password authentication. +target.description RUN_COOKIE => Configuration with an authentication cookie. +target.description RUN_MULTIPLE => Configuration with both password and cookie authentication. +target.description RUN_SOCKET => Configuration with a control socket. +target.description RUN_SCOOKIE => Configuration with a control socket and authentication cookie. +target.description RUN_PTRACE => Configuration with an open control port and 'DisableDebuggerAttachment 0' +target.description RUN_ALL => Runs integration tests for all connection configurations. # Version that we need to run the target. These need to match an enumeration of # stem.version.Requirement. -target.prereq CONN_PTRACE => DISABLE_DEBUGGER_ATTACHMENT +target.prereq RUN_PTRACE => DISABLE_DEBUGGER_ATTACHMENT # Torrc configuration options included with the target. Having this option set # means that each of these targets will have a dedicated integration test run. -target.torrc CONN_NONE => -target.torrc CONN_OPEN => PORT -target.torrc CONN_PASSWORD => PORT, PASSWORD -target.torrc CONN_COOKIE => PORT, COOKIE -target.torrc CONN_MULTIPLE => PORT, PASSWORD, COOKIE -target.torrc CONN_SOCKET => SOCKET -target.torrc CONN_SCOOKIE => SOCKET, COOKIE -target.torrc CONN_PTRACE => PORT, PTRACE +target.torrc RUN_NONE => +target.torrc RUN_OPEN => PORT +target.torrc RUN_PASSWORD => PORT, PASSWORD +target.torrc RUN_COOKIE => PORT, COOKIE +target.torrc RUN_MULTIPLE => PORT, PASSWORD, COOKIE +target.torrc RUN_SOCKET => SOCKET +target.torrc RUN_SCOOKIE => SOCKET, COOKIE +target.torrc RUN_PTRACE => PORT, PTRACE diff --git a/test/testrc.sample b/test/testrc.sample index d5a9326..8c89b04 100644 --- a/test/testrc.sample +++ b/test/testrc.sample @@ -12,37 +12,37 @@ # Path runtime logs are placed. Relative paths are expanded in reference to # 'run_tests.py'. Logging is disabled if set ot an empty value. # -# test.integ.target.online +# test.target.online # Runs tests with network activity. If set then we'll wait for tor to fully # bootstrap when starting, which won't happen without a network connection. # -# test.integ.target.relative_data_dir +# test.target.relative_data_dir # Uses a relative path for the tor data directory if set. # -# test.integ.target.connection.none -# test.integ.target.connection.open -# test.integ.target.connection.password -# test.integ.target.connection.cookie -# test.integ.target.connection.multiple -# test.integ.target.connection.socket -# test.integ.target.connection.scookie -# test.integ.target.connection.ptrace -# test.integ.target.connection.all +# test.target.run.none +# test.target.run.open +# test.target.run.password +# test.target.run.cookie +# test.target.run.multiple +# test.target.run.socket +# test.target.run.scookie +# test.target.run.ptrace +# test.target.run.all # Runs the integration test suite for all of the given connection and # authentication configurations. If the 'all' option is set then the other # flags are ignored. test.integ.test_directory ./test/data test.integ.log ./test/data/log -test.integ.target.online false -test.integ.target.relative_data_dir false -test.integ.target.connection.none false -test.integ.target.connection.open true -test.integ.target.connection.password false -test.integ.target.connection.cookie false -test.integ.target.connection.muiltipe false -test.integ.target.connection.socket false -test.integ.target.connection.scookie false -test.integ.target.connection.ptrace false -test.integ.target.connection.all false +test.target.online false +test.target.relative_data_dir false +test.target.run.none false +test.target.run.open true +test.target.run.password false +test.target.run.cookie false +test.target.run.muiltipe false +test.target.run.socket false +test.target.run.scookie false +test.target.run.ptrace false +test.target.run.all false [View Less]

1 0

[stem/master] Overhaul of run_tests.py
by atagar＠torproject.org 23 Jan '12

23 Jan '12

commit e7cb3b59298088b4a8adf35364ff6d775559d27a Author: Damian Johnson <atagar(a)torproject.org> Date: Sun Jan 22 19:10:41 2012 -0800 Overhaul of run_tests.py Several substantial changes to the run_tests.py script to improve readability: - splitting arg parsing from the rest of the main function - adding a config sync method to keep config dictinaries in sync with the main configuration (this will be especially important for arm since it allows for … [View More]proper runtime configuration editing) - moving remaining print functions into test/output.py - lots of general cleanup Remaining todo items from this... - still need to add testing for the config listeners - we should note the module that failed with the failures at the end - we still need multi-line config entries - the --no-color option was added but not yet implemented - the RUN_NONE target looks to be broken --- run_tests.py | 243 +++++++++++++++++++++++++-------------------------- stem/util/conf.py | 88 ++++++++++++++++--- test/output.py | 34 +++++++ test/settings.cfg | 8 +- test/testrc.sample | 14 +++ 5 files changed, 247 insertions(+), 140 deletions(-) diff --git a/run_tests.py b/run_tests.py index df9c538..364d411 100755 --- a/run_tests.py +++ b/run_tests.py @@ -1,7 +1,7 @@ #!/usr/bin/env python """ -Runs unit and integration tests. +Runs unit and integration tests. For usage information run this with '--help'. """ import os @@ -33,9 +33,37 @@ import stem.util.log as log import stem.util.term as term OPT = "uic:l:t:h" -OPT_EXPANDED = ["unit", "integ", "config=", "targets=", "log=", "tor=", "help"] +OPT_EXPANDED = ["unit", "integ", "config=", "targets=", "log=", "tor=", "no-color", "help"] DIVIDER = "=" * 70 +CONFIG = { + "test.arg.unit": False, + "test.arg.integ": False, + "test.arg.log": None, + "test.arg.tor": "tor", + "test.arg.no_color": False, + "target.config": {}, + "target.description": {}, + "target.prereq": {}, + "target.torrc": {}, +} + +TARGETS = stem.util.enum.Enum(*[(v, v) for v in ( + "ONLINE", + "RELATIVE", + "RUN_NONE", + "RUN_OPEN", + "RUN_PASSWORD", + "RUN_COOKIE", + "RUN_MULTIPLE", + "RUN_SOCKET", + "RUN_SCOOKIE", + "RUN_PTRACE", + "RUN_ALL", +)]) + +DEFAULT_RUN_TARGET = TARGETS.RUN_OPEN + # Tests are ordered by the dependencies so the lowest level tests come first. # This is because a problem in say, controller message parsing, will cause all # higher level tests to fail too. Hence we want the test that most narrowly @@ -61,18 +89,7 @@ INTEG_TESTS = ( test.integ.connection.connect.TestConnect, ) -# Integration tests above the basic suite. -TARGETS = stem.util.enum.Enum(*[(v, v) for v in ("ONLINE", "RELATIVE", "RUN_NONE", "RUN_OPEN", "RUN_PASSWORD", "RUN_COOKIE", "RUN_MULTIPLE", "RUN_SOCKET", "RUN_SCOOKIE", "RUN_PTRACE", "RUN_ALL")]) - -CONFIG = { - "target.config": {}, - "target.description": {}, - "target.prereq": {}, - "target.torrc": {}, -} - -DEFAULT_RUN_TARGET = TARGETS.RUN_OPEN - +# TODO: move into settings.cfg when we have multi-line options HELP_MSG = """Usage runTests.py [OPTION] Runs tests for the stem library. @@ -83,74 +100,34 @@ Runs tests for the stem library. -l, --log RUNLEVEL includes logging output with test results, runlevels: TRACE, DEBUG, INFO, NOTICE, WARN, ERROR --tor PATH custom tor binary to run testing against + --no-color displays testing output without color -h, --help presents this help - Integration targets: - %s -""" - -# TODO: add an option to disable output coloring? - -HEADER_ATTR = (term.Color.CYAN, term.Attr.BOLD) -CATEGORY_ATTR = (term.Color.GREEN, term.Attr.BOLD) -DEFAULT_TEST_ATTR = (term.Color.CYAN,) - -TEST_OUTPUT_ATTR = { - "... ok": (term.Color.GREEN,), - "... FAIL": (term.Color.RED, term.Attr.BOLD), - "... ERROR": (term.Color.RED, term.Attr.BOLD), - "... skipped": (term.Color.BLUE,), -} - -def print_divider(msg, is_header = False): - attr = HEADER_ATTR if is_header else CATEGORY_ATTR - print term.format("%s\n%s\n%s\n" % (DIVIDER, msg.center(70), DIVIDER), *attr) - -def print_logging(logging_buffer): - if not logging_buffer.is_empty(): - for entry in logging_buffer: - print term.format(entry.replace("\n", "\n "), term.Color.MAGENTA) - - print + Integration targets:""" -if __name__ == '__main__': - # loads the builtin testing configuration - settings_path = os.path.join(test.runner.STEM_BASE, "test", "settings.cfg") - - test_config = stem.util.conf.get_config("test") - test_config.load(settings_path) - test_config.update(CONFIG) - - # parses target.torrc as csv values and convert to runner Torrc enums - for target in CONFIG["target.torrc"]: - CONFIG["target.torrc"][target] = [] - - for opt in test_config.get_str_csv("target.torrc", [], sub_key = target): - if opt in test.runner.Torrc.keys(): - CONFIG["target.torrc"][target].append(test.runner.Torrc[opt]) - else: - print "'%s' isn't a test.runner.Torrc enumeration" % opt - sys.exit(1) +def load_user_configuration(test_config): + """ + Parses our commandline arguments, loading our custom test configuration if + '--config' was provided and then appending arguments to that. This does some + sanity checking on the input, printing an error and quitting if validation + fails. + """ - start_time = time.time() - run_unit_tests = False - run_integ_tests = False - config_path = None - override_targets = [] - logging_runlevel = None - tor_cmd = "tor" + arg_overrides, config_path = {}, None - # parses user input, noting any issues try: opts, args = getopt.getopt(sys.argv[1:], OPT, OPT_EXPANDED) except getopt.GetoptError, exc: - print str(exc) + " (for usage provide --help)" + print "%s (for usage provide --help)" % exc sys.exit(1) for opt, arg in opts: - if opt in ("-u", "--unit"): run_unit_tests = True - elif opt in ("-i", "--integ"): run_integ_tests = True - elif opt in ("-c", "--config"): config_path = os.path.abspath(arg) + if opt in ("-u", "--unit"): + arg_overrides["test.arg.unit"] = "true" + elif opt in ("-i", "--integ"): + arg_overrides["test.arg.integ"] = "true" + elif opt in ("-c", "--config"): + config_path = os.path.abspath(arg) elif opt in ("-t", "--targets"): integ_targets = arg.split(",") @@ -164,67 +141,75 @@ if __name__ == '__main__': print "Invalid integration target: %s" % target sys.exit(1) else: - override_targets.append(target) + target_config = test_config.get("target.config", {}).get(target) + if target_config: arg_overrides[target_config] = "true" elif opt in ("-l", "--log"): - logging_runlevel = arg.upper() - - if not logging_runlevel in log.LOG_VALUES: - print "'%s' isn't a logging runlevel, use one of the following instead:" % arg - print " TRACE, DEBUG, INFO, NOTICE, WARN, ERROR" - sys.exit(1) + arg_overrides["test.arg.log"] = arg.upper() elif opt in ("--tor"): - if not os.path.exists(arg): - print "Unable to start tor, '%s' does not exists." % arg - sys.exit(1) - - tor_cmd = arg + arg_overrides["test.arg.tor"] = arg elif opt in ("-h", "--help"): # Prints usage information and quits. This includes a listing of the # valid integration targets. + print HELP_MSG + # gets the longest target length so we can show the entries in columns - target_name_length = max([len(name) for name in TARGETS]) - description_format = "%%-%is - %%s" % target_name_length + target_name_length = max(map(len, TARGETS)) + description_format = " %%-%is - %%s" % target_name_length - target_lines = [] for target in TARGETS: - target_lines.append(description_format % (target, CONFIG["target.description"].get(target, ""))) + print description_format % (target, CONFIG["target.description"].get(target, "")) + + print - print HELP_MSG % "\n ".join(target_lines) sys.exit() - if not run_unit_tests and not run_integ_tests: - print "Nothing to run (for usage provide --help)\n" - sys.exit() + # load a testrc if '--config' was given, then apply arguments if config_path: - print_divider("TESTING CONFIG", True) - print - try: - sys.stdout.write(term.format("Loading test configuration (%s)... " % config_path, term.Color.BLUE, term.Attr.BOLD)) test_config.load(config_path) - sys.stdout.write(term.format("done\n", term.Color.BLUE, term.Attr.BOLD)) - - for config_key in test_config.keys(): - key_entry = " %s => " % config_key - - # if there's multiple values then list them on separate lines - value_div = ",\n" + (" " * len(key_entry)) - value_entry = value_div.join(test_config.get_value(config_key, multiple = True)) - - sys.stdout.write(term.format(key_entry + value_entry + "\n", term.Color.BLUE)) except IOError, exc: - sys.stdout.write(term.format("failed (%s)\n" % exc, term.Color.RED, term.Attr.BOLD)) - - print + print "Unable to load testing configuration at '%s': %s" % (config_path, exc) + sys.exit(1) - # Set the configuration flag for our '--target' arguments. This is meant to - # override our configuration flags if both set a target. + for key, value in arg_overrides.items(): + test_config.set(key, value) - for target in override_targets: - target_config = CONFIG["target.config"].get(target) - if target_config: test_config.set(target_config, "true") + # basic validation on user input + + log_config = CONFIG["test.arg.log"] + if log_config and not log_config in log.LOG_VALUES: + print "'%s' isn't a logging runlevel, use one of the following instead:" % log_config + print " TRACE, DEBUG, INFO, NOTICE, WARN, ERROR" + sys.exit(1) + + tor_config = CONFIG["test.arg.tor"] + if not os.path.exists(tor_config) and not stem.util.system.is_available(tor_config): + print "Unable to start tor, '%s' does not exists." % tor_config + sys.exit(1) + +if __name__ == '__main__': + start_time = time.time() + + # loads and validates our various configurations + test_config = stem.util.conf.get_config("test") + test_config.sync(CONFIG) + + settings_path = os.path.join(test.runner.STEM_BASE, "test", "settings.cfg") + test_config.load(settings_path) + + load_user_configuration(test_config) + + if not CONFIG["test.arg.unit"] and not CONFIG["test.arg.integ"]: + print "Nothing to run (for usage provide --help)\n" + sys.exit() + + # if we have verbose logging then provide the testing config + our_level = stem.util.log.logging_level(CONFIG["test.arg.log"]) + info_level = stem.util.log.logging_level(stem.util.log.INFO) + + if our_level <= info_level: test.output.print_config(test_config) error_tracker = test.output.ErrorTracker() output_filters = ( @@ -235,14 +220,14 @@ if __name__ == '__main__': ) stem_logger = log.get_logger() - logging_buffer = log.LogBuffer(logging_runlevel) + logging_buffer = log.LogBuffer(CONFIG["test.arg.log"]) stem_logger.addHandler(logging_buffer) - if run_unit_tests: - print_divider("UNIT TESTS", True) + if CONFIG["test.arg.unit"]: + test.output.print_divider("UNIT TESTS", True) for test_class in UNIT_TESTS: - print_divider(test_class.__module__) + test.output.print_divider(test_class.__module__) suite = unittest.TestLoader().loadTestsFromTestCase(test_class) test_results = StringIO.StringIO() unittest.TextTestRunner(test_results, verbosity=2).run(suite) @@ -250,12 +235,12 @@ if __name__ == '__main__': sys.stdout.write(test.output.apply_filters(test_results.getvalue(), *output_filters)) print - print_logging(logging_buffer) + test.output.print_logging(logging_buffer) print - if run_integ_tests: - print_divider("INTEGRATION TESTS", True) + if CONFIG["test.arg.integ"]: + test.output.print_divider("INTEGRATION TESTS", True) integ_runner = test.runner.get_runner() # Queue up all the targets with torrc options we want to run against. @@ -288,7 +273,7 @@ if __name__ == '__main__': if target_prereq: # lazy loaded to skip system call if we don't have any prereqs if not our_version: - our_version = stem.version.get_system_tor_version(tor_cmd) + our_version = stem.version.get_system_tor_version(CONFIG["test.arg.tor"]) if our_version < stem.version.Requirement[target_prereq]: skip_targets.append(target) @@ -297,13 +282,23 @@ if __name__ == '__main__': if target in skip_targets: continue try: - integ_runner.start(tor_cmd, extra_torrc_opts = CONFIG["target.torrc"].get(target, [])) + # converts the 'target.torrc' csv into a list of test.runner.Torrc enums + torrc_opts = [] + + for opt in test_config.get_str_csv("target.torrc", [], sub_key = target): + if opt in test.runner.Torrc.keys(): + torrc_opts.append(test.runner.Torrc[opt]) + else: + print "'%s' isn't a test.runner.Torrc enumeration" % opt + sys.exit(1) + + integ_runner.start(CONFIG["test.arg.tor"], extra_torrc_opts = torrc_opts) print term.format("Running tests...", term.Color.BLUE, term.Attr.BOLD) print for test_class in INTEG_TESTS: - print_divider(test_class.__module__) + test.output.print_divider(test_class.__module__) suite = unittest.TestLoader().loadTestsFromTestCase(test_class) test_results = StringIO.StringIO() unittest.TextTestRunner(test_results, verbosity=2).run(suite) @@ -311,7 +306,7 @@ if __name__ == '__main__': sys.stdout.write(test.output.apply_filters(test_results.getvalue(), *output_filters)) print - print_logging(logging_buffer) + test.output.print_logging(logging_buffer) except OSError: pass finally: diff --git a/stem/util/conf.py b/stem/util/conf.py index d4e9775..68a332a 100644 --- a/stem/util/conf.py +++ b/stem/util/conf.py @@ -18,6 +18,8 @@ Config - Custom configuration. |- save - writes the current configuration to a file |- clear - empties our loaded configuration contents |- update - replaces mappings in a dictionary with the config's values + |- add_listener - notifies the given listener when an update occures + |- sync - keeps a dictionary synchronized with our config |- keys - provides keys in the loaded configuration |- set - sets the given key/value pair |- unused_keys - provides keys that have never been requested @@ -33,15 +35,28 @@ import stem.util.log as log CONFS = {} # mapping of identifier to singleton instances of configs +class SyncListener: + def __init__(self, config_dict, interceptor): + self.config_dict = config_dict + self.interceptor = interceptor + + def update(self, config, key): + if key in self.config_dict: + new_value = config.get(key, self.config_dict[key]) + if new_value == self.config_dict[key]: return # no change + + if self.interceptor: + interceptor_value = self.interceptor(key, new_value) + if interceptor_value: new_value = interceptor_value + + self.config_dict[key] = new_value + # TODO: methods that will be needed if we want to allow for runtime # customization... # # Config.set(key, value) - accepts any type that the get() method does, # updating our contents with the string conversion # -# Config.addListener(functor) - allow other classes to have callbacks for when -# the configuration is changed (either via load() or set()) -# # Config.save(path) - writes our current configurations, ideally merging them # with the file that exists there so commenting and such are preserved @@ -131,6 +146,7 @@ class Config(): self._path = None # location we last loaded from or saved to self._contents = {} # configuration key/value pairs self._raw_contents = [] # raw contents read from configuration file + self._listeners = [] # functors to be notified of config changes # used for both _contents and _raw_contents access self._contents_lock = threading.RLock() @@ -240,6 +256,45 @@ class Config(): if type(val) == type(conf_mappings[entry]): conf_mappings[entry] = val + def add_listener(self, listener, backfill = True): + """ + Registers the given function to be notified of configuration updates. + Listeners are expected to be functors which accept (config, key). + + Arguments: + listener (functor) - function to be notified when our configuration is + changed + backfill (bool) - calls the function with our current values if true + """ + + self._contents_lock.acquire() + self._listeners.append(listener) + + if backfill: + for key in self.keys(): + listener(key) + + self._contents_lock.release() + + def sync(self, config_dict, interceptor = None): + """ + Synchronizes a dictionary with our current configuration (like the 'update' + method), and registers it to be updated whenever our configuration changes. + + If an interceptor is provided then this is called just prior to assigning + new values to the config_dict. The interceptor function is expected to + accept the (key, value) for the new values and return what we should + actually insert into the dictionary. If this returns None then the value is + updated as normal. + + Arguments: + config_dict (dict) - dictionary to keep synchronized with our + configuration + interceptor (functor) - function referred to prior to assigning values + """ + + self.add_listener(SyncListener(config_dict, interceptor).update) + def keys(self): """ Provides all keys in the currently loaded configuration. @@ -273,15 +328,24 @@ class Config(): the values are appended """ - if isinstance(value, str): - if not overwrite and key in self._contents: self._contents[key].append(value) - else: self._contents[key] = [value] - elif isinstance(value, list) or isinstance(value, tuple): - if not overwrite and key in self._contents: - self._contents[key] += value - else: self._contents[key] = value - else: - raise ValueError("Config.set() only accepts str, list, or tuple. Provided value was a '%s'" % type(value)) + try: + self._contents_lock.acquire() + + if isinstance(value, str): + if not overwrite and key in self._contents: self._contents[key].append(value) + else: self._contents[key] = [value] + + for listener in self._listeners: listener(self, key) + elif isinstance(value, list) or isinstance(value, tuple): + if not overwrite and key in self._contents: + self._contents[key] += value + else: self._contents[key] = value + + for listener in self._listeners: listener(self, key) + else: + raise ValueError("Config.set() only accepts str, list, or tuple. Provided value was a '%s'" % type(value)) + finally: + self._contents_lock.release() def get(self, key, default = None): """ diff --git a/test/output.py b/test/output.py index 6c00d28..327482a 100644 --- a/test/output.py +++ b/test/output.py @@ -9,6 +9,10 @@ import logging import stem.util.enum import stem.util.term as term +DIVIDER = "=" * 70 +HEADER_ATTR = (term.Color.CYAN, term.Attr.BOLD) +CATEGORY_ATTR = (term.Color.GREEN, term.Attr.BOLD) + LineType = stem.util.enum.Enum("OK", "FAIL", "ERROR", "SKIPPED", "CONTENT") LINE_ENDINGS = { @@ -26,6 +30,36 @@ LINE_ATTR = { LineType.CONTENT: (term.Color.CYAN,), } +def print_divider(msg, is_header = False): + attr = HEADER_ATTR if is_header else CATEGORY_ATTR + print term.format("%s\n%s\n%s\n" % (DIVIDER, msg.center(70), DIVIDER), *attr) + +def print_logging(logging_buffer): + if not logging_buffer.is_empty(): + for entry in logging_buffer: + print term.format(entry.replace("\n", "\n "), term.Color.MAGENTA) + + print + +def print_config(test_config): + print_divider("TESTING CONFIG", True) + + try: + print term.format("Test configuration... ", term.Color.BLUE, term.Attr.BOLD) + + for config_key in test_config.keys(): + key_entry = " %s => " % config_key + + # if there's multiple values then list them on separate lines + value_div = ",\n" + (" " * len(key_entry)) + value_entry = value_div.join(test_config.get_value(config_key, multiple = True)) + + print term.format(key_entry + value_entry, term.Color.BLUE) + except IOError, exc: + sys.stdout.write(term.format("failed (%s)\n" % exc, term.Color.RED, term.Attr.BOLD)) + + print + def apply_filters(testing_output, *filters): """ Gets the tests results, possably processed through a series of filters. The diff --git a/test/settings.cfg b/test/settings.cfg index 3766dd4..bf90ee0 100644 --- a/test/settings.cfg +++ b/test/settings.cfg @@ -13,8 +13,8 @@ # Configuration option with which the target is synced. If an option is set via # both the config and '--target' argument then the argument takes precedence. -target.config ONLINE => test.target.online -target.config RELATIVE => test.target.relative_data_dir +target.config ONLINE => test.target.online +target.config RELATIVE => test.target.relative_data_dir target.config RUN_NONE => test.target.run.none target.config RUN_OPEN => test.target.run.open target.config RUN_PASSWORD => test.target.run.password @@ -27,8 +27,8 @@ target.config RUN_ALL => test.target.run.all # The '--help' description of the target. -target.description ONLINE => Includes tests that require network activity. -target.description RELATIVE => Uses a relative path for tor's data directory. +target.description ONLINE => Includes tests that require network activity. +target.description RELATIVE => Uses a relative path for tor's data directory. target.description RUN_NONE => Configuration without a way for controllers to connect. target.description RUN_OPEN => Configuration with an open control port (default). target.description RUN_PASSWORD => Configuration with password authentication. diff --git a/test/testrc.sample b/test/testrc.sample index 8c89b04..25ca9aa 100644 --- a/test/testrc.sample +++ b/test/testrc.sample @@ -1,5 +1,12 @@ # Integration Test Settings # +# test.arg.unit +# test.arg.integ +# test.arg.log +# test.arg.tor +# test.arg.no_color +# Default values for runner arguments. +# # test.integ.test_directory # Path used for our data directory and any temporary test resources. Relative # paths are expanded in reference to the location of 'run_tests.py'. @@ -32,8 +39,15 @@ # authentication configurations. If the 'all' option is set then the other # flags are ignored. +test.arg.unit false +test.arg.integ false +test.arg.log +test.arg.tor tor +test.arg.no_color false + test.integ.test_directory ./test/data test.integ.log ./test/data/log + test.target.online false test.target.relative_data_dir false test.target.run.none false [View Less]

1 0

[tor/master] Use !SOCKET_OK to test if a socket is uninitialized
by nickm＠torproject.org 23 Jan '12

23 Jan '12

commit 9ce9836f853d8a31cde1c6ddd7c4e865099a582a Author: Sebastian Hahn <sebastian(a)torproject.org> Date: Mon Jan 23 02:06:04 2012 +0100 Use !SOCKET_OK to test if a socket is uninitialized This fixes a compile warning on Windows. Fixes bug 4946, not in any released version. --- src/or/dns.c | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diff --git a/src/or/dns.c b/src/or/dns.c index b74b056..ddf9841 100644 --- a/src/or/dns.c +++ b/src/or/dns.c @@ -… [View More]

1 0

r25333: {website} add a new mirror and update the mirrors table. (in website/trunk: . include)
by Andrew Lewman 23 Jan '12

23 Jan '12

Author: phobos Date: 2012-01-23 02:10:55 +0000 (Mon, 23 Jan 2012) New Revision: 25333 Modified: website/trunk/include/mirrors-table.wmi website/trunk/update-mirrors.pl Log: add a new mirror and update the mirrors table. Modified: website/trunk/include/mirrors-table.wmi =================================================================== --- website/trunk/include/mirrors-table.wmi 2012-01-21 20:59:32 UTC (rev 25332) +++ website/trunk/include/mirrors-table.wmi 2012-01-23 02:10:55 UTC (rev … [View More]

1 0

[tor/master] update comment to reflect our TOR_INVALID_SOCKET convention
by arma＠torproject.org 23 Jan '12

23 Jan '12

commit 2de0eeaa720a118f1644656586b27e67dfd24ef2 Author: Roger Dingledine <arma(a)torproject.org> Date: Sun Jan 22 19:39:29 2012 -0500 update comment to reflect our TOR_INVALID_SOCKET convention --- src/or/or.h | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/src/or/or.h b/src/or/or.h index 8b0b3fe..9ac8717 100644 --- a/src/or/or.h +++ b/src/or/or.h @@ -1020,7 +1020,8 @@ typedef struct connection_t { /** CONNECT/SOCKS proxy client handshake state (for … [View More]

1 0