[tor-bugs] #14944 [Stem]: Allow just a single test to be run

[Tor Bug Tracker & Wiki]

#14944: Allow just a single test to be run
-----------------------------+---------------------------
     Reporter:  atagar       |      Owner:  atagar
         Type:  enhancement  |     Status:  new
     Priority:  minor        |  Milestone:
    Component:  Stem         |    Version:
   Resolution:               |   Keywords:  testing, easy
Actual Points:               |  Parent ID:
       Points:               |
-----------------------------+---------------------------

Comment (by clv):

 Hi,

 I pushed what I've done here:
 https://github.com/leivaburto/stem/commit/27c8673552ced6ca9daa3674249daa45698a202e

 I basically changed from `loadTestsFromTestCase` to
 `loadTestsFromTestCase`, added a try-except on run_tests and modified
 _get_tests in util.py in order to:

  * Look for the given string (module_substring) in modules list.

  * If nothing found, remove last fragment of the string (what is supposed
 to be a method) and try again. If something is found after that, it will
 return the module found + the name of the method.

 To test the changes I made, I tried with the following arguments:

  * --all
  * --all --test test.integ.process (process module)
  * --all --test process
  * --integ --test test.integ.process
  * --integ --test process
  * -all --test test.integ.process.test_quiet_arguments (process module +
 single method)
  * -all --test process.test_quiet_arguments
  * --integ --test test.integ.process.test_quiet_arguments
  * --integ --test process.test_quiet_arguments

 Aditionally, there a couple of things to notice:

 This will show a fail msg:

  * --integ --test process.test_quiet_argumentss (double 's' at the end,
 non existing method)

 This will show nothing (as it does now)

  * --integ --test processs (triple 's' at the end, non existing module)

 Finally, there is one thing I haven't been able to solve: testing a non
 existing module  inside a sub folder. For example:

  * --all --test test.integ.descriptor.microdescriptors (extra 's' at the
 end, non existing module)

 As this module is not found, the program will remove the last fragment
 (microdescriptors) and try to look for test.integ.descriptor, which is the
 base for 5 sub modules that do exist, and will find all of them and try
 for the "method" microdescriptors on them, which will finally produce 5
 "failed messages". The problem here is that tere is no way of knowing
 beforehand if the given string corresponds to a submodule or a method...
 so I've been thinking that there are two possible solutions for this:

  1. List all method names in settings.cfg, as it is done now for module
 names.
  1. Add a new argument for testing single methods, so we know beforehand
 that we were asked for that and not something else. It could be something
 like `--test-method`

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/14944#comment:3>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online