On Mon, 18 Aug 2014, Gunes Acar wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Hi,
Recently, I had some issues when running the TBB test suite. I could submit a patch if you agree that the following can/should be improved:
1-) Make it possible to pass newlayout=0 from the command line and add documentation for the "newlayout" option.
There is a --no-newlayout command line option: https://people.torproject.org/~boklm/tbbtests-doc/usage.html
I noticed that I forgot to regenerate the html version of the doc after it was changed, so maybe it is the reason you didn't see it.
The error messages I got due to the "newlayout" option was very hard to trace.
Indeed, I remember having the same problem previously, and the error message in this case was not very helpful. It could be improved.
2-) The description for the "enable-tests" option says" "Only run the list of tests selected"
This is not completely true since 'tor_bootstrap' will run regardless of that option (managed by the property "always => 1".)
We can either fix the doc or ignore "always=1" when the "enable-tests" option is set.
In the begining, all the tests depended on 'tor_bootstrap' being run, so I added the "always => 1" property to avoid listing it in 'enable-tests' everytime. Now that we have other tests that don't require 'tor_bootstrap', this property has become a little annoying, so maybe it could just be removed.
An other possible improvement could be to add support for dependencies between tests, so that when a test is enabled, its dependencies are enabled too. But just removing the 'always' property would probably be good enough for now.
3-) When the suite skips a test due to "enable-tests", it should print out smt. like "Skipping the test..." or shouldn't print the test name at all. Currently if prints the following even though the tests were skipped:
This would be a good change.
Thanks for reporting those problems. Patches for any of those issues are welcome. Let me know if you need some help.
Nicolas