[tor-qa] fix for some tbb test suite issues ?

[Nicolas Vigier]

On Mon, 18 Aug 2014, Gunes Acar wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Hi,
> 
> Recently, I had some issues when running the TBB test suite. I could
> submit a patch if you agree that the following can/should be improved:
> 
> 1-) Make it possible to pass newlayout=0 from the command line and add
> documentation for the "newlayout" option.

There is a --no-newlayout command line option:
https://people.torproject.org/~boklm/tbbtests-doc/usage.html

I noticed that I forgot to regenerate the html version of the doc after
it was changed, so maybe it is the reason you didn't see it.

> 
> The error messages I got due to the "newlayout" option was very hard
> to trace.

Indeed, I remember having the same problem previously, and the error
message in this case was not very helpful. It could be improved.

> 
> 2-) The description for the "enable-tests" option says" "Only run the
> list of tests selected"
> 
> This is not completely true since 'tor_bootstrap' will run regardless
> of that option (managed by the property "always => 1".)
> 
> We can either fix the doc or ignore "always=1" when the "enable-tests"
> option is set.

In the begining, all the tests depended on 'tor_bootstrap' being run, so
I added the "always => 1" property to avoid listing it in 'enable-tests'
everytime. Now that we have other tests that don't require 'tor_bootstrap',
this property has become a little annoying, so maybe it could just be
removed.

An other possible improvement could be to add support for dependencies
between tests, so that when a test is enabled, its dependencies are
enabled too. But just removing the 'always' property would probably be
good enough for now.

> 
> 3-) When the suite skips a test due to "enable-tests", it should print
> out smt. like "Skipping the test..." or shouldn't print the test name
> at all. Currently if prints the following even though the tests were
> skipped:

This would be a good change.


Thanks for reporting those problems. Patches for any of those issues
are welcome. Let me know if you need some help.

Nicolas

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.torproject.org/pipermail/tor-qa/attachments/20140819/20b25c34/attachment.sig>