Hello,
I am currently investigating how we can run the Mozilla xpcshell tests on our version of firefox, and the other unit tests.
The xpcshell tests are included on the firefox source tree, and require a complete build tree, built without the '--disable-tests' option. So we cannot run those tests on the version that is shipped in the bundle.
To run those tests, I can see different options:
1. Run the tests as part of the gitian build process and save the output in a file that can be analyzed later.
The bundle build process is already very long, and we probably don't want to make it even longer. Running the xpcshell tests on my laptop (i5-2520M CPU @ 2.50GHz) took 2.5 hours. So this is probably not a good idea.
2. Save the build tree in a tarball at the end of the build, and make it available with the bundle files. We can then use it to run the tests as part of our test suite.
The main problem with this is that the obj directory at the end of the build is taking 6GB. It's possible to reduce it to a 888MB tar.xz file, but it takes 50 minutes to create it on my laptop.
3. Build the browser separatly, in the VM that we use for running the tests. We enough test VMs to be able to do that.
This also allow us to run those tests on any commit.
The main problem with this is that we are not running the tests on the same binaries that are shipped in the bundle.
So I'm thinking about doing 3. Does anybody have any other idea or comment about this ?
Nicolas