[tor-bugs] #28176 [Applications/Tor Browser]: Cleanup and add the testsuite to tor-browser-build.git

[Tor Bug Tracker & Wiki]

#28176: Cleanup and add the testsuite to tor-browser-build.git
-------------------------------------+-------------------------------------
     Reporter:  boklm                |      Owner:  boklm
         Type:  task                 |     Status:  assigned
     Priority:  Medium               |  Milestone:
    Component:  Applications/Tor     |    Version:
  Browser                            |   Keywords:  tbb-rbm, tbb-qa,
     Severity:  Normal               |  TorBrowserTeam201810
Actual Points:                       |  Parent ID:
       Points:                       |   Reviewer:
      Sponsor:                       |
-------------------------------------+-------------------------------------
 Currently we have a Tor Browser testsuite in a separate git repository, at
 https://gitweb.torproject.org/user/boklm/tor-browser-bundle-
 testsuite.git/.

 Rather than using a separate repository, I think it might be better to
 integrate that testsuite directly into `tor-browser-build.git`, as a sub-
 directory:
 - quite often some Tor Browser changes need corresponding changes in the
 testsuite. Having both in the same repository would make it possible to do
 both changes with the same commit (although we might still need two
 commits when the change is in `tor-browser.git`), and use the same review
 process.
 - some people don't know that we have a testsuite, or where it is. Having
 it in the same repository would make it easier to find and know that it
 exists.
 - having the testsuite in the same repository would make it easier to
 integrate it with the build process. For example we could add some
 makefile command such as `make run-testsuite-testbuild`, which would make
 it easier for developers to run the testsuite after doing a build.

 One option would be to copy the current testsuite to some `tor-browser-
 build.git` sub-directory, and make necessary changes to fix it and
 integrate it better. However I think that over time the testsuite
 accumulated a lot of complexity that is not really needed. Adding the
 current testsuite and fixing it will probably result in adding even more
 complexity. I think that an other option is to do some important cleanup
 first and only import minimal testsuite tools, and then cleanup and import
 each of the tests one by one.

 So I think the main two options are:
 1) copy the current testsuite, fix all the tests, then try to integrate it
 better, and to clean it.
 2) rewrite minimal testsuite tools, avoiding the complexity from the
 previous version, and import each test one by one after fixing and
 cleaning them.

 To me it seems easier to re-start from a minimal testsuite and re-add the
 needed features and tests (sometimes copying them from the old testsuite,
 and sometimes re-writting them from scratch if it makes more sense) than
 taking the current testsuite and trying to clean it little by little, so I
 think option 2) is better.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/28176>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online