[tbb-bugs] #17662 [Quality Assurance and Testing]: Have a test to check that Tor Browser updater is working

[Tor Bug Tracker & Wiki]

#17662: Have a test to check that Tor Browser updater is working
-------------------------------------+-------------------------------------
     Reporter:  boklm                |      Owner:  boklm
         Type:  task                 |     Status:  new
     Priority:  Medium               |  Milestone:
    Component:  Quality Assurance    |    Version:
  and Testing                        |   Keywords:  TorBrowserTeam201511,
     Severity:  Normal               |  tbb-testcase
Actual Points:                       |  Parent ID:
       Points:                       |    Sponsor:
-------------------------------------+-------------------------------------
 We should have a test to check that the Tor Browser updater is not broken
 and will be able to update the browser to futur releases when they are
 available. This test would help when making updater changes such as
 #17442.

 Mozilla has some marionette tests that we can use for that:
 https://github.com/mozilla/firefox-ui-tests/blob/mozilla-
 central/firefox_ui_tests/update/direct/test_direct_update.py
 https://github.com/mozilla/firefox-ui-tests/blob/mozilla-
 central/firefox_ui_tests/update/fallback/test_fallback_update.py

 Those tests take as arguments a channel name, a target version (the
 expected version after the update is installed), a target buildid.

 Mozilla QA team is posting on their wiki the config file they use when
 testing each new release:
 https://wiki.mozilla.org/QA/Desktop_Firefox/Releases/Configs/Fx42RC2

 Using this config they are testing that previous releases can be updated
 to the new release, but I could not find tests to check if the new release
 has a working updater that will be able to update to a futur release.

 To be able to check that the updater can update to an other release, I
 think we could create a "test-updater" channel, which provides a signed
 mar file with a large version number, so that the updater always accept
 this update. However the problem would be that an adversary can use this
 mar file to update users to an old browser. Maybe we can include a non-
 working browser in the mar file to avoid this, but this is still not very
 good.

 An alternative could be to have a "test-updater" channel which provides an
 old version, and patch our updater to have a preference that allows
 downgrades. We can then set this preference when testing the updater on
 this channel.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/17662>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online