[tbb-dev] Tor Browser 7.0 updater tests

[Mark Smith]

Over the past 1.5 days, Kathy and I used installers and MAR files from
https://people.torproject.org/~gk/builds/7.0 to perform various updater
tests on the following platforms:
  * 64-bit Ubuntu 14.04.5 LTS (desktop PC).
  * 32-bit Ubuntu 16.04.2 LTS (VM)
  * OSX 10.12.5 (Retina MacBook Pro)
  * 32-bit Windows 7 Pro (VM).
  * 64-bit Windows 10.0.15063 (Lenovo touch screen laptop)
We hosted XML update manifests (not the ones generated by the build
process though) and MAR files on our own HTTP server.

We did not find any problems during any of our tests. Here is a
description of the two kinds of tests we did:

(a) Upgrades from Tor Browser 6.5.2 to 7.0. This was mainly a test of
the 6.5.2 updater and the 7.0 MAR files. We tested various combinations of:
  * staged updates (Linux, OSX)
  * unstaged updates (all platforms)
  * incremental updates
  * full updates
  * en-US locale
  * es-ES locale
  * with various built-in bridges enabled: obfs3, obfs4, meek-azure.
While we did not exhaustively test every combination on each platform,
we feel good about the amount of testing that we did.

(b) "Upgrades" from 7.0 to 7.0. This was done to exercise the 7.0
updater and ensure that it works in at least the full update case. To do
this kind of testing we trick the update service into thinking there is
a 7.0.1 update available (via tweaks to the XML update manifest on our
HTTP server) and use a complete 7.0 MAR file for the "update." We tested
scenarios similar to the ones I mentioned under (a) above, except
partial updates are not an option in this case.

-- 
Mark Smith
Pearl Crescent
http://pearlcrescent.com/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: OpenPGP digital signature
URL: <http://lists.torproject.org/pipermail/tbb-dev/attachments/20170606/e28d85a8/attachment.sig>