[tor-bugs] #9444 [Tor bundles/installation]: Create deterministic TorBrowserBundles with Pluggable Transports

[Tor Bug Tracker & Wiki]

#9444: Create deterministic TorBrowserBundles with Pluggable Transports
------------------------------------------+--------------------------
     Reporter:  bastik                    |      Owner:  erinn
         Type:  task                      |     Status:  needs_review
     Priority:  normal                    |  Milestone:
    Component:  Tor bundles/installation  |    Version:
   Resolution:                            |   Keywords:  flashproxy
Actual Points:                            |  Parent ID:
       Points:                            |
------------------------------------------+--------------------------

Comment (by dcf):

 I built 35dfeaff54a692f9dfe21032289e8e13be75a760 twice in a row, and as
 suspected, the linux and mac packages are identical while the windows are
 not.
 {{{
 --- 3.0-beta-1-pt20131125-1/sha256sums.txt      2013-11-25
 22:11:49.000000000 -0800
 +++ 3.0-beta-1-pt20131125-2/sha256sums.txt      2013-11-25
 22:11:59.000000000 -0800
 @@ -1,4 +1,4 @@
  5ea95bffd85b8836276e818dc61532b79f6747270eb7741723b3dd71b32cc814
 TorPluggableTransportsBrowserBundle-3.0-beta-1-osx32_en-US.zip
 -821084a3ed69c296d56114c9aa0542801df1442b8dfe2c072947c9733c9e12ef  tor-
 pluggable-transports-browser-install-3.0-beta-1_en-US.exe
 +e34417ae86c46e02f5dabff8a017052aa85c4b83d5c6c2a273754e868f0143fb  tor-
 pluggable-transports-browser-install-3.0-beta-1_en-US.exe
  0bcfeed50712e15057604f3a8e4a639d62466e7aac4084712a7c2f7982fc54ef  tor-
 pluggable-transports-browser-linux32-3.0-beta-1_en-US.tar.xz
  3a9fc11456ffeca7519440c9131302bdbdf0f276cf62470241272389b2c87eff  tor-
 pluggable-transports-browser-linux64-3.0-beta-1_en-US.tar.xz
 }}}
 Packages are here:
  * https://people.torproject.org/~dcf/pt-bundle/3.0-beta-1-pt20131125-1/
  * https://people.torproject.org/~dcf/pt-bundle/3.0-beta-1-pt20131125-2/
 What differs is some compiled Python extensions (.pyd files) and the zip
 files created by py2exe:
 {{{
 $ diff -Nru 1/ 2/
 Binary files 1/Tor/Crypto.Cipher._AES.pyd and 2/Tor/Crypto.Cipher._AES.pyd
 differ
 Binary files 1/Tor/Crypto.Util._counter.pyd and
 2/Tor/Crypto.Util._counter.pyd differ
 Binary files 1/Tor/M2Crypto.__m2crypto.pyd and
 2/Tor/M2Crypto.__m2crypto.pyd differ
 Binary files 1/Tor/obfsproxy.zip and 2/Tor/obfsproxy.zip differ
 Binary files 1/Tor/py2exe-flashproxy.zip and 2/Tor/py2exe-flashproxy.zip
 differ
 Binary files 1/Tor/zope.interface._zope_interface_coptimizations.pyd and
 2/Tor/zope.interface._zope_interface_coptimizations.pyd differ
 }}}
 I assume that the zips can be fixed by re-dzipping them.

 A diff of one of the compiled extensions shows that a few bytes differ,
 some of which look like timestamps:
 {{{
 $ diff -u <(xxd 1/Tor/Crypto.Cipher._AES.pyd) <(xxd
 2/Tor/Crypto.Cipher._AES.pyd)
 --- /dev/fd/63  2013-11-26 06:14:39.309446996 +0000
 +++ /dev/fd/62  2013-11-26 06:14:39.309446996 +0000
 @@ -6,12 +6,12 @@
  0000050: 6973 2070 726f 6772 616d 2063 616e 6e6f  is program canno
  0000060: 7420 6265 2072 756e 2069 6e20 444f 5320  t be run in DOS
  0000070: 6d6f 6465 2e0d 0d0a 2400 0000 0000 0000  mode....$.......
 -0000080: 5045 0000 4c01 0900 9209 9352 0000 0000  PE..L......R....
 +0000080: 5045 0000 4c01 0900 5d20 9452 0000 0000  PE..L...] .R....
  0000090: 0000 0000 e000 0e23 0b01 0216 004c 0000  .......#.....L..
  00000a0: 0096 0000 0004 0000 2014 0000 0010 0000  ........ .......
  00000b0: 0060 0000 0000 5c6d 0010 0000 0002 0000  .`....\m........
  00000c0: 0400 0000 0100 0000 0400 0000 0000 0000  ................
 -00000d0: 0000 0100 0004 0000 04c3 0000 0300 0000  ................
 +00000d0: 0000 0100 0004 0000 9cf0 0000 0300 0000  ................
  00000e0: 0000 2000 0010 0000 0000 1000 0010 0000  .. .............
  00000f0: 0000 0000 1000 0000 00b0 0000 4800 0000  ............H...
  0000100: 00c0 0000 9c09 0000 0000 0000 0000 0000  ................
 @@ -2110,7 +2110,7 @@
  00083d0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
  00083e0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
  00083f0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 -0008400: 0000 0000 9209 9352 0000 0000 28b0 0000  .......R....(...
 +0008400: 0000 0000 5d20 9452 0000 0000 28b0 0000  ....] .R....(...
  0008410: 0100 0000 0100 0000 0100 0000 34b0 0000  ............4...
  0008420: 38b0 0000 3cb0 0000 5f41 4553 2e70 7964  8...<..._AES.pyd
  0008430: 0000 0000 7046 0000 3eb0 0000 0000 696e  ....pF..>.....in
 }}}
 Erinn, do you have any interest in looking at these reproducibility
 issues? Even if we can't run Wine under libfaketime, perhaps we can build
 it into the gcc and dllwrap wine-wrappers.

-- 
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/9444#comment:22>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online