[tor-bugs] #9444 [Tor bundles/installation]: Create deterministic TorBrowserBundles with Pluggable Transports

[Tor Bug Tracker & Wiki]

#9444: Create deterministic TorBrowserBundles with Pluggable Transports
------------------------------------------+--------------------------
     Reporter:  bastik                    |      Owner:  erinn
         Type:  task                      |     Status:  needs_review
     Priority:  normal                    |  Milestone:
    Component:  Tor bundles/installation  |    Version:
   Resolution:                            |   Keywords:  flashproxy
Actual Points:                            |  Parent ID:
       Points:                            |
------------------------------------------+--------------------------

Comment (by dcf):

 I found three remaining sources of nondeterminism and fixed them. I think
 that all the builds are reproducible now. I'm currently doing builds to
 make sure.
  1. Compiled Python extension modules (`.pyd` files) contain PE executable
 timestamps. This is solved by [https://gitweb.torproject.org/user/dcf/tor-
 browser-bundle.git/commitdiff/bf5e9a79fc63162e1d18c34146b329a7341d39b6
 enabling libfaketime in the wine-wrappers]. This is what Erinn and I tried
 to do in comment:24; what we were doing wrong was setting `LD_PRELOAD`
 without also setting `FAKETIME`. I also tried changing the timestamps
 using a `strip` or `objcopy` command after the fact; that changed the time
 in the header, but left a timestamp in the `.edata` section.
  2. Byte-compiled `.pyc` files [http://benno.id.au/blog/2013/01/15/python-
 determinism contain the timestamp] (the mtime) of the `.py` file they were
 compiled from. This is solved by [https://gitweb.torproject.org/user/dcf
 /tor-browser-
 bundle.git/commitdiff/bd4b3de712e345adc20ec9e959759f03da5826bc setting the
 timestamps] of Python source files before they are byte-compiled by
 py2exe.
  3. In addition to byte-compiling existing `.py` files, py2exe
 [http://sourceforge.net/p/py2exe/svn/HEAD/tree/tags/release_0_6_9/py2exe/build_exe.py#l1178
 dynamically generates a stub loader] for every `.pyd` extension module.
 These stubs are created in the filesystem at py2exe time, so there's no
 opportunity to change their timestamps. I added a script to
 [https://gitweb.torproject.org/user/dcf/tor-browser-
 bundle.git/commitdiff/2b21c0b477c31ee3b14cb4bc42238850d6f95ef8 rewrite
 .pyc timestamps] and had it [https://gitweb.torproject.org/user/dcf/tor-
 browser-bundle.git/commitdiff/ca4b110a02d4b8a243698e745871baea62dc1f73
 called on py2exe-generated zip files]. It has the side effect of re-
 dzipping py2exe's zip files.
 The build I'm currently doing is using
 ca4b110a02d4b8a243698e745871baea62dc1f73. If all goes well, the sha256sum
 of the windows bundle should be
 {{{
 814647f814e455932460ea99724e385e4cd3b8b491e48aa4d1d3ebc3d29334b6  tor-
 pluggable-transports-browser-install-3.0-beta-1_en-US.exe
 }}}

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/9444#comment:27>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online