[tor-bugs] #9444 [Tor bundles/installation]: Create deterministic TorBrowserBundles with Pluggable Transports

[Tor Bug Tracker & Wiki]

#9444: Create deterministic TorBrowserBundles with Pluggable Transports
------------------------------------------+--------------------------
     Reporter:  bastik                    |      Owner:  erinn
         Type:  task                      |     Status:  needs_review
     Priority:  normal                    |  Milestone:
    Component:  Tor bundles/installation  |    Version:
   Resolution:                            |   Keywords:  flashproxy
Actual Points:                            |  Parent ID:
       Points:                            |
------------------------------------------+--------------------------
Changes (by dcf):

 * status:  new => needs_review


Comment:

 In the "release early" spirit, here is some process on this ticket.
 {{{
 git clone https://www.bamsoftware.com/git/tor-browser-bundle.git
 }}}
 Or, to reuse your existing local repo,
 {{{
 git remote add bamsoftware https://www.bamsoftware.com/git/tor-browser-
 bundle.git
 git fetch bamsoftware
 git checkout -b pt bamsoftware/pt
 git diff master...pt
 }}}
 Commit 0085efb6edeb77036e0f17310199ce3b2fea9590 in that repo has some
 first steps towards building a pluggable transports bundle. When you run
 `make`, you will end up with new extra files
 {{{
 tor-pluggable-transports-browser-linux32-3.0-beta-1_en-US.tar.xz
 tor-pluggable-transports-browser-linux64-3.0-beta-1_en-US.tar.xz
 }}}
 These archives contain an installation of pyptlib (which is standing as a
 proof-of-concept placeholder for all pluggable transport files). I
 implemented it only for the linux bundles so far.

 A summary of the changes:
  1. A new `gitian-pluggable-transports.yml` builds pyptlib and `pluggable-
 transports-linux$GBUILD_BITS-gbuilt.zip`.
  2. A new `gitian-ptbundle.yml` (almost completely a copy of `gitian-
 bundle.yml`) does all the things that `gitian-bundle.yml` does, plus it
 copies the contents of `pluggable-transports-linux$GBUILD_BITS-gbuilt.zip`
 and names the output file `tor-pluggable-transports-browser-linux`...
 instead of `tor-browser-linux`....
  3. Two new steps in `mkbundle-linux.sh` (cheekily labeled 4/3 and 5/3)
 call the above descriptors.

 Some questions that came up as I was doing it. They don't all need
 answers.
  * Is this overall the right way to do it? Should there be a `gitian-
 ptbundle.yml` that is almost a copy of `gitian-bundle.yml`? It wasn't
 clear to me how to factor out the stuff in common.
  * We will need to make changes to `torrc`. [https://gitweb.torproject.org
 /pluggable-
 transports/bundle.git/blob/50b44927d460825c6b920d4f7d53e357984d794a
 :/bundle-torrc-gnulinux (For example.)] What's a good way to do that? I
 don't want to modify e.g. `Bundle-Data/linux/Data/Tor/torrc` because
 that's used by the vanilla bundles. Really all we need is to append some
 lines to the end of the `torrc` that is normally installed (currently an
 empty file).
  * `gitian-tor.yml` builds libssl. Pluggable transports need libssl too.
 How do we handle this situation? Have each descriptor build its own copy
 of libssl, but copy only one into the package?
  * Should pluggable transport binaries, libraries, and Python packages be
 installed into `/Tor` alongside the `tor` binary, or somewhere else? In
 the bundles we have been building, they are all in the same directory
 (`App`).
  * My plan is to build all the pluggable transports (flashproxy and
 obfsproxy, with all their dependencies) in the single descriptor `gitian-
 pluggable-transports.yml`. Should each subprogram instead get its own
 descriptor? Would that mean duplicating descriptor code to build the
 dependencies they have in common?

 Some tips for others wanting to help: The full gitian build takes a long
 time. You can rebuild only selected parts as long as you don't do `make
 clean` (or `make`, which does `make clean`). Use `make build` or manually
 run `./mkbundle-linux.sh` instead. Do this to watch build progress and see
 what went wrong when there's a problem:
 {{{
 tail -F ~/gitian-builder/var/build.log
 }}}
 Delete the appropriate `*gbuilt*` files from `~/gitian-builder/inputs/` to
 get `mkbundle-linux.sh` to rebuild certain things. All this is actually
 covered in [https://gitweb.torproject.org/builders/tor-browser-
 bundle.git/blob/HEAD:/gitian/README.build README.build].

-- 
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/9444#comment:8>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online