[tor-bugs] #9444 [Tor bundles/installation]: Create deterministic TorBrowserBundles with Pluggable Transports

[Tor Bug Tracker & Wiki]

#9444: Create deterministic TorBrowserBundles with Pluggable Transports
------------------------------------------+--------------------------
     Reporter:  bastik                    |      Owner:  erinn
         Type:  task                      |     Status:  needs_review
     Priority:  normal                    |  Milestone:
    Component:  Tor bundles/installation  |    Version:
   Resolution:                            |   Keywords:  flashproxy
Actual Points:                            |  Parent ID:
       Points:                            |
------------------------------------------+--------------------------

Comment (by dcf):

 I'm having trouble making Python work for the Windows bundles.

 We need to compile Python extensions against python.h and libpython. On
 linux we use Ubuntu's python-dev. On mac, the Python "framework" from the
 SDK works. My first thought was to cross-compile Python for Windows.
 Unfortunately, cross-compiling Python seems hard and will require at least
 a bunch of patches from the Python bugtracker.
  * [http://bugs.python.org/issue3871 Issue 3871: cross and native build of
 python for mingw* hosts]
  * [http://bugs.python.org/issue6335 Issue 6335: Add support for mingw]
  * [https://github.com/mingwandroid/crucifixion-freedom crucifixion-
 freedom: Cross compilation patches and build scripts for CPython]
  * [https://code.google.com/p/mingw-and-ndk/ mingw-and-ndk: Cross
 compilation patches for Python 2.7.3 and 3.3.0, targeting
 MinGW/Darwin/Linux and MinGW builds generated from these]

 So instead I started trying to build against the
 [http://www.python.org/download/releases/2.7.6/ Windows MSI installer].
 The [http://wiki.winehq.org/msiexec msiexec] program from Wine is able to
 extract the MSI, and Ubuntu precise has
 [http://packages.ubuntu.com/precise/wine1.4 wine1.4].

 With Python so installed in `$INSTDIR/python`, I was able to cross-compile
 pycrypto using something like this:
 {{{
 sh configure --host=i686-w64-mingw32
 CC="i686-w64-mingw32" LDSHARED="i686-w64-mingw32 -shared" python setup.py
 build_ext -I$INSTDIR/python/include -L$INSTDIR/python/libs -lpython27
 python setup.py install --install-lib $INSTDIR/python/Lib
 }}}

 However compiling Twisted is not so easy. Twisted has a bunch of compile-
 time tests like
 [https://twistedmatrix.com/trac/browser/tags/releases/twisted-13.2.0/twisted/python/dist.py#L328
 os.name == 'nt'] and
 [https://twistedmatrix.com/trac/browser/tags/releases/twisted-13.2.0/twisted/topfiles/setup.py#L35
 sys.platform != 'win32'] that control preprocessor definitions, what files
 get compiled, and who knows what else. Naively trying to cross-compile as
 above chokes on
 [https://twistedmatrix.com/trac/browser/tags/releases/twisted-13.2.0/twisted/python/sendmsg.c#L16
 an include of <sys/socket.h>] in a POSIX-only source file. We could
 probably find a way to fake `os.name` and `sys.platform`, but I worry
 about what other side effects that may have.

 The most obvious thing to do is just to build all extension modules in
 Wine, seeing as we already need Wine to extract the Python MSI. infinity0
 [https://gitweb.torproject.org/pluggable-
 transports/bundle.git/commit/9a39d8a00dcbe3d71279fd5e5de6e9ef65275053 had
 luck building the 2.x bundle] using Wine, though that was using
 [https://gitweb.torproject.org/pluggable-
 transports/bundle.git/blob/9a39d8a00dcbe3d71279fd5e5de6e9ef65275053
 :/bundle-windows.txt prebuilt Twisted etc.] Only problem is I'm getting
 Wine page faults when I try it.
 {{{
 ubuntu at ubuntu:~/build/setuptools-1.4$ wine ~/install/python/python.exe
 setup.py install
 running install
 running bdist_egg
 running egg_info
 writing requirements to setuptools.egg-info\requires.txt
 writing setuptools.egg-info\PKG-INFO
 writing top-level names to setuptools.egg-info\top_level.txt
 writing dependency_links to setuptools.egg-info\dependency_links.txt
 writing entry points to setuptools.egg-info\entry_points.txt
 wine: Unhandled page fault on read access to 0x3e892268 at address
 0x1e044a38 (thread 002f), starting debugger...
 Application tried to create a window, but no driver could be loaded.
 Make sure that your X server is running and that $DISPLAY is set
 correctly.
 Unhandled exception: page fault on read access to 0x3e892268 in 32-bit
 code (0x1e044a38).
 <register dump and stack trace>
 }}}
 So my plan now is to use binary versions of compiled extension modules (at
 least Twisted), and hope that py2exe will work under the version of Wine
 that comes with precise.

-- 
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/9444#comment:18>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online