[tor-bugs] #9444 [Tor bundles/installation]: Create deterministic TorBrowserBundles with Pluggable Transports
Tor Bug Tracker & Wiki
blackhole at torproject.org
Sun Nov 24 23:14:54 UTC 2013
#9444: Create deterministic TorBrowserBundles with Pluggable Transports
------------------------------------------+--------------------------
Reporter: bastik | Owner: erinn
Type: task | Status: needs_review
Priority: normal | Milestone:
Component: Tor bundles/installation | Version:
Resolution: | Keywords: flashproxy
Actual Points: | Parent ID:
Points: |
------------------------------------------+--------------------------
Comment (by dcf):
These are noteworthy characteristics of the windows pluggable transports
build:
* Python packages are built within Wine.
* The Wine used is wine1.6 from the [https://launchpad.net/~ubuntu-
wine/+archive/ppa Ubuntu Wine Team PPA], not the wine1.4 that is in main
from [http://packages.ubuntu.com/search?keywords=wine precise onward].
* Windows Python is installed using the
[http://python.org/download/releases/2.7.5/ binary MSI installer]. py2exe
is also copied from a binary installer. Other Python packages (PyCrypto,
Twisted, etc.) are compiled from source.
* Wrapper executables for gcc, dllwrap, and swig are compiled early and
used by later build steps. These allow the Unixy cross-compiling tools to
be called by Python distutils running under Wine.
Here's a summary of things that had to be overcome in order to make the
windows build work, which motivate the above characteristics.
Python is hard to cross-compile, as mentioned in comment:18. So rather
it's installed using a binary MSI package. I cherry-picked a couple of
commits ([https://gitweb.torproject.org/builders/tor-browser-
bundle.git/commit/67639197f6001d3440774914b2da637849379155 67639197]
[https://gitweb.torproject.org/builders/tor-browser-
bundle.git/commit/9a2042d4b5d0963cc26f8beb97bfb92f6ce11284 9a2042d4]) to
get the Python version that was added to master since I made my branch.
Everything needs to be built under Wine—including pure Python packages
with no compiled extensions. The reason for this is that packages'
`setup.py` files make decisions based on things like the `os.name` and
`sys.platform` of the Python they are being built with, not that of the
cross host they are intended to run on. Some packages make decisions
regarding which source files to include or exclude; for example PyCrypto
starts by compiling a file called `winrand.c` when `sys.platform ==
'win32'`, and omits it otherwise.
The [http://packages.ubuntu.com/search?keywords=wine wine1.4] in Ubuntu
has an issue that often manifests when running Python code, leading to the
page fault that was seen in comment:18. There are a few bug reports:
* [http://bugs.winehq.org/show_bug.cgi?id=29764 blender 2.61 crashes on
start (python3.2)]
* [http://bugs.winehq.org/show_bug.cgi?id=30515 EVE Online crashes when
viewing certain ship models (T3 ships)]
The crash happens
[http://hg.python.org/cpython/file/694e2708b4a8/Modules/posixmodule.c#l1078
here in the _PyVerify_fd function]. This function is called by many Python
functions that deal with file descriptor numbers, for example `io.open`
and `os.fstat`. This code is enough to crash Python run under Wine 1.4:
{{{
>>> import os
>>> os.fstat(4)
wine: Unhandled page fault on read access to 0x3e892268 at address
0x1e044a38 (thread 000b), starting debugger...
Application tried to create a window, but no driver could be loaded.
Make sure that your X server is running and that $DISPLAY is set
correctly.
Unhandled exception: page fault on read access to 0x3e892268 in 32-bit
code (0x1e044a38).
Register dump:
CS:0073 SS:007b DS:007b ES:007b FS:0033 GS:003b
EIP:1e044a38 ESP:0042fbb0 EBP:001ce5f0 EFLAGS:00010202( R- -- I - - -
)
EAX:1ffffffc EBX:00000004 ECX:1e892268 EDX:00000003
ESI:00000000 EDI:00000004
Stack dump:
0x0042fbb0: 1e048370 00193c60 00000000 1e04839a
0x0042fbc0: 00000004 00000004 7bca6ff4 00110000
0x0042fbd0: 756d043d 00129030 001945a0 00158b58
0x0042fbe0: 00124a58 1e08e62b 00000000 756d043d
0x0042fbf0: 00159c90 001945a0 00000000 001d1c98
0x0042fc00: 1e08e62b 00159c90 001945a0 756d043d
Backtrace:
}}}
What goes wrong is that `_PyVerify_fd` tries to find out the size of a
data structure (`_msize(__pioinfo[0])`) that it expects to be heap-
allocated, but [http://bugs.winehq.org/show_bug.cgi?id=29764#c8 Wine does
it differently]. ([http://bugs.python.org/issue4804 This] is the Python
issue that introduced that behavior.) The crash is actually inside
`RtlSizeHeap` inside `_msize`; here is a Wine debug trace:
{{{
462888.985:0011:Call msvcrt._msize(1e892268) ret=1e044a0d
462888.985:0011:Call ntdll.RtlSizeHeap(00110000,00000000,1e892268)
ret=6c6f2247
462888.985:0011:warn:heap:validate_block_pointer Heap 0x110000: pointer
0x1e892268 is not inside heap
462888.985:0011:trace:heap:RtlSizeHeap (0x110000,70000062,0x1e892268):
returning ffffffff
462888.985:0011:Ret ntdll.RtlSizeHeap() retval=ffffffff ret=6c6f2247
462888.985:0011:warn:msvcrt:_msize :Probably called with non wine-
allocated memory, ret = -1
462888.985:0011:Ret msvcrt._msize() retval=ffffffff ret=1e044a0d
462888.985:0011:trace:seh:raise_exception code=c0000005 flags=0
addr=0x1e044a38 ip=1e044a38 tid=0011
}}}
I initially tried to patch around calls that end up in `_PyVerify_fd`, but
they are pervasive. A newer Wine fixes it. Installing the newer Wine
involves installing an
[http://keyserver.ubuntu.com:11371/pks/lookup?op=get&search=0x5A9A06AEF9CB8DB0
APT GPG key] and adding the PPA to `sources.list`.
Python distutils has a mingw32 compiler profile that is activated with
`--compiler=mingw32`. Sadly,
[http://hg.python.org/cpython/file/fbebc90abcd1/Lib/distutils/cygwinccompiler.py#l302
it hardcodes "gcc" and some compiler flags], and there is no way to change
them short of editing the distutils source. (See also the
[http://hg.python.org/cpython/file/fbebc90abcd1/Lib/distutils/cygwinccompiler.py#l138
similar cygwin profile], where it is made explicit in a comment: "Hard-
code GCC because that's what this is all about. XXX optimization, warnings
etc. should be customizable." The usual `CC` and `CFLAGS` environment
variables are honored
[http://hg.python.org/cpython/file/fbebc90abcd1/Lib/distutils/sysconfig.py#l171
only by the unix profile]; they are completely ignored by mingw32.
Aha, you say, I will copy/symlink i686-w64-mingw32-gcc so it is available
as gcc.exe. But! among the flags hardcoded by distutils is `-mno-cygwin`,
which [http://bugs.python.org/issue12641 was removed from GCC in 2010] and
now causes an error when used. Also, when running inside Wine, distutils
will pass to the compiler Windows paths with backslashes like
build\lib.win32-2.7\Crypto\Util\strxor.o instead of
build/lib.win32-2.7/Crypto/Util/strxor.o. (The `-mno-cygwin` issue is
[http://bugs.python.org/issue12641 fixed in Python 2.7.6], but the problem
with Windows paths remains.)
Aha, you say, I will install a small Python wrapper that removes `-mno-
cygwin` and rewrites paths (using the [http://wiki.winehq.org/winepath
winepath] utility), then execs the real compiler. But simply installing a
Python script as gcc.exe doesn't really work. It works in that it runs and
does what you want, but (perhaps because it is not a native .exe) Wine
detaches it from the foreground when it is run. distutils repeatedly calls
gcc to produce a .o file, then calls dllwrap on the .o file. When gcc
backgrounds itself immediately, distutils doesn't wait for it to finish
and write the .o file that dllwrap needs, so all the dllwrap calls fail.
Aha, you say, I will install a batch file called gcc.bat, which will
invoke the Windows python.exe on the gcc wrapper. This might work, except
for that distutils
[http://hg.python.org/cpython/file/fbebc90abcd1/Lib/distutils/spawn.py#l161
requires that all executables have a file name ending in ".exe"].
So you compile your Python wrappers into native binaries using py2exe.
Then you wonder for a while why your processes still seem to be getting
backgrounded. It turns out that your call to
[http://docs.python.org/2/library/subprocess.html#subprocess.call
subprocess.call] is failing with "WindowsError: [Error 6] Invalid handle"
when it does a
[http://docs.python.org/2/library/subprocess.html#subprocess.Popen.wait
Popen.wait]. This causes the Python wrapper to return prematurely, while
the gcc process is still running. Instead of waiting for the process to
terminate, you can read from its stderr stream, and wait until it closes.
Then finally things start working. You still get a message on the terminal
when the command exits:
{{{
Exception WindowsError: (6, 'Invalid handle') in <bound method
Popen.__del__ of <subprocess.Popen object at 0x001580D0>> ignored
}}}
but it doesn't happen until the command is actually finished.
Incidentally, this is why py2exe is installed from a binary package
instead of from source—it would require py2exe-compiled wrappers itself,
in order to be compiled. Writing the wrappers in C, for example, would
allow py2exe to be built from source.
I found I needed wrappers for gcc, dllwrap, and swig. (swig only for
m2crypto.) These Wine wrappers, I put in a subdirectory `build-helpers
/wine-wrappers`. I changed the `build-helpers` copy commands into
recursive copies to get the whole subdirectory. But I noticed this message
from gbuild:
{{{
sha256sum: wine-wrappers: Is a directory
}}}
This means that the Wine wrappers aren't included in the `gitian-
builder/result/pluggable-transports-windows-res.yml`. I don't know what's
the best way to handle that.
Current versions of Wine [http://bugs.winehq.org/show_bug.cgi?id=31237
don't work with libfaketime]. We selectively disable libfaketime for Wine
commands using `LD_PRELOAD=`. Otherwise, you get this error for every Wine
command: (this bug was hard to figure out because I could SSH into the
virtual host and the same commands would be fine when run manually!)
{{{
wine: created the configuration directory '/home/ubuntu/.wine'
wine: Unhandled page fault on read access to 0x00000000 at address (nil)
(thread 000b),
err:seh:start_debugger Couldn't start debugger ("winedbg --auto 10 104")
(2)
Read the Wine Developers Guide on how to set up winedbg or another
debugger
wine: Unhandled page fault on read access to 0x00000000 at address (nil)
(thread 0011),
err:seh:start_debugger Couldn't start debugger ("winedbg --auto 16 68")
(2)
Read the Wine Developers Guide on how to set up winedbg or another
debugger
}}}
(This is a different page fault than the `_PyVerify_fd` one above; this
one is at 0x00000000 and the other is at some nonzero address.)
--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/9444#comment:20>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online
More information about the tor-bugs
mailing list