[tor-bugs] #27827 [Obfuscation/Snowflake]: Reproducibility issue of the snowflake osx64 build

Tor Bug Tracker & Wiki blackhole at torproject.org
Sun Sep 23 21:15:40 UTC 2018


#27827: Reproducibility issue of the snowflake osx64 build
-----------------------------------+------------------------
 Reporter:  boklm                  |          Owner:  (none)
     Type:  defect                 |         Status:  new
 Priority:  Medium                 |      Milestone:
Component:  Obfuscation/Snowflake  |        Version:
 Severity:  Normal                 |     Resolution:
 Keywords:                         |  Actual Points:
Parent ID:                         |         Points:
 Reviewer:                         |        Sponsor:
-----------------------------------+------------------------

Old description:

> The build of Snowflake for MacOS is often producing the same result, but
> not always.
>
> Arthur has been rebuilding Snowflake 8 times, with 4 different results:
> https://gist.github.com/arthuredelstein/73860df088c565ea0b2ca6eef586063a

New description:

 The build of Snowflake for MacOS is often producing the same result, but
 not always.

 Arthur has been rebuilding Snowflake 8 times, with 4 different results:
 https://gist.github.com/arthuredelstein/73860df088c565ea0b2ca6eef586063a
 {{{
 fish script:

 for x in (seq 8)
     rm out/snowflake/snowflake-6077141f4aff-osx-x86_64-3b578d.tar.gz
     ./rbm/rbm build snowflake  --target alpha --target torbrowser-osx-
 x86_64
     tar xvf out/snowflake/snowflake-6077141f4aff-osx-x86_64-3b578d.tar.gz
     echo (sha256sum ./Contents/MacOS/Tor/PluggableTransports/snowflake-
 client
 end

 Results:

 b060b42cfd0c8fb2781dbb0fd45d42804dbb414473fec0597d9c2fb7d6d12aa8
 ./Contents/MacOS/Tor/PluggableTransports/snowflake-client
 1ee0dd2a0b228988e22c663d62b696b23a6ac48dc742a57dfa8f854aa3992bc3
 ./Contents/MacOS/Tor/PluggableTransports/snowflake-client
 22557c38d913e478e480dd3581efc00019fe2989c4273d9207f1719c34b6e399
 ./Contents/MacOS/Tor/PluggableTransports/snowflake-client
 22557c38d913e478e480dd3581efc00019fe2989c4273d9207f1719c34b6e399
 ./Contents/MacOS/Tor/PluggableTransports/snowflake-client
 1ee0dd2a0b228988e22c663d62b696b23a6ac48dc742a57dfa8f854aa3992bc3
 ./Contents/MacOS/Tor/PluggableTransports/snowflake-client
 1ee0dd2a0b228988e22c663d62b696b23a6ac48dc742a57dfa8f854aa3992bc3
 ./Contents/MacOS/Tor/PluggableTransports/snowflake-client
 1ee0dd2a0b228988e22c663d62b696b23a6ac48dc742a57dfa8f854aa3992bc3
 ./Contents/MacOS/Tor/PluggableTransports/snowflake-client
 6d008bc7d29e8543608491b67d4b11da7bd6589741d9f52ac5fd50dd39d84f29
 ./Contents/MacOS/Tor/PluggableTransports/snowflake-client
 }}}

--

Comment (by dcf):

 The difference is different ordering of some symbols/path in the
 snowflake-client binary. Notice how (except for `cgo-gcc-prolog`), it's
 different orderings of three blocks, color coded below.

 Notice also that each block contains a `/tmp/go-buildXXXXXXXXX` string.
 These come from [https://gitweb.torproject.org/builders/tor-browser-
 build.git/tree/projects/snowflake/build?h=tbb-8.5a2-build3#n38 the
 snowflake build descriptor]: the `XXXXXXXXX` overwrites a random number.
 What I suspect is happening is, the go compiler is generating three
 distinct random `/tmp/go-buildXXXXXXXXX` paths and sorting the blocks on
 them; then the build descriptor overwrites them all.

 The overwriting is a workaround for the upstream Go bug
 [https://github.com/golang/go/issues/9206 #9206], which is now marked
 closed. The first thing I would try is upgrading to a newer Go and seeing
 if it just gets fixed.

 {{{#!th
 1ee0dd2a0b228988e22c663d62b696b23a6ac48dc742a57dfa8f854aa3992bc3
 }}}
 {{{#!th
 22557c38d913e478e480dd3581efc00019fe2989c4273d9207f1719c34b6e399
 }}}
 {{{#!th
 6d008bc7d29e8543608491b67d4b11da7bd6589741d9f52ac5fd50dd39d84f29
 }}}
 |-----------------------------------
 {{{#!td
 {{{#!html
 <small>
 <pre style="background-color: red">
 /tmp/go-build/net/_obj/
 _cgo_export.c
 /tmp/go-buildXXXXXXXXX/net/_obj/_cgo_export.o
 __cgo_9f1b05c52f96_Cfunc__Cmalloc
 cgo_resnew.cgo2.c
 /tmp/go-buildXXXXXXXXX/net/_obj/cgo_resnew.cgo2.o
 __cgo_9f1b05c52f96_C2func_getnameinfo
 <span style="background-color: gray">cgo-gcc-prolog</span>
 __cgo_9f1b05c52f96_Cfunc_getnameinfo
 cgo_unix.cgo2.c
 /tmp/go-buildXXXXXXXXX/net/_obj/cgo_unix.cgo2.o
 __cgo_9f1b05c52f96_C2func_getaddrinfo
 __cgo_9f1b05c52f96_Cfunc_free
 __cgo_9f1b05c52f96_Cfunc_freeaddrinfo
 __cgo_9f1b05c52f96_Cfunc_gai_strerror
 __cgo_9f1b05c52f96_Cfunc_getaddrinfo
 </pre><pre style="background-color: green">
 /tmp/go-build/crypto/x509/_obj/
 root_cgo_darwin.cgo2.c
 /tmp/go-buildXXXXXXXXX/crypto/x509/_obj/root_cgo_darwin.cgo2.o
 _FetchPEMRoots_MountainLion
 /var/tmp/dist/go/src/crypto/x509/root_cgo_darwin.go
 _useOldCode
 _FetchPEMRoots
 __cgo_62033c69288a_Cfunc_CFDataGetBytePtr
 __cgo_62033c69288a_Cfunc_CFDataGetLength
 __cgo_62033c69288a_Cfunc_CFRelease
 __cgo_62033c69288a_Cfunc_FetchPEMRoots
 </pre><pre style="background-color: lightblue">
 /var/tmp/dist/go/src/runtime/cgo/
 gcc_context.c
 /tmp/go-buildXXXXXXXXX/runtime/cgo/_obj/gcc_context.o
 __cgo_release_context
 gcc_darwin_amd64.c
 /tmp/go-buildXXXXXXXXX/runtime/cgo/_obj/gcc_darwin_amd64.o
 _x_cgo_init
 __cgo_sys_thread_start
 _threadentry
 _k1
 gcc_libinit.c
 /tmp/go-buildXXXXXXXXX/runtime/cgo/_obj/gcc_libinit.o
 _x_cgo_sys_thread_create
 __cgo_try_pthread_create
 __cgo_wait_runtime_init_done
 _x_cgo_notify_runtime_init_done
 _x_cgo_set_context_function
 __cgo_get_context_function
 _runtime_init_mu
 _runtime_init_cond
 _runtime_init_done
 _cgo_context_function
 gcc_setenv.c
 /tmp/go-buildXXXXXXXXX/runtime/cgo/_obj/gcc_setenv.o
 _x_cgo_setenv
 _x_cgo_unsetenv
 gcc_util.c
 /tmp/go-buildXXXXXXXXX/runtime/cgo/_obj/gcc_util.o
 _x_cgo_thread_start
 gcc_amd64.S
 /tmp/go-buildXXXXXXXXX/runtime/cgo/_obj/gcc_amd64.o
 _crosscall_amd64
 </pre>
 }}}
 }}}
 {{{#!td
 {{{#!html
 <small>
 <pre style="background-color: red">
 /tmp/go-build/net/_obj/
 _cgo_export.c
 /tmp/go-buildXXXXXXXXX/net/_obj/_cgo_export.o
 __cgo_9f1b05c52f96_Cfunc__Cmalloc
 cgo_resnew.cgo2.c
 /tmp/go-buildXXXXXXXXX/net/_obj/cgo_resnew.cgo2.o
 __cgo_9f1b05c52f96_C2func_getnameinfo
 <span style="background-color: gray">cgo-gcc-prolog</span>
 __cgo_9f1b05c52f96_Cfunc_getnameinfo
 cgo_unix.cgo2.c
 /tmp/go-buildXXXXXXXXX/net/_obj/cgo_unix.cgo2.o
 __cgo_9f1b05c52f96_C2func_getaddrinfo
 __cgo_9f1b05c52f96_Cfunc_free
 __cgo_9f1b05c52f96_Cfunc_freeaddrinfo
 __cgo_9f1b05c52f96_Cfunc_gai_strerror
 __cgo_9f1b05c52f96_Cfunc_getaddrinfo
 </pre><pre style="background-color: lightblue">
 /var/tmp/dist/go/src/runtime/cgo/
 gcc_context.c
 /tmp/go-buildXXXXXXXXX/runtime/cgo/_obj/gcc_context.o
 __cgo_release_context
 gcc_darwin_amd64.c
 /tmp/go-buildXXXXXXXXX/runtime/cgo/_obj/gcc_darwin_amd64.o
 _x_cgo_init
 __cgo_sys_thread_start
 _threadentry
 _k1
 gcc_libinit.c
 /tmp/go-buildXXXXXXXXX/runtime/cgo/_obj/gcc_libinit.o
 _x_cgo_sys_thread_create
 __cgo_try_pthread_create
 __cgo_wait_runtime_init_done
 _x_cgo_notify_runtime_init_done
 _x_cgo_set_context_function
 __cgo_get_context_function
 _runtime_init_mu
 _runtime_init_cond
 _runtime_init_done
 _cgo_context_function
 gcc_setenv.c
 /tmp/go-buildXXXXXXXXX/runtime/cgo/_obj/gcc_setenv.o
 _x_cgo_setenv
 _x_cgo_unsetenv
 gcc_util.c
 /tmp/go-buildXXXXXXXXX/runtime/cgo/_obj/gcc_util.o
 _x_cgo_thread_start
 gcc_amd64.S
 /tmp/go-buildXXXXXXXXX/runtime/cgo/_obj/gcc_amd64.o
 _crosscall_amd64
 </pre><pre style="background-color: green">
 /tmp/go-build/crypto/x509/_obj/
 root_cgo_darwin.cgo2.c
 /tmp/go-buildXXXXXXXXX/crypto/x509/_obj/root_cgo_darwin.cgo2.o
 _FetchPEMRoots_MountainLion
 /var/tmp/dist/go/src/crypto/x509/root_cgo_darwin.go
 _useOldCode
 _FetchPEMRoots
 __cgo_62033c69288a_Cfunc_CFDataGetBytePtr
 __cgo_62033c69288a_Cfunc_CFDataGetLength
 __cgo_62033c69288a_Cfunc_CFRelease
 __cgo_62033c69288a_Cfunc_FetchPEMRoots
 </pre>
 }}}
 }}}
 {{{#!td
 {{{#!html
 <small>
 <pre style="background-color: lightblue">
 /var/tmp/dist/go/src/runtime/cgo/
 gcc_context.c
 /tmp/go-buildXXXXXXXXX/runtime/cgo/_obj/gcc_context.o
 __cgo_release_context
 gcc_darwin_amd64.c
 /tmp/go-buildXXXXXXXXX/runtime/cgo/_obj/gcc_darwin_amd64.o
 _x_cgo_init
 __cgo_sys_thread_start
 _threadentry
 _k1
 gcc_libinit.c
 /tmp/go-buildXXXXXXXXX/runtime/cgo/_obj/gcc_libinit.o
 _x_cgo_sys_thread_create
 __cgo_try_pthread_create
 __cgo_wait_runtime_init_done
 _x_cgo_notify_runtime_init_done
 _x_cgo_set_context_function
 __cgo_get_context_function
 _runtime_init_mu
 _runtime_init_cond
 _runtime_init_done
 _cgo_context_function
 gcc_setenv.c
 /tmp/go-buildXXXXXXXXX/runtime/cgo/_obj/gcc_setenv.o
 _x_cgo_setenv
 _x_cgo_unsetenv
 gcc_util.c
 /tmp/go-buildXXXXXXXXX/runtime/cgo/_obj/gcc_util.o
 _x_cgo_thread_start
 gcc_amd64.S
 /tmp/go-buildXXXXXXXXX/runtime/cgo/_obj/gcc_amd64.o
 _crosscall_amd64
 </pre><pre style="background-color: green">
 /tmp/go-build/crypto/x509/_obj/
 root_cgo_darwin.cgo2.c
 /tmp/go-buildXXXXXXXXX/crypto/x509/_obj/root_cgo_darwin.cgo2.o
 _FetchPEMRoots_MountainLion
 /var/tmp/dist/go/src/crypto/x509/root_cgo_darwin.go
 _useOldCode
 _FetchPEMRoots
 __cgo_62033c69288a_Cfunc_CFDataGetBytePtr
 <span style="background-color: gray">cgo-gcc-prolog</span>
 __cgo_62033c69288a_Cfunc_CFDataGetLength
 __cgo_62033c69288a_Cfunc_CFRelease
 __cgo_62033c69288a_Cfunc_FetchPEMRoots
 </pre><pre style="background-color: red">
 /tmp/go-build/net/_obj/
 _cgo_export.c
 /tmp/go-buildXXXXXXXXX/net/_obj/_cgo_export.o
 __cgo_9f1b05c52f96_Cfunc__Cmalloc
 cgo_resnew.cgo2.c
 /tmp/go-buildXXXXXXXXX/net/_obj/cgo_resnew.cgo2.o
 __cgo_9f1b05c52f96_C2func_getnameinfo
 __cgo_9f1b05c52f96_Cfunc_getnameinfo
 cgo_unix.cgo2.c
 /tmp/go-buildXXXXXXXXX/net/_obj/cgo_unix.cgo2.o
 __cgo_9f1b05c52f96_C2func_getaddrinfo
 __cgo_9f1b05c52f96_Cfunc_free
 __cgo_9f1b05c52f96_Cfunc_freeaddrinfo
 __cgo_9f1b05c52f96_Cfunc_gai_strerror
 __cgo_9f1b05c52f96_Cfunc_getaddrinfo
 </pre>
 }}}
 }}}

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/27827#comment:3>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online


More information about the tor-bugs mailing list