December 2014 - tbb-commits - lists.torproject.org

[tor-browser-bundle/master] Bump versions for nightlies.
by gk＠torproject.org 02 Dec '14

02 Dec '14

commit 6800ca4a0e89a59afecae2b931ab6795b3292eaf Author: Georg Koppen <gk(a)torproject.org> Date: Tue Dec 2 13:25:47 2014 +0000 Bump versions for nightlies. --- gitian/versions.nightly | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/gitian/versions.nightly b/gitian/versions.nightly index e6ea2a1..1b6aff4 100755 --- a/gitian/versions.nightly +++ b/gitian/versions.nightly @@ -4,7 +4,7 @@ BUILD_PT_BUNDLES=1 VERIFY_TAGS=0 -FIREFOX_VERSION=31.2.0esr … [View More]

1 0

[tor-browser-bundle/master] Bump lang pack version, too.
by gk＠torproject.org 02 Dec '14

02 Dec '14

commit ad84f4205a95dfdbf72a9949014c469c3316f72c Author: Georg Koppen <gk(a)torproject.org> Date: Tue Dec 2 13:29:02 2014 +0000 Bump lang pack version, too. --- gitian/versions.nightly | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/gitian/versions.nightly b/gitian/versions.nightly index 1b6aff4..2ba0141 100755 --- a/gitian/versions.nightly +++ b/gitian/versions.nightly @@ -37,7 +37,7 @@ GITIAN_TAG=tor-browser-builder-3.x-7 OPENSSL_VER=1.0.1i GMP_VER=5.1.3 … [View More]

1 0

[tor-browser-bundle/master] Peparing 4.5-alpha-2.
by gk＠torproject.org 02 Dec '14

02 Dec '14

commit 7af137f526da8495bbaebffc59077f2e9422d71a Author: Georg Koppen <gk(a)torproject.org> Date: Tue Dec 2 13:55:13 2014 +0000 Peparing 4.5-alpha-2. Bump Torbutton, NoScript and HTTPS Everywhere version. Update config.yml and ChangeLog. --- Bundle-Data/Docs/ChangeLog.txt | 30 ++++++++++++++++++++++++++++++ gitian/versions.alpha | 18 +++++++++--------- tools/update-responses/config.yml | 19 +++++++++---------- 3 files changed, 48 insertions(+), … [View More]19 deletions(-) diff --git a/Bundle-Data/Docs/ChangeLog.txt b/Bundle-Data/Docs/ChangeLog.txt index f226adc..f1511b0 100644 --- a/Bundle-Data/Docs/ChangeLog.txt +++ b/Bundle-Data/Docs/ChangeLog.txt @@ -1,3 +1,33 @@ +Tor Browser 4.5-alpha-2 -- Dec 5 2014 + * All Platforms + * Update Firefox to 31.3.0esr + * Update NoScript to 2.6.9.5 + * Update HTTPS Everywhere to 5.0development.1 + * Update Torbutton to 1.8.1.2 + * Bug 13672: Make circuit display optional + * Bug 13671: Make bridges visible on circuit display + * Bug 9387: Incorporate user feedback + * Bug 13784: Remove third party authentication tokens + +Tor Browser 4.0.2 -- Dec 2 2014 + * All Platforms + * Update Firefox to 31.3.0esr + * Update NoScript to 2.6.9.5 + * Update HTTPS Everywhere to 4.0.2 + * Update Torbutton to 1.7.0.2 + * Bug 13019: Synchronize locale spoofing pref with our Firefox patch + * Bug 13746: Properly link Torbutton UI to thirdparty pref. + * Bug 13742: Fix domain isolation for content cache and disk-enabled +browsing mode + * Bug 5926: Prevent JS engine locale leaks (by setting the C library +locale) + * Bug 13504: Remove unreliable/unreachable non-public bridges + * Bug 13435: Remove our custom POODLE fix + * Windows + * Bug 13443: Re-enable DirectShow; fix crash with mingw patch. + * Bug 13558: Fix crash on Windows XP during download folder changing + * Bug 13594: Fix update failure for Windows XP users + Tor Browser 4.5-alpha-1 -- Nov 14 2014 * All Platforms * Bug 3455: Patch Firefox SOCKS and proxy filters to allow user+pass isolation diff --git a/gitian/versions.alpha b/gitian/versions.alpha index edb59c2..a25af43 100755 --- a/gitian/versions.alpha +++ b/gitian/versions.alpha @@ -1,17 +1,17 @@ -TORBROWSER_VERSION=4.5-alpha-1 +TORBROWSER_VERSION=4.5-alpha-2 BUNDLE_LOCALES="ar de es-ES fa fr it ko nl pl pt-PT ru tr vi zh-CN" BUILD_PT_BUNDLES=1 VERIFY_TAGS=1 -FIREFOX_VERSION=31.2.0esr +FIREFOX_VERSION=31.3.0esr TORBROWSER_UPDATE_CHANNEL=alpha -TORBROWSER_TAG=tor-browser-${FIREFOX_VERSION}-4.5-1-build6 +TORBROWSER_TAG=tor-browser-${FIREFOX_VERSION}-4.5-1-build1 TOR_TAG=tor-0.2.6.1-alpha TORLAUNCHER_TAG=0.2.7.0.1 -TORBUTTON_TAG=1.8.1.1 +TORBUTTON_TAG=1.8.1.2 HTTPSE_TAG=3.5.3 # XXX: HTTPSE_VER is used instead, pending #11630 NSIS_TAG=v0.2 ZLIB_TAG=v1.2.8 @@ -36,7 +36,7 @@ GITIAN_TAG=tor-browser-builder-3.x-7 OPENSSL_VER=1.0.1i GMP_VER=5.1.3 FIREFOX_LANG_VER=$FIREFOX_VERSION -FIREFOX_LANG_BUILD=build1 +FIREFOX_LANG_BUILD=build2 BINUTILS_VER=2.24 GCC_VER=4.9.1 PYTHON_VER=2.7.5 @@ -50,13 +50,13 @@ PY2EXE_VER=0.6.9 SETUPTOOLS_VER=1.4 LXML_VER=3.3.5 PARSLEY_VER=1.2 -HTTPSE_VER=5.0development.0 +HTTPSE_VER=5.0development.1 GO_VER=1.3.3 ## File names for the source packages OPENSSL_PACKAGE=openssl-${OPENSSL_VER}.tar.gz GMP_PACKAGE=gmp-${GMP_VER}.tar.bz2 -NOSCRIPT_PACKAGE=noscript_security_suite-2.6.9.3-sm+fn+fx.xpi +NOSCRIPT_PACKAGE=noscript_security_suite-2.6.9.5-sm+fn+fx.xpi HTTPSE_PACKAGE=https-everywhere-${HTTPSE_VER}.xpi TOOLCHAIN4_PACKAGE=x86_64-apple-darwin10.tar.xz TOOLCHAIN4_OLD_PACKAGE=multiarch-darwin11-cctools127.2-gcc42-5666.3-llvmgcc42-2336.1-Linux-120724.tar.xz @@ -86,8 +86,8 @@ GMP_HASH=752079520b4690531171d0f4532e40f08600215feefede70b24fabdc6f1ab160 OSXSDK_HASH=6602d8d5ddb371fbc02e2a5967d9bd0cd7358d46f9417753c8234b923f2ea6fc TOOLCHAIN4_HASH=7b71bfe02820409b994c5c33a7eab81a81c72550f5da85ff7af70da3da244645 TOOLCHAIN4_OLD_HASH=65c1b2d302358a6b95a26c6828a66908a199276193bb0b268f2dcc1a997731e9 -NOSCRIPT_HASH=d399d724953fe032ccca758e577c2136b8fd6c377cea0e3f42ed999b796d290a -HTTPSE_HASH=6feb4982d13f89656b61435b6b49e1996c61bc79e52d046d02312879376028e3 +NOSCRIPT_HASH=6db840d4ecf60985aa9eb7ef0142e67d78503d76a778f8f4e7e33e1810a2701 +HTTPSE_HASH=d97ec822641d4ebf404fee1179497e1d80f2b8344d5fa485169ef045ba184db0 MSVCR100_HASH=1221a09484964a6f38af5e34ee292b9afefccb3dc6e55435fd3aaf7c235d9067 PYCRYPTO_HASH=f2ce1e989b272cfcb677616763e0a2e7ec659effa67a88aa92b3a65528f60a3c ARGPARSE_HASH=ddaf4b0a618335a32b6664d4ae038a1de8fbada3b25033f9021510ed2b3941a4 diff --git a/tools/update-responses/config.yml b/tools/update-responses/config.yml index 557b197..6b0018d 100644 --- a/tools/update-responses/config.yml +++ b/tools/update-responses/config.yml @@ -6,20 +6,19 @@ build_targets: osx32: Darwin_x86-gcc3 osx64: Darwin_x86_64-gcc3 channels: - release: 4.0 - alpha: 4.5-alpha-1 + release: 4.0.2 + alpha: 4.5-alpha-2 versions: - 4.0: - platformVersion: 31.2.0 + 4.0.2: + platformVersion: 31.3.0 detailsURL: https://www.torproject.org/projects/torbrowser.html.en - download_url: https://www.torproject.org/dist/torbrowser/4.0 - 4.5-alpha-1: - platformVersion: 31.2.0 + download_url: https://www.torproject.org/dist/torbrowser/4.0.2 + 4.5-alpha-2: + platformVersion: 31.3.0 detailsURL: https://www.torproject.org/projects/torbrowser.html.en - download_url: https://www.torproject.org/dist/torbrowser/4.5-alpha-1 + download_url: https://www.torproject.org/dist/torbrowser/4.5-alpha-2 incremental_from: - - 4.0 - - 4.0.1 + - 4.5-alpha-1 migrate_archs: osx32: osx64 osx32: [View Less]

1 0

[tor-browser-bundle/maint-4.0] Bumping tor-browser tag for build2.
by gk＠torproject.org 01 Dec '14

01 Dec '14

commit 8501e871a574856e962bc1732dfea05d206c3656 Author: Georg Koppen <gk(a)torproject.org> Date: Mon Dec 1 09:32:56 2014 +0000 Bumping tor-browser tag for build2. --- Bundle-Data/Docs/ChangeLog.txt | 1 + gitian/versions | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/Bundle-Data/Docs/ChangeLog.txt b/Bundle-Data/Docs/ChangeLog.txt index 93d20d7..50f7974 100644 --- a/Bundle-Data/Docs/ChangeLog.txt +++ b/Bundle-Data/Docs/ChangeLog.txt @@ -9,… [View More]

1 0

[tor-browser/tor-browser-31.3.0esr-4.0-1] Bug 13435: Remove our custom POODLE fix.
by gk＠torproject.org 01 Dec '14

01 Dec '14

commit 91ef6ad6972aac82b7ce539664a27fd471251733 Author: Georg Koppen <gk(a)torproject.org> Date: Mon Dec 1 09:20:53 2014 +0000 Bug 13435: Remove our custom POODLE fix. --- browser/app/profile/000-tor-browser.js | 2 -- 1 file changed, 2 deletions(-) diff --git a/browser/app/profile/000-tor-browser.js b/browser/app/profile/000-tor-browser.js index 43b894d..db85aa0 100644 --- a/browser/app/profile/000-tor-browser.js +++ b/browser/app/profile/000-tor-browser.js @@ -171,8 +171,6 @@… [View More]

1 0

[tor-browser/tor-browser-31.3.0esr-4.0-1] Revert "Bug 762358 - Re-run configure when mozconfig changed in a significant way. r=gps a=NPOTB"
by gk＠torproject.org 01 Dec '14

01 Dec '14

commit 811add0090f95de6e1fa3700fb55e8bc57802a52 Author: Georg Koppen <gk(a)torproject.org> Date: Mon Dec 1 08:59:35 2014 +0100 Revert "Bug 762358 - Re-run configure when mozconfig changed in a significant way. r=gps a=NPOTB" This reverts commit 87d09ba73a620931ffbe2576064aae55a961b097. It seems it breaks our deterministic build setup running |configure| again during |make build|. The former does not like being run under libfaketime but the latter needs to be in … [View More]order to get reproducible builds. --- build/autoconf/altoptions.m4 | 43 +------ build/autoconf/mozconfig-find | 76 +++++++++++++ build/autoconf/mozconfig2client-mk | 76 +++++++++++++ build/autoconf/mozconfig2configure | 103 +++++++++++++++++ client.mk | 50 ++++++--- python/mozbuild/mozbuild/base.py | 38 ++----- python/mozbuild/mozbuild/mach_commands.py | 118 ++++---------------- python/mozbuild/mozbuild/mozconfig.py | 101 ++++++----------- python/mozbuild/mozbuild/mozconfig_loader | 8 -- python/mozbuild/mozbuild/test/backend/common.py | 8 -- .../mozbuild/test/frontend/test_emitter.py | 8 -- python/mozbuild/mozbuild/test/test_base.py | 75 +++---------- python/mozbuild/mozbuild/test/test_mozconfig.py | 56 +--------- testing/xpcshell/selftest.py | 1 - 14 files changed, 381 insertions(+), 380 deletions(-) diff --git a/build/autoconf/altoptions.m4 b/build/autoconf/altoptions.m4 index 6c40ad6..3dc8c6e 100644 --- a/build/autoconf/altoptions.m4 +++ b/build/autoconf/altoptions.m4 @@ -116,42 +116,7 @@ echo " dnl MOZ_READ_MYCONFIG() - Read in 'myconfig.sh' file AC_DEFUN([MOZ_READ_MOZCONFIG], [AC_REQUIRE([AC_INIT_BINSH])dnl -inserted= -dnl Shell is hard, so here is what the following does: -dnl - Reset $@ (command line arguments) -dnl - Add the configure options from mozconfig to $@ one by one -dnl - Add the original command line arguments after that, one by one -dnl -dnl There are several tricks involved: -dnl - It is not possible to preserve the whitespaces in $@ by assigning to -dnl another variable, so the two first steps above need to happen in the first -dnl iteration of the third step. -dnl - We always want the configure options to be added, so the loop must be -dnl iterated at least once, so we add a dummy argument first, and discard it. -dnl - something | while read line ... makes the while run in a subshell, meaning -dnl that anything it does is not propagated to the main shell, so we can't do -dnl set -- foo there. As a consequence, what the while loop reading mach -dnl environment output does is output a set of shell commands for the main shell -dnl to eval. -dnl - Extra care is due when lines from mach environment output contain special -dnl shell characters, so we use ' for quoting and ensure no ' end up in between -dnl the quoting mark unescaped. -dnl Some of the above is directly done in mach environment --format=configure. -failed_eval() { - echo "Failed eval'ing the following:" - $(dirname [$]0)/[$1]/mach environment --format=configure - exit 1 -} - -set -- dummy "[$]@" -for ac_option -do - if test -z "$inserted"; then - set -- - eval "$($(dirname [$]0)/[$1]/mach environment --format=configure)" || failed_eval - inserted=1 - else - set -- "[$]@" "$ac_option" - fi -done -]) +# Read in '.mozconfig' script to set the initial options. +# See the mozconfig2configure script for more details. +_AUTOCONF_TOOLS_DIR=`dirname [$]0`/[$1]/build/autoconf +. $_AUTOCONF_TOOLS_DIR/mozconfig2configure]) diff --git a/build/autoconf/mozconfig-find b/build/autoconf/mozconfig-find new file mode 100755 index 0000000..97dd90c --- /dev/null +++ b/build/autoconf/mozconfig-find @@ -0,0 +1,76 @@ +#! /bin/sh +# +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this +# file, You can obtain one at http://mozilla.org/MPL/2.0/. + +# mozconfigfind - Loads options from .mozconfig onto configure's +# command-line. The .mozconfig file is searched for in the +# order: +# If $MOZCONFIG is set, use that. +# If one of $TOPSRCDIR/.mozconfig or $TOPSRCDIR/mozconfig exists, use it. +# If both exist, or if various legacy locations contain a mozconfig, error. +# Otherwise, use the default build options. +# +topsrcdir=$1 + +abspath() { + if uname -s | grep -q MINGW; then + # We have no way to figure out whether we're in gmake or pymake right + # now. gmake gives us Unix-style paths while pymake gives us Windows-style + # paths, so attempt to handle both. + regexes='^$[A-Za-z]:\|\\\\\|\/$ ^\/' + else + regexes='^\/' + fi + + for regex in $regexes; do + if echo $1 | grep -q $regex; then + echo $1 + return + fi + done + + # If we're at this point, we have a relative path + echo `pwd`/$1 +} + +if [ -n "$MOZCONFIG" ] && ! [ -f "$MOZCONFIG" ]; then + echo "Specified MOZCONFIG \"$MOZCONFIG\" does not exist!" 1>&2 + exit 1 +fi + +if [ -n "$MOZ_MYCONFIG" ]; then + echo "Your environment currently has the MOZ_MYCONFIG variable set to \"$MOZ_MYCONFIG\". MOZ_MYCONFIG is no longer supported. Please use MOZCONFIG instead." 1>&2 + exit 1 +fi + +if [ -z "$MOZCONFIG" ] && [ -f "$topsrcdir/.mozconfig" ] && [ -f "$topsrcdir/mozconfig" ]; then + echo "Both \$topsrcdir/.mozconfig and \$topsrcdir/mozconfig are supported, but you must choose only one. Please remove the other." 1>&2 + exit 1 +fi + +for _config in "$MOZCONFIG" \ + "$topsrcdir/.mozconfig" \ + "$topsrcdir/mozconfig" +do + if test -f "$_config"; then + abspath $_config + exit 0 + fi +done + +# We used to support a number of other implicit .mozconfig locations. We now +# detect if we were about to use any of these locations and issue an error if we +# find any. +for _config in "$topsrcdir/mozconfig.sh" \ + "$topsrcdir/myconfig.sh" \ + "$HOME/.mozconfig" \ + "$HOME/.mozconfig.sh" \ + "$HOME/.mozmyconfig.sh" +do + if test -f "$_config"; then + echo "You currently have a mozconfig at \"$_config\". This implicit location is no longer supported. Please move it to $topsrcdir/.mozconfig or specify it explicitly via \$MOZCONFIG." 1>&2 + exit 1 + fi +done diff --git a/build/autoconf/mozconfig2client-mk b/build/autoconf/mozconfig2client-mk new file mode 100755 index 0000000..aaf8de1 --- /dev/null +++ b/build/autoconf/mozconfig2client-mk @@ -0,0 +1,76 @@ +#! /bin/sh +# +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this +# file, You can obtain one at http://mozilla.org/MPL/2.0/. + +# mozconfig2client-mk - Translates .mozconfig into options for client.mk. +# Prints defines to stdout. +# +# See mozconfig2configure for more details + +print_header() { + cat <<EOF +# gmake +# This file is automatically generated for client.mk. +# Do not edit. Edit $FOUND_MOZCONFIG instead. + +EOF +} + +ac_add_options() { + for _opt + do + case "$_opt" in + --target=*) + echo $_opt | sed s/--target/CONFIG_GUESS/ + ;; + *) + echo "# $_opt is used by configure (not client.mk)" + ;; + esac + done +} + +ac_add_app_options() { + echo "# $* is used by configure (not client.mk)" +} + +mk_add_options() { + for _opt + do + # Escape shell characters, space, tab, dollar, quote, backslash, + # and substitute '@<word>@' with '$(<word>)'. + _opt=`echo "$_opt" | sed -e 's/$[\"\\]$/\\\\\1/g; s/@$[^@]*$@/\$(\1)/g;'` + echo $_opt; + done +} + +# Main +#-------------------------------------------------- + +scriptdir=`dirname $0` +topsrcdir=$1 + +# If the path changes, configure should be rerun +echo "# PATH=$PATH" + +# If FOUND_MOZCONFIG isn't set, look for it and make sure the script doesn't error out +isfoundset=${FOUND_MOZCONFIG+yes} +if [ -z $isfoundset ]; then + FOUND_MOZCONFIG=`$scriptdir/mozconfig-find $topsrcdir` + if [ $? -ne 0 ]; then + echo '$(error Fix above errors before continuing.)' + else + isfoundset=yes + fi +fi + +if [ -n $isfoundset ]; then + if [ "$FOUND_MOZCONFIG" ] + then + print_header + . "$FOUND_MOZCONFIG" + echo "FOUND_MOZCONFIG := $FOUND_MOZCONFIG" + fi +fi diff --git a/build/autoconf/mozconfig2configure b/build/autoconf/mozconfig2configure new file mode 100755 index 0000000..99623b6 --- /dev/null +++ b/build/autoconf/mozconfig2configure @@ -0,0 +1,103 @@ +#! /bin/sh +# +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this +# file, You can obtain one at http://mozilla.org/MPL/2.0/. + +# mozconfig2configure - Loads options from .mozconfig onto configure's +# command-line. See mozconfig-find for how the config file is +# found +# +# The options from .mozconfig are inserted into the command-line +# before the real command-line options. This way the real options +# will override any .mozconfig options. +# +# .mozconfig is a shell script. To add an option to configure's +# command-line use the pre-defined function, ac_add_options, +# +# ac_add_options <configure-option> [<configure-option> ... ] +# +# For example, +# +# ac_add_options --with-pthreads --enable-debug +# +# ac_add_options can be called multiple times in .mozconfig. +# Each call adds more options to configure's command-line. + +# Note: $_AUTOCONF_TOOLS_DIR must be defined in the script that includes this. + +ac_add_options() { + for _opt + do + # Escape shell characters, space, tab, dollar, quote, backslash, parentheses. + _opt=`echo $_opt | sed -e 's/$[\ \ \$\"\\\($]\)/\\\\\1/g;s/@$[^@]*$@/\$\1/g;'` + _opt=`echo $_opt | sed -e 's/@$[^@]*$@/\$(\1)/g'` + + # Avoid adding duplicates + case "$ac_options" in + # Note that all options in $ac_options are enclosed in quotes, + # so there will always be a last character to match [^-A-Za-z0-9_] + *"\"$_opt[^-A-Za-z0-9_]"* ) ;; + * ) mozconfig_ac_options="$mozconfig_ac_options $_opt" ;; + esac + done +} + +ac_add_app_options() { + APP=$1 + shift; + if [ "$APP" = "$MOZ_BUILD_APP" ]; then + ac_add_options "$*"; + fi +} + +mk_add_options() { + # These options are for client.mk + # configure can safely ignore them. + : +} + +ac_echo_options() { + echo "Adding configure options from $FOUND_MOZCONFIG:" + eval "set -- $mozconfig_ac_options" + for _opt + do + echo " $_opt" + done +} + +# Main +#-------------------------------------------------- +topsrcdir=$(cd `dirname $0`; pwd -W 2>/dev/null || pwd) +ac_options= +mozconfig_ac_options= + +# Save the real command-line options +for _opt +do + # Escape shell characters, space, tab, dollar, quote, backslash. + _opt=`echo $_opt | sed -e 's/$[\ \ \$\"\\]$/\\\\\1/g;'` + ac_options="$ac_options \"$_opt\"" +done + + +# If FOUND_MOZCONFIG isn't set, look for it and make sure the script doesn't error out +isfoundset=${FOUND_MOZCONFIG+yes} +if [ -z $isfoundset ]; then + FOUND_MOZCONFIG=`$_AUTOCONF_TOOLS_DIR/mozconfig-find $topsrcdir` + if [ $? -ne 0 ]; then + echo "Fix above errors before continuing." 1>&2 + exit 1 + fi +fi + +if [ "$FOUND_MOZCONFIG" ]; then + . "$FOUND_MOZCONFIG" +fi +export FOUND_MOZCONFIG + +if [ "$mozconfig_ac_options" ]; then + ac_echo_options 1>&2 +fi + +eval "set -- $mozconfig_ac_options $ac_options" diff --git a/client.mk b/client.mk index 5cbaf6b..f737c5b 100644 --- a/client.mk +++ b/client.mk @@ -50,6 +50,7 @@ endif ifndef TOPSRCDIR ifeq (,$(wildcard client.mk)) TOPSRCDIR := $(patsubst %/,%,$(dir $(MAKEFILE_LIST))) +MOZ_OBJDIR = . else TOPSRCDIR := $(CWD) endif @@ -98,6 +99,8 @@ endif # See build pages, http://www.mozilla.org/build/ for how to set up mozconfig. +MOZCONFIG_LOADER := build/autoconf/mozconfig2client-mk + define CR @@ -108,9 +111,7 @@ endef # before evaluation. $(shell) replacing newlines with spaces, || is always # followed by a space (since sed doesn't remove newlines), except on the # last line, so replace both '|| ' and '||'. -# Also, make MOZ_PGO available to mozconfig when passed on make command line. -# Likewise for MOZ_CURRENT_PROJECT. -MOZCONFIG_CONTENT := $(subst ||,$(CR),$(subst || ,$(CR),$(shell $(addprefix MOZ_CURRENT_PROJECT=,$(MOZ_CURRENT_PROJECT)) MOZ_PGO=$(MOZ_PGO) $(TOPSRCDIR)/mach environment --format=client.mk | sed 's/$$/||/'))) +MOZCONFIG_CONTENT := $(subst ||,$(CR),$(subst || ,$(CR),$(shell $(TOPSRCDIR)/$(MOZCONFIG_LOADER) $(TOPSRCDIR) | sed 's/$$/||/'))) $(eval $(MOZCONFIG_CONTENT)) export FOUND_MOZCONFIG @@ -141,18 +142,35 @@ ifeq (,$(findstring -j,$(MOZ_MAKE_FLAGS))) endif +ifndef MOZ_OBJDIR + MOZ_OBJDIR = obj-$(CONFIG_GUESS) +else +# On Windows Pymake builds check MOZ_OBJDIR doesn't start with "/" + ifneq (,$(findstring mingw,$(CONFIG_GUESS))) + ifeq (1_a,$(.PYMAKE)_$(firstword a$(subst /, ,$(MOZ_OBJDIR)))) + $(error For Windows Pymake builds, MOZ_OBJDIR must be a Windows [and not MSYS] style path.) + endif + endif +endif + ifdef MOZ_BUILD_PROJECTS ifdef MOZ_CURRENT_PROJECT + OBJDIR = $(MOZ_OBJDIR)/$(MOZ_CURRENT_PROJECT) + MOZ_MAKE = $(MAKE) $(MOZ_MAKE_FLAGS) -C $(OBJDIR) BUILD_PROJECT_ARG = MOZ_BUILD_APP=$(MOZ_CURRENT_PROJECT) - export MOZ_CURRENT_PROJECT else + OBJDIR = $(error Cannot find the OBJDIR when MOZ_CURRENT_PROJECT is not set.) MOZ_MAKE = $(error Cannot build in the OBJDIR when MOZ_CURRENT_PROJECT is not set.) endif -endif # MOZ_BUILD_PROJECTS +else # MOZ_BUILD_PROJECTS + +OBJDIR = $(MOZ_OBJDIR) MOZ_MAKE = $(MAKE) $(MOZ_MAKE_FLAGS) -C $(OBJDIR) +endif # MOZ_BUILD_PROJECTS + # 'configure' scripts generated by autoconf. CONFIGURES := $(TOPSRCDIR)/configure CONFIGURES += $(TOPSRCDIR)/js/src/configure @@ -184,7 +202,7 @@ WANT_MOZCONFIG_MK = 1 endif ifdef WANT_MOZCONFIG_MK -# For now, only output "export" lines from mach environment --format=client.mk output. +# For now, only output "export" lines from mozconfig2client-mk output. MOZCONFIG_MK_LINES := $(filter export||%,$(MOZCONFIG_OUT_LINES)) $(OBJDIR)/.mozconfig.mk: $(FOUND_MOZCONFIG) $(call mkdir_deps,$(OBJDIR)) $(OBJDIR)/CLOBBER $(if $(MOZCONFIG_MK_LINES),( $(foreach line,$(MOZCONFIG_MK_LINES), echo '$(subst ||, ,$(line))';) )) > $@ @@ -217,11 +235,17 @@ everything: clean build # This is up here, outside of the MOZ_CURRENT_PROJECT logic so that this # is usable in multi-pass builds, where you might not have a runnable # application until all the build passes and postflight scripts have run. +ifdef MOZ_OBJDIR + PGO_OBJDIR = $(MOZ_OBJDIR) +else + PGO_OBJDIR := $(TOPSRCDIR) +endif + profiledbuild:: $(MAKE) -f $(TOPSRCDIR)/client.mk realbuild MOZ_PROFILE_GENERATE=1 MOZ_PGO_INSTRUMENTED=1 - $(MAKE) -C $(OBJDIR) package MOZ_PGO_INSTRUMENTED=1 MOZ_INTERNAL_SIGNING_FORMAT= MOZ_EXTERNAL_SIGNING_FORMAT= - rm -f $(OBJDIR)/jarlog/en-US.log - MOZ_PGO_INSTRUMENTED=1 JARLOG_FILE=jarlog/en-US.log EXTRA_TEST_ARGS=10 $(MAKE) -C $(OBJDIR) pgo-profile-run + $(MAKE) -C $(PGO_OBJDIR) package MOZ_PGO_INSTRUMENTED=1 MOZ_INTERNAL_SIGNING_FORMAT= MOZ_EXTERNAL_SIGNING_FORMAT= + rm -f ${PGO_OBJDIR}/jarlog/en-US.log + MOZ_PGO_INSTRUMENTED=1 JARLOG_FILE=jarlog/en-US.log EXTRA_TEST_ARGS=10 $(MAKE) -C $(PGO_OBJDIR) pgo-profile-run $(MAKE) -f $(TOPSRCDIR)/client.mk maybe_clobber_profiledbuild $(MAKE) -f $(TOPSRCDIR)/client.mk realbuild MOZ_PROFILE_USE=1 @@ -302,7 +326,6 @@ CONFIG_STATUS_DEPS := \ $(TOPSRCDIR)/build/virtualenv_packages.txt \ $(TOPSRCDIR)/python/mozbuild/mozbuild/virtualenv.py \ $(TOPSRCDIR)/testing/mozbase/packages.txt \ - $(OBJDIR)/.mozconfig.json \ $(NULL) CONFIGURE_ENV_ARGS += \ @@ -330,13 +353,8 @@ configure-preqs = \ $(call mkdir_deps,$(OBJDIR)) \ $(if $(MOZ_BUILD_PROJECTS),$(call mkdir_deps,$(MOZ_OBJDIR))) \ save-mozconfig \ - $(OBJDIR)/.mozconfig.json \ $(NULL) -CREATE_MOZCONFIG_JSON := $(shell $(TOPSRCDIR)/mach environment --format=json -o $(OBJDIR)/.mozconfig.json) -$(OBJDIR)/.mozconfig.json: $(call mkdir_deps,$(OBJDIR)) - @$(TOPSRCDIR)/mach environment --format=json -o $@ - save-mozconfig: $(FOUND_MOZCONFIG) -cp $(FOUND_MOZCONFIG) $(OBJDIR)/.mozconfig @@ -355,7 +373,7 @@ $(OBJDIR)/config.status: $(CONFIG_STATUS_DEPS) else $(OBJDIR)/Makefile: $(CONFIG_STATUS_DEPS) endif - @$(MAKE) -f $(TOPSRCDIR)/client.mk configure CREATE_MOZCONFIG_JSON= + @$(MAKE) -f $(TOPSRCDIR)/client.mk configure ifneq (,$(CONFIG_STATUS)) $(OBJDIR)/config/autoconf.mk: $(TOPSRCDIR)/config/autoconf.mk.in diff --git a/python/mozbuild/mozbuild/base.py b/python/mozbuild/mozbuild/base.py index 14c9972..d00d987 100644 --- a/python/mozbuild/mozbuild/base.py +++ b/python/mozbuild/mozbuild/base.py @@ -161,8 +161,7 @@ class MozbuildObject(ProcessExecutionMixin): # environment. If no mozconfig is present, the config will not have # much defined. loader = MozconfigLoader(topsrcdir) - current_project = os.environ.get('MOZ_CURRENT_PROJECT') - config = loader.read_mozconfig(mozconfig, moz_build_app=current_project) + config = loader.read_mozconfig(mozconfig) config_topobjdir = MozbuildObject.resolve_mozconfig_topobjdir( topsrcdir, config) @@ -172,30 +171,13 @@ class MozbuildObject(ProcessExecutionMixin): # inside an objdir you probably want to perform actions on that objdir, # not another one. This prevents accidental usage of the wrong objdir # when the current objdir is ambiguous. - # However, if the found mozconfig resolves to another objdir that - # doesn't exist, we may be in a subtree like when building mozilla/ - # under c-c, and the objdir was defined as a relative path. Try again - # adjusting for that. - if topobjdir and config_topobjdir: - if not os.path.exists(config_topobjdir): - config_topobjdir = MozbuildObject.resolve_mozconfig_topobjdir( - os.path.dirname(topsrcdir), config) - if current_project: - config_topobjdir = os.path.join(config_topobjdir, - current_project) - config_topobjdir = os.path.join(config_topobjdir, - os.path.basename(topsrcdir)) - elif current_project: - config_topobjdir = os.path.join(config_topobjdir, current_project) - - _config_topobjdir = config_topobjdir - mozilla_dir = os.path.join(_config_topobjdir, 'mozilla') - if not samepath(topobjdir, _config_topobjdir) \ - and (not os.path.exists(mozilla_dir) or not samepath(topobjdir, + mozilla_dir = os.path.join(config_topobjdir, 'mozilla') + if not samepath(topobjdir, config_topobjdir) \ + and (os.path.exists(mozilla_dir) and not samepath(topobjdir, mozilla_dir)): - raise ObjdirMismatchException(topobjdir, _config_topobjdir) + raise ObjdirMismatchException(topobjdir, config_topobjdir) topobjdir = topobjdir or config_topobjdir if topobjdir: @@ -251,8 +233,7 @@ class MozbuildObject(ProcessExecutionMixin): """ if self._mozconfig is None: loader = MozconfigLoader(self.topsrcdir) - self._mozconfig = loader.read_mozconfig( - moz_build_app=os.environ.get('MOZ_CURRENT_PROJECT')) + self._mozconfig = loader.read_mozconfig() return self._mozconfig @@ -567,13 +548,8 @@ class MachCommandBase(MozbuildObject): # more reliable than mozconfig when cwd is inside an objdir. topsrcdir = context.topdir topobjdir = None - detect_virtualenv_mozinfo = True - if hasattr(context, 'detect_virtualenv_mozinfo'): - detect_virtualenv_mozinfo = getattr(context, - 'detect_virtualenv_mozinfo') try: - dummy = MozbuildObject.from_environment(cwd=context.cwd, - detect_virtualenv_mozinfo=detect_virtualenv_mozinfo) + dummy = MozbuildObject.from_environment(cwd=context.cwd) topsrcdir = dummy.topsrcdir topobjdir = dummy._topobjdir except BuildEnvironmentNotFoundException: diff --git a/python/mozbuild/mozbuild/mach_commands.py b/python/mozbuild/mozbuild/mach_commands.py index 0b19815..0bfe4a6 100644 --- a/python/mozbuild/mozbuild/mach_commands.py +++ b/python/mozbuild/mozbuild/mach_commands.py @@ -10,8 +10,6 @@ import operator import os import sys -import mozpack.path as mozpath - from mach.decorators import ( CommandArgument, CommandProvider, @@ -919,53 +917,37 @@ class Makefiles(MachCommandBase): class MachDebug(MachCommandBase): @Command('environment', category='build-dev', description='Show info about the mach and build environment.') - @CommandArgument('--format', default='pretty', - choices=['pretty', 'client.mk', 'configure', 'json'], - help='Print data in the given format.') - @CommandArgument('--output', '-o', type=str, - help='Output to the given file.') @CommandArgument('--verbose', '-v', action='store_true', help='Print verbose output.') - def environment(self, format, output=None, verbose=False): - func = getattr(self, '_environment_%s' % format.replace('.', '_')) - - if output: - # We want to preserve mtimes if the output file already exists - # and the content hasn't changed. - from mozbuild.util import FileAvoidWrite - with FileAvoidWrite(output) as out: - return func(out, verbose) - return func(sys.stdout, verbose) - - def _environment_pretty(self, out, verbose): + def environment(self, verbose=False): state_dir = self._mach_context.state_dir import platform - print('platform:\n\t%s' % platform.platform(), file=out) - print('python version:\n\t%s' % sys.version, file=out) - print('python prefix:\n\t%s' % sys.prefix, file=out) - print('mach cwd:\n\t%s' % self._mach_context.cwd, file=out) - print('os cwd:\n\t%s' % os.getcwd(), file=out) - print('mach directory:\n\t%s' % self._mach_context.topdir, file=out) - print('state directory:\n\t%s' % state_dir, file=out) + print('platform:\n\t%s' % platform.platform()) + print('python version:\n\t%s' % sys.version) + print('python prefix:\n\t%s' % sys.prefix) + print('mach cwd:\n\t%s' % self._mach_context.cwd) + print('os cwd:\n\t%s' % os.getcwd()) + print('mach directory:\n\t%s' % self._mach_context.topdir) + print('state directory:\n\t%s' % state_dir) - print('object directory:\n\t%s' % self.topobjdir, file=out) + print('object directory:\n\t%s' % self.topobjdir) if self.mozconfig['path']: - print('mozconfig path:\n\t%s' % self.mozconfig['path'], file=out) + print('mozconfig path:\n\t%s' % self.mozconfig['path']) if self.mozconfig['configure_args']: - print('mozconfig configure args:', file=out) + print('mozconfig configure args:') for arg in self.mozconfig['configure_args']: - print('\t%s' % arg, file=out) + print('\t%s' % arg) if self.mozconfig['make_extra']: - print('mozconfig extra make args:', file=out) + print('mozconfig extra make args:') for arg in self.mozconfig['make_extra']: - print('\t%s' % arg, file=out) + print('\t%s' % arg) if self.mozconfig['make_flags']: - print('mozconfig make flags:', file=out) + print('mozconfig make flags:') for arg in self.mozconfig['make_flags']: - print('\t%s' % arg, file=out) + print('\t%s' % arg) config = None @@ -976,70 +958,14 @@ class MachDebug(MachCommandBase): pass if config: - print('config topsrcdir:\n\t%s' % config.topsrcdir, file=out) - print('config topobjdir:\n\t%s' % config.topobjdir, file=out) + print('config topsrcdir:\n\t%s' % config.topsrcdir) + print('config topobjdir:\n\t%s' % config.topobjdir) if verbose: - print('config substitutions:', file=out) + print('config substitutions:') for k in sorted(config.substs): - print('\t%s: %s' % (k, config.substs[k]), file=out) + print('\t%s: %s' % (k, config.substs[k])) - print('config defines:', file=out) + print('config defines:') for k in sorted(config.defines): - print('\t%s' % k, file=out) - - def _environment_client_mk(self, out, verbose): - if self.mozconfig['make_extra']: - for arg in self.mozconfig['make_extra']: - print(arg, file=out) - objdir = mozpath.normsep(self.topobjdir) - print('MOZ_OBJDIR=%s' % objdir, file=out) - if 'MOZ_CURRENT_PROJECT' in os.environ: - objdir = mozpath.join(objdir, os.environ['MOZ_CURRENT_PROJECT']) - print('OBJDIR=%s' % objdir, file=out) - if self.mozconfig['path']: - print('FOUND_MOZCONFIG=%s' % mozpath.normsep(self.mozconfig['path']), - file=out) - - def _environment_configure(self, out, verbose): - if self.mozconfig['path']: - # Replace ' with '"'"', so that shell quoting e.g. - # a'b becomes 'a'"'"'b'. - quote = lambda s: s.replace("'", """'"'"'""") - print('echo Adding configure options from %s' % - mozpath.normsep(self.mozconfig['path']), file=out) - if self.mozconfig['configure_args']: - for arg in self.mozconfig['configure_args']: - quoted_arg = quote(arg) - print("echo ' %s'" % quoted_arg, file=out) - print("""set -- "$@" '%s'""" % quoted_arg, file=out) - for key, value in self.mozconfig['env']['added'].items(): - print("export %s='%s'" % (key, quote(value)), file=out) - for key, (old, value) in self.mozconfig['env']['modified'].items(): - print("export %s='%s'" % (key, quote(value)), file=out) - for key, value in self.mozconfig['vars']['added'].items(): - print("%s='%s'" % (key, quote(value)), file=out) - for key, (old, value) in self.mozconfig['vars']['modified'].items(): - print("%s='%s'" % (key, quote(value)), file=out) - for key in self.mozconfig['env']['removed'].keys() + \ - self.mozconfig['vars']['removed'].keys(): - print("unset %s" % key, file=out) - - def _environment_json(self, out, verbose): - import json - class EnvironmentEncoder(json.JSONEncoder): - def default(self, obj): - if isinstance(obj, MozbuildObject): - result = { - 'topsrcdir': obj.topsrcdir, - 'topobjdir': obj.topobjdir, - 'mozconfig': obj.mozconfig, - } - if verbose: - result['substs'] = obj.substs - result['defines'] = obj.defines - return result - elif isinstance(obj, set): - return list(obj) - return json.JSONEncoder.default(self, obj) - json.dump(self, cls=EnvironmentEncoder, sort_keys=True, fp=out) + print('\t%s' % k) diff --git a/python/mozbuild/mozbuild/mozconfig.py b/python/mozbuild/mozbuild/mozconfig.py index 9feca70..3c1e3eb 100644 --- a/python/mozbuild/mozbuild/mozconfig.py +++ b/python/mozbuild/mozbuild/mozconfig.py @@ -65,11 +65,7 @@ class MozconfigLoader(ProcessExecutionMixin): DEPRECATED_TOPSRCDIR_PATHS = ('mozconfig.sh', 'myconfig.sh') DEPRECATED_HOME_PATHS = ('.mozconfig', '.mozconfig.sh', '.mozmyconfig.sh') - IGNORE_SHELL_VARIABLES = {'_'} - - ENVIRONMENT_VARIABLES = { - 'CC', 'CXX', 'CFLAGS', 'CXXFLAGS', 'LDFLAGS', 'MOZ_OBJDIR', - } + IGNORE_SHELL_VARIABLES = ('_') def __init__(self, topsrcdir): self.topsrcdir = topsrcdir @@ -200,7 +196,6 @@ class MozconfigLoader(ProcessExecutionMixin): 'make_flags': None, 'make_extra': None, 'env': None, - 'vars': None, } if path is None: @@ -236,49 +231,36 @@ class MozconfigLoader(ProcessExecutionMixin): parsed = self._parse_loader_output(output) - def diff_vars(vars_before, vars_after): - set1 = set(vars_before.keys()) - self.IGNORE_SHELL_VARIABLES - set2 = set(vars_after.keys()) - self.IGNORE_SHELL_VARIABLES - added = set2 - set1 - removed = set1 - set2 - maybe_modified = set1 & set2 - changed = { - 'added': {}, - 'removed': {}, - 'modified': {}, - 'unmodified': {}, - } - - for key in added: - changed['added'][key] = vars_after[key] - - for key in removed: - changed['removed'][key] = vars_before[key] - - for key in maybe_modified: - if vars_before[key] != vars_after[key]: - changed['modified'][key] = ( - vars_before[key], vars_after[key]) - elif key in self.ENVIRONMENT_VARIABLES: - # In order for irrelevant environment variable changes not - # to incur in re-running configure, only a set of - # environment variables are stored when they are - # unmodified. Otherwise, changes such as using a different - # terminal window, or even rebooting, would trigger - # reconfigures. - changed['unmodified'][key] = vars_after[key] - - return changed - - result['env'] = diff_vars(parsed['env_before'], parsed['env_after']) - - # Environment variables also appear as shell variables, but that's - # uninteresting duplication of information. Filter them out. - filt = lambda x, y: {k: v for k, v in x.items() if k not in y} - result['vars'] = diff_vars( - filt(parsed['vars_before'], parsed['env_before']), - filt(parsed['vars_after'], parsed['env_after']) - ) + all_variables = set(parsed['vars_before'].keys()) + all_variables |= set(parsed['vars_after'].keys()) + + changed = { + 'added': {}, + 'removed': {}, + 'modified': {}, + 'unmodified': {}, + } + + for key in all_variables: + if key in self.IGNORE_SHELL_VARIABLES: + continue + + if key not in parsed['vars_before']: + changed['added'][key] = parsed['vars_after'][key] + continue + + if key not in parsed['vars_after']: + changed['removed'][key] = parsed['vars_before'][key] + continue + + if parsed['vars_before'][key] != parsed['vars_after'][key]: + changed['modified'][key] = ( + parsed['vars_before'][key], parsed['vars_after'][key]) + continue + + changed['unmodified'][key] = parsed['vars_after'][key] + + result['env'] = changed result['configure_args'] = [self._expand(o) for o in parsed['ac']] @@ -286,9 +268,6 @@ class MozconfigLoader(ProcessExecutionMixin): result['configure_args'].extend(self._expand(o) for o in parsed['ac_app'][moz_build_app]) - if 'MOZ_OBJDIR' in parsed['env_before']: - result['topobjdir'] = parsed['env_before']['MOZ_OBJDIR'] - mk = [self._expand(o) for o in parsed['mk']] for o in mk: @@ -318,8 +297,6 @@ class MozconfigLoader(ProcessExecutionMixin): ac_app_options = defaultdict(list) before_source = {} after_source = {} - env_before_source = {} - env_after_source = {} current = None current_type = None @@ -362,14 +339,7 @@ class MozconfigLoader(ProcessExecutionMixin): assert current_type is not None - vars_mapping = { - 'BEFORE_SOURCE': before_source, - 'AFTER_SOURCE': after_source, - 'ENV_BEFORE_SOURCE': env_before_source, - 'ENV_AFTER_SOURCE': env_after_source, - } - - if current_type in vars_mapping: + if current_type in ('BEFORE_SOURCE', 'AFTER_SOURCE'): # mozconfigs are sourced using the Bourne shell (or at least # in Bourne shell mode). This means |set| simply lists # variables from the current shell (not functions). (Note that @@ -430,7 +400,10 @@ class MozconfigLoader(ProcessExecutionMixin): assert name is not None - vars_mapping[current_type][name] = value + if current_type == 'BEFORE_SOURCE': + before_source[name] = value + else: + after_source[name] = value current = [] @@ -444,8 +417,6 @@ class MozconfigLoader(ProcessExecutionMixin): 'ac_app': ac_app_options, 'vars_before': before_source, 'vars_after': after_source, - 'env_before': env_before_source, - 'env_after': env_after_source, } def _expand(self, s): diff --git a/python/mozbuild/mozbuild/mozconfig_loader b/python/mozbuild/mozbuild/mozconfig_loader index 569c690..d12c7fa 100755 --- a/python/mozbuild/mozbuild/mozconfig_loader +++ b/python/mozbuild/mozbuild/mozconfig_loader @@ -44,10 +44,6 @@ mk_add_options() { done } -echo "------BEGIN_ENV_BEFORE_SOURCE" -env -echo "------END_ENV_BEFORE_SOURCE" - echo "------BEGIN_BEFORE_SOURCE" set echo "------END_BEFORE_SOURCE" @@ -62,7 +58,3 @@ echo "------BEGIN_AFTER_SOURCE" set echo "------END_AFTER_SOURCE" -echo "------BEGIN_ENV_AFTER_SOURCE" -env -echo "------END_ENV_AFTER_SOURCE" - diff --git a/python/mozbuild/mozbuild/test/backend/common.py b/python/mozbuild/mozbuild/test/backend/common.py index b6e1566..7cfec3c 100644 --- a/python/mozbuild/mozbuild/test/backend/common.py +++ b/python/mozbuild/mozbuild/test/backend/common.py @@ -83,14 +83,6 @@ CONFIGS = DefaultOnReadDict({ class BackendTester(unittest.TestCase): - def setUp(self): - self._old_env = dict(os.environ) - os.environ.pop('MOZ_OBJDIR', None) - - def tearDown(self): - os.environ.clear() - os.environ.update(self._old_env) - def _get_environment(self, name): """Obtain a new instance of a ConfigEnvironment for a known profile. diff --git a/python/mozbuild/mozbuild/test/frontend/test_emitter.py b/python/mozbuild/mozbuild/test/frontend/test_emitter.py index 9e5fef1..25b69d4 100644 --- a/python/mozbuild/mozbuild/test/frontend/test_emitter.py +++ b/python/mozbuild/mozbuild/test/frontend/test_emitter.py @@ -41,14 +41,6 @@ data_path = mozpath.join(data_path, 'data') class TestEmitterBasic(unittest.TestCase): - def setUp(self): - self._old_env = dict(os.environ) - os.environ.pop('MOZ_OBJDIR', None) - - def tearDown(self): - os.environ.clear() - os.environ.update(self._old_env) - def reader(self, name): config = MockConfig(mozpath.join(data_path, name), extra_substs=dict( ENABLE_TESTS='1', diff --git a/python/mozbuild/mozbuild/test/test_base.py b/python/mozbuild/mozbuild/test/test_base.py index 3735213..24118cd 100644 --- a/python/mozbuild/mozbuild/test/test_base.py +++ b/python/mozbuild/mozbuild/test/test_base.py @@ -22,15 +22,14 @@ from mozbuild.base import ( BadEnvironmentException, MachCommandBase, MozbuildObject, - ObjdirMismatchException, PathArgument, ) from mozbuild.backend.configenvironment import ConfigEnvironment -from buildconfig import topsrcdir, topobjdir curdir = os.path.dirname(__file__) +topsrcdir = os.path.abspath(os.path.join(curdir, '..', '..', '..', '..')) log_manager = LoggingManager() @@ -39,15 +38,14 @@ class TestMozbuildObject(unittest.TestCase): self._old_cwd = os.getcwd() self._old_env = dict(os.environ) os.environ.pop('MOZCONFIG', None) - os.environ.pop('MOZ_OBJDIR', None) def tearDown(self): os.chdir(self._old_cwd) os.environ.clear() os.environ.update(self._old_env) - def get_base(self, topobjdir=None): - return MozbuildObject(topsrcdir, None, log_manager, topobjdir=topobjdir) + def get_base(self): + return MozbuildObject(topsrcdir, None, log_manager) def test_objdir_config_guess(self): base = self.get_base() @@ -74,6 +72,7 @@ class TestMozbuildObject(unittest.TestCase): 'foo')) self.assertTrue(base.topobjdir.endswith('foo')) + @unittest.skip('Failing on buildbot.') def test_objdir_config_status(self): """Ensure @CONFIG_GUESS@ is handled when loading mozconfig.""" base = self.get_base() @@ -107,17 +106,16 @@ class TestMozbuildObject(unittest.TestCase): mozconfig=mozconfig, ), fh) - os.environ[b'MOZCONFIG'] = mozconfig.encode('utf-8') + os.environ[b'MOZCONFIG'] = mozconfig os.chdir(topobjdir) - obj = MozbuildObject.from_environment( - detect_virtualenv_mozinfo=False) + obj = MozbuildObject.from_environment() self.assertEqual(obj.topobjdir, topobjdir) finally: - os.chdir(self._old_cwd) shutil.rmtree(d) + @unittest.skip('Failing on buildbot.') def test_relative_objdir(self): """Relative defined objdirs are loaded properly.""" d = os.path.realpath(tempfile.mkdtemp()) @@ -136,18 +134,16 @@ class TestMozbuildObject(unittest.TestCase): mozconfig=mozconfig, ), fh) - os.environ[b'MOZCONFIG'] = mozconfig.encode('utf-8') + os.environ[b'MOZCONFIG'] = mozconfig child = os.path.join(topobjdir, 'foo', 'bar') os.makedirs(child) os.chdir(child) - obj = MozbuildObject.from_environment( - detect_virtualenv_mozinfo=False) + obj = MozbuildObject.from_environment() self.assertEqual(obj.topobjdir, topobjdir) finally: - os.chdir(self._old_cwd) shutil.rmtree(d) @unittest.skipIf(not hasattr(os, 'symlink'), 'symlinks not available.') @@ -181,9 +177,9 @@ class TestMozbuildObject(unittest.TestCase): self.assertEqual(obj.topobjdir, topobjdir_real) finally: - os.chdir(self._old_cwd) shutil.rmtree(d) + @unittest.skip('Failed on buildbot (bug 853954)') def test_mach_command_base_inside_objdir(self): """Ensure a MachCommandBase constructed from inside the objdir works.""" @@ -212,7 +208,6 @@ class TestMozbuildObject(unittest.TestCase): context.topdir = topsrcdir context.settings = None context.log_manager = None - context.detect_virtualenv_mozinfo=False o = MachCommandBase(context) @@ -220,9 +215,9 @@ class TestMozbuildObject(unittest.TestCase): self.assertEqual(o.topsrcdir, topsrcdir) finally: - os.chdir(self._old_cwd) shutil.rmtree(d) + @unittest.skip('Failing on buildbot.') def test_objdir_is_srcdir_rejected(self): """Ensure the srcdir configurations are rejected.""" d = os.path.realpath(tempfile.mkdtemp()) @@ -240,41 +235,6 @@ class TestMozbuildObject(unittest.TestCase): MozbuildObject.from_environment(detect_virtualenv_mozinfo=False) finally: - os.chdir(self._old_cwd) - shutil.rmtree(d) - - def test_objdir_mismatch(self): - """Ensure MachCommandBase throwing on objdir mismatch.""" - d = os.path.realpath(tempfile.mkdtemp()) - - try: - real_topobjdir = os.path.join(d, 'real-objdir') - os.makedirs(real_topobjdir) - - topobjdir = os.path.join(d, 'objdir') - os.makedirs(topobjdir) - - topsrcdir = os.path.join(d, 'srcdir') - os.makedirs(topsrcdir) - - mozconfig = os.path.join(d, 'mozconfig') - with open(mozconfig, 'wt') as fh: - fh.write('mk_add_options MOZ_OBJDIR=%s' % real_topobjdir) - - mozinfo = os.path.join(topobjdir, 'mozinfo.json') - with open(mozinfo, 'wt') as fh: - json.dump(dict( - topsrcdir=topsrcdir, - mozconfig=mozconfig, - ), fh) - - os.chdir(topobjdir) - - with self.assertRaises(ObjdirMismatchException): - MozbuildObject.from_environment(detect_virtualenv_mozinfo=False) - - finally: - os.chdir(self._old_cwd) shutil.rmtree(d) def test_config_guess(self): @@ -286,8 +246,9 @@ class TestMozbuildObject(unittest.TestCase): self.assertIsNotNone(result) self.assertGreater(len(result), 0) + @unittest.skip('Failing on buildbot (bug 853954).') def test_config_environment(self): - base = self.get_base(topobjdir=topobjdir) + base = self.get_base() ce = base.config_environment self.assertIsInstance(ce, ConfigEnvironment) @@ -298,17 +259,15 @@ class TestMozbuildObject(unittest.TestCase): self.assertIsInstance(base.defines, dict) self.assertIsInstance(base.substs, dict) + @unittest.skip('Failing on buildbot (bug 853954).') def test_get_binary_path(self): - base = self.get_base(topobjdir=topobjdir) + base = self.get_base() platform = sys.platform # We should ideally use the config.status from the build. Let's install # a fake one. - substs = [ - ('MOZ_APP_NAME', 'awesomeapp'), - ('MOZ_BUILD_APP', 'awesomeapp'), - ] + substs = [('MOZ_APP_NAME', 'awesomeapp')] if sys.platform.startswith('darwin'): substs.append(('OS_ARCH', 'Darwin')) substs.append(('BIN_SUFFIX', '')) @@ -343,7 +302,7 @@ class TestMozbuildObject(unittest.TestCase): if platform.startswith('darwin'): self.assertTrue(p.endswith('awesomeapp/Nightly.app/Contents/MacOS/awesomeapp')) elif platform.startswith(('win32', 'cygwin')): - self.assertTrue(p.endswith('awesomeapp\\awesomeapp.exe')) + self.assertTrue(p.endswith('awesomeapp/awesomeapp.exe')) else: self.assertTrue(p.endswith('awesomeapp/awesomeapp')) diff --git a/python/mozbuild/mozbuild/test/test_mozconfig.py b/python/mozbuild/mozbuild/test/test_mozconfig.py index 744424e..429523d 100644 --- a/python/mozbuild/mozbuild/test/test_mozconfig.py +++ b/python/mozbuild/mozbuild/test/test_mozconfig.py @@ -29,7 +29,6 @@ class TestMozconfigLoader(unittest.TestCase): def setUp(self): self._old_env = dict(os.environ) os.environ.pop('MOZCONFIG', None) - os.environ.pop('MOZ_OBJDIR', None) os.environ.pop('CC', None) os.environ.pop('CXX', None) self._temp_dirs = set() @@ -244,7 +243,6 @@ class TestMozconfigLoader(unittest.TestCase): 'make_flags': None, 'make_extra': None, 'env': None, - 'vars': None, }) def test_read_empty_mozconfig(self): @@ -258,10 +256,9 @@ class TestMozconfigLoader(unittest.TestCase): self.assertEqual(result['make_extra'], []) for f in ('added', 'removed', 'modified'): - self.assertEqual(len(result['vars'][f]), 0) self.assertEqual(len(result['env'][f]), 0) - self.assertEqual(result['env']['unmodified'], {}) + self.assertGreater(len(result['env']['unmodified']), 0) def test_read_capture_ac_options(self): """Ensures ac_add_options calls are captured.""" @@ -319,22 +316,6 @@ class TestMozconfigLoader(unittest.TestCase): self.assertEqual(result['make_flags'], '-j8') self.assertEqual(result['make_extra'], ['FOO=BAR BAZ', 'BIZ=1']) - def test_read_empty_mozconfig_objdir_environ(self): - os.environ[b'MOZ_OBJDIR'] = b'obj-firefox' - with NamedTemporaryFile(mode='w') as mozconfig: - result = self.get_loader().read_mozconfig(mozconfig.name) - self.assertEqual(result['topobjdir'], 'obj-firefox') - - def test_read_capture_mk_options_objdir_environ(self): - """Ensures mk_add_options calls are captured and override the environ.""" - os.environ[b'MOZ_OBJDIR'] = b'obj-firefox' - with NamedTemporaryFile(mode='w') as mozconfig: - mozconfig.write('mk_add_options MOZ_OBJDIR=/foo/bar\n') - mozconfig.flush() - - result = self.get_loader().read_mozconfig(mozconfig.name) - self.assertEqual(result['topobjdir'], '/foo/bar') - def test_read_moz_objdir_substitution(self): """Ensure @TOPSRCDIR@ substitution is recognized in MOZ_OBJDIR.""" with NamedTemporaryFile(mode='w') as mozconfig: @@ -356,10 +337,9 @@ class TestMozconfigLoader(unittest.TestCase): result = self.get_loader().read_mozconfig(mozconfig.name) - self.assertEqual(result['vars']['added'], { + self.assertEqual(result['env']['added'], { 'CC': '/usr/local/bin/clang', 'CXX': '/usr/local/bin/clang++'}) - self.assertEqual(result['env']['added'], {}) def test_read_exported_variables(self): """Exported variables are caught as new variables.""" @@ -369,7 +349,6 @@ class TestMozconfigLoader(unittest.TestCase): result = self.get_loader().read_mozconfig(mozconfig.name) - self.assertEqual(result['vars']['added'], {}) self.assertEqual(result['env']['added'], { 'MY_EXPORTED': 'woot'}) @@ -383,25 +362,10 @@ class TestMozconfigLoader(unittest.TestCase): result = self.get_loader().read_mozconfig(mozconfig.name) - self.assertEqual(result['vars']['modified'], {}) self.assertEqual(result['env']['modified'], { 'CC': ('/usr/bin/gcc', '/usr/local/bin/clang') }) - def test_read_unmodified_variables(self): - """Variables modified by mozconfig are detected.""" - os.environ[b'CC'] = b'/usr/bin/gcc' - - with NamedTemporaryFile(mode='w') as mozconfig: - mozconfig.flush() - - result = self.get_loader().read_mozconfig(mozconfig.name) - - self.assertEqual(result['vars']['unmodified'], {}) - self.assertEqual(result['env']['unmodified'], { - 'CC': '/usr/bin/gcc' - }) - def test_read_removed_variables(self): """Variables unset by the mozconfig are detected.""" os.environ[b'CC'] = b'/usr/bin/clang' @@ -412,7 +376,6 @@ class TestMozconfigLoader(unittest.TestCase): result = self.get_loader().read_mozconfig(mozconfig.name) - self.assertEqual(result['vars']['removed'], {}) self.assertEqual(result['env']['removed'], { 'CC': '/usr/bin/clang'}) @@ -425,11 +388,10 @@ class TestMozconfigLoader(unittest.TestCase): result = self.get_loader().read_mozconfig(mozconfig.name) - self.assertEqual(result['vars']['added'], { + self.assertEqual(result['env']['added'], { 'multi': 'foo\nbar', 'single': '1' }) - self.assertEqual(result['env']['added'], {}) def test_read_topsrcdir_defined(self): """Ensure $topsrcdir references work as expected.""" @@ -440,25 +402,19 @@ class TestMozconfigLoader(unittest.TestCase): loader = self.get_loader() result = loader.read_mozconfig(mozconfig.name) - self.assertEqual(result['vars']['added']['TEST'], + self.assertEqual(result['env']['added']['TEST'], loader.topsrcdir.replace(os.sep, '/')) - self.assertEqual(result['env']['added'], {}) def test_read_empty_variable_value(self): """Ensure empty variable values are parsed properly.""" with NamedTemporaryFile(mode='w') as mozconfig: mozconfig.write('EMPTY=\n') - mozconfig.write('export EXPORT_EMPTY=\n') mozconfig.flush() result = self.get_loader().read_mozconfig(mozconfig.name) - self.assertEqual(result['vars']['added'], { - 'EMPTY': '', - }) - self.assertEqual(result['env']['added'], { - 'EXPORT_EMPTY': '' - }) + self.assertIn('EMPTY', result['env']['added']) + self.assertEqual(result['env']['added']['EMPTY'], '') def test_read_load_exception(self): """Ensure non-0 exit codes in mozconfigs are handled properly.""" diff --git a/testing/xpcshell/selftest.py b/testing/xpcshell/selftest.py index 78e1aa9..df2ba24 100644 --- a/testing/xpcshell/selftest.py +++ b/testing/xpcshell/selftest.py @@ -12,7 +12,6 @@ from StringIO import StringIO from xml.etree.ElementTree import ElementTree from mozbuild.base import MozbuildObject -os.environ.pop('MOZ_OBJDIR') build_obj = MozbuildObject.from_environment() from runxpcshelltests import XPCShellTests [View Less]

1 0

[tor-browser-bundle/master] Adding gk's key for verifying tags, too.
by gk＠torproject.org 01 Dec '14

01 Dec '14

commit 08a199b9e868d02caff80a06f591d1a116f40483 Author: Georg Koppen <gk(a)torproject.org> Date: Mon Dec 1 09:18:32 2014 +0000 Adding gk's key for verifying tags, too. --- gitian/gpg/torbutton.gpg | Bin 15716 -> 44645 bytes 1 file changed, 0 insertions(+), 0 deletions(-) diff --git a/gitian/gpg/torbutton.gpg b/gitian/gpg/torbutton.gpg index 7f14ee3..ebc37d7 100644 Binary files a/gitian/gpg/torbutton.gpg and b/gitian/gpg/torbutton.gpg differ

1 0

[tor-browser-bundle/maint-4.0] Adding gk's key for verifying tags, too.
by gk＠torproject.org 01 Dec '14

01 Dec '14

commit 8e37ce5fba29567a91dc9d79b44187a4e400b8cf Author: Georg Koppen <gk(a)torproject.org> Date: Mon Dec 1 09:16:59 2014 +0000 Adding gk's key for verifying tags, too. --- gitian/gpg/torbutton.gpg | Bin 15716 -> 44645 bytes 1 file changed, 0 insertions(+), 0 deletions(-) diff --git a/gitian/gpg/torbutton.gpg b/gitian/gpg/torbutton.gpg index 7f14ee3..ebc37d7 100644 Binary files a/gitian/gpg/torbutton.gpg and b/gitian/gpg/torbutton.gpg differ

1 0