- tbb-commits - lists.torproject.org

[tor-browser-bundle/maint-4.0] Bumping tor-browser tag for build2.
by gk＠torproject.org 01 Dec '14

01 Dec '14

commit 8501e871a574856e962bc1732dfea05d206c3656 Author: Georg Koppen <gk(a)torproject.org> Date: Mon Dec 1 09:32:56 2014 +0000 Bumping tor-browser tag for build2. --- Bundle-Data/Docs/ChangeLog.txt | 1 + gitian/versions | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/Bundle-Data/Docs/ChangeLog.txt b/Bundle-Data/Docs/ChangeLog.txt index 93d20d7..50f7974 100644 --- a/Bundle-Data/Docs/ChangeLog.txt +++ b/Bundle-Data/Docs/ChangeLog.txt @@ -9,6 +9,7 @@ Tor Browser 4.0.2 -- Dec 2 2014 * Bug 13742: Fix domain isolation for content cache and disk-enabled browsing mode * Bug 5926: Prevent JS engine locale leaks (by setting the C library locale) * Bug 13504: Remove unreliable/unreachable non-public bridges + * Bug 13435: Remove our custom POODLE fix * Windows * Bug 13443: Re-enable DirectShow; fix crash with mingw patch. * Bug 13558: Fix crash on Windows XP during download folder changing diff --git a/gitian/versions b/gitian/versions index ad8fd74..6f9d97b 100755 --- a/gitian/versions +++ b/gitian/versions @@ -8,7 +8,7 @@ FIREFOX_VERSION=31.3.0esr TORBROWSER_UPDATE_CHANNEL=release -TORBROWSER_TAG=tor-browser-${FIREFOX_VERSION}-4.0-1-build1 +TORBROWSER_TAG=tor-browser-${FIREFOX_VERSION}-4.0-1-build2 TOR_TAG=tor-0.2.5.10 TORLAUNCHER_TAG=0.2.7.0.1 TORBUTTON_TAG=1.7.0.2

1 0

[tor-browser/tor-browser-31.3.0esr-4.0-1] Bug 13435: Remove our custom POODLE fix.
by gk＠torproject.org 01 Dec '14

01 Dec '14

commit 91ef6ad6972aac82b7ce539664a27fd471251733 Author: Georg Koppen <gk(a)torproject.org> Date: Mon Dec 1 09:20:53 2014 +0000 Bug 13435: Remove our custom POODLE fix. --- browser/app/profile/000-tor-browser.js | 2 -- 1 file changed, 2 deletions(-) diff --git a/browser/app/profile/000-tor-browser.js b/browser/app/profile/000-tor-browser.js index 43b894d..db85aa0 100644 --- a/browser/app/profile/000-tor-browser.js +++ b/browser/app/profile/000-tor-browser.js @@ -171,8 +171,6 @@ pref("media.audio_data.enabled", false); // Enable TLS 1.1 and 1.2: // https://trac.torproject.org/projects/tor/ticket/11253 pref("security.tls.version.max", 3); -// POODLE hotfix: Disable SSLv3 -pref("security.tls.version.min", 1); #ifdef TOR_BROWSER_VERSION #expand pref("torbrowser.version", __TOR_BROWSER_VERSION__);

1 0

[tor-browser/tor-browser-31.3.0esr-4.0-1] Revert "Bug 762358 - Re-run configure when mozconfig changed in a significant way. r=gps a=NPOTB"
by gk＠torproject.org 01 Dec '14

01 Dec '14

commit 811add0090f95de6e1fa3700fb55e8bc57802a52 Author: Georg Koppen <gk(a)torproject.org> Date: Mon Dec 1 08:59:35 2014 +0100 Revert "Bug 762358 - Re-run configure when mozconfig changed in a significant way. r=gps a=NPOTB" This reverts commit 87d09ba73a620931ffbe2576064aae55a961b097. It seems it breaks our deterministic build setup running |configure| again during |make build|. The former does not like being run under libfaketime but the latter needs to be in order to get reproducible builds. --- build/autoconf/altoptions.m4 | 43 +------ build/autoconf/mozconfig-find | 76 +++++++++++++ build/autoconf/mozconfig2client-mk | 76 +++++++++++++ build/autoconf/mozconfig2configure | 103 +++++++++++++++++ client.mk | 50 ++++++--- python/mozbuild/mozbuild/base.py | 38 ++----- python/mozbuild/mozbuild/mach_commands.py | 118 ++++---------------- python/mozbuild/mozbuild/mozconfig.py | 101 ++++++----------- python/mozbuild/mozbuild/mozconfig_loader | 8 -- python/mozbuild/mozbuild/test/backend/common.py | 8 -- .../mozbuild/test/frontend/test_emitter.py | 8 -- python/mozbuild/mozbuild/test/test_base.py | 75 +++---------- python/mozbuild/mozbuild/test/test_mozconfig.py | 56 +--------- testing/xpcshell/selftest.py | 1 - 14 files changed, 381 insertions(+), 380 deletions(-) diff --git a/build/autoconf/altoptions.m4 b/build/autoconf/altoptions.m4 index 6c40ad6..3dc8c6e 100644 --- a/build/autoconf/altoptions.m4 +++ b/build/autoconf/altoptions.m4 @@ -116,42 +116,7 @@ echo " dnl MOZ_READ_MYCONFIG() - Read in 'myconfig.sh' file AC_DEFUN([MOZ_READ_MOZCONFIG], [AC_REQUIRE([AC_INIT_BINSH])dnl -inserted= -dnl Shell is hard, so here is what the following does: -dnl - Reset $@ (command line arguments) -dnl - Add the configure options from mozconfig to $@ one by one -dnl - Add the original command line arguments after that, one by one -dnl -dnl There are several tricks involved: -dnl - It is not possible to preserve the whitespaces in $@ by assigning to -dnl another variable, so the two first steps above need to happen in the first -dnl iteration of the third step. -dnl - We always want the configure options to be added, so the loop must be -dnl iterated at least once, so we add a dummy argument first, and discard it. -dnl - something | while read line ... makes the while run in a subshell, meaning -dnl that anything it does is not propagated to the main shell, so we can't do -dnl set -- foo there. As a consequence, what the while loop reading mach -dnl environment output does is output a set of shell commands for the main shell -dnl to eval. -dnl - Extra care is due when lines from mach environment output contain special -dnl shell characters, so we use ' for quoting and ensure no ' end up in between -dnl the quoting mark unescaped. -dnl Some of the above is directly done in mach environment --format=configure. -failed_eval() { - echo "Failed eval'ing the following:" - $(dirname [$]0)/[$1]/mach environment --format=configure - exit 1 -} - -set -- dummy "[$]@" -for ac_option -do - if test -z "$inserted"; then - set -- - eval "$($(dirname [$]0)/[$1]/mach environment --format=configure)" || failed_eval - inserted=1 - else - set -- "[$]@" "$ac_option" - fi -done -]) +# Read in '.mozconfig' script to set the initial options. +# See the mozconfig2configure script for more details. +_AUTOCONF_TOOLS_DIR=`dirname [$]0`/[$1]/build/autoconf +. $_AUTOCONF_TOOLS_DIR/mozconfig2configure]) diff --git a/build/autoconf/mozconfig-find b/build/autoconf/mozconfig-find new file mode 100755 index 0000000..97dd90c --- /dev/null +++ b/build/autoconf/mozconfig-find @@ -0,0 +1,76 @@ +#! /bin/sh +# +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this +# file, You can obtain one at http://mozilla.org/MPL/2.0/. + +# mozconfigfind - Loads options from .mozconfig onto configure's +# command-line. The .mozconfig file is searched for in the +# order: +# If $MOZCONFIG is set, use that. +# If one of $TOPSRCDIR/.mozconfig or $TOPSRCDIR/mozconfig exists, use it. +# If both exist, or if various legacy locations contain a mozconfig, error. +# Otherwise, use the default build options. +# +topsrcdir=$1 + +abspath() { + if uname -s | grep -q MINGW; then + # We have no way to figure out whether we're in gmake or pymake right + # now. gmake gives us Unix-style paths while pymake gives us Windows-style + # paths, so attempt to handle both. + regexes='^$[A-Za-z]:\|\\\\\|\/$ ^\/' + else + regexes='^\/' + fi + + for regex in $regexes; do + if echo $1 | grep -q $regex; then + echo $1 + return + fi + done + + # If we're at this point, we have a relative path + echo `pwd`/$1 +} + +if [ -n "$MOZCONFIG" ] && ! [ -f "$MOZCONFIG" ]; then + echo "Specified MOZCONFIG \"$MOZCONFIG\" does not exist!" 1>&2 + exit 1 +fi + +if [ -n "$MOZ_MYCONFIG" ]; then + echo "Your environment currently has the MOZ_MYCONFIG variable set to \"$MOZ_MYCONFIG\". MOZ_MYCONFIG is no longer supported. Please use MOZCONFIG instead." 1>&2 + exit 1 +fi + +if [ -z "$MOZCONFIG" ] && [ -f "$topsrcdir/.mozconfig" ] && [ -f "$topsrcdir/mozconfig" ]; then + echo "Both \$topsrcdir/.mozconfig and \$topsrcdir/mozconfig are supported, but you must choose only one. Please remove the other." 1>&2 + exit 1 +fi + +for _config in "$MOZCONFIG" \ + "$topsrcdir/.mozconfig" \ + "$topsrcdir/mozconfig" +do + if test -f "$_config"; then + abspath $_config + exit 0 + fi +done + +# We used to support a number of other implicit .mozconfig locations. We now +# detect if we were about to use any of these locations and issue an error if we +# find any. +for _config in "$topsrcdir/mozconfig.sh" \ + "$topsrcdir/myconfig.sh" \ + "$HOME/.mozconfig" \ + "$HOME/.mozconfig.sh" \ + "$HOME/.mozmyconfig.sh" +do + if test -f "$_config"; then + echo "You currently have a mozconfig at \"$_config\". This implicit location is no longer supported. Please move it to $topsrcdir/.mozconfig or specify it explicitly via \$MOZCONFIG." 1>&2 + exit 1 + fi +done diff --git a/build/autoconf/mozconfig2client-mk b/build/autoconf/mozconfig2client-mk new file mode 100755 index 0000000..aaf8de1 --- /dev/null +++ b/build/autoconf/mozconfig2client-mk @@ -0,0 +1,76 @@ +#! /bin/sh +# +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this +# file, You can obtain one at http://mozilla.org/MPL/2.0/. + +# mozconfig2client-mk - Translates .mozconfig into options for client.mk. +# Prints defines to stdout. +# +# See mozconfig2configure for more details + +print_header() { + cat <<EOF +# gmake +# This file is automatically generated for client.mk. +# Do not edit. Edit $FOUND_MOZCONFIG instead. + +EOF +} + +ac_add_options() { + for _opt + do + case "$_opt" in + --target=*) + echo $_opt | sed s/--target/CONFIG_GUESS/ + ;; + *) + echo "# $_opt is used by configure (not client.mk)" + ;; + esac + done +} + +ac_add_app_options() { + echo "# $* is used by configure (not client.mk)" +} + +mk_add_options() { + for _opt + do + # Escape shell characters, space, tab, dollar, quote, backslash, + # and substitute '@<word>@' with '$(<word>)'. + _opt=`echo "$_opt" | sed -e 's/$[\"\\]$/\\\\\1/g; s/@$[^@]*$@/\$(\1)/g;'` + echo $_opt; + done +} + +# Main +#-------------------------------------------------- + +scriptdir=`dirname $0` +topsrcdir=$1 + +# If the path changes, configure should be rerun +echo "# PATH=$PATH" + +# If FOUND_MOZCONFIG isn't set, look for it and make sure the script doesn't error out +isfoundset=${FOUND_MOZCONFIG+yes} +if [ -z $isfoundset ]; then + FOUND_MOZCONFIG=`$scriptdir/mozconfig-find $topsrcdir` + if [ $? -ne 0 ]; then + echo '$(error Fix above errors before continuing.)' + else + isfoundset=yes + fi +fi + +if [ -n $isfoundset ]; then + if [ "$FOUND_MOZCONFIG" ] + then + print_header + . "$FOUND_MOZCONFIG" + echo "FOUND_MOZCONFIG := $FOUND_MOZCONFIG" + fi +fi diff --git a/build/autoconf/mozconfig2configure b/build/autoconf/mozconfig2configure new file mode 100755 index 0000000..99623b6 --- /dev/null +++ b/build/autoconf/mozconfig2configure @@ -0,0 +1,103 @@ +#! /bin/sh +# +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this +# file, You can obtain one at http://mozilla.org/MPL/2.0/. + +# mozconfig2configure - Loads options from .mozconfig onto configure's +# command-line. See mozconfig-find for how the config file is +# found +# +# The options from .mozconfig are inserted into the command-line +# before the real command-line options. This way the real options +# will override any .mozconfig options. +# +# .mozconfig is a shell script. To add an option to configure's +# command-line use the pre-defined function, ac_add_options, +# +# ac_add_options <configure-option> [<configure-option> ... ] +# +# For example, +# +# ac_add_options --with-pthreads --enable-debug +# +# ac_add_options can be called multiple times in .mozconfig. +# Each call adds more options to configure's command-line. + +# Note: $_AUTOCONF_TOOLS_DIR must be defined in the script that includes this. + +ac_add_options() { + for _opt + do + # Escape shell characters, space, tab, dollar, quote, backslash, parentheses. + _opt=`echo $_opt | sed -e 's/$[\ \ \$\"\\\($]\)/\\\\\1/g;s/@$[^@]*$@/\$\1/g;'` + _opt=`echo $_opt | sed -e 's/@$[^@]*$@/\$(\1)/g'` + + # Avoid adding duplicates + case "$ac_options" in + # Note that all options in $ac_options are enclosed in quotes, + # so there will always be a last character to match [^-A-Za-z0-9_] + *"\"$_opt[^-A-Za-z0-9_]"* ) ;; + * ) mozconfig_ac_options="$mozconfig_ac_options $_opt" ;; + esac + done +} + +ac_add_app_options() { + APP=$1 + shift; + if [ "$APP" = "$MOZ_BUILD_APP" ]; then + ac_add_options "$*"; + fi +} + +mk_add_options() { + # These options are for client.mk + # configure can safely ignore them. + : +} + +ac_echo_options() { + echo "Adding configure options from $FOUND_MOZCONFIG:" + eval "set -- $mozconfig_ac_options" + for _opt + do + echo " $_opt" + done +} + +# Main +#-------------------------------------------------- +topsrcdir=$(cd `dirname $0`; pwd -W 2>/dev/null || pwd) +ac_options= +mozconfig_ac_options= + +# Save the real command-line options +for _opt +do + # Escape shell characters, space, tab, dollar, quote, backslash. + _opt=`echo $_opt | sed -e 's/$[\ \ \$\"\\]$/\\\\\1/g;'` + ac_options="$ac_options \"$_opt\"" +done + + +# If FOUND_MOZCONFIG isn't set, look for it and make sure the script doesn't error out +isfoundset=${FOUND_MOZCONFIG+yes} +if [ -z $isfoundset ]; then + FOUND_MOZCONFIG=`$_AUTOCONF_TOOLS_DIR/mozconfig-find $topsrcdir` + if [ $? -ne 0 ]; then + echo "Fix above errors before continuing." 1>&2 + exit 1 + fi +fi + +if [ "$FOUND_MOZCONFIG" ]; then + . "$FOUND_MOZCONFIG" +fi +export FOUND_MOZCONFIG + +if [ "$mozconfig_ac_options" ]; then + ac_echo_options 1>&2 +fi + +eval "set -- $mozconfig_ac_options $ac_options" diff --git a/client.mk b/client.mk index 5cbaf6b..f737c5b 100644 --- a/client.mk +++ b/client.mk @@ -50,6 +50,7 @@ endif ifndef TOPSRCDIR ifeq (,$(wildcard client.mk)) TOPSRCDIR := $(patsubst %/,%,$(dir $(MAKEFILE_LIST))) +MOZ_OBJDIR = . else TOPSRCDIR := $(CWD) endif @@ -98,6 +99,8 @@ endif # See build pages, http://www.mozilla.org/build/ for how to set up mozconfig. +MOZCONFIG_LOADER := build/autoconf/mozconfig2client-mk + define CR @@ -108,9 +111,7 @@ endef # before evaluation. $(shell) replacing newlines with spaces, || is always # followed by a space (since sed doesn't remove newlines), except on the # last line, so replace both '|| ' and '||'. -# Also, make MOZ_PGO available to mozconfig when passed on make command line. -# Likewise for MOZ_CURRENT_PROJECT. -MOZCONFIG_CONTENT := $(subst ||,$(CR),$(subst || ,$(CR),$(shell $(addprefix MOZ_CURRENT_PROJECT=,$(MOZ_CURRENT_PROJECT)) MOZ_PGO=$(MOZ_PGO) $(TOPSRCDIR)/mach environment --format=client.mk | sed 's/$$/||/'))) +MOZCONFIG_CONTENT := $(subst ||,$(CR),$(subst || ,$(CR),$(shell $(TOPSRCDIR)/$(MOZCONFIG_LOADER) $(TOPSRCDIR) | sed 's/$$/||/'))) $(eval $(MOZCONFIG_CONTENT)) export FOUND_MOZCONFIG @@ -141,18 +142,35 @@ ifeq (,$(findstring -j,$(MOZ_MAKE_FLAGS))) endif +ifndef MOZ_OBJDIR + MOZ_OBJDIR = obj-$(CONFIG_GUESS) +else +# On Windows Pymake builds check MOZ_OBJDIR doesn't start with "/" + ifneq (,$(findstring mingw,$(CONFIG_GUESS))) + ifeq (1_a,$(.PYMAKE)_$(firstword a$(subst /, ,$(MOZ_OBJDIR)))) + $(error For Windows Pymake builds, MOZ_OBJDIR must be a Windows [and not MSYS] style path.) + endif + endif +endif + ifdef MOZ_BUILD_PROJECTS ifdef MOZ_CURRENT_PROJECT + OBJDIR = $(MOZ_OBJDIR)/$(MOZ_CURRENT_PROJECT) + MOZ_MAKE = $(MAKE) $(MOZ_MAKE_FLAGS) -C $(OBJDIR) BUILD_PROJECT_ARG = MOZ_BUILD_APP=$(MOZ_CURRENT_PROJECT) - export MOZ_CURRENT_PROJECT else + OBJDIR = $(error Cannot find the OBJDIR when MOZ_CURRENT_PROJECT is not set.) MOZ_MAKE = $(error Cannot build in the OBJDIR when MOZ_CURRENT_PROJECT is not set.) endif -endif # MOZ_BUILD_PROJECTS +else # MOZ_BUILD_PROJECTS + +OBJDIR = $(MOZ_OBJDIR) MOZ_MAKE = $(MAKE) $(MOZ_MAKE_FLAGS) -C $(OBJDIR) +endif # MOZ_BUILD_PROJECTS + # 'configure' scripts generated by autoconf. CONFIGURES := $(TOPSRCDIR)/configure CONFIGURES += $(TOPSRCDIR)/js/src/configure @@ -184,7 +202,7 @@ WANT_MOZCONFIG_MK = 1 endif ifdef WANT_MOZCONFIG_MK -# For now, only output "export" lines from mach environment --format=client.mk output. +# For now, only output "export" lines from mozconfig2client-mk output. MOZCONFIG_MK_LINES := $(filter export||%,$(MOZCONFIG_OUT_LINES)) $(OBJDIR)/.mozconfig.mk: $(FOUND_MOZCONFIG) $(call mkdir_deps,$(OBJDIR)) $(OBJDIR)/CLOBBER $(if $(MOZCONFIG_MK_LINES),( $(foreach line,$(MOZCONFIG_MK_LINES), echo '$(subst ||, ,$(line))';) )) > $@ @@ -217,11 +235,17 @@ everything: clean build # This is up here, outside of the MOZ_CURRENT_PROJECT logic so that this # is usable in multi-pass builds, where you might not have a runnable # application until all the build passes and postflight scripts have run. +ifdef MOZ_OBJDIR + PGO_OBJDIR = $(MOZ_OBJDIR) +else + PGO_OBJDIR := $(TOPSRCDIR) +endif + profiledbuild:: $(MAKE) -f $(TOPSRCDIR)/client.mk realbuild MOZ_PROFILE_GENERATE=1 MOZ_PGO_INSTRUMENTED=1 - $(MAKE) -C $(OBJDIR) package MOZ_PGO_INSTRUMENTED=1 MOZ_INTERNAL_SIGNING_FORMAT= MOZ_EXTERNAL_SIGNING_FORMAT= - rm -f $(OBJDIR)/jarlog/en-US.log - MOZ_PGO_INSTRUMENTED=1 JARLOG_FILE=jarlog/en-US.log EXTRA_TEST_ARGS=10 $(MAKE) -C $(OBJDIR) pgo-profile-run + $(MAKE) -C $(PGO_OBJDIR) package MOZ_PGO_INSTRUMENTED=1 MOZ_INTERNAL_SIGNING_FORMAT= MOZ_EXTERNAL_SIGNING_FORMAT= + rm -f ${PGO_OBJDIR}/jarlog/en-US.log + MOZ_PGO_INSTRUMENTED=1 JARLOG_FILE=jarlog/en-US.log EXTRA_TEST_ARGS=10 $(MAKE) -C $(PGO_OBJDIR) pgo-profile-run $(MAKE) -f $(TOPSRCDIR)/client.mk maybe_clobber_profiledbuild $(MAKE) -f $(TOPSRCDIR)/client.mk realbuild MOZ_PROFILE_USE=1 @@ -302,7 +326,6 @@ CONFIG_STATUS_DEPS := \ $(TOPSRCDIR)/build/virtualenv_packages.txt \ $(TOPSRCDIR)/python/mozbuild/mozbuild/virtualenv.py \ $(TOPSRCDIR)/testing/mozbase/packages.txt \ - $(OBJDIR)/.mozconfig.json \ $(NULL) CONFIGURE_ENV_ARGS += \ @@ -330,13 +353,8 @@ configure-preqs = \ $(call mkdir_deps,$(OBJDIR)) \ $(if $(MOZ_BUILD_PROJECTS),$(call mkdir_deps,$(MOZ_OBJDIR))) \ save-mozconfig \ - $(OBJDIR)/.mozconfig.json \ $(NULL) -CREATE_MOZCONFIG_JSON := $(shell $(TOPSRCDIR)/mach environment --format=json -o $(OBJDIR)/.mozconfig.json) -$(OBJDIR)/.mozconfig.json: $(call mkdir_deps,$(OBJDIR)) - @$(TOPSRCDIR)/mach environment --format=json -o $@ - save-mozconfig: $(FOUND_MOZCONFIG) -cp $(FOUND_MOZCONFIG) $(OBJDIR)/.mozconfig @@ -355,7 +373,7 @@ $(OBJDIR)/config.status: $(CONFIG_STATUS_DEPS) else $(OBJDIR)/Makefile: $(CONFIG_STATUS_DEPS) endif - @$(MAKE) -f $(TOPSRCDIR)/client.mk configure CREATE_MOZCONFIG_JSON= + @$(MAKE) -f $(TOPSRCDIR)/client.mk configure ifneq (,$(CONFIG_STATUS)) $(OBJDIR)/config/autoconf.mk: $(TOPSRCDIR)/config/autoconf.mk.in diff --git a/python/mozbuild/mozbuild/base.py b/python/mozbuild/mozbuild/base.py index 14c9972..d00d987 100644 --- a/python/mozbuild/mozbuild/base.py +++ b/python/mozbuild/mozbuild/base.py @@ -161,8 +161,7 @@ class MozbuildObject(ProcessExecutionMixin): # environment. If no mozconfig is present, the config will not have # much defined. loader = MozconfigLoader(topsrcdir) - current_project = os.environ.get('MOZ_CURRENT_PROJECT') - config = loader.read_mozconfig(mozconfig, moz_build_app=current_project) + config = loader.read_mozconfig(mozconfig) config_topobjdir = MozbuildObject.resolve_mozconfig_topobjdir( topsrcdir, config) @@ -172,30 +171,13 @@ class MozbuildObject(ProcessExecutionMixin): # inside an objdir you probably want to perform actions on that objdir, # not another one. This prevents accidental usage of the wrong objdir # when the current objdir is ambiguous. - # However, if the found mozconfig resolves to another objdir that - # doesn't exist, we may be in a subtree like when building mozilla/ - # under c-c, and the objdir was defined as a relative path. Try again - # adjusting for that. - if topobjdir and config_topobjdir: - if not os.path.exists(config_topobjdir): - config_topobjdir = MozbuildObject.resolve_mozconfig_topobjdir( - os.path.dirname(topsrcdir), config) - if current_project: - config_topobjdir = os.path.join(config_topobjdir, - current_project) - config_topobjdir = os.path.join(config_topobjdir, - os.path.basename(topsrcdir)) - elif current_project: - config_topobjdir = os.path.join(config_topobjdir, current_project) - - _config_topobjdir = config_topobjdir - mozilla_dir = os.path.join(_config_topobjdir, 'mozilla') - if not samepath(topobjdir, _config_topobjdir) \ - and (not os.path.exists(mozilla_dir) or not samepath(topobjdir, + mozilla_dir = os.path.join(config_topobjdir, 'mozilla') + if not samepath(topobjdir, config_topobjdir) \ + and (os.path.exists(mozilla_dir) and not samepath(topobjdir, mozilla_dir)): - raise ObjdirMismatchException(topobjdir, _config_topobjdir) + raise ObjdirMismatchException(topobjdir, config_topobjdir) topobjdir = topobjdir or config_topobjdir if topobjdir: @@ -251,8 +233,7 @@ class MozbuildObject(ProcessExecutionMixin): """ if self._mozconfig is None: loader = MozconfigLoader(self.topsrcdir) - self._mozconfig = loader.read_mozconfig( - moz_build_app=os.environ.get('MOZ_CURRENT_PROJECT')) + self._mozconfig = loader.read_mozconfig() return self._mozconfig @@ -567,13 +548,8 @@ class MachCommandBase(MozbuildObject): # more reliable than mozconfig when cwd is inside an objdir. topsrcdir = context.topdir topobjdir = None - detect_virtualenv_mozinfo = True - if hasattr(context, 'detect_virtualenv_mozinfo'): - detect_virtualenv_mozinfo = getattr(context, - 'detect_virtualenv_mozinfo') try: - dummy = MozbuildObject.from_environment(cwd=context.cwd, - detect_virtualenv_mozinfo=detect_virtualenv_mozinfo) + dummy = MozbuildObject.from_environment(cwd=context.cwd) topsrcdir = dummy.topsrcdir topobjdir = dummy._topobjdir except BuildEnvironmentNotFoundException: diff --git a/python/mozbuild/mozbuild/mach_commands.py b/python/mozbuild/mozbuild/mach_commands.py index 0b19815..0bfe4a6 100644 --- a/python/mozbuild/mozbuild/mach_commands.py +++ b/python/mozbuild/mozbuild/mach_commands.py @@ -10,8 +10,6 @@ import operator import os import sys -import mozpack.path as mozpath - from mach.decorators import ( CommandArgument, CommandProvider, @@ -919,53 +917,37 @@ class Makefiles(MachCommandBase): class MachDebug(MachCommandBase): @Command('environment', category='build-dev', description='Show info about the mach and build environment.') - @CommandArgument('--format', default='pretty', - choices=['pretty', 'client.mk', 'configure', 'json'], - help='Print data in the given format.') - @CommandArgument('--output', '-o', type=str, - help='Output to the given file.') @CommandArgument('--verbose', '-v', action='store_true', help='Print verbose output.') - def environment(self, format, output=None, verbose=False): - func = getattr(self, '_environment_%s' % format.replace('.', '_')) - - if output: - # We want to preserve mtimes if the output file already exists - # and the content hasn't changed. - from mozbuild.util import FileAvoidWrite - with FileAvoidWrite(output) as out: - return func(out, verbose) - return func(sys.stdout, verbose) - - def _environment_pretty(self, out, verbose): + def environment(self, verbose=False): state_dir = self._mach_context.state_dir import platform - print('platform:\n\t%s' % platform.platform(), file=out) - print('python version:\n\t%s' % sys.version, file=out) - print('python prefix:\n\t%s' % sys.prefix, file=out) - print('mach cwd:\n\t%s' % self._mach_context.cwd, file=out) - print('os cwd:\n\t%s' % os.getcwd(), file=out) - print('mach directory:\n\t%s' % self._mach_context.topdir, file=out) - print('state directory:\n\t%s' % state_dir, file=out) + print('platform:\n\t%s' % platform.platform()) + print('python version:\n\t%s' % sys.version) + print('python prefix:\n\t%s' % sys.prefix) + print('mach cwd:\n\t%s' % self._mach_context.cwd) + print('os cwd:\n\t%s' % os.getcwd()) + print('mach directory:\n\t%s' % self._mach_context.topdir) + print('state directory:\n\t%s' % state_dir) - print('object directory:\n\t%s' % self.topobjdir, file=out) + print('object directory:\n\t%s' % self.topobjdir) if self.mozconfig['path']: - print('mozconfig path:\n\t%s' % self.mozconfig['path'], file=out) + print('mozconfig path:\n\t%s' % self.mozconfig['path']) if self.mozconfig['configure_args']: - print('mozconfig configure args:', file=out) + print('mozconfig configure args:') for arg in self.mozconfig['configure_args']: - print('\t%s' % arg, file=out) + print('\t%s' % arg) if self.mozconfig['make_extra']: - print('mozconfig extra make args:', file=out) + print('mozconfig extra make args:') for arg in self.mozconfig['make_extra']: - print('\t%s' % arg, file=out) + print('\t%s' % arg) if self.mozconfig['make_flags']: - print('mozconfig make flags:', file=out) + print('mozconfig make flags:') for arg in self.mozconfig['make_flags']: - print('\t%s' % arg, file=out) + print('\t%s' % arg) config = None @@ -976,70 +958,14 @@ class MachDebug(MachCommandBase): pass if config: - print('config topsrcdir:\n\t%s' % config.topsrcdir, file=out) - print('config topobjdir:\n\t%s' % config.topobjdir, file=out) + print('config topsrcdir:\n\t%s' % config.topsrcdir) + print('config topobjdir:\n\t%s' % config.topobjdir) if verbose: - print('config substitutions:', file=out) + print('config substitutions:') for k in sorted(config.substs): - print('\t%s: %s' % (k, config.substs[k]), file=out) + print('\t%s: %s' % (k, config.substs[k])) - print('config defines:', file=out) + print('config defines:') for k in sorted(config.defines): - print('\t%s' % k, file=out) - - def _environment_client_mk(self, out, verbose): - if self.mozconfig['make_extra']: - for arg in self.mozconfig['make_extra']: - print(arg, file=out) - objdir = mozpath.normsep(self.topobjdir) - print('MOZ_OBJDIR=%s' % objdir, file=out) - if 'MOZ_CURRENT_PROJECT' in os.environ: - objdir = mozpath.join(objdir, os.environ['MOZ_CURRENT_PROJECT']) - print('OBJDIR=%s' % objdir, file=out) - if self.mozconfig['path']: - print('FOUND_MOZCONFIG=%s' % mozpath.normsep(self.mozconfig['path']), - file=out) - - def _environment_configure(self, out, verbose): - if self.mozconfig['path']: - # Replace ' with '"'"', so that shell quoting e.g. - # a'b becomes 'a'"'"'b'. - quote = lambda s: s.replace("'", """'"'"'""") - print('echo Adding configure options from %s' % - mozpath.normsep(self.mozconfig['path']), file=out) - if self.mozconfig['configure_args']: - for arg in self.mozconfig['configure_args']: - quoted_arg = quote(arg) - print("echo ' %s'" % quoted_arg, file=out) - print("""set -- "$@" '%s'""" % quoted_arg, file=out) - for key, value in self.mozconfig['env']['added'].items(): - print("export %s='%s'" % (key, quote(value)), file=out) - for key, (old, value) in self.mozconfig['env']['modified'].items(): - print("export %s='%s'" % (key, quote(value)), file=out) - for key, value in self.mozconfig['vars']['added'].items(): - print("%s='%s'" % (key, quote(value)), file=out) - for key, (old, value) in self.mozconfig['vars']['modified'].items(): - print("%s='%s'" % (key, quote(value)), file=out) - for key in self.mozconfig['env']['removed'].keys() + \ - self.mozconfig['vars']['removed'].keys(): - print("unset %s" % key, file=out) - - def _environment_json(self, out, verbose): - import json - class EnvironmentEncoder(json.JSONEncoder): - def default(self, obj): - if isinstance(obj, MozbuildObject): - result = { - 'topsrcdir': obj.topsrcdir, - 'topobjdir': obj.topobjdir, - 'mozconfig': obj.mozconfig, - } - if verbose: - result['substs'] = obj.substs - result['defines'] = obj.defines - return result - elif isinstance(obj, set): - return list(obj) - return json.JSONEncoder.default(self, obj) - json.dump(self, cls=EnvironmentEncoder, sort_keys=True, fp=out) + print('\t%s' % k) diff --git a/python/mozbuild/mozbuild/mozconfig.py b/python/mozbuild/mozbuild/mozconfig.py index 9feca70..3c1e3eb 100644 --- a/python/mozbuild/mozbuild/mozconfig.py +++ b/python/mozbuild/mozbuild/mozconfig.py @@ -65,11 +65,7 @@ class MozconfigLoader(ProcessExecutionMixin): DEPRECATED_TOPSRCDIR_PATHS = ('mozconfig.sh', 'myconfig.sh') DEPRECATED_HOME_PATHS = ('.mozconfig', '.mozconfig.sh', '.mozmyconfig.sh') - IGNORE_SHELL_VARIABLES = {'_'} - - ENVIRONMENT_VARIABLES = { - 'CC', 'CXX', 'CFLAGS', 'CXXFLAGS', 'LDFLAGS', 'MOZ_OBJDIR', - } + IGNORE_SHELL_VARIABLES = ('_') def __init__(self, topsrcdir): self.topsrcdir = topsrcdir @@ -200,7 +196,6 @@ class MozconfigLoader(ProcessExecutionMixin): 'make_flags': None, 'make_extra': None, 'env': None, - 'vars': None, } if path is None: @@ -236,49 +231,36 @@ class MozconfigLoader(ProcessExecutionMixin): parsed = self._parse_loader_output(output) - def diff_vars(vars_before, vars_after): - set1 = set(vars_before.keys()) - self.IGNORE_SHELL_VARIABLES - set2 = set(vars_after.keys()) - self.IGNORE_SHELL_VARIABLES - added = set2 - set1 - removed = set1 - set2 - maybe_modified = set1 & set2 - changed = { - 'added': {}, - 'removed': {}, - 'modified': {}, - 'unmodified': {}, - } - - for key in added: - changed['added'][key] = vars_after[key] - - for key in removed: - changed['removed'][key] = vars_before[key] - - for key in maybe_modified: - if vars_before[key] != vars_after[key]: - changed['modified'][key] = ( - vars_before[key], vars_after[key]) - elif key in self.ENVIRONMENT_VARIABLES: - # In order for irrelevant environment variable changes not - # to incur in re-running configure, only a set of - # environment variables are stored when they are - # unmodified. Otherwise, changes such as using a different - # terminal window, or even rebooting, would trigger - # reconfigures. - changed['unmodified'][key] = vars_after[key] - - return changed - - result['env'] = diff_vars(parsed['env_before'], parsed['env_after']) - - # Environment variables also appear as shell variables, but that's - # uninteresting duplication of information. Filter them out. - filt = lambda x, y: {k: v for k, v in x.items() if k not in y} - result['vars'] = diff_vars( - filt(parsed['vars_before'], parsed['env_before']), - filt(parsed['vars_after'], parsed['env_after']) - ) + all_variables = set(parsed['vars_before'].keys()) + all_variables |= set(parsed['vars_after'].keys()) + + changed = { + 'added': {}, + 'removed': {}, + 'modified': {}, + 'unmodified': {}, + } + + for key in all_variables: + if key in self.IGNORE_SHELL_VARIABLES: + continue + + if key not in parsed['vars_before']: + changed['added'][key] = parsed['vars_after'][key] + continue + + if key not in parsed['vars_after']: + changed['removed'][key] = parsed['vars_before'][key] + continue + + if parsed['vars_before'][key] != parsed['vars_after'][key]: + changed['modified'][key] = ( + parsed['vars_before'][key], parsed['vars_after'][key]) + continue + + changed['unmodified'][key] = parsed['vars_after'][key] + + result['env'] = changed result['configure_args'] = [self._expand(o) for o in parsed['ac']] @@ -286,9 +268,6 @@ class MozconfigLoader(ProcessExecutionMixin): result['configure_args'].extend(self._expand(o) for o in parsed['ac_app'][moz_build_app]) - if 'MOZ_OBJDIR' in parsed['env_before']: - result['topobjdir'] = parsed['env_before']['MOZ_OBJDIR'] - mk = [self._expand(o) for o in parsed['mk']] for o in mk: @@ -318,8 +297,6 @@ class MozconfigLoader(ProcessExecutionMixin): ac_app_options = defaultdict(list) before_source = {} after_source = {} - env_before_source = {} - env_after_source = {} current = None current_type = None @@ -362,14 +339,7 @@ class MozconfigLoader(ProcessExecutionMixin): assert current_type is not None - vars_mapping = { - 'BEFORE_SOURCE': before_source, - 'AFTER_SOURCE': after_source, - 'ENV_BEFORE_SOURCE': env_before_source, - 'ENV_AFTER_SOURCE': env_after_source, - } - - if current_type in vars_mapping: + if current_type in ('BEFORE_SOURCE', 'AFTER_SOURCE'): # mozconfigs are sourced using the Bourne shell (or at least # in Bourne shell mode). This means |set| simply lists # variables from the current shell (not functions). (Note that @@ -430,7 +400,10 @@ class MozconfigLoader(ProcessExecutionMixin): assert name is not None - vars_mapping[current_type][name] = value + if current_type == 'BEFORE_SOURCE': + before_source[name] = value + else: + after_source[name] = value current = [] @@ -444,8 +417,6 @@ class MozconfigLoader(ProcessExecutionMixin): 'ac_app': ac_app_options, 'vars_before': before_source, 'vars_after': after_source, - 'env_before': env_before_source, - 'env_after': env_after_source, } def _expand(self, s): diff --git a/python/mozbuild/mozbuild/mozconfig_loader b/python/mozbuild/mozbuild/mozconfig_loader index 569c690..d12c7fa 100755 --- a/python/mozbuild/mozbuild/mozconfig_loader +++ b/python/mozbuild/mozbuild/mozconfig_loader @@ -44,10 +44,6 @@ mk_add_options() { done } -echo "------BEGIN_ENV_BEFORE_SOURCE" -env -echo "------END_ENV_BEFORE_SOURCE" - echo "------BEGIN_BEFORE_SOURCE" set echo "------END_BEFORE_SOURCE" @@ -62,7 +58,3 @@ echo "------BEGIN_AFTER_SOURCE" set echo "------END_AFTER_SOURCE" -echo "------BEGIN_ENV_AFTER_SOURCE" -env -echo "------END_ENV_AFTER_SOURCE" - diff --git a/python/mozbuild/mozbuild/test/backend/common.py b/python/mozbuild/mozbuild/test/backend/common.py index b6e1566..7cfec3c 100644 --- a/python/mozbuild/mozbuild/test/backend/common.py +++ b/python/mozbuild/mozbuild/test/backend/common.py @@ -83,14 +83,6 @@ CONFIGS = DefaultOnReadDict({ class BackendTester(unittest.TestCase): - def setUp(self): - self._old_env = dict(os.environ) - os.environ.pop('MOZ_OBJDIR', None) - - def tearDown(self): - os.environ.clear() - os.environ.update(self._old_env) - def _get_environment(self, name): """Obtain a new instance of a ConfigEnvironment for a known profile. diff --git a/python/mozbuild/mozbuild/test/frontend/test_emitter.py b/python/mozbuild/mozbuild/test/frontend/test_emitter.py index 9e5fef1..25b69d4 100644 --- a/python/mozbuild/mozbuild/test/frontend/test_emitter.py +++ b/python/mozbuild/mozbuild/test/frontend/test_emitter.py @@ -41,14 +41,6 @@ data_path = mozpath.join(data_path, 'data') class TestEmitterBasic(unittest.TestCase): - def setUp(self): - self._old_env = dict(os.environ) - os.environ.pop('MOZ_OBJDIR', None) - - def tearDown(self): - os.environ.clear() - os.environ.update(self._old_env) - def reader(self, name): config = MockConfig(mozpath.join(data_path, name), extra_substs=dict( ENABLE_TESTS='1', diff --git a/python/mozbuild/mozbuild/test/test_base.py b/python/mozbuild/mozbuild/test/test_base.py index 3735213..24118cd 100644 --- a/python/mozbuild/mozbuild/test/test_base.py +++ b/python/mozbuild/mozbuild/test/test_base.py @@ -22,15 +22,14 @@ from mozbuild.base import ( BadEnvironmentException, MachCommandBase, MozbuildObject, - ObjdirMismatchException, PathArgument, ) from mozbuild.backend.configenvironment import ConfigEnvironment -from buildconfig import topsrcdir, topobjdir curdir = os.path.dirname(__file__) +topsrcdir = os.path.abspath(os.path.join(curdir, '..', '..', '..', '..')) log_manager = LoggingManager() @@ -39,15 +38,14 @@ class TestMozbuildObject(unittest.TestCase): self._old_cwd = os.getcwd() self._old_env = dict(os.environ) os.environ.pop('MOZCONFIG', None) - os.environ.pop('MOZ_OBJDIR', None) def tearDown(self): os.chdir(self._old_cwd) os.environ.clear() os.environ.update(self._old_env) - def get_base(self, topobjdir=None): - return MozbuildObject(topsrcdir, None, log_manager, topobjdir=topobjdir) + def get_base(self): + return MozbuildObject(topsrcdir, None, log_manager) def test_objdir_config_guess(self): base = self.get_base() @@ -74,6 +72,7 @@ class TestMozbuildObject(unittest.TestCase): 'foo')) self.assertTrue(base.topobjdir.endswith('foo')) + @unittest.skip('Failing on buildbot.') def test_objdir_config_status(self): """Ensure @CONFIG_GUESS@ is handled when loading mozconfig.""" base = self.get_base() @@ -107,17 +106,16 @@ class TestMozbuildObject(unittest.TestCase): mozconfig=mozconfig, ), fh) - os.environ[b'MOZCONFIG'] = mozconfig.encode('utf-8') + os.environ[b'MOZCONFIG'] = mozconfig os.chdir(topobjdir) - obj = MozbuildObject.from_environment( - detect_virtualenv_mozinfo=False) + obj = MozbuildObject.from_environment() self.assertEqual(obj.topobjdir, topobjdir) finally: - os.chdir(self._old_cwd) shutil.rmtree(d) + @unittest.skip('Failing on buildbot.') def test_relative_objdir(self): """Relative defined objdirs are loaded properly.""" d = os.path.realpath(tempfile.mkdtemp()) @@ -136,18 +134,16 @@ class TestMozbuildObject(unittest.TestCase): mozconfig=mozconfig, ), fh) - os.environ[b'MOZCONFIG'] = mozconfig.encode('utf-8') + os.environ[b'MOZCONFIG'] = mozconfig child = os.path.join(topobjdir, 'foo', 'bar') os.makedirs(child) os.chdir(child) - obj = MozbuildObject.from_environment( - detect_virtualenv_mozinfo=False) + obj = MozbuildObject.from_environment() self.assertEqual(obj.topobjdir, topobjdir) finally: - os.chdir(self._old_cwd) shutil.rmtree(d) @unittest.skipIf(not hasattr(os, 'symlink'), 'symlinks not available.') @@ -181,9 +177,9 @@ class TestMozbuildObject(unittest.TestCase): self.assertEqual(obj.topobjdir, topobjdir_real) finally: - os.chdir(self._old_cwd) shutil.rmtree(d) + @unittest.skip('Failed on buildbot (bug 853954)') def test_mach_command_base_inside_objdir(self): """Ensure a MachCommandBase constructed from inside the objdir works.""" @@ -212,7 +208,6 @@ class TestMozbuildObject(unittest.TestCase): context.topdir = topsrcdir context.settings = None context.log_manager = None - context.detect_virtualenv_mozinfo=False o = MachCommandBase(context) @@ -220,9 +215,9 @@ class TestMozbuildObject(unittest.TestCase): self.assertEqual(o.topsrcdir, topsrcdir) finally: - os.chdir(self._old_cwd) shutil.rmtree(d) + @unittest.skip('Failing on buildbot.') def test_objdir_is_srcdir_rejected(self): """Ensure the srcdir configurations are rejected.""" d = os.path.realpath(tempfile.mkdtemp()) @@ -240,41 +235,6 @@ class TestMozbuildObject(unittest.TestCase): MozbuildObject.from_environment(detect_virtualenv_mozinfo=False) finally: - os.chdir(self._old_cwd) - shutil.rmtree(d) - - def test_objdir_mismatch(self): - """Ensure MachCommandBase throwing on objdir mismatch.""" - d = os.path.realpath(tempfile.mkdtemp()) - - try: - real_topobjdir = os.path.join(d, 'real-objdir') - os.makedirs(real_topobjdir) - - topobjdir = os.path.join(d, 'objdir') - os.makedirs(topobjdir) - - topsrcdir = os.path.join(d, 'srcdir') - os.makedirs(topsrcdir) - - mozconfig = os.path.join(d, 'mozconfig') - with open(mozconfig, 'wt') as fh: - fh.write('mk_add_options MOZ_OBJDIR=%s' % real_topobjdir) - - mozinfo = os.path.join(topobjdir, 'mozinfo.json') - with open(mozinfo, 'wt') as fh: - json.dump(dict( - topsrcdir=topsrcdir, - mozconfig=mozconfig, - ), fh) - - os.chdir(topobjdir) - - with self.assertRaises(ObjdirMismatchException): - MozbuildObject.from_environment(detect_virtualenv_mozinfo=False) - - finally: - os.chdir(self._old_cwd) shutil.rmtree(d) def test_config_guess(self): @@ -286,8 +246,9 @@ class TestMozbuildObject(unittest.TestCase): self.assertIsNotNone(result) self.assertGreater(len(result), 0) + @unittest.skip('Failing on buildbot (bug 853954).') def test_config_environment(self): - base = self.get_base(topobjdir=topobjdir) + base = self.get_base() ce = base.config_environment self.assertIsInstance(ce, ConfigEnvironment) @@ -298,17 +259,15 @@ class TestMozbuildObject(unittest.TestCase): self.assertIsInstance(base.defines, dict) self.assertIsInstance(base.substs, dict) + @unittest.skip('Failing on buildbot (bug 853954).') def test_get_binary_path(self): - base = self.get_base(topobjdir=topobjdir) + base = self.get_base() platform = sys.platform # We should ideally use the config.status from the build. Let's install # a fake one. - substs = [ - ('MOZ_APP_NAME', 'awesomeapp'), - ('MOZ_BUILD_APP', 'awesomeapp'), - ] + substs = [('MOZ_APP_NAME', 'awesomeapp')] if sys.platform.startswith('darwin'): substs.append(('OS_ARCH', 'Darwin')) substs.append(('BIN_SUFFIX', '')) @@ -343,7 +302,7 @@ class TestMozbuildObject(unittest.TestCase): if platform.startswith('darwin'): self.assertTrue(p.endswith('awesomeapp/Nightly.app/Contents/MacOS/awesomeapp')) elif platform.startswith(('win32', 'cygwin')): - self.assertTrue(p.endswith('awesomeapp\\awesomeapp.exe')) + self.assertTrue(p.endswith('awesomeapp/awesomeapp.exe')) else: self.assertTrue(p.endswith('awesomeapp/awesomeapp')) diff --git a/python/mozbuild/mozbuild/test/test_mozconfig.py b/python/mozbuild/mozbuild/test/test_mozconfig.py index 744424e..429523d 100644 --- a/python/mozbuild/mozbuild/test/test_mozconfig.py +++ b/python/mozbuild/mozbuild/test/test_mozconfig.py @@ -29,7 +29,6 @@ class TestMozconfigLoader(unittest.TestCase): def setUp(self): self._old_env = dict(os.environ) os.environ.pop('MOZCONFIG', None) - os.environ.pop('MOZ_OBJDIR', None) os.environ.pop('CC', None) os.environ.pop('CXX', None) self._temp_dirs = set() @@ -244,7 +243,6 @@ class TestMozconfigLoader(unittest.TestCase): 'make_flags': None, 'make_extra': None, 'env': None, - 'vars': None, }) def test_read_empty_mozconfig(self): @@ -258,10 +256,9 @@ class TestMozconfigLoader(unittest.TestCase): self.assertEqual(result['make_extra'], []) for f in ('added', 'removed', 'modified'): - self.assertEqual(len(result['vars'][f]), 0) self.assertEqual(len(result['env'][f]), 0) - self.assertEqual(result['env']['unmodified'], {}) + self.assertGreater(len(result['env']['unmodified']), 0) def test_read_capture_ac_options(self): """Ensures ac_add_options calls are captured.""" @@ -319,22 +316,6 @@ class TestMozconfigLoader(unittest.TestCase): self.assertEqual(result['make_flags'], '-j8') self.assertEqual(result['make_extra'], ['FOO=BAR BAZ', 'BIZ=1']) - def test_read_empty_mozconfig_objdir_environ(self): - os.environ[b'MOZ_OBJDIR'] = b'obj-firefox' - with NamedTemporaryFile(mode='w') as mozconfig: - result = self.get_loader().read_mozconfig(mozconfig.name) - self.assertEqual(result['topobjdir'], 'obj-firefox') - - def test_read_capture_mk_options_objdir_environ(self): - """Ensures mk_add_options calls are captured and override the environ.""" - os.environ[b'MOZ_OBJDIR'] = b'obj-firefox' - with NamedTemporaryFile(mode='w') as mozconfig: - mozconfig.write('mk_add_options MOZ_OBJDIR=/foo/bar\n') - mozconfig.flush() - - result = self.get_loader().read_mozconfig(mozconfig.name) - self.assertEqual(result['topobjdir'], '/foo/bar') - def test_read_moz_objdir_substitution(self): """Ensure @TOPSRCDIR@ substitution is recognized in MOZ_OBJDIR.""" with NamedTemporaryFile(mode='w') as mozconfig: @@ -356,10 +337,9 @@ class TestMozconfigLoader(unittest.TestCase): result = self.get_loader().read_mozconfig(mozconfig.name) - self.assertEqual(result['vars']['added'], { + self.assertEqual(result['env']['added'], { 'CC': '/usr/local/bin/clang', 'CXX': '/usr/local/bin/clang++'}) - self.assertEqual(result['env']['added'], {}) def test_read_exported_variables(self): """Exported variables are caught as new variables.""" @@ -369,7 +349,6 @@ class TestMozconfigLoader(unittest.TestCase): result = self.get_loader().read_mozconfig(mozconfig.name) - self.assertEqual(result['vars']['added'], {}) self.assertEqual(result['env']['added'], { 'MY_EXPORTED': 'woot'}) @@ -383,25 +362,10 @@ class TestMozconfigLoader(unittest.TestCase): result = self.get_loader().read_mozconfig(mozconfig.name) - self.assertEqual(result['vars']['modified'], {}) self.assertEqual(result['env']['modified'], { 'CC': ('/usr/bin/gcc', '/usr/local/bin/clang') }) - def test_read_unmodified_variables(self): - """Variables modified by mozconfig are detected.""" - os.environ[b'CC'] = b'/usr/bin/gcc' - - with NamedTemporaryFile(mode='w') as mozconfig: - mozconfig.flush() - - result = self.get_loader().read_mozconfig(mozconfig.name) - - self.assertEqual(result['vars']['unmodified'], {}) - self.assertEqual(result['env']['unmodified'], { - 'CC': '/usr/bin/gcc' - }) - def test_read_removed_variables(self): """Variables unset by the mozconfig are detected.""" os.environ[b'CC'] = b'/usr/bin/clang' @@ -412,7 +376,6 @@ class TestMozconfigLoader(unittest.TestCase): result = self.get_loader().read_mozconfig(mozconfig.name) - self.assertEqual(result['vars']['removed'], {}) self.assertEqual(result['env']['removed'], { 'CC': '/usr/bin/clang'}) @@ -425,11 +388,10 @@ class TestMozconfigLoader(unittest.TestCase): result = self.get_loader().read_mozconfig(mozconfig.name) - self.assertEqual(result['vars']['added'], { + self.assertEqual(result['env']['added'], { 'multi': 'foo\nbar', 'single': '1' }) - self.assertEqual(result['env']['added'], {}) def test_read_topsrcdir_defined(self): """Ensure $topsrcdir references work as expected.""" @@ -440,25 +402,19 @@ class TestMozconfigLoader(unittest.TestCase): loader = self.get_loader() result = loader.read_mozconfig(mozconfig.name) - self.assertEqual(result['vars']['added']['TEST'], + self.assertEqual(result['env']['added']['TEST'], loader.topsrcdir.replace(os.sep, '/')) - self.assertEqual(result['env']['added'], {}) def test_read_empty_variable_value(self): """Ensure empty variable values are parsed properly.""" with NamedTemporaryFile(mode='w') as mozconfig: mozconfig.write('EMPTY=\n') - mozconfig.write('export EXPORT_EMPTY=\n') mozconfig.flush() result = self.get_loader().read_mozconfig(mozconfig.name) - self.assertEqual(result['vars']['added'], { - 'EMPTY': '', - }) - self.assertEqual(result['env']['added'], { - 'EXPORT_EMPTY': '' - }) + self.assertIn('EMPTY', result['env']['added']) + self.assertEqual(result['env']['added']['EMPTY'], '') def test_read_load_exception(self): """Ensure non-0 exit codes in mozconfigs are handled properly.""" diff --git a/testing/xpcshell/selftest.py b/testing/xpcshell/selftest.py index 78e1aa9..df2ba24 100644 --- a/testing/xpcshell/selftest.py +++ b/testing/xpcshell/selftest.py @@ -12,7 +12,6 @@ from StringIO import StringIO from xml.etree.ElementTree import ElementTree from mozbuild.base import MozbuildObject -os.environ.pop('MOZ_OBJDIR') build_obj = MozbuildObject.from_environment() from runxpcshelltests import XPCShellTests

1 0

[tor-browser-bundle/master] Adding gk's key for verifying tags, too.
by gk＠torproject.org 01 Dec '14

01 Dec '14

commit 08a199b9e868d02caff80a06f591d1a116f40483 Author: Georg Koppen <gk(a)torproject.org> Date: Mon Dec 1 09:18:32 2014 +0000 Adding gk's key for verifying tags, too. --- gitian/gpg/torbutton.gpg | Bin 15716 -> 44645 bytes 1 file changed, 0 insertions(+), 0 deletions(-) diff --git a/gitian/gpg/torbutton.gpg b/gitian/gpg/torbutton.gpg index 7f14ee3..ebc37d7 100644 Binary files a/gitian/gpg/torbutton.gpg and b/gitian/gpg/torbutton.gpg differ

1 0

[tor-browser-bundle/maint-4.0] Adding gk's key for verifying tags, too.
by gk＠torproject.org 01 Dec '14

01 Dec '14

commit 8e37ce5fba29567a91dc9d79b44187a4e400b8cf Author: Georg Koppen <gk(a)torproject.org> Date: Mon Dec 1 09:16:59 2014 +0000 Adding gk's key for verifying tags, too. --- gitian/gpg/torbutton.gpg | Bin 15716 -> 44645 bytes 1 file changed, 0 insertions(+), 0 deletions(-) diff --git a/gitian/gpg/torbutton.gpg b/gitian/gpg/torbutton.gpg index 7f14ee3..ebc37d7 100644 Binary files a/gitian/gpg/torbutton.gpg and b/gitian/gpg/torbutton.gpg differ

1 0

[tor-browser-bundle/maint-4.0] Leave the custom POODLE fix in.
by gk＠torproject.org 29 Nov '14

29 Nov '14

commit a5d9326573df0960c7b9184dd172a23d5476e1a6 Author: Georg Koppen <gk(a)torproject.org> Date: Sat Nov 29 08:40:32 2014 +0000 Leave the custom POODLE fix in. --- Bundle-Data/Docs/ChangeLog.txt | 1 - 1 file changed, 1 deletion(-) diff --git a/Bundle-Data/Docs/ChangeLog.txt b/Bundle-Data/Docs/ChangeLog.txt index 50f7974..93d20d7 100644 --- a/Bundle-Data/Docs/ChangeLog.txt +++ b/Bundle-Data/Docs/ChangeLog.txt @@ -9,7 +9,6 @@ Tor Browser 4.0.2 -- Dec 2 2014 * Bug 13742: Fix domain isolation for content cache and disk-enabled browsing mode * Bug 5926: Prevent JS engine locale leaks (by setting the C library locale) * Bug 13504: Remove unreliable/unreachable non-public bridges - * Bug 13435: Remove our custom POODLE fix * Windows * Bug 13443: Re-enable DirectShow; fix crash with mingw patch. * Bug 13558: Fix crash on Windows XP during download folder changing

1 0

[tor-browser-bundle/maint-4.0] We are going with a build1.
by gk＠torproject.org 29 Nov '14

29 Nov '14

commit 97f1899c10ab8cf0e59f91f951da5bbc934833a5 Author: Georg Koppen <gk(a)torproject.org> Date: Sat Nov 29 08:34:49 2014 +0000 We are going with a build1. --- gitian/versions | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/gitian/versions b/gitian/versions index 6f9d97b..ad8fd74 100755 --- a/gitian/versions +++ b/gitian/versions @@ -8,7 +8,7 @@ FIREFOX_VERSION=31.3.0esr TORBROWSER_UPDATE_CHANNEL=release -TORBROWSER_TAG=tor-browser-${FIREFOX_VERSION}-4.0-1-build2 +TORBROWSER_TAG=tor-browser-${FIREFOX_VERSION}-4.0-1-build1 TOR_TAG=tor-0.2.5.10 TORLAUNCHER_TAG=0.2.7.0.1 TORBUTTON_TAG=1.7.0.2

1 0

[tor-browser-bundle/maint-4.0] update_responses: add support for arch migration
by gk＠torproject.org 28 Nov '14

28 Nov '14

commit c17dc8aa81fb10d27dc1b9a8c3ca4116414cccfa Author: Nicolas Vigier <boklm(a)torproject.org> Date: Thu Nov 6 23:07:16 2014 +0100 update_responses: add support for arch migration --- tools/update-responses/update_responses | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/tools/update-responses/update_responses b/tools/update-responses/update_responses index a42e700..c788586 100755 --- a/tools/update-responses/update_responses +++ b/tools/update-responses/update_responses @@ -219,6 +219,14 @@ sub write_responses { get_version_files($config, $version); create_missing_incremental_mars($config, $version); my $files = $config->{versions}{$version}{files}; + my $migrate_archs = $config->{versions}{$version}{migrate_archs} // {}; + foreach my $old_os (keys %$migrate_archs) { + my $new_os = $migrate_archs->{$old_os}; + foreach my $lang (keys %{$files->{$new_os}}) { + $files->{$old_os}{$lang}{complete} = + $files->{$new_os}{$lang}{complete}; + } + } foreach my $os (keys %$files) { foreach my $lang (keys %{$files->{$os}}) { my $resp = get_response($config, $version, $os,

1 0

[tor-browser-bundle/maint-4.0] Updated config.yml for 4.0.2
by gk＠torproject.org 28 Nov '14

28 Nov '14

commit 6ea30dc92d1d4682706ad88971f324a6383434b7 Author: Georg Koppen <gk(a)torproject.org> Date: Fri Nov 28 12:53:13 2014 +0000 Updated config.yml for 4.0.2 --- tools/update-responses/config.yml | 27 +++++++++++++++++---------- 1 file changed, 17 insertions(+), 10 deletions(-) diff --git a/tools/update-responses/config.yml b/tools/update-responses/config.yml index 2132373..bdecdec 100644 --- a/tools/update-responses/config.yml +++ b/tools/update-responses/config.yml @@ -4,20 +4,27 @@ build_targets: linux64: Linux_x86_64-gcc3 win32: WINNT_x86-gcc3 osx32: Darwin_x86-gcc3 + osx64: Darwin_x86_64-gcc3 channels: - release: 4.0.1 - alpha: 4.0.1 + release: 4.0.2 + alpha: 4.5-alpha-1 versions: - 4.0: - platformVersion: 31.2.0 + 4.0.2: + platformVersion: 31.3.0 detailsURL: https://www.torproject.org/projects/torbrowser.html.en - download_url: https://www.torproject.org/dist/torbrowser/4.0 - 4.0.1: + download_url: https://www.torproject.org/dist/torbrowser/4.0.2 + incremental_from: + - 4.0 + - 4.0.1 + 4.5-alpha-1: platformVersion: 31.2.0 detailsURL: https://www.torproject.org/projects/torbrowser.html.en - download_url: https://www.torproject.org/dist/torbrowser/4.0.1 + download_url: https://www.torproject.org/dist/torbrowser/4.5-alpha-1 incremental_from: - 4.0 -# osx32: -# minSupportedOSVersion: 10.7 -# unsupported: 1 + - 4.0.1 + migrate_archs: + osx32: osx64 + osx32: + minSupportedOSVersion: 10.8 + detailsURL: https://blog.torproject.org/blog/end-life-plan-tor-browser-32-bit-macs#upda…

1 0

[tor-browser-bundle/maint-4.0] Remove default obfs3 bridges #4 and #5, and scramblesuit bridge #1.
by gk＠torproject.org 28 Nov '14

28 Nov '14

commit d54502befc3bffef55541e9db18e3428d548cde1 Author: Isis Lovecruft <isis(a)torproject.org> Date: Tue Oct 21 20:36:40 2014 +0000 Remove default obfs3 bridges #4 and #5, and scramblesuit bridge #1. These bridges were removed because: 1. They have never provided their bridge descriptors for metrics. See #13504: https://bugs.torproject.org/13504 2. They are either unreachable or unreliable. 2.a. The two obfs3 bridges are unreachable: Oct 21 20:25:53.000 [warn] Proxy Client: unable to connect to 209.141.36.236:45496 ("TTL expired") Oct 21 20:25:53.000 [warn] Proxy Client: unable to connect to 208.79.90.242:35658 ("TTL expired") 2.b. The scramblesuit bridge is rejecting connections: Oct 21 20:26:57.000 [warn] Proxy Client: unable to connect to 188.40.121.112:39707 ("Connection refused") * REMOVE default bridges obfs3.4, obfs3.5, and scramblesuit.1. * RENAME other bridges' preference names to keep the lists sequential. * FIXES part of #13504, sort of kind of. We still don't have historical metrics for these bridges for the period they were active. --- Bundle-Data/PTConfigs/bridge_prefs.js | 11 ++++------- 1 file changed, 4 insertions(+), 7 deletions(-) diff --git a/Bundle-Data/PTConfigs/bridge_prefs.js b/Bundle-Data/PTConfigs/bridge_prefs.js index d9c2049..20ec235 100644 --- a/Bundle-Data/PTConfigs/bridge_prefs.js +++ b/Bundle-Data/PTConfigs/bridge_prefs.js @@ -5,10 +5,8 @@ pref("extensions.torlauncher.default_bridge_recommended_type", "obfs3"); pref("extensions.torlauncher.default_bridge.obfs3.1", "obfs3 83.212.101.3:80 A09D536DD1752D542E1FBB3C9CE4449D51298239"); pref("extensions.torlauncher.default_bridge.obfs3.2", "obfs3 169.229.59.74:31493 AF9F66B7B04F8FF6F32D455F05135250A16543C9"); pref("extensions.torlauncher.default_bridge.obfs3.3", "obfs3 169.229.59.75:46328 AF9F66B7B04F8FF6F32D455F05135250A16543C9"); -pref("extensions.torlauncher.default_bridge.obfs3.4", "obfs3 209.141.36.236:45496 58D91C3A631F910F32E18A55441D5A0463BA66E2"); -pref("extensions.torlauncher.default_bridge.obfs3.5", "obfs3 208.79.90.242:35658 BA61757846841D64A83EA2514C766CB92F1FB41F"); -pref("extensions.torlauncher.default_bridge.obfs3.6", "obfs3 109.105.109.163:38980 1E05F577A0EC0213F971D81BF4D86A9E4E8229ED"); -pref("extensions.torlauncher.default_bridge.obfs3.7", "obfs3 109.105.109.163:47779 4C331FA9B3D1D6D8FB0D8FBBF0C259C360D97E6A"); +pref("extensions.torlauncher.default_bridge.obfs3.4", "obfs3 109.105.109.163:38980 1E05F577A0EC0213F971D81BF4D86A9E4E8229ED"); +pref("extensions.torlauncher.default_bridge.obfs3.5", "obfs3 109.105.109.163:47779 4C331FA9B3D1D6D8FB0D8FBBF0C259C360D97E6A"); pref("extensions.torlauncher.default_bridge.flashproxy.1", "flashproxy 0.0.1.0:1"); pref("extensions.torlauncher.default_bridge.flashproxy.2", "flashproxy 0.0.1.0:2"); @@ -26,9 +24,8 @@ pref("extensions.torlauncher.default_bridge.fte.6", "fte 128.105.214.163:8080 A1 pref("extensions.torlauncher.default_bridge.fte-ipv6.1", "fte [2001:49f0:d002:1::2]:80 B629B0B607C8AC9349B5646C24E9D242184F5B6E"); pref("extensions.torlauncher.default_bridge.fte-ipv6.2", "fte [2001:49f0:d00a:1::c]:80 2BD466989944867075E872310EBAD65BC88C8AEF"); -pref("extensions.torlauncher.default_bridge.scramblesuit.1", "scramblesuit 188.40.121.112:39707 5DE8D363D8F150C99E1A2D7237368D614838132C password=L5POGQONBPS2HZUR6GXBIDS4CMIYYOTI"); -pref("extensions.torlauncher.default_bridge.scramblesuit.2", "scramblesuit 188.226.213.208:54278 AA5A86C1490296EF4FACA946CC5A182FCD1C5B1E password=MD2VRP7WXAMSG7MKIGMHI4CB4BMSNO7T"); -pref("extensions.torlauncher.default_bridge.scramblesuit.3", "scramblesuit 83.212.101.3:443 A09D536DD1752D542E1FBB3C9CE4449D51298239 password=XTCXLG2JAMJKZW2POLBAOWOQETQSMASH"); +pref("extensions.torlauncher.default_bridge.scramblesuit.1", "scramblesuit 188.226.213.208:54278 AA5A86C1490296EF4FACA946CC5A182FCD1C5B1E password=MD2VRP7WXAMSG7MKIGMHI4CB4BMSNO7T"); +pref("extensions.torlauncher.default_bridge.scramblesuit.2", "scramblesuit 83.212.101.3:443 A09D536DD1752D542E1FBB3C9CE4449D51298239 password=XTCXLG2JAMJKZW2POLBAOWOQETQSMASH"); pref("extensions.torlauncher.default_bridge.meek-google.1", "meek 0.0.2.0:1 url=https://meek-reflect.appspot.com/ front=www.google.com"); pref("extensions.torlauncher.default_bridge.meek-amazon.1", "meek 0.0.2.0:2 url=https://d2zfqthxsdq309.cloudfront.net/ front=a0.awsstatic.com");

1 0