- tbb-commits - lists.torproject.org

[tor-browser-build/master] Include mar-tools zip in the output files
by boklm＠torproject.org 21 Aug '17

21 Aug '17

commit f3fed13feabaead01099d009f099120a8677c650 Author: Nicolas Vigier <boklm(a)torproject.org> Date: Mon Aug 21 18:10:03 2017 +0200 Include mar-tools zip in the output files --- projects/tor-browser/build | 2 ++ 1 file changed, 2 insertions(+) diff --git a/projects/tor-browser/build b/projects/tor-browser/build index 77a02ea..31c2d6d 100644 --- a/projects/tor-browser/build +++ b/projects/tor-browser/build @@ -275,6 +275,8 @@ cd $distdir [% END %] rm -rf $distdir/${PKG_DIR} +cp $rootdir/[% c('input_files_by_name/firefox') %]/mar-tools-*.zip "$OUTDIR"/ + # If we did not create a multi-lingual package above, create a package for # each locale. [% IF ! c("var/multi_lingual") %]

1 0

[tor-browser-build/master] Add a README.MAKEFILE file to describe makefile rules
by boklm＠torproject.org 15 Aug '17

15 Aug '17

commit f291e750afda75d68ba29bbcb998fba67e3f8012 Author: Nicolas Vigier <boklm(a)torproject.org> Date: Tue Aug 15 13:22:39 2017 +0200 Add a README.MAKEFILE file to describe makefile rules --- README | 6 +++++ README.MAKEFILE | 79 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 85 insertions(+) diff --git a/README b/README index 751ccdd..de79d35 100644 --- a/README +++ b/README @@ -212,3 +212,9 @@ Hacking on the Tor Browser build The file README.HACKING tries to list the main things to know when making changes to the Tor Browser build. +Description of makefile rules +----------------------------- + +You can find a description of the makefile rules in the README.MAKEFILE +file. + diff --git a/README.MAKEFILE b/README.MAKEFILE new file mode 100644 index 0000000..ef33904 --- /dev/null +++ b/README.MAKEFILE @@ -0,0 +1,79 @@ +Description of makefile rules +============================= + + +release +------- +Build Tor Browser for the release channel, for all supported platforms. +The resulting build can be found in directory release/$version. + +release-{linux-x86_64,linux-i686,windows-i686,osx-x86_64} +--------------------------------------------------------- +Same as release, for the selected platform only. + +alpha +----- +Build Tor Browser for the alpha channel, for all supported platforms. +The resulting build can be found in directoryp alpha/$version. + +alpha-{linux-x86_64,linux-i686,windows-i686,osx-x86_64} +------------------------------------------------------- +Same as alpha, for the selected platform only. + +nightly +------- +Build Tor Browser for the nightly channel, for all supported platforms. +Many of the components will be based on the master branch. You might +want to run "make fetch" before, to fetch the latest commits from all +components. + +The resulting build can be found in directory nightly/$date + +nightly-{linux-x86_64,linux-i686,windows-i686,osx-x86_64} +--------------------------------------------------------- +Same as nightly, for the selected platform only. + +alpha_nightly +------------- +Build Tor Browser for the alpha channel, for all supported platforms. +This is the same as 'make alpha', but the output directory is different. +The resulting build can be found in directory alpha_nightly/$date + +alpha_nightly-{linux-x86_64,linux-i686,windows-i686,osx-x86_64} +--------------------------------------------------------------- +The same as alpha_nightly, for the selected platform only. + +testbuild +--------- +Build a Tor Browser test build. To make the build faster, only the en-US +locale is created, without mar files. By default it is based on the +alpha channel, but you can change that by editing rbm.conf.local. +The resulting build can be found in directory testbuild. + +fetch +----- +Fetch new commits from all components. This is useful when you want to +build the nightly channel. + +clean +----- +Clean files created during previous builds that are no longer used in +current builds. Before using this command, you first need to configure +in rbm.local.conf the channel(s) that you are building. + +In other software, the clean rule is used to remove all build files, +but this is not the case here, where only obsolete build files are +removed. It is usually not necessary to remove all build files before +starting a new build as the need for rebuilding components is detected +automatically. If you want to remove all build files, you can remove +the out/ directory. + +clean-dry-run +------------- +Print the files that would be removed when running 'make clean'. + +signtag-{release,alpha} +----------------------- +Create a git signed tag for the selected channel, using the version and +build number defined as var/torbrowser_version and var/torbrowser_build. +

1 0

[tor-browser-build/master] Update rbm for the build_log_append option
by boklm＠torproject.org 11 Aug '17

11 Aug '17

commit 89656909bd3d54722aa6ee1627b93858c5cdd93c Author: Nicolas Vigier <boklm(a)torproject.org> Date: Fri Aug 11 13:02:32 2017 +0200 Update rbm for the build_log_append option --- rbm | 2 +- rbm.local.conf.example | 5 +++++ 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/rbm b/rbm index 6902007..0f5857b 160000 --- a/rbm +++ b/rbm @@ -1 +1 @@ -Subproject commit 690200770631f4adcfbbe7988379eb39698e11ab +Subproject commit 0f5857bb0486187fcd35c553dec9ae6d3657a98c diff --git a/rbm.local.conf.example b/rbm.local.conf.example index e30d8a8..62d13b2 100644 --- a/rbm.local.conf.example +++ b/rbm.local.conf.example @@ -21,6 +21,11 @@ ### stdout and stderr. #build_log: '-' +### By default, the logs from previous builds are kept in the log files. +### If you set build_log_append to 0, then previous logs are cleaned +### when starting a new build. +#build_log_append: 0 + buildconf: buildconf: 1

1 0

[rbm/master] Add the build_log_append option
by boklm＠torproject.org 11 Aug '17

11 Aug '17

commit 0f5857bb0486187fcd35c553dec9ae6d3657a98c Author: Nicolas Vigier <boklm(a)torproject.org> Date: Fri Aug 11 12:56:25 2017 +0200 Add the build_log_append option --- NEWS | 2 ++ doc/rbm_config.asc | 5 +++++ lib/RBM.pm | 3 ++- lib/RBM/DefaultConfig.pm | 1 + 4 files changed, 10 insertions(+), 1 deletion(-) diff --git a/NEWS b/NEWS index c43dc2d..d517840 100644 --- a/NEWS +++ b/NEWS @@ -1,3 +1,5 @@ +- Add the build_log_append option + - Add the rbm_tmp_dir option - Add the input_files_paths option diff --git a/doc/rbm_config.asc b/doc/rbm_config.asc index dc3b5e4..1935cd3 100644 --- a/doc/rbm_config.asc +++ b/doc/rbm_config.asc @@ -149,6 +149,11 @@ build_log:: The file where the build logs will be written. If the value is `-` (the default), the logs will be output on stdout and stderr. +build_log_append:: + If you set to +build_log_append+ to 0, the log file (defined in + +build_log+) will be cleaned when starting a new build. The + default is 1. + fetch:: The value should be 0 or 1, depending on whether the commits from the remote git or hg repository should be fetched diff --git a/lib/RBM.pm b/lib/RBM.pm index c1240cd..df29a01 100644 --- a/lib/RBM.pm +++ b/lib/RBM.pm @@ -1020,10 +1020,11 @@ sub build_run { } my $build_log = project_config($project, "build_log", $options); if ($build_log ne '-') { + my $append = project_config($project, "build_log_append", $options); $build_log = path($build_log); make_path(dirname($build_log)); my $now = localtime; - write_file($build_log, {append => 1}, "Starting build: $now\n"); + write_file($build_log, {append => $append}, "Starting build: $now\n"); print "Build log: $build_log\n"; } chdir $srcdir; diff --git a/lib/RBM/DefaultConfig.pm b/lib/RBM/DefaultConfig.pm index 63021e9..b1f97cb 100644 --- a/lib/RBM/DefaultConfig.pm +++ b/lib/RBM/DefaultConfig.pm @@ -126,6 +126,7 @@ our %default_config = ( rpmspec => '[% SET tmpl = project _ ".spec"; INCLUDE $tmpl -%]', build => '[% INCLUDE build -%]', build_log => '-', + build_log_append => '1', notmpl => [ qw(projects_dir) ], describe => \&git_describe, abbrev_lenght => '12',

1 0

[tor-browser-build/master] Display container's path when opening debug shell
by boklm＠torproject.org 11 Aug '17

11 Aug '17

commit b9512ca6c0fddee7aa61e6c1672df59ddaa0aacc Author: Nicolas Vigier <boklm(a)torproject.org> Date: Fri Aug 11 12:22:54 2017 +0200 Display container's path when opening debug shell --- README.HACKING | 5 ++--- rbm.conf | 3 +++ 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/README.HACKING b/README.HACKING index 8dde2f2..ea5d500 100644 --- a/README.HACKING +++ b/README.HACKING @@ -256,7 +256,6 @@ in the build containers, but vi can be used instead. When you press ctrl+d (twice if you opened a bash shell) the debug shell is closed and the container containing the failed build is removed. -If you want to make a backup of the container's rootfs with the failed -build to look at it later, you should be able to find it in -/tmp/$UID/rbm-containers. +The path to the container should be printed on the screen in case you +want to backup its rootfs to be able to look at it later. diff --git a/rbm.conf b/rbm.conf index 523a005..93c5e5b 100644 --- a/rbm.conf +++ b/rbm.conf @@ -314,6 +314,9 @@ runc: remote_exec: | #!/bin/sh set -e + [% IF c("interactive") -%] + echo Container directory: [% shell_quote(c("var/container/dir")) %] + [% END -%] mkdir -p '[% c("var/container/dir", { error_if_undef => 1 }) %]'/rootfs/rbm echo '#!/bin/sh' > '[% c("var/container/dir") %]'/rootfs/rbm/cmd echo [% shell_quote(c('exec_cmd')) %] >> '[% c("var/container/dir") %]'/rootfs/rbm/cmd

1 0

[tor-browser-build/master] Avoid using directory with predictable name in /tmp
by boklm＠torproject.org 11 Aug '17

11 Aug '17

commit 36296855fd4e553fb871c2f03a2fc62597d562a5 Author: Nicolas Vigier <boklm(a)torproject.org> Date: Fri Aug 11 12:28:57 2017 +0200 Avoid using directory with predictable name in /tmp Update rbm with support for the rbm_tmp_dir option, and use it in the containers directory. --- rbm | 2 +- rbm.conf | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/rbm b/rbm index f494d16..6902007 160000 --- a/rbm +++ b/rbm @@ -1 +1 @@ -Subproject commit f494d16a1b86700661a029f5eca00c6216bf08d1 +Subproject commit 690200770631f4adcfbbe7988379eb39698e11ab diff --git a/rbm.conf b/rbm.conf index 93c5e5b..5e1e66d 100644 --- a/rbm.conf +++ b/rbm.conf @@ -36,7 +36,7 @@ var: build: [% c("build", { filename => 'f', output_dir => '/out' }) %] container: - dir: '[% c("tmp_dir") %]/[% c("uid") %]/rbm-containers/[% sha256(c("build_id")) %]' + dir: '[% c("rbm_tmp_dir") %]/rbm-containers/[% sha256(c("build_id")) %]' user: rbm input_files_list: | [% FOREACH file IN c("input_files_by_name").keys.sort -%]

1 0

[rbm/master] Add the rbm_tmp_dir option
by boklm＠torproject.org 11 Aug '17

11 Aug '17

commit 690200770631f4adcfbbe7988379eb39698e11ab Author: Nicolas Vigier <boklm(a)torproject.org> Date: Fri Aug 11 12:25:55 2017 +0200 Add the rbm_tmp_dir option --- NEWS | 2 ++ doc/rbm_config.asc | 5 +++++ lib/RBM/DefaultConfig.pm | 13 +++++++++++++ 3 files changed, 20 insertions(+) diff --git a/NEWS b/NEWS index 4eac374..c43dc2d 100644 --- a/NEWS +++ b/NEWS @@ -1,3 +1,5 @@ +- Add the rbm_tmp_dir option + - Add the input_files_paths option - Add the ENV option diff --git a/doc/rbm_config.asc b/doc/rbm_config.asc index 2e9b8ab..dc3b5e4 100644 --- a/doc/rbm_config.asc +++ b/doc/rbm_config.asc @@ -136,6 +136,11 @@ tmp_dir:: This directory will contains some scripts that will be executed, so it should not be on a partition mounted as noexec. +rbm_tmp_dir:: + A directory created inside +tmp_dir+ using +File::Temp+, that + you can use to store temporary files. This directory is removed + automatically when rbm exits. + output_dir:: The directory where output files (tarballs, spec files or packages) are created. The default value is +out+. diff --git a/lib/RBM/DefaultConfig.pm b/lib/RBM/DefaultConfig.pm index cf0ed74..63021e9 100644 --- a/lib/RBM/DefaultConfig.pm +++ b/lib/RBM/DefaultConfig.pm @@ -13,6 +13,7 @@ use File::Basename; use RBM; use Cwd qw(getcwd); use IO::CaptureOutput qw(capture_exec); +use File::Temp; sub git_describe { my ($project, $options) = @_; @@ -101,10 +102,22 @@ sub docker_version { RBM::exit_error("Could not find docker version"); } +sub rbm_tmp_dir { + my ($project, $options) = @_; + CORE::state $rbm_tmp_dir; + return $rbm_tmp_dir->dirname if $rbm_tmp_dir; + my $tmp_dir = RBM::project_config($project, 'tmp_dir', $options) + || RBM::exit_error('No tmp_dir specified'); + $rbm_tmp_dir = File::Temp->newdir(TEMPLATE => 'rbm-XXXXXX', + DIR => $tmp_dir); + return $rbm_tmp_dir->dirname; +} + our %default_config = ( sysconf_file => '/etc/rbm.conf', localconf_file=> 'rbm.local.conf', tmp_dir => '[% GET ENV.TMPDIR ? ENV.TMPDIR : "/tmp"; %]', + rbm_tmp_dir => \&rbm_tmp_dir, projects_dir => 'projects', output_dir => 'out', git_clone_dir => 'git_clones',

1 0

[tor-browser-build/master] Update tor-browser.git branch used in nightly
by boklm＠torproject.org 10 Aug '17

10 Aug '17

commit 8a999d86a27b8edd4dfc3ec8a497f13804d37f74 Author: Nicolas Vigier <boklm(a)torproject.org> Date: Thu Aug 10 12:43:52 2017 +0200 Update tor-browser.git branch used in nightly --- projects/firefox/config | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/projects/firefox/config b/projects/firefox/config index e2cf727..1360772 100644 --- a/projects/firefox/config +++ b/projects/firefox/config @@ -27,7 +27,7 @@ targets: torbrowser_update_channel: release nightly: - git_hash: 'tor-browser-[% c("var/firefox_version") %]-[% c("var/torbrowser_branch") %]-1' + git_hash: 'tor-browser-[% c("var/firefox_version") %]-[% c("var/torbrowser_branch") %]-2' tag_gpg_id: 0 var: torbrowser_update_channel: default

1 0

[tor-browser-build/master] README.HACKING: explain which mozconfig files we are using
by boklm＠torproject.org 10 Aug '17

10 Aug '17

commit 2c5b8113fa1fcbfaac8f5b3310b7e54a7ca38af2 Author: Nicolas Vigier <boklm(a)torproject.org> Date: Thu Aug 10 12:24:20 2017 +0200 README.HACKING: explain which mozconfig files we are using --- README.HACKING | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/README.HACKING b/README.HACKING index 221cc90..8dde2f2 100644 --- a/README.HACKING +++ b/README.HACKING @@ -222,6 +222,21 @@ commit you want to build. You will also need to comment the 'tag_gpg_id: 1' line (unless git_hash is pointing to a signed git tag). +The Firefox mozconfig files +--------------------------- + +In the gitian build, we are using the mozconfig files included in the +tor-browser.git repository (the .mozconfig, .mozconfig-mac and +.mozconfig-mingw files). + +In the rbm build however, we need to make some small modifications to +those files, so we are instead using mozconfig files stored in the +projects/firefox/ directory and ignoring the .mozconfig files from the +tor-browser.git repository. + +This could change in the future, when we are not using gitian anymore. + + Debugging a build error -----------------------

1 0

[tor-browser-build/master] Synchronize mozconfig files with their version from tor-browser.git
by boklm＠torproject.org 10 Aug '17

10 Aug '17

commit aa96f1ab37786d11529f8266a1eeca07b04e7016 Author: Nicolas Vigier <boklm(a)torproject.org> Date: Thu Aug 10 12:39:09 2017 +0200 Synchronize mozconfig files with their version from tor-browser.git The main change is the removal of the --enable-jemalloc=4 option which was done with #22482. --- projects/firefox/mozconfig-linux-i686 | 3 +-- projects/firefox/mozconfig-linux-x86_64 | 3 +-- projects/firefox/mozconfig-osx-x86_64 | 3 +-- projects/firefox/mozconfig-windows-i686 | 2 +- 4 files changed, 4 insertions(+), 7 deletions(-) diff --git a/projects/firefox/mozconfig-linux-i686 b/projects/firefox/mozconfig-linux-i686 index 8b4ecb3..a7d75fa 100755 --- a/projects/firefox/mozconfig-linux-i686 +++ b/projects/firefox/mozconfig-linux-i686 @@ -31,7 +31,6 @@ ac_add_options --disable-debug ac_add_options --disable-maintenance-service ac_add_options --disable-crashreporter ac_add_options --disable-webrtc -# Let's not compile EME at least until we can enable ClearKey and only Clearkey. +# Let's make sure no preference is enabling either Adobe's or Google's CDM. ac_add_options --disable-eme #ac_add_options --disable-ctypes -ac_add_options --enable-jemalloc=4 diff --git a/projects/firefox/mozconfig-linux-x86_64 b/projects/firefox/mozconfig-linux-x86_64 index cde5b49..7d9ab1c 100755 --- a/projects/firefox/mozconfig-linux-x86_64 +++ b/projects/firefox/mozconfig-linux-x86_64 @@ -23,7 +23,6 @@ ac_add_options --disable-debug ac_add_options --disable-maintenance-service ac_add_options --disable-crashreporter ac_add_options --disable-webrtc -# Let's not compile EME at least until we can enable ClearKey and only Clearkey. +# Let's make sure no preference is enabling either Adobe's or Google's CDM. ac_add_options --disable-eme #ac_add_options --disable-ctypes -ac_add_options --enable-jemalloc=4 diff --git a/projects/firefox/mozconfig-osx-x86_64 b/projects/firefox/mozconfig-osx-x86_64 index 1658e18..bc7a30e 100644 --- a/projects/firefox/mozconfig-osx-x86_64 +++ b/projects/firefox/mozconfig-osx-x86_64 @@ -50,7 +50,6 @@ ac_add_options --disable-crashreporter ac_add_options --disable-maintenance-service ac_add_options --disable-webrtc ac_add_options --disable-tests -# Let's not compile EME at least until we can enable ClearKey and only Clearkey. -# (Apart from that there is no Adobe CRM module for OS X right now) +# Let's make sure no preference is enabling either Adobe's or Google's CDM. ac_add_options --disable-eme # ac_add_options --disable-ctypes diff --git a/projects/firefox/mozconfig-windows-i686 b/projects/firefox/mozconfig-windows-i686 index de86c49..af30573 100644 --- a/projects/firefox/mozconfig-windows-i686 +++ b/projects/firefox/mozconfig-windows-i686 @@ -21,7 +21,7 @@ ac_add_options --enable-verify-mar # We can't build the sandbox code with mingw-w64: # https://bugzilla.mozilla.org/show_bug.cgi?id=1042426 ac_add_options --disable-sandbox -# EME is not running without sandbox. Thus, don't compile it in the first place. +# Let's make sure no preference is enabling either Adobe's or Google's CDM. ac_add_options --disable-eme ac_add_options --disable-crashreporter ac_add_options --disable-maintenance-service

1 0