- tbb-commits - lists.torproject.org

[tor-browser/tor-browser-78.8.0esr-10.0-1] fixup! Bug 40193: Add `AT_EMPTY_PATH` definition
by gk＠torproject.org 02 Mar '21

02 Mar '21

commit 17a77d61e488593854cf2e818756ce24ceb15fbc Author: Matthew Finkel <sysrqb(a)torproject.org> Date: Mon Mar 1 20:42:26 2021 +0000 fixup! Bug 40193: Add `AT_EMPTY_PATH` definition --- security/sandbox/linux/SandboxFilter.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/security/sandbox/linux/SandboxFilter.cpp b/security/sandbox/linux/SandboxFilter.cpp index 0ffcff9b5b32..2ca3fbb45027 100644 --- a/security/sandbox/linux/SandboxFilter.cpp +++ b/security/sandbox/linux/SandboxFilter.cpp @@ -74,7 +74,7 @@ using namespace sandbox::bpf_dsl; #define FMODE_NONOTIFY 0x4000000 #ifndef AT_EMPTY_PATH -# define AT_EMPTY_PATH 0x100 /* Allow empty relative pathname */ +# define AT_EMPTY_PATH 0x1000 /* Allow empty relative pathname */ #endif #ifndef F_LINUX_SPECIFIC_BASE

1 0

[Git][tpo/applications/android-components] Pushed new branch android-components-73.0.3-10.5-1
by Matthew Finkel 01 Mar '21

01 Mar '21

Matthew Finkel pushed new branch android-components-73.0.3-10.5-1 at The Tor Project / Applications / android-components -- View it on GitLab: https://gitlab.torproject.org/tpo/applications/android-components/-/tree/an… You're receiving this email because of your account on gitlab.torproject.org.

1 0

[tor-browser-build/master] Bug 40245: Rebuild all container images
by gk＠torproject.org 01 Mar '21

01 Mar '21

commit c41852357ce7335b04a1dccc04a058dab2653632 Author: Nicolas Vigier <boklm(a)torproject.org> Date: Mon Mar 1 17:32:45 2021 +0100 Bug 40245: Rebuild all container images In #40240 we found that package updates were not installed in containers created using debootstrap, but are now installed in the containers created with mmdebstrap. Some of those updates are causing differences in the build of some components. To be able to build both the stable and master branches from the same tor-browser-build directory without sharing containers, we update the filenames of all containers. --- projects/container-image/config | 1 + 1 file changed, 1 insertion(+) diff --git a/projects/container-image/config b/projects/container-image/config index e0b8b13..e1274f2 100644 --- a/projects/container-image/config +++ b/projects/container-image/config @@ -24,6 +24,7 @@ targets: pre: | #!/bin/sh + # Version: 2 # [% c('var/container/suite') %] set -e export DEBIAN_FRONTEND=noninteractive

1 0

[tor-browser-build/master] Merge remote-tracking branch 'gitlab/merge-requests/230'
by gk＠torproject.org 01 Mar '21

01 Mar '21

commit 926e276a89e117b15a951bef467746f3b9517aca Merge: aadd6fe c418523 Author: Georg Koppen <gk(a)torproject.org> Date: Mon Mar 1 19:18:42 2021 +0000 Merge remote-tracking branch 'gitlab/merge-requests/230' projects/container-image/config | 1 + 1 file changed, 1 insertion(+)

1 0

[tor-browser-build/master] Bug 40243: Remove tools/ansible/roles/tbb-nightly-signing
by gk＠torproject.org 01 Mar '21

01 Mar '21

commit 04f288b9181cc97f48ec58fb794e8de66711faf1 Author: Nicolas Vigier <boklm(a)torproject.org> Date: Mon Mar 1 17:09:02 2021 +0100 Bug 40243: Remove tools/ansible/roles/tbb-nightly-signing Revert "Bug 25102: Setup nightly signing" This reverts commit 17804f5ada032276eabf7e33d41feef46ca511d7. --- tools/ansible/Makefile | 3 -- tools/ansible/README | 7 --- tools/ansible/ansible.cfg | 6 --- tools/ansible/inventory | 1 - .../roles/tbb-nightly-signing/defaults/main.yml | 7 --- .../roles/tbb-nightly-signing/tasks/main.yml | 54 ---------------------- tools/ansible/tbb-nightly-signing.yml | 6 --- 7 files changed, 84 deletions(-) diff --git a/tools/ansible/Makefile b/tools/ansible/Makefile index 97a63c1..ea63a44 100644 --- a/tools/ansible/Makefile +++ b/tools/ansible/Makefile @@ -6,6 +6,3 @@ fpcentral: boklm-tbb-nightly-build: ansible-playbook --vault-password-file=~/ansible-vault/boklm-tbb-nightly -i inventory boklm-tbb-nightly-build.yml - -tbb-nightly-signing: - ANSIBLE_CONFIG='$((a)D)/ansible.cfg' ansible-playbook -i inventory tbb-nightly-signing.yml diff --git a/tools/ansible/README b/tools/ansible/README index 5407a73..6056372 100644 --- a/tools/ansible/README +++ b/tools/ansible/README @@ -25,13 +25,6 @@ boklm-tbb-nightly-build: For more details, see also this ticket: https://trac.torproject.org/projects/tor/ticket/33948 -tbb-nightly-signing: - This target is used to deploy a nightly signing machine. The - configuration of nightly signing is done in the following files: - tools/ansible/roles/tbb-nightly-signing/defaults/main.yml - tools/signing/nightly/config.yml - tools/signing/nightly/update-responses-base-config.yml - Adding, removing, updating users on the Tor Browser team build machine ====================================================================== diff --git a/tools/ansible/ansible.cfg b/tools/ansible/ansible.cfg deleted file mode 100644 index 0663746..0000000 --- a/tools/ansible/ansible.cfg +++ /dev/null @@ -1,6 +0,0 @@ -[ssh_connection] -; When connecting to a v3 onion, we get the error: -; "unix_listener: [...] too long for Unix domain socket" -; We solve this by using %n (The original remote hostname, as given on -; the command line) instead of %h (The remote hostname) in the control path. -control_path=%(directory)s/%%r-%%n-%%r diff --git a/tools/ansible/inventory b/tools/ansible/inventory index 47fda66..fc25842 100644 --- a/tools/ansible/inventory +++ b/tools/ansible/inventory @@ -1,7 +1,6 @@ build-sunet-a ansible_ssh_user=root ansible_ssh_host=build-sunet-a.torproject.net fpcentral ansible_become=True ansible_become_method=sudo ansible_become_user=fpcentral ansible_ssh_host=forrestii.torproject.org allow_world_readable_tmpfiles=True boklm-tbb-nightly-build ansible_ssh_user=root ansible_become_method=su -tbb-nightly-signing ansible_ssh_user=root ansible_become_method=su [tbb-build] build-sunet-a diff --git a/tools/ansible/roles/tbb-nightly-signing/defaults/main.yml b/tools/ansible/roles/tbb-nightly-signing/defaults/main.yml deleted file mode 100644 index cbe3b82..0000000 --- a/tools/ansible/roles/tbb-nightly-signing/defaults/main.yml +++ /dev/null @@ -1,7 +0,0 @@ ---- -nightly_signing_user: nightly-signing -nightly_signing_cron_hour: '*' -nightly_signing_cron_minute: '0,30' -tor_browser_build_dir: "/home/{{ nightly_signing_user }}/tor-browser-build" -tor_browser_build_git_url: https://git.torproject.org/builders/tor-browser-build.git -tor_browser_build_commit: 8d66414b7860751ffec6a83a6bc6dbfbd94f801a diff --git a/tools/ansible/roles/tbb-nightly-signing/tasks/main.yml b/tools/ansible/roles/tbb-nightly-signing/tasks/main.yml deleted file mode 100644 index 3cc96ba..0000000 --- a/tools/ansible/roles/tbb-nightly-signing/tasks/main.yml +++ /dev/null @@ -1,54 +0,0 @@ ---- -- name: Install dependencies - apt: - name: "{{ item }}" - state: present - with_items: - - git - - libdatetime-perl - - libfindbin-libs-perl - - libfile-slurp-perl - - libxml-writer-perl - - libio-captureoutput-perl - - libparallel-forkmanager-perl - - libxml-libxml-perl - - libwww-perl - - libjson-perl - - libyaml-libyaml-perl - - libyaml-perl - - libtemplate-perl - - libio-handle-util-perl - - libio-all-perl - - libio-captureoutput-perl - - libpath-tiny-perl - - libstring-shellquote-perl - - libsort-versions-perl - - libdigest-sha-perl - - libdata-uuid-perl - - libdata-dump-perl - - libfile-copy-recursive-perl - - libnss3-tools - - rsync - -- name: create nightly-signing user - user: - name: "{{ nightly_signing_user }}" - comment: "Tor Browser Nightly Signing" - createhome: yes - home: "/home/{{ nightly_signing_user }}" - -- name: clone tor-browser-build - become: yes - become_user: "{{ nightly_signing_user }}" - git: - repo: "{{ tor_browser_build_git_url }}" - dest: "{{ tor_browser_build_dir }}" - version: "{{ tor_browser_build_commit }}" - -- name: add cron to sign nighly build - cron: - name: tbb-sign-nightly-build - user: "{{ nightly_signing_user }}" - hour: "{{ nightly_signing_cron_hour }}" - minute: "{{ nightly_signing_cron_minute }}" - job: "torsocks /home/{{ nightly_signing_user }}/tor-browser-build/tools/signing/nightly/sign-nightly" diff --git a/tools/ansible/tbb-nightly-signing.yml b/tools/ansible/tbb-nightly-signing.yml deleted file mode 100644 index bc0c681..0000000 --- a/tools/ansible/tbb-nightly-signing.yml +++ /dev/null @@ -1,6 +0,0 @@ ---- - -- hosts: tbb-nightly-signing - roles: - - role: tbb-nightly-signing - - role: unattended-upgrades

1 0

[tor-browser-build/master] Bug 40244: Remove tools/ansible/roles/tbb-nightly-build
by gk＠torproject.org 01 Mar '21

01 Mar '21

commit aadd6feb0302552da49eac8cae440c1510a436f3 Author: Nicolas Vigier <boklm(a)torproject.org> Date: Mon Mar 1 17:19:03 2021 +0100 Bug 40244: Remove tools/ansible/roles/tbb-nightly-build --- tools/ansible/Makefile | 3 - tools/ansible/README | 8 -- tools/ansible/boklm-tbb-nightly-build.yml | 10 -- tools/ansible/group_vars/boklm-tbb-nightly/dma.yml | 9 -- .../boklm-tbb-nightly/tbb-nightly-build.yml | 6 - tools/ansible/inventory | 4 - .../roles/tbb-nightly-build/defaults/main.yml | 13 -- .../roles/tbb-nightly-build/files/prune-old-builds | 136 --------------------- .../roles/tbb-nightly-build/handlers/main.yml | 5 - .../ansible/roles/tbb-nightly-build/tasks/main.yml | 99 --------------- .../roles/tbb-nightly-build/templates/nginx.conf | 20 --- .../tbb-nightly-build/templates/rbm.local.conf | 114 ----------------- .../tbb-nightly-build/templates/start-tbb-nightly | 5 - .../tbb-nightly-build.auth_basic_user_file | 1 - .../tbb-nightly-build/templates/testsuite-config | 34 ------ .../tbb-nightly-build/templates/www-index.html | 10 -- .../vaulted_vars/boklm-tbb-nightly/dma-auth.yml | 10 -- 17 files changed, 487 deletions(-) diff --git a/tools/ansible/Makefile b/tools/ansible/Makefile index ea63a44..608f932 100644 --- a/tools/ansible/Makefile +++ b/tools/ansible/Makefile @@ -3,6 +3,3 @@ ansible-tbb-build: fpcentral: ANSIBLE_CONFIG='$((a)D)/ansible-fpcentral.cfg' ansible-playbook -i inventory --ask-become-pass fpcentral.yml - -boklm-tbb-nightly-build: - ansible-playbook --vault-password-file=~/ansible-vault/boklm-tbb-nightly -i inventory boklm-tbb-nightly-build.yml diff --git a/tools/ansible/README b/tools/ansible/README index 6056372..2cdcfcb 100644 --- a/tools/ansible/README +++ b/tools/ansible/README @@ -17,14 +17,6 @@ fpcentral: you need to be in the fpcentral tpo ldap group. Your ldap password will be asked, to sudo to the fpcentral user. -boklm-tbb-nightly-build: - This target is used by boklm to deploy a nightly build machine. If - anybody else wants to set up their own nightly builds, it is possible to - do it by adding a new host to the inventory file and making copies of - group_vars/boklm-tbb-nightly/ and boklm-tbb-nightly-build.yml. - For more details, see also this ticket: - https://trac.torproject.org/projects/tor/ticket/33948 - Adding, removing, updating users on the Tor Browser team build machine ====================================================================== diff --git a/tools/ansible/boklm-tbb-nightly-build.yml b/tools/ansible/boklm-tbb-nightly-build.yml deleted file mode 100644 index dd9e1dc..0000000 --- a/tools/ansible/boklm-tbb-nightly-build.yml +++ /dev/null @@ -1,10 +0,0 @@ ---- - -- hosts: boklm-tbb-nightly-build - roles: - - role: tbb-builder - - role: tbb-nightly-build - - role: unattended-upgrades - - role: mta - vars_files: - - vaulted_vars/boklm-tbb-nightly/dma-auth.yml diff --git a/tools/ansible/group_vars/boklm-tbb-nightly/dma.yml b/tools/ansible/group_vars/boklm-tbb-nightly/dma.yml deleted file mode 100644 index 0148da5..0000000 --- a/tools/ansible/group_vars/boklm-tbb-nightly/dma.yml +++ /dev/null @@ -1,9 +0,0 @@ ---- -dma_conf: | - SMARTHOST mail.riseup.net - PORT 587 - AUTHPATH /etc/dma/auth.conf - SECURETRANSFER - STARTTLS - MAILNAME /etc/mailname - MASQUERADE boklm-tbb-nightly(a)riseup.net diff --git a/tools/ansible/group_vars/boklm-tbb-nightly/tbb-nightly-build.yml b/tools/ansible/group_vars/boklm-tbb-nightly/tbb-nightly-build.yml deleted file mode 100644 index 6addb1d..0000000 --- a/tools/ansible/group_vars/boklm-tbb-nightly/tbb-nightly-build.yml +++ /dev/null @@ -1,6 +0,0 @@ ---- -nightly_build_hostname: f4amtbsowhix7rrf.onion -nightly_build_url: 'http://{{ nightly_build_hostname }}' -nightly_build_email_from: "'Tor Browser Nightly Builds (boklm) <boklm-tbb-nightly(a)riseup.net>'," -nightly_build_email_to: "[ 'boklm(a)torproject.org', 'gk(a)torproject.org', 'sysrqb(a)torproject.org', ]," -nightly_build_sign_build: 1 diff --git a/tools/ansible/inventory b/tools/ansible/inventory index fc25842..32a5805 100644 --- a/tools/ansible/inventory +++ b/tools/ansible/inventory @@ -1,12 +1,8 @@ build-sunet-a ansible_ssh_user=root ansible_ssh_host=build-sunet-a.torproject.net fpcentral ansible_become=True ansible_become_method=sudo ansible_become_user=fpcentral ansible_ssh_host=forrestii.torproject.org allow_world_readable_tmpfiles=True -boklm-tbb-nightly-build ansible_ssh_user=root ansible_become_method=su [tbb-build] build-sunet-a [fpcentral] fpcentral - -[boklm-tbb-nightly] -boklm-tbb-nightly-build diff --git a/tools/ansible/roles/tbb-nightly-build/defaults/main.yml b/tools/ansible/roles/tbb-nightly-build/defaults/main.yml deleted file mode 100644 index 1e45b1a..0000000 --- a/tools/ansible/roles/tbb-nightly-build/defaults/main.yml +++ /dev/null @@ -1,13 +0,0 @@ ---- -nightly_build_user: tbb-nightly -nightly_build_cron_hour: 2 -nightly_build_cron_minute: 20 -nightly_build_keep_builds: 3 -testsuite_dir: "/home/{{ nightly_build_user }}/tbb-testsuite" -testsuite_git_url: https://git.torproject.org/tor-browser-bundle-testsuite.git -testsuite_git_commit: 71bce1264f10e8f184095aad54cf27e81f56a7a4 -nightly_build_wwwdir: "/home/{{ nightly_build_user }}/www" -nightly_build_nginx_enable: true -nightly_build_nginx_listen: 127.0.0.1:80 -nightly_build_nginx_auth_basic_enable: true -nightly_build_nginx_auth_basic_content: tor-guest:ezmjAG/jVTat. diff --git a/tools/ansible/roles/tbb-nightly-build/files/prune-old-builds b/tools/ansible/roles/tbb-nightly-build/files/prune-old-builds deleted file mode 100755 index 852a9da..0000000 --- a/tools/ansible/roles/tbb-nightly-build/files/prune-old-builds +++ /dev/null @@ -1,136 +0,0 @@ -#!/usr/bin/perl -w - -# Copyright (c) 2019, The Tor Project, Inc. -# -# Redistribution and use in source and binary forms, with or without -# modification, are permitted provided that the following conditions are -# met: -# -# * Redistributions of source code must retain the above copyright -# notice, this list of conditions and the following disclaimer. -# -# * Redistributions in binary form must reproduce the above -# copyright notice, this list of conditions and the following disclaimer -# in the documentation and/or other materials provided with the -# distribution. -# -# * Neither the names of the copyright owners nor the names of its -# contributors may be used to endorse or promote products derived from -# this software without specific prior written permission. -# -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS -# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT -# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR -# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT -# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, -# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT -# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, -# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY -# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT -# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE -# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - - -# 'prune-old-builds' is a script to prune old builds. -# -# -# Usage: -# $ ./prune-old-builds [options] <directory> -# -# -# Available options: -# -# --dry-run -# Don't delete anything, but say what would be deleted. -# -# --prefix <prefix> -# Prefix of the directories to be removed. Default is 'tbb-nightly.'. -# -# --separator <c> -# Separator character to separate the year, month, day in the directory -# names. Default is '.'. -# -# --days <n> -# Number of days that we should keep. Default is 6. -# -# --weeks <n> -# Number of monday builds that we should keep. Default is 3. -# -# --months <n> -# Number of 1st day of the month builds that we should keep. -# Default is 3. - -use strict; -use Getopt::Long; -use DateTime; -use DateTime::Duration; -use File::Path qw(remove_tree); - -my %options = ( - days => 6, - weeks => 3, - months => 3, - prefix => 'tbb-nightly.', - separator => '.', -); - -sub keep_builds { - my %res; - - my $day = DateTime::Duration->new(days => 1); - my $week = DateTime::Duration->new(weeks => 1); - my $month = DateTime::Duration->new(months => 1); - - my $n = $options{days}; - my $dt = DateTime->now; - while ($n) { - $res{ $options{prefix} . $dt->ymd($options{separator}) } = 1; - $dt = $dt - $day; - $n--; - } - - my $w = $options{weeks}; - while ($dt->day_of_week != 1) { - $dt = $dt - $day; - } - while ($w) { - $res{ $options{prefix} . $dt->ymd($options{separator}) } = 1; - $dt = $dt - $week; - $w--; - } - - my $m = $options{months}; - $dt = DateTime->now; - while ($dt->day != 1) { - $dt = $dt - $day; - } - while ($m) { - $res{ $options{prefix} . $dt->ymd($options{separator}) } = 1; - $dt = $dt - $month; - $m--; - } - - return \%res; -} - -sub clean_directory { - my ($directory) = @_; - my $k = keep_builds; - chdir $directory || die "Error entering $directory"; - foreach my $file (glob "$options{prefix}*") { - next unless $file =~ m/^$options{prefix}\d{4}$options{separator}\d{2}$options{separator}\d{2}$/; - next if $k->{$file}; - if ($options{'dry-run'}) { - print "Would remove $file\n"; - } else { - remove_tree($file); - } - } -} - -my @opts = qw(days=i weeks=i months=i prefix=s dry-run!); -Getopt::Long::GetOptions(\%options, @opts); -die "Missing argument: directory to clean" unless @ARGV; -foreach my $dir (@ARGV) { - clean_directory($dir); -} diff --git a/tools/ansible/roles/tbb-nightly-build/handlers/main.yml b/tools/ansible/roles/tbb-nightly-build/handlers/main.yml deleted file mode 100644 index 3350d73..0000000 --- a/tools/ansible/roles/tbb-nightly-build/handlers/main.yml +++ /dev/null @@ -1,5 +0,0 @@ ---- -- name: restart nginx - service: - name: nginx - state: restarted diff --git a/tools/ansible/roles/tbb-nightly-build/tasks/main.yml b/tools/ansible/roles/tbb-nightly-build/tasks/main.yml deleted file mode 100644 index d5254b6..0000000 --- a/tools/ansible/roles/tbb-nightly-build/tasks/main.yml +++ /dev/null @@ -1,99 +0,0 @@ ---- -- name: create tbb-nightly user - user: - name: "{{ nightly_build_user }}" - comment: "Tor Browser Nightly Builds" - groups: tbb-build - createhome: yes - home: "/home/{{ nightly_build_user }}" - -- name: clone tor browser testsuite - become: yes - become_user: "{{ nightly_build_user }}" - git: - repo: "{{ testsuite_git_url }}" - dest: "{{ testsuite_dir }}" - version: "{{ testsuite_git_commit }}" - -- name: install testsuite dependencies - command: "{{ testsuite_dir }}/install-deps" - -- name: add testsuite config - template: - src: testsuite-config - dest: "{{ testsuite_dir }}/config/tbb-nightly" - mode: 0644 - owner: "{{ nightly_build_user }}" - -- name: create rbm-config directory - file: - path: '{{ testsuite_dir }}/rbm-config/' - state: directory - owner: '{{ nightly_build_user }}' - mode: 0755 - -- name: add rbm config - template: - src: rbm.local.conf - dest: "{{ testsuite_dir }}/rbm-config/tbb-nightly.rbm.local.conf" - mode: 0644 - owner: "{{ nightly_build_user }}" - -- name: prune-old-builds - copy: - src: prune-old-builds - dest: "/home/{{ nightly_build_user }}/prune-old-builds" - mode: 0755 - owner: "{{ nightly_build_user }}" - -- name: add start-tbb-nightly script - template: - src: start-tbb-nightly - dest: "/home/{{ nightly_build_user }}/start-tbb-nightly" - mode: 0755 - owner: "{{ nightly_build_user }}" - -- name: add cron to start nighly build - cron: - name: tbb-nightly-build - user: "{{ nightly_build_user }}" - hour: "{{ nightly_build_cron_hour }}" - minute: "{{ nightly_build_cron_minute }}" - job: "/home/{{ nightly_build_user }}/start-tbb-nightly" - -- name: create wwwdir - file: - path: '{{ nightly_build_wwwdir }}' - state: directory - owner: '{{ nightly_build_user }}' - mode: 0755 - -- name: add index page - template: - src: www-index.html - dest: "{{ nightly_build_wwwdir }}/index.html" - mode: 0644 - owner: "{{ nightly_build_user }}" - -- name: create tbb-nightly-build.auth_basic_user_file - template: - src: tbb-nightly-build.auth_basic_user_file - dest: /etc/nginx/tbb-nightly-build.auth_basic_user_file - mode: 0644 - owner: root - when: nightly_build_nginx_auth_basic_enable - -- name: install nginx - apt: - name: nginx - state: present - when: nightly_build_nginx_enable - -- name: add nginx config - template: - src: nginx.conf - dest: /etc/nginx/sites-enabled/tbb-nightly-build - mode: 0644 - notify: - - restart nginx - when: nightly_build_nginx_enable diff --git a/tools/ansible/roles/tbb-nightly-build/templates/nginx.conf b/tools/ansible/roles/tbb-nightly-build/templates/nginx.conf deleted file mode 100644 index 62ca8e3..0000000 --- a/tools/ansible/roles/tbb-nightly-build/templates/nginx.conf +++ /dev/null @@ -1,20 +0,0 @@ -server { - listen {{ nightly_build_nginx_listen }}; - server_name {{ nightly_build_hostname }}; - root {{ nightly_build_wwwdir }}; - index index.html; - location / { - try_files $uri $uri/ =404; - } - location /reports { - alias {{ testsuite_dir }}/reports; - } - location /tor-browser-builds { - alias {{ testsuite_dir }}/tor-browser-builds; - autoindex on; - {% if nightly_build_nginx_auth_basic_enable %} - auth_basic "Use tor-guest as username and password"; - auth_basic_user_file /etc/nginx/tbb-nightly-build.auth_basic_user_file; - {% endif %} - } -} diff --git a/tools/ansible/roles/tbb-nightly-build/templates/rbm.local.conf b/tools/ansible/roles/tbb-nightly-build/templates/rbm.local.conf deleted file mode 100644 index 4380480..0000000 --- a/tools/ansible/roles/tbb-nightly-build/templates/rbm.local.conf +++ /dev/null @@ -1,114 +0,0 @@ ---- -### This file is used to override options from rbm.conf to adapt them -### to your local setup. -### -### Copy this file as rbm.local.conf to enable it, and uncomment the -### options you want to modify. - - -### The tmp_dir option defines where temporary files are stored. The -### builds are made from this directory, so using a directory on a fast -### disk can improve build time. By default we are using a tmp directory -### under the tor-browser-build directory. -#tmp_dir: /tmp - -### The debug option defines whether a debugging shell should be opened -### automatically in the build directory/container in case of build -### failure. If you are doing automated builds, you might want to disable -### this. -#debug: 0 - -### The build_log option defines in which file the build logs of each -### component are stored. If you set it to '-' the logs are output on -### stdout and stderr. -#build_log: '-' - -### By default, the logs from previous builds are kept in the log files. -### If you set build_log_append to 0, then previous logs are cleaned -### when starting a new build. -#build_log_append: 0 - -buildconf: - buildconf: 1 - - ### The buildconf/num_procs option can be used to select the number of - ### build processes to run simultaneously. You can also use the - ### RBM_NUM_PROCS environment variable. The default is 4. - #num_procs: 8 - - ### The buildconf/git_signtag_opt option is useful when you tag a release. - ### It is used to set the 'git tag' argument to select the keyid for - ### signing the tag. - #git_signtag_opt: '-u keyid' - -var: - local_conf: 1 - - ### The var/sign_build option defines if you want to sign the - ### sha256sums-unsigned-build.txt and - ### sha256sums-unsigned-build.incrementals.txt files with gpg. - {% if nightly_build_sign_build is defined %}sign_build: {{ nightly_build_sign_build }}{% endif %} - - - ### The var/sign_build_gpg_opts option can be used to define some gpg - ### options to select the key to use to sign the sha256sums-unsigned-build.txt - ### and sha256sums-unsigned-build.incrementals.txt files. - #sign_build_gpg_opts: '--local-user XXXXXXXX' - - ### The clean configuration is used by the cleaning script to find the - ### branches and build targets you are using, to compute the list of - ### files that should be kept. - ### - ### If you only do alpha builds for all platforms, you can use the - ### following configuration: - clean: - HEAD: - - project: release - target: - - nightly - - torbrowser-all - # - ### If you are doing 'release' builds in the maint-7.0 branch and - ### 'alpha' builds in the master branch, you can use the following - ### configuration: - #clean: - # master: - # - project: release - # target: - # - alpha - # - torbrowser-all - # maint-7.0: - # - project: release - # target: - # - release - # - torbrowser-all - -targets: - - ### testbuild is based on alpha by default. Uncomment this if you want it - ### to be based on nightly. - #torbrowser-testbuild: - # - testbuild - # - nightly - - testbuild: - var: - testbuild: 1 - - ### Uncomment this if you want to create mar files in your test build. - #build_mar: 1 - - nightly: - - ### By default 'fetch' is set to 1 for nightly builds, meaning that new - ### commits will be fetched automatically during each build. You can - ### disable this during development if you want to do rebuilds to test - ### a specific change, but don't want rebuilds caused by unrelated - ### changes, or if you want to decide at which point new commits are - ### fetched. When 'fetch' is set to 'if_needed', new commits will only - ### be fetched if the selected commit (or branch, or tag) is not present, - ### which means that existing branches won't be updated. In that case - ### you can fetch new commits by running 'make fetch'. - fetch: 'if_needed' - -# vim: filetype=yaml sw=2 diff --git a/tools/ansible/roles/tbb-nightly-build/templates/start-tbb-nightly b/tools/ansible/roles/tbb-nightly-build/templates/start-tbb-nightly deleted file mode 100644 index fafc1d9..0000000 --- a/tools/ansible/roles/tbb-nightly-build/templates/start-tbb-nightly +++ /dev/null @@ -1,5 +0,0 @@ -#!/bin/sh -cd {{ testsuite_dir }} -export RBM_NO_DEBUG=1 -./tbb-testsuite --config=tbb-nightly "$@" -/home/{{ nightly_build_user }}/prune-old-builds --days {{ nightly_build_keep_builds }} ./tor-browser-builds diff --git a/tools/ansible/roles/tbb-nightly-build/templates/tbb-nightly-build.auth_basic_user_file b/tools/ansible/roles/tbb-nightly-build/templates/tbb-nightly-build.auth_basic_user_file deleted file mode 100644 index 45d673c..0000000 --- a/tools/ansible/roles/tbb-nightly-build/templates/tbb-nightly-build.auth_basic_user_file +++ /dev/null @@ -1 +0,0 @@ -{{ nightly_build_nginx_auth_basic_content }} diff --git a/tools/ansible/roles/tbb-nightly-build/templates/testsuite-config b/tools/ansible/roles/tbb-nightly-build/templates/testsuite-config deleted file mode 100644 index 90b08ac..0000000 --- a/tools/ansible/roles/tbb-nightly-build/templates/testsuite-config +++ /dev/null @@ -1,34 +0,0 @@ -# vim: filetype=perl expandtab -use strict; -use FindBin; -use DateTime; -use TBBTestSuite::TestSuite::TorBrowserBuild; - -my $date = DateTime->now->ymd; -my $tbb_version = 'tbb-nightly.' . DateTime->now->ymd('.'); -my $name = "tor-browser-$date"; - -if (-d "$options->{'reports-dir'}/r/$name") { - print "Doing nothing: $name already done\n"; - return ( args => [] ); -} - -my $testsuite = TBBTestSuite::TestSuite::TorBrowserBuild->new({ - tbb_version => $tbb_version, - publish_dir => "$FindBin::Bin/tor-browser-builds/$tbb_version", - publish_url => "{{ nightly_build_url }}/tor-browser-builds/$tbb_version", - rbm_local_conf => "$FindBin::Bin/rbm-config/tbb-nightly.rbm.local.conf", - make_clean => 1, - }); - -my %res = ( - name => $name, - args => [ $testsuite ], - tags => [ 'nightly' ], - 'reports-url' => '{{ nightly_build_url }}/reports/', - 'email-subject' => '[build result: [% success ? "ok" : "failed" %]] [% options.name %]', - {% if nightly_build_email_to is defined %}'email-report' => 1,{% endif %} - {% if nightly_build_email_to is defined %}'email-to' => {{ nightly_build_email_to }}{% endif %} - {% if nightly_build_email_from is defined %}'email-from' => {{ nightly_build_email_from }}{% endif %} -); -%res; diff --git a/tools/ansible/roles/tbb-nightly-build/templates/www-index.html b/tools/ansible/roles/tbb-nightly-build/templates/www-index.html deleted file mode 100644 index 05050d8..0000000 --- a/tools/ansible/roles/tbb-nightly-build/templates/www-index.html +++ /dev/null @@ -1,10 +0,0 @@ -<html> -<head> - <title>Tor Browser Nightly Builds</title> -</head> -<body> - <h1>Tor Browser Nightly Builds</title> - <a href="reports/"><h3>Build Reports</h3></a> - <a href="tor-browser-builds/"><h3>Build files</h3></a> -</body> -</html> diff --git a/tools/ansible/vaulted_vars/boklm-tbb-nightly/dma-auth.yml b/tools/ansible/vaulted_vars/boklm-tbb-nightly/dma-auth.yml deleted file mode 100644 index 254291c..0000000 --- a/tools/ansible/vaulted_vars/boklm-tbb-nightly/dma-auth.yml +++ /dev/null @@ -1,10 +0,0 @@ -$ANSIBLE_VAULT;1.1;AES256 -64353537366566623534653938363036396164303631616138313130663766626463303034336564 -6339346639633765383534653561646366626665393333340a343533636436333838633039363265 -33393762363563323338356634396137393466616336326337323761643332363438313735646135 -6633326462616261310a663738306463613237326164663533326230316662333935333361636334 -61336433633964643631653230633861393131646532666536653738376261386535356636666262 -30303761333230623662323037376130386134373939613861343233363038636464623132363135 -66386532346165303839346563383934633462386534383330636432356166666238383332353930 -39316439653733376239343661373265303033323237366132366161316535636165336539333130 -3033

1 0

[Git][tpo/applications/fenix] Pushed new branch tor-browser-87.0.0b2-10.5-1
by Matthew Finkel 01 Mar '21

01 Mar '21

Matthew Finkel pushed new branch tor-browser-87.0.0b2-10.5-1 at The Tor Project / Applications / fenix -- View it on GitLab: https://gitlab.torproject.org/tpo/applications/fenix/-/tree/tor-browser-87.… You're receiving this email because of your account on gitlab.torproject.org.

1 0

[tor-browser-build/master] Bug 40209: Sign mar files from tb-build-01
by gk＠torproject.org 27 Feb '21

27 Feb '21

commit bf04408b47b53fb7f7a93eeeb8d9f9555326f0a9 Author: Nicolas Vigier <boklm(a)torproject.org> Date: Thu Feb 25 18:00:20 2021 +0100 Bug 40209: Sign mar files from tb-build-01 --- keyring/torbrowser-nightly.gpg | Bin 3523 -> 1189 bytes tools/signing/nightly/config.yml | 2 +- 2 files changed, 1 insertion(+), 1 deletion(-) diff --git a/keyring/torbrowser-nightly.gpg b/keyring/torbrowser-nightly.gpg index 952db15..9edccf6 100644 Binary files a/keyring/torbrowser-nightly.gpg and b/keyring/torbrowser-nightly.gpg differ diff --git a/tools/signing/nightly/config.yml b/tools/signing/nightly/config.yml index 540728c..bf1db20 100644 --- a/tools/signing/nightly/config.yml +++ b/tools/signing/nightly/config.yml @@ -2,7 +2,7 @@ martools_version: 9.0.2 martools_url: https://archive.torproject.org/tor-package-archive/torbrowser/ martools_gpg_keyring: keyring/torbrowser.gpg -builds_url: http://f4amtbsowhix7rrf.onion/tor-browser-builds +builds_url: /srv/tbb-nightlies-master.torproject.org/htdocs/nightly-builds/tor-browser-builds builds_url_auth_basic_username: tor-guest builds_url_auth_basic_password: tor-guest publish_dirs:

1 0

[tor-browser-build/master] Bug 23631: Use rootless containers
by gk＠torproject.org 25 Feb '21

25 Feb '21

commit 90c077db1d398041c75e6e4ebc56fb80570eb027 Author: Nicolas Vigier <boklm(a)torproject.org> Date: Tue Jun 16 20:03:56 2020 +0200 Bug 23631: Use rootless containers --- README | 20 +- README.HACKING | 23 ++ projects/common/runc-config.json | 267 --------------------- projects/container-image/config | 4 +- projects/debootstrap-image/config | 80 ------ .../{debootstrap-image => mmdebstrap-image}/build | 0 projects/mmdebstrap-image/config | 59 +++++ .../mmdebstrap-image/mmdebstrap-no-user-ns.patch | 30 +++ projects/mmdebstrap/build | 8 + projects/mmdebstrap/config | 5 + rbm | 2 +- rbm.conf | 92 +++---- 12 files changed, 172 insertions(+), 418 deletions(-) diff --git a/README b/README index e37f20d..8ebccf6 100644 --- a/README +++ b/README @@ -4,12 +4,10 @@ Tor Browser Build Installing build dependencies ----------------------------- -To build Tor Browser, you need a Linux distribution that has support -for runc (such as Debian Buster, Ubuntu 16.04, Fedora 30, etc ...). - -Your user account should have sudo access, which is required to be able -to extract container file systems, start containers and copy files to and -from containers. +To build Tor Browser, you need a recent Linux distribution with support +for user_namespaces(7) (such as Debian Buster, Ubuntu 16.04, Fedora 30, +etc ...). You will need to install the uidmap package, providing the +newuidmap and newgidmap commands. The sources of most components are downloaded using git, which needs to be installed. @@ -40,12 +38,20 @@ If you are running Debian or Ubuntu, you can install them with: libio-captureoutput-perl libjson-perl libpath-tiny-perl \ libstring-shellquote-perl libsort-versions-perl \ libdigest-sha-perl libdata-uuid-perl libdata-dump-perl \ - libfile-copy-recursive-perl libfile-slurp-perl git runc + libfile-copy-recursive-perl libfile-slurp-perl git \ + uidmap The build system is based on rbm, which is included as a git submodule in the rbm/ directory. You can fetch the rbm git submodule by running 'make submodule-update'. +The build uses user_namespaces(7), which are disabled by default on Debian. +To enable them you can use the following command as root: + + # sysctl -w kernel.unprivileged_userns_clone=1 + +You can enable them permanently by adding the setting to /etc/sysctl.d/ + Starting a build ---------------- diff --git a/README.HACKING b/README.HACKING index 0d3eeb8..506502d 100644 --- a/README.HACKING +++ b/README.HACKING @@ -305,6 +305,29 @@ The path to the container should be printed on the screen in case you want to backup its rootfs to be able to look at it later. +Manually removing old containers +-------------------------------- + +When a build finishes or when you exit a debugging shell, the old +container should automatically be removed. In some cases however, for +example your computer is rebooted in the middle of a build, some old +container directories may be left in the tmp directory. Some of the +files in the container directories are owned by subordinate user ids +(see the subuid man page), which will prevent you from removing them +with your normal user id. To remove them you can open a container +shell (a new User namespace) using the following command: + + $ ./rbm/container run -- /bin/bash + +From this shell you should be able to remove the old containers +directories in the tmp directory. + +It is also possible to pass the rm command directly without opening a +shell: + + $ ./rbm/container run -- rm -Rf ./tmp/rbm-* + + Testing an rbm patch -------------------- diff --git a/projects/common/runc-config.json b/projects/common/runc-config.json deleted file mode 100644 index b078b97..0000000 --- a/projects/common/runc-config.json +++ /dev/null @@ -1,267 +0,0 @@ -{ - "ociVersion": "1.0.0[% IF !c("var_p/runc_spec100") %]-rc1[% END %]", - "platform": { - "os": "linux", - "arch": "amd64" - }, - "process": { - "terminal": [% IF c("interactive") %]true[% ELSE %]false[% END %], - "user": { - "uid": 0, - "gid": 0 - }, - "args": [ - "/rbm/run" - ], - "env": [ - "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", - "TERM=xterm" - ], - "cwd": "/", -[% IF c("var_p/runc_spec100") -%] - "capabilities": { - "bounding": [ - "CAP_AUDIT_WRITE", - "CAP_KILL", - "CAP_NET_BIND_SERVICE", - "CAP_SETGID", - "CAP_SETUID", - "CAP_MKNOD", - "CAP_SYS_CHROOT", -[% IF c("var/container/CAP_SYS_ADMIN") -%] - "CAP_SYS_ADMIN", -[% END -%] - "CAP_FSETID", - "CAP_FOWNER", - "CAP_DAC_OVERRIDE", - "CAP_CHOWN" - ], - "effective": [ - "CAP_AUDIT_WRITE", - "CAP_KILL", - "CAP_NET_BIND_SERVICE", - "CAP_SETGID", - "CAP_SETUID", - "CAP_MKNOD", - "CAP_SYS_CHROOT", -[% IF c("var/container/CAP_SYS_ADMIN") -%] - "CAP_SYS_ADMIN", -[% END -%] - "CAP_FSETID", - "CAP_FOWNER", - "CAP_DAC_OVERRIDE", - "CAP_CHOWN" - ], - "inheritable": [ - "CAP_AUDIT_WRITE", - "CAP_KILL", - "CAP_NET_BIND_SERVICE", - "CAP_SETGID", - "CAP_SETUID", - "CAP_MKNOD", - "CAP_SYS_CHROOT", -[% IF c("var/container/CAP_SYS_ADMIN") -%] - "CAP_SYS_ADMIN", -[% END -%] - "CAP_FSETID", - "CAP_FOWNER", - "CAP_DAC_OVERRIDE", - "CAP_CHOWN" - ], - "permitted": [ - "CAP_AUDIT_WRITE", - "CAP_KILL", - "CAP_NET_BIND_SERVICE", - "CAP_SETGID", - "CAP_SETUID", - "CAP_MKNOD", - "CAP_SYS_CHROOT", -[% IF c("var/container/CAP_SYS_ADMIN") -%] - "CAP_SYS_ADMIN", -[% END -%] - "CAP_FSETID", - "CAP_FOWNER", - "CAP_DAC_OVERRIDE", - "CAP_CHOWN" - ], - "ambient": [ - "CAP_AUDIT_WRITE", - "CAP_KILL", - "CAP_NET_BIND_SERVICE", - "CAP_SETGID", - "CAP_SETUID", - "CAP_MKNOD", - "CAP_SYS_CHROOT", -[% IF c("var/container/CAP_SYS_ADMIN") -%] - "CAP_SYS_ADMIN", -[% END -%] - "CAP_FSETID", - "CAP_FOWNER", - "CAP_DAC_OVERRIDE", - "CAP_CHOWN" - ] - }, -[% ELSE -%] - "capabilities": [ - "CAP_AUDIT_WRITE", - "CAP_KILL", - "CAP_NET_BIND_SERVICE", - "CAP_SETGID", - "CAP_SETUID", - "CAP_MKNOD", - "CAP_SYS_CHROOT", -[% IF c("var/container/CAP_SYS_ADMIN") -%] - "CAP_SYS_ADMIN", -[% END -%] - "CAP_FSETID", - "CAP_FOWNER", - "CAP_DAC_OVERRIDE", - "CAP_CHOWN" - ], -[% END -%] - "noNewPrivileges": true - }, - "root": { - "path": "rootfs", - "readonly": false - }, - "hostname": "runc", - "mounts": [ - { - "destination": "/proc", - "type": "proc", - "source": "proc" - }, - { - "type": "bind", - "source": "/etc/resolv.conf", - "destination": "/etc/resolv.conf", - "options": [ - "rbind", - "ro" - ] - }, - { - "destination": "/dev", - "type": "tmpfs", - "source": "tmpfs", - "options": [ - "nosuid", - "strictatime", - "mode=755", - "size=65536k" - ] - }, - { - "destination": "/dev/pts", - "type": "devpts", - "source": "devpts", - "options": [ - "nosuid", - "noexec", - "newinstance", - "ptmxmode=0666", - "mode=0620", - "gid=5" - ] - }, - { - "destination": "/dev/shm", - "type": "tmpfs", - "source": "shm", - "options": [ - "nosuid", - "noexec", - "nodev", - "mode=1777", - "size=65536k" - ] - }, - { - "destination": "/dev/mqueue", - "type": "mqueue", - "source": "mqueue", - "options": [ - "nosuid", - "noexec", - "nodev" - ] - }, - { - "destination": "/sys", - "type": "sysfs", - "source": "sysfs", - "options": [ - "nosuid", - "noexec", - "nodev", - "ro" - ] - }, - { - "destination": "/sys/fs/cgroup", - "type": "cgroup", - "source": "cgroup", - "options": [ - "nosuid", - "noexec", - "nodev", - "relatime", - "ro" - ] - } - ], - "hooks": {}, - "linux": { - "resources": { - "devices": [ - { - "allow": false, - "access": "rwm" - } - ] - }, - "namespaces": [ - { - "type": "pid" - }, - { - "type": "ipc" - }, - { - "type": "uts" - }, -[% IF c("var/container/disable_network/" _ c("exec_name")) -%] - { - "type": "network", - "path": "/var/run/netns/rbm-[% sha256(c("build_id", { error_if_undef => 1 })) %]" - }, -[% END -%] - { - "type": "mount" - } - ], - "maskedPaths": [ - "/proc/kcore", - "/proc/latency_stats", - "/proc/timer_stats", -[% IF c("var_p/runc_spec100") -%] - "/proc/timer_list", - "/sys/firmware", -[% END -%] - "/proc/sched_debug" - ], - "readonlyPaths": [ - "/proc/asound", - "/proc/bus", - "/proc/fs", - "/proc/irq", - "/proc/sys", - "/proc/sysrq-trigger" - ] - }, - "solaris": { - "cappedCPU": {}, - "cappedMemory": {} - } -} diff --git a/projects/container-image/config b/projects/container-image/config index 7520b8e..e0b8b13 100644 --- a/projects/container-image/config +++ b/projects/container-image/config @@ -70,9 +70,9 @@ remote_get: | SET dst = shell_quote(c('get_dst', { error_if_undef => 1 })); -%] mkdir -p "[% dst %]" - sudo tar -C "[% c("var/container/dir") %]/rootfs" -czf "[% dst %]/[% c("filename") %]" . + [% c("rbmdir") %]/container archive '[% c("var/container/dir") %]' "[% dst %]/[% c("filename") %]" input_files: - - project: debootstrap-image + - project: mmdebstrap-image target: - '[% c("var/container/suite") %]-[% c("var/container/arch") %]' diff --git a/projects/debootstrap-image/config b/projects/debootstrap-image/config deleted file mode 100644 index 5098e29..0000000 --- a/projects/debootstrap-image/config +++ /dev/null @@ -1,80 +0,0 @@ -# vim: filetype=yaml sw=2 -filename: 'container-image_[% c("var/container/suite") %]-[% c("var/container/arch") %]-[% c("version") %].tar.gz' -version: 3 -pkg_type: build - -var: - ubuntu_version: 20.04.1 - - container: - use_container: 1 - # We need CAP_SYS_ADMIN for debootstrap to work - CAP_SYS_ADMIN: 1 - -pre: | - #!/bin/sh - set -e - export DEBIAN_FRONTEND=noninteractive - apt-get update -y -q - apt-get install -y -q debian-archive-keyring ubuntu-keyring debootstrap - debootstrap --arch=[% c("var/container/arch") %] [% c("var/container/debootstrap_opt") %] [% c("var/container/suite") %] base-image [% c("var/container/debootstrap_mirror") %] - [% IF c("var/apt_package_filename") || c("var/apt_utils_package_filename") || c("var/libapt_inst_package_filename") || c("var/libapt_pkg_package_filename") -%] - mkdir ./base-image/apt-update - mv [% c("var/apt_package_filename") %] [% c("var/apt_utils_package_filename") %] \ - [% c("var/libapt_inst_package_filename") %] [% c("var/libapt_pkg_package_filename") %] \ - ./base-image/apt-update - mount proc ./base-image/proc -t proc - mount sysfs ./base-image/sys -t sysfs - chroot ./base-image dpkg -i -R /apt-update - umount ./base-image/proc - umount ./base-image/sys - [% END -%] - [% IF c("var/minimal_apt_version") -%] - apt_version=$(dpkg --admindir=$(pwd)/base-image/var/lib/dpkg -s apt | grep '^Version: ' | cut -d ' ' -f 2) - echo "apt version: $apt_version" - dpkg --compare-versions "$apt_version" ge '[% c("var/minimal_apt_version") %]' - [% END -%] - tar -C ./base-image -czf [% dest_dir %]/[% c("filename") %] . - -targets: - jessie-amd64: - var: - minimal_apt_version: 1.0.9.8.7 - # CVE-2020-27350 - apt_packages_baseurl: https://deb.freexian.com/extended-lts/pool/main/a/apt - apt_package_filename: apt_1.0.9.8.7_amd64.deb - apt_package_sha256sum: 02d5c6240e3410234e91facd4279c1acdce1523b583e05fac7ad64bdc75db7c7 - apt_utils_package_filename: apt-utils_1.0.9.8.7_amd64.deb - apt_utils_package_sha256sum: aa6f0cc7cf2e7cc6e5c56782019cd651d26b01d280aef4d25288587e198ff2d3 - libapt_inst_package_filename: libapt-inst1.5_1.0.9.8.7_amd64.deb - libapt_inst_package_sha256sum: a85c99a64f3d6cdde5bf0849649c2d5f21c7513987854628659ea014bc3eb214 - libapt_pkg_package_filename: libapt-pkg4.12_1.0.9.8.7_amd64.deb - libapt_pkg_package_sha256sum: 0db2410376516f3420594647afc72dc99b48a44877eae4e8c79444a8244e988f - - container: - suite: jessie - arch: amd64 - - buster-amd64: - var: - minimal_apt_version: 1.8.2 - container: - suite: buster - arch: amd64 - -input_files: - - URL: 'https://cdimage.ubuntu.com/ubuntu-base/releases/[% c("var/ubuntu_version") %]/release/ubuntu-base-[% c("var/ubuntu_version") %]-base-amd64.tar.gz' - filename: 'container-image_ubuntu-base-[% c("var/ubuntu_version") %]-base-amd64.tar.gz' - sha256sum: 97f6cbcf41f39272c84e32b6f926ed5d6906c6e463a557025181f4aa191ad667 - - URL: '[% c("var/apt_packages_baseurl") %]/[% c("var/apt_package_filename") %]' - sha256sum: '[% c("var/apt_package_sha256sum") %]' - enable: '[% c("var/apt_package_filename") %]' - - URL: '[% c("var/apt_packages_baseurl") %]/[% c("var/apt_utils_package_filename") %]' - sha256sum: '[% c("var/apt_utils_package_sha256sum") %]' - enable: '[% c("var/apt_utils_package_filename") %]' - - URL: '[% c("var/apt_packages_baseurl") %]/[% c("var/libapt_inst_package_filename") %]' - sha256sum: '[% c("var/libapt_inst_package_sha256sum") %]' - enable: '[% c("var/libapt_inst_package_filename") %]' - - URL: '[% c("var/apt_packages_baseurl") %]/[% c("var/libapt_pkg_package_filename") %]' - sha256sum: '[% c("var/libapt_pkg_package_sha256sum") %]' - enable: '[% c("var/libapt_pkg_package_filename") %]' diff --git a/projects/debootstrap-image/build b/projects/mmdebstrap-image/build similarity index 100% rename from projects/debootstrap-image/build rename to projects/mmdebstrap-image/build diff --git a/projects/mmdebstrap-image/config b/projects/mmdebstrap-image/config new file mode 100644 index 0000000..102053d --- /dev/null +++ b/projects/mmdebstrap-image/config @@ -0,0 +1,59 @@ +# vim: filetype=yaml sw=2 +filename: 'container-image_[% c("var/container/suite") %]-[% c("var/container/arch") %]-[% c("version") %].tar.gz' +version: 1 +pkg_type: build + +var: + ubuntu_version: 20.04.1 + + container: + use_container: 1 + +pre: | + #!/bin/sh + set -e + rootdir=$(pwd) + export DEBIAN_FRONTEND=noninteractive + apt-get update -y -q + apt-get install -y -q debian-archive-keyring ubuntu-keyring mmdebstrap gnupg patch + + export SOURCE_DATE_EPOCH='[% c("timestamp") %]' + tar -xf [% c('input_files_by_name/mmdebstrap') %] + # As a user namespace inside an other user namespace doesn't seem to work, + # we patch mmdebstrap to avoid creating a new user namespace + patch -d mmdebstrap -p1 < mmdebstrap-no-user-ns.patch + ./mmdebstrap/mmdebstrap --mode=unshare [% c("var/container/mmdebstrap_opt") %] [% c("var/container/suite") %] output.tar.gz [% c("var/container/debian_mirror") %] + + [% IF c("var/minimal_apt_version") -%] + mkdir base-image + tar -C base-image -xf output.tar.gz ./var/lib/dpkg + apt_version=$(dpkg --admindir=$rootdir/base-image/var/lib/dpkg -s apt | grep '^Version: ' | cut -d ' ' -f 2) + echo "apt version: $apt_version" + dpkg --compare-versions "$apt_version" ge '[% c("var/minimal_apt_version") %]' + [% END -%] + + mv output.tar.gz [% dest_dir %]/[% c("filename") %] + +targets: + jessie-amd64: + var: + minimal_apt_version: 1.0.9.8.6 + + container: + suite: jessie + arch: amd64 + + buster-amd64: + var: + minimal_apt_version: 1.8.2 + container: + suite: buster + arch: amd64 + +input_files: + - project: mmdebstrap + name: mmdebstrap + - URL: 'https://cdimage.ubuntu.com/ubuntu-base/releases/[% c("var/ubuntu_version") %]/release/ubuntu-base-[% c("var/ubuntu_version") %]-base-amd64.tar.gz' + filename: 'container-image_ubuntu-base-[% c("var/ubuntu_version") %]-base-amd64.tar.gz' + sha256sum: 97f6cbcf41f39272c84e32b6f926ed5d6906c6e463a557025181f4aa191ad667 + - filename: mmdebstrap-no-user-ns.patch diff --git a/projects/mmdebstrap-image/mmdebstrap-no-user-ns.patch b/projects/mmdebstrap-image/mmdebstrap-no-user-ns.patch new file mode 100644 index 0000000..15c547e --- /dev/null +++ b/projects/mmdebstrap-image/mmdebstrap-no-user-ns.patch @@ -0,0 +1,30 @@ +diff --git a/mmdebstrap b/mmdebstrap +index 9b7d892..5d85a74 100755 +--- a/mmdebstrap ++++ b/mmdebstrap +@@ -204,6 +204,7 @@ sub get_tar_compressor { + } + + sub test_unshare { ++ return 1; + my $verbose = shift; + if ($EFFECTIVE_USER_ID == 0) { + my $msg = "cannot use unshare mode when executing as root"; +@@ -382,7 +383,7 @@ sub get_unshare_cmd { + my $idmap = shift; + + my $unshare_flags +- = $CLONE_NEWUSER | $CLONE_NEWNS | $CLONE_NEWPID | $CLONE_NEWUTS ++ = $CLONE_NEWNS | $CLONE_NEWPID | $CLONE_NEWUTS + | $CLONE_NEWIPC; + + if (0) { +@@ -457,6 +458,8 @@ sub get_unshare_cmd { + $gidmapcmd .= " $hostid $nsid $range"; + } + } ++ $uidmapcmd = ""; ++ $gidmapcmd = ""; + my $idmapcmd = ''; + if ($uidmapcmd ne "") { + 0 == system "newuidmap $ppid $uidmapcmd" diff --git a/projects/mmdebstrap/build b/projects/mmdebstrap/build new file mode 100644 index 0000000..ac0f21b --- /dev/null +++ b/projects/mmdebstrap/build @@ -0,0 +1,8 @@ +#!/bin/bash +[% c("var/set_default_env") -%] +tar xf [% project %]-[% c("version") %].tar.gz +mv [% project %]-[% c("version") %] [% project %] +[% c('tar', { + tar_src => [ project ], + tar_args => '-czf ' _ dest_dir _ '/' _ c('filename'), + }) %] diff --git a/projects/mmdebstrap/config b/projects/mmdebstrap/config new file mode 100644 index 0000000..8fbfc97 --- /dev/null +++ b/projects/mmdebstrap/config @@ -0,0 +1,5 @@ +# vim: filetype=yaml sw=2 +version: 655857e5259436755f34b908d07a6ebd4c8b0303 +git_url: http://gitlab.mister-muffin.de/josch/mmdebstrap.git +git_hash: '[% c("version") %]' +filename: '[% project %]-src-[% c("version") %]-[% c("var/build_id") %].tar.gz' diff --git a/rbm b/rbm index 15c8476..cfb4018 160000 --- a/rbm +++ b/rbm @@ -1 +1 @@ -Subproject commit 15c84760ebe3f2677d1212aa00ef6f7485446a69 +Subproject commit cfb4018d34400e43837e77754519b3c2761b7ae6 diff --git a/rbm.conf b/rbm.conf index 14a55a6..ba13cc0 100644 --- a/rbm.conf +++ b/rbm.conf @@ -495,13 +495,13 @@ gpg_wrapper: | exec [% c('gpg_bin') %] [% c('gpg_args') %] --with-fingerprint [% gpg_kr %] "$@" fi -remote_start: '[% IF c("var/container/use_container") && ! c("var/container/global_disable") %][% c("runc/remote_start") %][% END %]' -remote_exec: '[% IF c("var/container/use_container") && ! c("var/container/global_disable") %][% c("runc/remote_exec") %][% END %]' -remote_put: '[% IF c("var/container/use_container") && ! c("var/container/global_disable") %][% c("runc/remote_put") %][% END %]' -remote_get: '[% IF c("var/container/use_container") && ! c("var/container/global_disable") %][% c("runc/remote_get") %][% END %]' -remote_finish: '[% IF c("var/container/use_container") && ! c("var/container/global_disable") %][% c("runc/remote_finish") %][% END %]' +remote_start: '[% IF c("var/container/use_container") && ! c("var/container/global_disable") %][% c("container/remote_start") %][% END %]' +remote_exec: '[% IF c("var/container/use_container") && ! c("var/container/global_disable") %][% c("container/remote_exec") %][% END %]' +remote_put: '[% IF c("var/container/use_container") && ! c("var/container/global_disable") %][% c("container/remote_put") %][% END %]' +remote_get: '[% IF c("var/container/use_container") && ! c("var/container/global_disable") %][% c("container/remote_get") %][% END %]' +remote_finish: '[% IF c("var/container/use_container") && ! c("var/container/global_disable") %][% c("container/remote_finish") %][% END %]' -runc: +container: remote_start: | #!/bin/sh set -e @@ -511,11 +511,9 @@ runc: ls -l '[% c("remote_srcdir") %]' >&2 exit 1 fi - mkdir -p '[% c("var/container/dir") %]'/rootfs/rbm - sudo tar -C '[% c("var/container/dir") %]'/rootfs -xf $(ls -1 '[% c("remote_srcdir", { error_if_undef => 1 }) %]/container-image_'*) - [% SET user = c("var/container/user") -%] - [% c("remote_exec", { exec_as_root => 1, exec_cmd => 'id ' _ user - _ ' >/dev/null 2>&1 || adduser -m ' _ user _ ' || useradd -m ' _ user }) %] + [% c("rbmdir") %]/container extract '[% c("var/container/dir") %]' '[% c("remote_srcdir", { error_if_undef => 1 }) %]/container-image_'* + test -d '[% c("var/container/dir") %]'/home/rbm || \ + [% c("rbmdir") %]/container run --chroot='[% c("var/container/dir") %]' -- /usr/sbin/useradd -m [% c("var/container/user") %] remote_exec: | #!/bin/sh @@ -523,26 +521,27 @@ runc: [% IF c("interactive") -%] echo Container directory: [% shell_quote(c("var/container/dir")) %] [% END -%] - mkdir -p '[% c("var/container/dir", { error_if_undef => 1 }) %]'/rootfs/rbm - echo '#!/bin/sh' > '[% c("var/container/dir") %]'/rootfs/rbm/cmd - echo [% shell_quote(c('exec_cmd')) %] >> '[% c("var/container/dir") %]'/rootfs/rbm/cmd - echo '#!/bin/sh' > '[% c("var/container/dir") %]'/rootfs/rbm/run - [% IF c('exec_as_root'); SET user = 'root'; ELSE; SET user = c("var/container/user", { error_if_undef => 1 }); END; %] - echo 'su - [% user %] -c /rbm/cmd' >> '[% c("var/container/dir") %]'/rootfs/rbm/run - chmod +x '[% c("var/container/dir") %]'/rootfs/rbm/cmd - chmod +x '[% c("var/container/dir") %]'/rootfs/rbm/run - cat > '[% c("var/container/dir") %]'/config.json << EOF - [% INCLUDE 'runc-config.json' %] - EOF - [% IF c("var/container/disable_network/" _ c("exec_name")) -%] - sudo ip netns add 'rbm-[% sha256(c("build_id", { error_if_undef => 1 })) %]' - # make sure the lo interface is up (see bug 31293) - sudo ip netns exec 'rbm-[% sha256(c("build_id", { error_if_undef => 1 })) %]' ip link set lo up - [% END -%] - sudo runc [% IF c("var_p/runc100") %]run[% ELSE %]start[% END %] -b '[% c("var/container/dir") %]' rbm-[% sha256(c("build_id", { error_if_undef => 1 })) %] [% IF c("runc_hide_stderr") %]2>/dev/null[% END %] + mkdir -p '[% c("var/container/dir", { error_if_undef => 1 }) %]'/rbm + echo '#!/bin/sh' > '[% c("var/container/dir") %]'/rbm/cmd + echo [% shell_quote(c('exec_cmd')) %] >> '[% c("var/container/dir") %]'/rbm/cmd + echo '#!/bin/sh' > '[% c("var/container/dir") %]'/rbm/run [% IF c("var/container/disable_network/" _ c("exec_name")) -%] - sudo ip netns delete 'rbm-[% sha256(c("build_id", { error_if_undef => 1 })) %]' + # Some programs such as gradle need the lo interface to be up. + # See for example tor-browser#31293 + echo 'ip link set lo up' >> '[% c("var/container/dir") %]'/rbm/run [% END -%] + [% IF c('exec_as_root'); SET user = 'root'; ELSE; SET user = c("var/container/user", { error_if_undef => 1 }); END; %] + echo 'su - [% user %] -c /rbm/cmd' >> '[% c("var/container/dir") %]'/rbm/run + chmod +x '[% c("var/container/dir") %]'/rbm/cmd + chmod +x '[% c("var/container/dir") %]'/rbm/run + [% + IF c("var/container/disable_network/" _ c("exec_name")); + SET disable_network = '--disable-network'; + ELSE; + SET disable_network = ''; + END; + -%] + [% c("rbmdir") %]/container run [% disable_network %] --chroot='[% c("var/container/dir") %]' -- /rbm/run remote_put: | #!/bin/sh @@ -551,12 +550,7 @@ runc: SET src = shell_quote(c('put_src', { error_if_undef => 1 })); SET dst = shell_quote(c('put_dst', { error_if_undef => 1 })); -%] - sudo mkdir -p '[% c("var/container/dir") %]'/rootfs/[% dst %] - sudo cp -aP [% src %] '[% c("var/container/dir") %]'/rootfs/[% dst %] - # On Ubuntu, the /root/.profile file contains a `mesg n` line which is - # producing some `stdin: is not a tty` messages. To hide them, we hide - # stderr from this part by setting runc_hide_stderr. - [% c("remote_exec", { exec_as_root => 1, exec_cmd => 'chown -R ' _ c("var/container/user") _ ' ' _ dst, runc_hide_stderr => 1 }) %] + [% c("rbmdir") %]/container put '[% c("var/container/dir") %]' [% src %] [% dst %] [% c("var/container/user") %] remote_get: | #!/bin/sh @@ -565,24 +559,12 @@ runc: SET src = shell_quote(c('get_src', { error_if_undef => 1 })); SET dst = shell_quote(c('get_dst', { error_if_undef => 1 })); -%] - mkdir -p [% dst %] - srcdir='[% c("var/container/dir", { error_if_undef => 1 }) %]'/rootfs/[% src %] - sudo chown -R $(whoami) "$srcdir" - if [ $(ls -1 "$srcdir"/* 2> /dev/null | wc -l) -gt 0 ] - then - for file in "$srcdir"/* - do - bname="$(basename "$file")" - test -e [% dst %]/"$bname" && rm -Rf [% dst %]/"$bname" - mv -f "$file" [% dst %]/ - done - fi + [% c("rbmdir") %]/container get '[% c("var/container/dir") %]' [% src %] [% dst %] remote_finish: | #!/bin/sh set -e - sudo rm -Rf '[% c("var/container/dir", { error_if_undef => 1 }) %]'/rootfs '[% c("var/container/dir", { error_if_undef => 1 }) %]'/config.json - rmdir '[% c("var/container/dir") %]' + [% c("rbmdir") %]/container remove '[% c("var/container/dir") %]' ENV: TZ: UTC @@ -592,18 +574,6 @@ ENV: use IO::CaptureOutput qw(capture_exec); ( var_p => { - # runc100 is true if we are using runc >= 1.0.0 - # we assume that any version that is not 0.1.1 is >= 1.0.0 - runc100 => sub { - my ($out) = capture_exec('sudo', 'runc', '--version'); - return !($out =~ m/^runc version 0.1.1/); - }, - # runc_spec100 is true if runc spec is at least 1.0.0 - # We will need to update this when there is a new spec version available - runc_spec100 => sub { - my ($out) = capture_exec('sudo', 'runc', '--version'); - return $out =~ m/^.*spec: 1\.[0-9]+\.[0-9]+(?:-dev)?$/m; - }, nightly_torbrowser_version => sub { state $version = ''; return $version if $version;

1 0

[tor-browser-bundle-testsuite/master] Bug 40015: Add config for tb-build-01.tpo nightly builds
by gk＠torproject.org 25 Feb '21

25 Feb '21

commit 3880bda3ef2f40b14632ceb445b5837a6ad643d5 Author: Nicolas Vigier <boklm(a)torproject.org> Date: Fri Jan 15 21:36:55 2021 +0100 Bug 40015: Add config for tb-build-01.tpo nightly builds --- config/tb-build-01.torproject.org | 38 +++++++ .../tb-build-01.torproject.org.rbm.local.conf | 113 +++++++++++++++++++++ 2 files changed, 151 insertions(+) diff --git a/config/tb-build-01.torproject.org b/config/tb-build-01.torproject.org new file mode 100644 index 0000000..62f8bc1 --- /dev/null +++ b/config/tb-build-01.torproject.org @@ -0,0 +1,38 @@ +# vim: filetype=perl expandtab +use strict; +use FindBin; +use DateTime; +use TBBTestSuite::TestSuite::TorBrowserBuild; + +my $date = DateTime->now->ymd; +my $tbb_version = 'tbb-nightly.' . DateTime->now->ymd('.'); +my $name = "tor-browser-$date"; + +my $builds_dir_root = '/home/tb-builder/nightly-builds'; +my $reports_dir = "$builds_dir_root/reports"; + +if (-d "$reports_dir/r/$name") { + print "Doing nothing: $name already done\n"; + return ( args => [] ); +} + +my $testsuite = TBBTestSuite::TestSuite::TorBrowserBuild->new({ + tbb_version => $tbb_version, + publish_dir => "$builds_dir_root/tor-browser-builds/$tbb_version", + publish_url => "https://nightlies.tbb.torproject.org/nightly-builds/tor-browser-builds/$tbb…", + rbm_local_conf => "$FindBin::Bin/rbm-config/tb-build-01.torproject.org.rbm.local.conf", + make_clean => 1, + }); + +my %res = ( + name => $name, + args => [ $testsuite ], + tags => [ 'nightly' ], + 'reports-dir' => $reports_dir, + 'reports-url' => 'https://nightlies.tbb.torproject.org/nightly-builds/reports/', + 'email-subject' => '[build result: [% success ? "ok" : "failed" %]] [% options.name %]', + 'email-report' => 1, + 'email-to' => [ 'boklm(a)torproject.org', 'gk(a)torproject.org', 'sysrqb(a)torproject.org', ], + 'email-from' => 'Tor Browser Nightly Builds <tb-builder(a)tb-build-01.torproject.org>', +); +%res; diff --git a/rbm-config/tb-build-01.torproject.org.rbm.local.conf b/rbm-config/tb-build-01.torproject.org.rbm.local.conf new file mode 100644 index 0000000..1d323d3 --- /dev/null +++ b/rbm-config/tb-build-01.torproject.org.rbm.local.conf @@ -0,0 +1,113 @@ +--- +### This file is used to override options from rbm.conf to adapt them +### to your local setup. +### +### Copy this file as rbm.local.conf to enable it, and uncomment the +### options you want to modify. + + +### The tmp_dir option defines where temporary files are stored. The +### builds are made from this directory, so using a directory on a fast +### disk can improve build time. By default we are using a tmp directory +### under the tor-browser-build directory. +#tmp_dir: /tmp + +### The debug option defines whether a debugging shell should be opened +### automatically in the build directory/container in case of build +### failure. If you are doing automated builds, you might want to disable +### this. +#debug: 0 + +### The build_log option defines in which file the build logs of each +### component are stored. If you set it to '-' the logs are output on +### stdout and stderr. +#build_log: '-' + +### By default, the logs from previous builds are kept in the log files. +### If you set build_log_append to 0, then previous logs are cleaned +### when starting a new build. +#build_log_append: 0 + +buildconf: + buildconf: 1 + + ### The buildconf/num_procs option can be used to select the number of + ### build processes to run simultaneously. You can also use the + ### RBM_NUM_PROCS environment variable. The default is 4. + num_procs: 8 + + ### The buildconf/git_signtag_opt option is useful when you tag a release. + ### It is used to set the 'git tag' argument to select the keyid for + ### signing the tag. + #git_signtag_opt: '-u keyid' + +var: + local_conf: 1 + + ### The var/sign_build option defines if you want to sign the + ### sha256sums-unsigned-build.txt and + ### sha256sums-unsigned-build.incrementals.txt files with gpg. + sign_build: 1 + + ### The var/sign_build_gpg_opts option can be used to define some gpg + ### options to select the key to use to sign the sha256sums-unsigned-build.txt + ### and sha256sums-unsigned-build.incrementals.txt files. + #sign_build_gpg_opts: '--local-user XXXXXXXX' + + ### The clean configuration is used by the cleaning script to find the + ### branches and build targets you are using, to compute the list of + ### files that should be kept. + ### + ### If you only do alpha builds for all platforms, you can use the + ### following configuration: + clean: + HEAD: + - project: release + target: + - nightly + - torbrowser-all + # + ### If you are doing 'release' builds in the maint-7.0 branch and + ### 'alpha' builds in the master branch, you can use the following + ### configuration: + #clean: + # master: + # - project: release + # target: + # - alpha + # - torbrowser-all + # maint-7.0: + # - project: release + # target: + # - release + # - torbrowser-all + +targets: + + ### testbuild is based on alpha by default. Uncomment this if you want it + ### to be based on nightly. + #torbrowser-testbuild: + # - testbuild + # - nightly + + testbuild: + var: + testbuild: 1 + + ### Uncomment this if you want to create mar files in your test build. + #build_mar: 1 + + nightly: + + ### By default 'fetch' is set to 1 for nightly builds, meaning that new + ### commits will be fetched automatically during each build. You can + ### disable this during development if you want to do rebuilds to test + ### a specific change, but don't want rebuilds caused by unrelated + ### changes, or if you want to decide at which point new commits are + ### fetched. When 'fetch' is set to 'if_needed', new commits will only + ### be fetched if the selected commit (or branch, or tag) is not present, + ### which means that existing branches won't be updated. In that case + ### you can fetch new commits by running 'make fetch'. + fetch: 'if_needed' + +# vim: filetype=yaml sw=2

1 0