Pier Angelo Vendrame pushed to branch main at The Tor Project / Applications / tor-browser-build

Commits:

8 changed files:

Changes:

  • projects/firefox/build
    ... ... @@ -62,9 +62,8 @@ tar -C /var/tmp/build -xf [% project %]-[% c('version') %].tar.[% c('compress_ta
    62 62
     mkdir -p $distdir/[% IF ! c("var/macos") %]Browser[% END %]
    
    63 63
     
    
    64 64
     cd /var/tmp/build/[% project %]-[% c("version") %]
    
    65
    -cat > .mozconfig << 'MOZCONFIG_EOF'
    
    66
    -[% INCLUDE mozconfig %]
    
    67
    -MOZCONFIG_EOF
    
    65
    +cp $rootdir/mozconfig ./
    
    66
    +
    
    68 67
     [% IF c("var/asan") -%]
    
    69 68
       # Without disabling LSan our build is blowing up:
    
    70 69
       # https://bugs.torproject.org/10599#comment:52
    

  • projects/firefox/config
    ... ... @@ -165,6 +165,9 @@ targets:
    165 165
     
    
    166 166
     input_files:
    
    167 167
       - project: container-image
    
    168
    +  - filename: 'mozconfig'
    
    169
    +    content: '[% INCLUDE "mozconfig.in" %]'
    
    170
    +    refresh_input: 1
    
    168 171
       - name: '[% c("var/compiler") %]'
    
    169 172
         project: '[% c("var/compiler") %]'
    
    170 173
         # Cross-binutils are already included in the cross-compiler
    

  • projects/firefox/mozconfig โ†’ projects/firefox/mozconfig.in

  • projects/geckoview/README.md
    ... ... @@ -4,41 +4,38 @@ tor-browser.git.
    4 4
     
    
    5 5
     # Custom ESR
    
    6 6
     
    
    7
    -Mozilla doesn't provide an ESR version for Firefox for Android, so it doesn't
    
    7
    +Mozilla doesn't provide an ESR version of Firefox for Android, so it doesn't
    
    8 8
     provide one for GeckoView as well.
    
    9 9
     However, we decided to stay on the same version as desktop Firefox, and to
    
    10 10
     backport the Android security bugs from the rapid release.
    
    11 11
     Therefore, `geckoview` and `firefox` should always build from the same tree.
    
    12 12
     
    
    13
    -# "Fat" AARs
    
    13
    +# Build steps
    
    14 14
     
    
    15
    -Theoretically, GeckoView is designed to be available also to products different
    
    16
    -from Mozilla's browsers.
    
    17
    -Therefore, Mozilla decided to
    
    18
    -[provide a single AAR](https://bugzilla.mozilla.org/show_bug.cgi?id=1508976)
    
    19
    -that targets all architectures, instead of one AAR for each architecture.
    
    15
    +At a certain point, Mozilla migrated Android frontend code (_Fenix_) to
    
    16
    +mozilla-central. Therefore, we now use only tor-browser.git.
    
    20 17
     
    
    21
    -Unifying AARs is a separate step, with its own `mozconfig`.
    
    22
    -Therefore, we do it in a separate step, that runs the `merge_aars` script.
    
    18
    +The build steps are:
    
    23 19
     
    
    24
    -Before Firefox 99, it was possible to use external AARs with the compile
    
    25
    -environment turned off.
    
    26
    -This configuration was not used by Mozilla and at a certain point they
    
    27
    -[broke it](https://bugzilla.mozilla.org/show_bug.cgi?id=1763770).
    
    28
    -The workaround is to use artifact builds, but we've never setup for our build
    
    29
    -environment.
    
    30
    -Instead, we've
    
    31
    -[patched the build system](https://gitlab.torproject.org/tpo/applications/tor-browser/-/merge_requests/271/diffs?commit_id=b620f9a965b2030ccb3e45015867326a18eb9c33)
    
    32
    -in tor-browser.git.
    
    20
    +1. Build GeckoView for each architecture
    
    21
    +2. Merge the various architecture into a single "fat" AAR
    
    22
    +3. Build Android Components
    
    23
    +4. Build Fenix
    
    24
    +
    
    25
    +We run 1. in a dedicated step (`build`), since it's architecture-specific, then
    
    26
    +we run the rest in a single RBM step (`build_apk`).
    
    27
    +
    
    28
    +We had to
    
    29
    +[patch the build system](https://gitlab.torproject.org/tpo/applications/tor-browser/-/merge_requests/271/diffs?commit_id=b620f9a965b2030ccb3e45015867326a18eb9c33)
    
    30
    +in tor-browser.git to make this possible, as upstream relies on artifact builds
    
    31
    +for this.
    
    32
    +Instead, we disable the compile environment.
    
    33 33
     
    
    34 34
     Notice that it isn't necessary to include all the architectures Mozilla and us
    
    35 35
     support (currently, aarch64, armv7 and x86_64).
    
    36
    -The merge automation also supports "merging" one architecture.
    
    37
    -We use this hack when `var/android_single_arch` is defined, which is the default
    
    38
    -only in testbuilds.
    
    39
    -The rationale is that we stay as close as possible to Mozilla for builds we
    
    40
    -release to users, but we don't need to wait for all the architectures to build
    
    41
    -for developer testbuilds.
    
    36
    +We do it to stay as close as possible to Mozilla for builds we release to users,
    
    37
    +but we do single-architecture builds in testbuilds, to reduce the time to make
    
    38
    +them.
    
    42 39
     
    
    43 40
     # Java dependencies
    
    44 41
     
    

  • projects/geckoview/build
    ... ... @@ -14,28 +14,12 @@ export PATH="/var/tmp/dist/rust/bin:/var/tmp/dist/cbindgen:/var/tmp/dist/nasm/bi
    14 14
     [% END -%]
    
    15 15
     
    
    16 16
     cd /var/tmp/build/[% project %]-[% c("version") %]
    
    17
    -cat > .mozconfig << 'MOZCONFIG_EOF'
    
    18
    -. $topsrcdir/mozconfig-[% c("var/osname") %]
    
    19
    -
    
    20
    -mk_add_options MOZ_PARALLEL_BUILD=[% c("num_procs") %]
    
    21
    -export MOZ_INCLUDE_SOURCE_INFO=1
    
    22
    -export MOZ_SOURCE_REPO="[% c('var/gitlab_project') %]"
    
    23
    -export MOZ_SOURCE_CHANGESET=[% c("var/git_commit") %]
    
    24
    -
    
    25
    -ac_add_options --enable-update-channel=[% c("var/variant") %]
    
    26
    -
    
    27
    -[% IF c("var/dev_artifacts") -%]
    
    28
    -  ac_add_options --enable-tests
    
    29
    -  # Necessary for package-tests step which expects dist/bin/jsapi-tests to exist.
    
    30
    -  ac_add_options --enable-js-shell
    
    31
    -[% END -%]
    
    32
    -
    
    33
    -MOZCONFIG_EOF
    
    34
    -echo "ac_add_options --with-branding=$branding_dir" >> .mozconfig
    
    17
    +cp $rootdir/mozconfig ./
    
    35 18
     
    
    36 19
     echo "Starting ./mach configure $(date)"
    
    37 20
     ./mach configure \
    
    38 21
       --with-base-browser-version=[% c("var/torbrowser_version") %] \
    
    22
    +  --with-branding=$branding_dir \
    
    39 23
       [% IF !c("var/rlbox") -%]--without-wasm-sandboxed-libraries[% END %]
    
    40 24
     
    
    41 25
     echo "Starting ./mach build $(date)"
    

  • projects/geckoview/build_apk
    ... ... @@ -22,21 +22,13 @@ export MOZ_ANDROID_FAT_AAR_ARM64_V8A=$rootdir/[% c('input_files_by_name/geckovie
    22 22
     export MOZ_ANDROID_FAT_AAR_X86_64=$rootdir/[% c('input_files_by_name/geckoview_x86_64') %]/geckoview/*.aar
    
    23 23
     
    
    24 24
     cd $builddir/[% project %]-[% c("version") %]
    
    25
    -ln -s mozconfig-android-all .mozconfig
    
    26
    -cat >> mozconfig-android-all << 'MOZCONFIG_EOF'
    
    27
    -mk_add_options MOZ_PARALLEL_BUILD=[% c("num_procs") %]
    
    28
    -export MOZ_INCLUDE_SOURCE_INFO=1
    
    29
    -export MOZ_SOURCE_REPO="[% c('var/gitlab_project') %]"
    
    30
    -export MOZ_SOURCE_CHANGESET=[% c("var/git_commit") %]
    
    31
    -
    
    32
    -ac_add_options --enable-update-channel=[% c("var/variant") %]
    
    33
    -MOZCONFIG_EOF
    
    34
    -echo "ac_add_options --with-branding=$branding_dir" >> mozconfig-android-all
    
    35
    -
    
    36
    -# We still need to specify --tor-browser-version due to bug 34005.
    
    25
    +cp $rootdir/mozconfig ./
    
    26
    +
    
    27
    +# We still need to specify --base-browser-version due to bug 34005.
    
    37 28
     ./mach configure \
    
    38 29
       --with-base-browser-version=[% c("var/torbrowser_version") %] \
    
    39 30
       --enable-update-channel=[% c("var/variant") %] \
    
    31
    +  --with-branding=$branding_dir \
    
    40 32
       [% IF !c("var/rlbox") -%]--without-wasm-sandboxed-libraries[% END %]
    
    41 33
     
    
    42 34
     ./mach build --verbose
    

  • projects/geckoview/config
    ... ... @@ -47,6 +47,7 @@ var:
    47 47
       variant: beta
    
    48 48
       has_l10n: '[% !c("var/testbuild") && !c("var/locales").empty %]'
    
    49 49
       l10n-changesets: '[% exec("git --no-pager show " _ c("git_hash") _ ":browser/locales/l10n-changesets.json", { exec_noco => 1 }) %]'
    
    50
    +  mozconfig_file: 'mozconfig-[% c("var/osname") %]'
    
    50 51
     
    
    51 52
     targets:
    
    52 53
       release:
    
    ... ... @@ -63,9 +64,14 @@ steps:
    63 64
         filename: 'geckoview-[% c("version") %]-apks-[% c("var/build_id") %]'
    
    64 65
         version: '[% c("abbrev") %]'
    
    65 66
         build_apk: '[% INCLUDE build_apk %]'
    
    67
    +    var:
    
    68
    +      mozconfig_file: 'mozconfig-all'
    
    66 69
         input_files:
    
    67 70
           - project: container-image
    
    68 71
             pkg_type: build
    
    72
    +      - filename: 'mozconfig'
    
    73
    +        content: '[% INCLUDE "mozconfig.in" %]'
    
    74
    +        refresh_input: 1
    
    69 75
           - name: '[% c("var/compiler") %]'
    
    70 76
             project: '[% c("var/compiler") %]'
    
    71 77
             pkg_type: build
    
    ... ... @@ -136,6 +142,9 @@ steps:
    136 142
     
    
    137 143
     input_files:
    
    138 144
       - project: container-image
    
    145
    +  - filename: 'mozconfig'
    
    146
    +    content: '[% INCLUDE "mozconfig.in" %]'
    
    147
    +    refresh_input: 1
    
    139 148
       - name: '[% c("var/compiler") %]'
    
    140 149
         project: '[% c("var/compiler") %]'
    
    141 150
       - project: gradle
    

  • projects/geckoview/mozconfig.in
    1
    +. $topsrcdir/[% c("var/mozconfig_file") %]
    
    2
    +
    
    3
    +mk_add_options MOZ_PARALLEL_BUILD=[% c("num_procs") %]
    
    4
    +export MOZ_INCLUDE_SOURCE_INFO=1
    
    5
    +export MOZ_SOURCE_REPO="[% c('var/gitlab_project') %]"
    
    6
    +export MOZ_SOURCE_CHANGESET=[% c("var/git_commit") %]
    
    7
    +
    
    8
    +ac_add_options --enable-update-channel=[% c("var/variant") %]
    
    9
    +
    
    10
    +[% IF c("var/dev_artifacts") -%]
    
    11
    +  ac_add_options --enable-tests
    
    12
    +  # Necessary for package-tests step which expects dist/bin/jsapi-tests to exist.
    
    13
    +  ac_add_options --enable-js-shell
    
    14
    +[% END -%]