- tbb-commits - lists.torproject.org

[Git][tpo/applications/tor-browser][base-browser-128.5.0esr-14.5-1] fixup! Firefox preference overrides.
by Pier Angelo Vendrame (＠pierov) 16 Dec '24

16 Dec '24

Pier Angelo Vendrame pushed to branch base-browser-128.5.0esr-14.5-1 at The Tor Project / Applications / Tor Browser Commits: fa621bc7 by Pier Angelo Vendrame at 2024-12-16T18:23:54+01:00 fixup! Firefox preference overrides. BB 43366: Do not use system accent color in inputs. We did not manage to write a reliable proof of concept for fingerprinting this, but I managed to get the color once (with Firefox on Android, which is the worse case, as the system offers to use colors from the wallpaper). - - - - - 1 changed file: - browser/app/profile/001-base-profile.js Changes: ===================================== browser/app/profile/001-base-profile.js ===================================== @@ -463,6 +463,9 @@ pref("pdfjs.disabled", false, locked); #endif // Bug 40057: Ensure system colors are not used for CSS4 colors pref("browser.display.use_system_colors", false); +// tor-browser#43366: do not use system accent color in inputs. +// See also https://bugzilla.mozilla.org/show_bug.cgi?id=1861362. +pref("widget.non-native-theme.use-theme-accent", false); // tor-browser#41943: defense-in-depth, but do not lock anymore (enabled in Firefox 119, http://bugzil.la/1851162) pref("javascript.options.spectre.disable_for_isolated_content", false); View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser/-/commit/fa621bc… -- View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser/-/commit/fa621bc… You're receiving this email because of your account on gitlab.torproject.org.

1 0

[Git][tpo/applications/tor-browser][tor-browser-128.5.0esr-14.5-1] fixup! Firefox preference overrides.
by Pier Angelo Vendrame (＠pierov) 16 Dec '24

16 Dec '24

Pier Angelo Vendrame pushed to branch tor-browser-128.5.0esr-14.5-1 at The Tor Project / Applications / Tor Browser Commits: 7f8c1d43 by Pier Angelo Vendrame at 2024-12-16T18:18:53+01:00 fixup! Firefox preference overrides. BB 43366: Do not use system accent color in inputs. We did not manage to write a reliable proof of concept for fingerprinting this, but I managed to get the color once (with Firefox on Android, which is the worse case, as the system offers to use colors from the wallpaper). - - - - - 1 changed file: - browser/app/profile/001-base-profile.js Changes: ===================================== browser/app/profile/001-base-profile.js ===================================== @@ -463,6 +463,9 @@ pref("pdfjs.disabled", false, locked); #endif // Bug 40057: Ensure system colors are not used for CSS4 colors pref("browser.display.use_system_colors", false); +// tor-browser#43366: do not use system accent color in inputs. +// See also https://bugzilla.mozilla.org/show_bug.cgi?id=1861362. +pref("widget.non-native-theme.use-theme-accent", false); // tor-browser#41943: defense-in-depth, but do not lock anymore (enabled in Firefox 119, http://bugzil.la/1851162) pref("javascript.options.spectre.disable_for_isolated_content", false); View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser/-/commit/7f8c1d4… -- View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser/-/commit/7f8c1d4… You're receiving this email because of your account on gitlab.torproject.org.

1 0

[Git][tpo/applications/tor-browser][tor-browser-128.5.0esr-14.5-1] fixup! Bug 40701: Add security warning when downloading a file
by morgan (＠morgan) 16 Dec '24

16 Dec '24

morgan pushed to branch tor-browser-128.5.0esr-14.5-1 at The Tor Project / Applications / Tor Browser Commits: 00d7026f by Henry Wilkes at 2024-12-16T11:27:38+00:00 fixup! Bug 40701: Add security warning when downloading a file Bug 43312: Ensure download panel warning uses the same width as the mozilla downloads list. We stop setting a `width` on the download panel because this can conflict with the default width set for the mozilla downloads list, set by localisers. Instead, we use the same width for our Tor Browser warning message using a wrapper element. We also ensure that the wrapper element has `min-width: 100%` for scenarios where the localiser width is less than the min-width of the ancestor panel. - - - - - 2 changed files: - browser/components/downloads/content/downloads.css - browser/components/downloads/content/downloadsPanel.inc.xhtml Changes: ===================================== browser/components/downloads/content/downloads.css ===================================== @@ -92,15 +92,19 @@ #downloadsPanel-mainView { min-width: 37em; padding: 0.62em; - /* If we don't set a width, #downloadsPanelTorWarning will request - * its max-content width. */ - width: 37em; } #downloadsPanelTorWarning { margin-block-end: var(--arrowpanel-menuitem-padding-block); } +#downloadsPanelTorWarningWrapper { + /* The wrapper element has its `width` attribute set by mozilla localisers. + * We want to ensure the element occupies the available width when the + * localiser width is smaller. See tor-browser#43312. */ + min-width: 100%; +} + #downloadsHistory, #downloadsFooterButtons { margin: 0; ===================================== browser/components/downloads/content/downloadsPanel.inc.xhtml ===================================== @@ -104,8 +104,25 @@ disablekeynav="true"> <panelview id="downloadsPanel-mainView"> - <html:moz-message-bar id="downloadsPanelTorWarning"> - </html:moz-message-bar> +  + <html:div + id="downloadsPanelTorWarningWrapper" + data-l10n-id="downloads-panel-items" + data-l10n-attrs="style" + > + <html:moz-message-bar id="downloadsPanelTorWarning"> + </html:moz-message-bar> + </html:div> <vbox class="panel-view-body-unscrollable"> <richlistbox id="downloadsListBox" data-l10n-id="downloads-panel-items" View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser/-/commit/00d7026… -- View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser/-/commit/00d7026… You're receiving this email because of your account on gitlab.torproject.org.

1 0

[Git][tpo/applications/tor-browser-build][maint-14.0] Bug 41311: Remove apparmor profile if abi 4.0 is not present
by morgan (＠morgan) 16 Dec '24

16 Dec '24

morgan pushed to branch maint-14.0 at The Tor Project / Applications / tor-browser-build Commits: 36c06687 by Nicolas Vigier at 2024-12-16T12:54:30+00:00 Bug 41311: Remove apparmor profile if abi 4.0 is not present When abi 4.0 file not available, loading the apparmor profile is failing. But the profile is only useful on Ubuntu 24.04 (and later), where the abi 4.0 file is present. We're doing something similar to: https://github.com/CollaboraOnline/online/commit/5f7b37c7412c3e1d81d9c50a0f… - - - - - 2 changed files: - projects/linux-packages/config - + projects/linux-packages/debian/postinst.in Changes: ===================================== projects/linux-packages/config ===================================== @@ -171,6 +171,10 @@ input_files: content: "[% INCLUDE 'debian/docs.in' %]" refresh_input: 1 enable: '[% c("var/build_deb_pkg") %]' + - filename: debian/postinst + content: "[% INCLUDE 'debian/postinst.in' %]" + refresh_input: 1 + enable: '[% c("var/build_deb_pkg") %]' - filename: debian/rules content: "[% INCLUDE 'debian/rules.in' %]" refresh_input: 1 ===================================== projects/linux-packages/debian/postinst.in ===================================== @@ -0,0 +1,12 @@ +#!/bin/sh + +set -e + +case "$1" in + install|upgrade|configure) + # If abi 4.0 is not present, then remove the apparmor profile config + if [ ! -e /etc/apparmor.d/abi/4.0 ]; then + rm -f /etc/apparmor.d/[% c("var/system_pkg/pkg_name") %] + fi + ;; +esac View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/commit/3… -- View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/commit/3… You're receiving this email because of your account on gitlab.torproject.org.

1 0

[Git][tpo/applications/tor-browser-build][main] Bug 41311: Remove apparmor profile if abi 4.0 is not present
by morgan (＠morgan) 16 Dec '24

16 Dec '24

morgan pushed to branch main at The Tor Project / Applications / tor-browser-build Commits: ee82c7f8 by Nicolas Vigier at 2024-12-12T12:13:31+01:00 Bug 41311: Remove apparmor profile if abi 4.0 is not present When abi 4.0 file not available, loading the apparmor profile is failing. But the profile is only useful on Ubuntu 24.04 (and later), where the abi 4.0 file is present. We're doing something similar to: https://github.com/CollaboraOnline/online/commit/5f7b37c7412c3e1d81d9c50a0f… - - - - - 2 changed files: - projects/linux-packages/config - + projects/linux-packages/debian/postinst.in Changes: ===================================== projects/linux-packages/config ===================================== @@ -201,6 +201,10 @@ input_files: content: "[% INCLUDE 'debian/docs.in' %]" refresh_input: 1 enable: '[% c("var/build_deb_pkg") %]' + - filename: debian/postinst + content: "[% INCLUDE 'debian/postinst.in' %]" + refresh_input: 1 + enable: '[% c("var/build_deb_pkg") %]' - filename: debian/rules content: "[% INCLUDE 'debian/rules.in' %]" refresh_input: 1 ===================================== projects/linux-packages/debian/postinst.in ===================================== @@ -0,0 +1,12 @@ +#!/bin/sh + +set -e + +case "$1" in + install|upgrade|configure) + # If abi 4.0 is not present, then remove the apparmor profile config + if [ ! -e /etc/apparmor.d/abi/4.0 ]; then + rm -f /etc/apparmor.d/[% c("var/system_pkg/pkg_name") %] + fi + ;; +esac View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/commit/e… -- View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/commit/e… You're receiving this email because of your account on gitlab.torproject.org.

1 0

[Git][tpo/applications/tor-browser-build][main] Bug 41317: Add the linux-aarch64 target to linux-packages
by boklm (＠boklm) 14 Dec '24

14 Dec '24

boklm pushed to branch main at The Tor Project / Applications / tor-browser-build Commits: 4c9640b6 by NoisyCoil at 2024-12-12T12:03:29+01:00 Bug 41317: Add the linux-aarch64 target to linux-packages Also fixing #41335 (linux-package project is not getting the channel target in testbuilds). Co-authored-by: Nicolas Vigier <boklm(a)torproject.org> - - - - - 6 changed files: - projects/linux-packages/build - projects/linux-packages/config - projects/linux-packages/debian/control.in - projects/linux-packages/rpm-package.spec - projects/release/build - projects/release/config Changes: ===================================== projects/linux-packages/build ===================================== @@ -57,6 +57,11 @@ umask 0022 $rootdir/[% c('input_files_by_name/linux-i686') %]/[% c("var/project-name") %]-linux-i686-[% c("version") %].tar.xz \ i386 [% END -%] + [% IF c("var/browser-linux-aarch64") -%] + extract_src_tar \ + $rootdir/[% c('input_files_by_name/linux-aarch64') %]/[% c("var/project-name") %]-linux-aarch64-[% c("version") %].tar.xz \ + arm64 + [% END -%] dpkg-buildpackage --unsigned-source --unsigned-changes --build=source [% FOREACH deb_arch = c("var_p/system_pkg/deb_archs") -%] @@ -92,6 +97,10 @@ EOF cp -a $rootdir/[% c('input_files_by_name/linux-i686') %]/[% c("var/project-name") %]-linux-i686-[% c("version") %].tar.xz \ SOURCES/[% c("var/system_pkg/pkg_name") %]-linux-i386-[% c("var/system_pkg/pkg_version") %].tar.xz [% END -%] + [% IF c("var/browser-linux-aarch64") -%] + cp -a $rootdir/[% c('input_files_by_name/linux-aarch64') %]/[% c("var/project-name") %]-linux-aarch64-[% c("version") %].tar.xz \ + SOURCES/[% c("var/system_pkg/pkg_name") %]-linux-aarch64-[% c("var/system_pkg/pkg_version") %].tar.xz + [% END -%] rpmdir=$(pwd) [% IF c("var/browser-linux-x86_64") -%] rpmbuild --define "_topdir $rpmdir" -v -bb SPECS/[% c("var/system_pkg/pkg_name") %].spec @@ -99,6 +108,9 @@ EOF [% IF c("var/browser-linux-i686") -%] rpmbuild --define "_topdir $rpmdir" --target i686-redhat-linux -v -bb SPECS/[% c("var/system_pkg/pkg_name") %].spec [% END -%] + [% IF c("var/browser-linux-aarch64") -%] + rpmbuild --define "_topdir $rpmdir" --target aarch64-redhat-linux -v -bb SPECS/[% c("var/system_pkg/pkg_name") %].spec + [% END -%] mv RPMS/*/*.rpm "$OUTDIR/" [% END %] ===================================== projects/linux-packages/config ===================================== @@ -19,6 +19,7 @@ var: deb_archs_list: - '[% IF c("var/browser-linux-x86_64") %]amd64[% END %]' - '[% IF c("var/browser-linux-i686") %]i386[% END %]' + - '[% IF c("var/browser-linux-aarch64") %]arm64[% END %]' targets: @@ -28,7 +29,7 @@ targets: build_deb_pkg: 1 pre_pkginst: | dpkg --add-architecture i386 - # some :i386 packages fail to install when /var/lib/dpkg/available + # some foreign-arch packages fail to install when /var/lib/dpkg/available # does not exist, so create it as an empty file echo > /var/lib/dpkg/available arch_deps: @@ -58,6 +59,31 @@ targets: - libpulse-dev:i386 - libx11-xcb-dev:i386 - libxt-dev:i386 + deb-arm64: + var: + build_deb_pkg: 1 + pre_pkginst: | + dpkg --add-architecture arm64 + # some foreign-arch packages fail to install when /var/lib/dpkg/available + # does not exist, so create it as an empty file + echo > /var/lib/dpkg/available + arch_deps: + # Packages needed to build the deb package + - dpkg-dev + - debhelper + - dh-exec + - dh-apparmor + # Packages needed to generate dependencies for the deb package + - linux-libc-dev:arm64 + - libasound2-dev:arm64 + - libfontconfig1-dev:arm64 + - libfreetype6-dev:arm64 + - libgconf2-dev:arm64 + - libgtk-3-dev:arm64 + - libpango1.0-dev:arm64 + - libpulse-dev:arm64 + - libx11-xcb-dev:arm64 + - libxt-dev:arm64 rpm: var: @@ -72,18 +98,15 @@ targets: container: suite: bookworm - browser-all: - - browser-linux-x86_64 - - browser-linux-i686 - - browser-all-desktop: browser-all - browser-linux-x86_64: var: browser-linux-x86_64: 1 browser-linux-i686: var: browser-linux-i686: '[% c("var/browser_type") != "mullvadbrowser" %]' + browser-linux-aarch64: + var: + browser-linux-aarch64: 1 torbrowser: var: @@ -139,6 +162,13 @@ input_files: - '[% c("var/build_target") %]' - '[% c("var/browser_type") %]-linux-i686' + - name: linux-aarch64 + project: browser + enable: '[% c("var/browser-linux-aarch64") %]' + target: + - '[% c("var/build_target") %]' + - '[% c("var/browser_type") %]-linux-aarch64' + - filename: Makefile content: "[% INCLUDE 'Makefile.in' %]" refresh_input: 1 ===================================== projects/linux-packages/debian/control.in ===================================== @@ -5,7 +5,7 @@ Section: web Build-Depends: debhelper (>= 9), dh-apparmor Package: [% c("var/system_pkg/pkg_name") %] -Architecture: [% c("var_p/system_pkg/deb_archs").join(" ") %] +Architecture: amd64 i386 arm64 Depends: ${shlibs:Depends}, Description: [% c('var/display_name') %] [% c("var/system_pkg/pkg_description") %] ===================================== projects/linux-packages/rpm-package.spec ===================================== @@ -21,6 +21,7 @@ %global privlibs %{privlibs}|libxul %global __provides_exclude ^(%{privlibs})\\.so %global __requires_exclude ^(%{privlibs})\\.so +%global __strip true Summary: [% c("var/display_name") %] Name: [% c("var/system_pkg/pkg_name") %] @@ -36,6 +37,9 @@ Source2: %{name}-linux-x86_64-%{version}.tar.xz [% IF c("var/browser-linux-i686") -%] Source3: %{name}-linux-i386-%{version}.tar.xz [% END -%] +[% IF c("var/browser-linux-aarch64") -%] +Source4: %{name}-linux-aarch64-%{version}.tar.xz +[% END -%] %description [% c("var/system_pkg/pkg_description") %] ===================================== projects/release/build ===================================== @@ -40,8 +40,13 @@ mkdir -p "$destdir" [% IF c("var/browser-linux-aarch64") -%] mv [% c('input_files_by_name/linux-aarch64') %]/* "$destdir"/ [% END -%] -[% IF c("var/linux-packages") -%] - mv [% c('input_files_by_name/deb-packages') %]/* "$destdir"/ +[% IF c("var/linux-packages") || c("var/linux-packages-aarch64") -%] + [% IF c("var/linux-packages") -%] + mv [% c('input_files_by_name/deb-packages') %]/* "$destdir"/ + [% END -%] + [% IF c("var/linux-packages-aarch64") -%] + mv [% c('input_files_by_name/deb-packages-aarch64') %]/* "$destdir"/ + [% END -%] mv [% c('input_files_by_name/rpm-packages') %]/* "$destdir"/ [% END -%] [% IF c("var/browser-src") -%] ===================================== projects/release/config ===================================== @@ -63,7 +63,7 @@ targets: browser-linux-aarch64: var: browser-linux-aarch64: '[% ! c("var/browser-all-target") || c("var/testbuild") || c("var/nightly") %]' - linux-packages: 0 + linux-packages-aarch64: '[% (! c("var/browser-all-target") || c("var/testbuild") || c("var/nightly")) && (c("var/mullvad-browser") || c("var/tor-browser-linux-packages")) %]' browser-windows-i686: var: browser-windows-i686: '[% c("var/browser_type") != "mullvadbrowser" %]' @@ -210,20 +210,40 @@ input_files: - name: deb-packages project: linux-packages enable: '[% c("var/linux-packages") %]' - # Add linux-x86_64 targets for container config - target_prepend: + # Add $browser_type-linux-x86_64 target for container config + target: + - '[% IF c("var/testbuild") %]testbuild[% END %]' + - '[% c("var/build_target") %]' + - '[% IF c("var/browser-linux-i686") %]browser-linux-i686[% END %]' + - '[% IF c("var/browser-linux-x86_64") %]browser-linux-x86_64[% END %]' - deb - - linux-x86_64 - - linux + - '[% c("var/browser_type") %]-linux-x86_64' + + # linux-aarch64 requires a separate container because of multi-arch + # conflicts between amd64 and arm64 + - name: deb-packages-aarch64 + project: linux-packages + enable: '[% c("var/linux-packages-aarch64") %]' + # Add $browser_type-linux-x86_64 target for container config + target: + - '[% IF c("var/testbuild") %]testbuild[% END %]' + - '[% c("var/build_target") %]' + - browser-linux-aarch64 + - deb-arm64 + - '[% c("var/browser_type") %]-linux-x86_64' - name: rpm-packages project: linux-packages - enable: '[% c("var/linux-packages") %]' - # Add linux-x86_64 targets for container config - target_prepend: + enable: '[% c("var/linux-packages") || c("var/linux-packages-aarch64") %]' + # Add $browser_type-linux-x86_64 target for container config + target: + - '[% IF c("var/testbuild") %]testbuild[% END %]' + - '[% c("var/build_target") %]' + - '[% IF c("var/browser-linux-i686") %]browser-linux-i686[% END %]' + - '[% IF c("var/browser-linux-x86_64") %]browser-linux-x86_64[% END %]' + - '[% IF c("var/linux-packages-aarch64") %]browser-linux-aarch64[% END %]' - rpm - - linux-x86_64 - - linux + - '[% c("var/browser_type") %]-linux-x86_64' - name: windows-i686 project: browser View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/commit/4… -- View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/commit/4… You're receiving this email because of your account on gitlab.torproject.org.

1 0

[Git][tpo/applications/tor-browser][tor-browser-128.5.0esr-14.5-1] 4 commits: fixup! Add CI for Base Browser
by Pier Angelo Vendrame (＠pierov) 12 Dec '24

12 Dec '24

Pier Angelo Vendrame pushed to branch tor-browser-128.5.0esr-14.5-1 at The Tor Project / Applications / Tor Browser Commits: 67cfb4f1 by Beatriz Rizental at 2024-12-11T20:00:05+01:00 fixup! Add CI for Base Browser Small refactor of file organization. - - - - - 94bb2ef1 by Beatriz Rizental at 2024-12-11T20:00:06+01:00 fixup! Add CI for Tor Browser Small refactor of file organization. - - - - - 19a0c991 by Beatriz Rizental at 2024-12-12T05:00:14+01:00 fixup! Add CI for Tor Browser Multi-platform startup test for nightly builds. This commit is desktop only. - - - - - 7dc11c34 by Beatriz Rizental at 2024-12-12T05:00:15+01:00 fixup! Add CI for Tor Browser Disable Windows and MacOS startup tests. These can only be enabled once we have Windows and MacOS runners available. Relates to: https://gitlab.torproject.org/tpo/tpa/team/-/issues/41829 - - - - - 11 changed files: - .gitlab-ci.yml - .gitlab/ci/scripts/helpers.py → .gitlab/ci/jobs/lint/helpers.py - .gitlab/ci/lint.yml → .gitlab/ci/jobs/lint/lint.yml - + .gitlab/ci/jobs/startup-test/startup-test.py - + .gitlab/ci/jobs/startup-test/startup-test.yml - .gitlab/ci/update-containers.yml → .gitlab/ci/jobs/update-containers.yml - .gitlab/ci/update-translations.yml → .gitlab/ci/jobs/update-translations.yml - + .gitlab/ci/mixins.yml - testing/mozbase/mozinstall/mozinstall/mozinstall.py - testing/mozbase/setup_development.py - testing/mozharness/scripts/does_it_crash.py Changes: ===================================== .gitlab-ci.yml ===================================== @@ -1,6 +1,7 @@ stages: - update-container-images - lint + - startup-test - update-translations variables: @@ -8,6 +9,8 @@ variables: LOCAL_REPO_PATH: /srv/apps-repos/tor-browser.git include: - - local: '.gitlab/ci/lint.yml' - - local: '.gitlab/ci/update-containers.yml' - - local: '.gitlab/ci/update-translations.yml' + - local: '.gitlab/ci/mixins.yml' + - local: '.gitlab/ci/jobs/lint/lint.yml' + - local: '.gitlab/ci/jobs/startup-test/startup-test.yml' + - local: '.gitlab/ci/jobs/update-containers.yml' + - local: '.gitlab/ci/jobs/update-translations.yml' ===================================== .gitlab/ci/scripts/helpers.py → .gitlab/ci/jobs/lint/helpers.py ===================================== ===================================== .gitlab/ci/lint.yml → .gitlab/ci/jobs/lint/lint.yml ===================================== @@ -1,11 +1,10 @@ .base: + extends: .with-local-repo-bash stage: lint image: $IMAGE_PATH interruptible: true variables: MOZBUILD_STATE_PATH: "$CI_PROJECT_DIR/.cache/mozbuild" - # A copy of the repository already is available in the runner. - GIT_STRATEGY: "none" cache: paths: - node_modules @@ -17,24 +16,11 @@ tags: # Run these jobs in the browser dedicated runners. - firefox - before_script: - - git init - - git remote add local "$LOCAL_REPO_PATH" - - git fetch --depth 500 local - - git remote add origin "$CI_REPOSITORY_URL" - - | - if [ -z "${CI_COMMIT_BRANCH:-$CI_MERGE_REQUEST_SOURCE_BRANCH_NAME}" ]; then - echo "No branch specified. Stopping the pipeline." - exit 1 - fi - - echo "Fetching from remote branch ${CI_COMMIT_BRANCH:-$CI_MERGE_REQUEST_SOURCE_BRANCH_NAME}" - - git fetch origin "${CI_COMMIT_BRANCH:-$CI_MERGE_REQUEST_SOURCE_BRANCH_NAME}" - - git checkout origin/${CI_COMMIT_BRANCH:-$CI_MERGE_REQUEST_SOURCE_BRANCH_NAME} eslint: extends: .base script: - - .gitlab/ci/scripts/helpers.py --get-changed-files | xargs -d '\n' ./mach lint -l eslint + - .gitlab/ci/jobs/lint/helpers.py --get-changed-files | xargs -d '\n' ./mach lint -l eslint rules: - if: $CI_PIPELINE_SOURCE == 'merge_request_event' changes: @@ -63,7 +49,7 @@ eslint: stylelint: extends: .base script: - - .gitlab/ci/scripts/helpers.py --get-changed-files | xargs -d '\n' ./mach lint -l stylelint + - .gitlab/ci/jobs/lint/helpers.py --get-changed-files | xargs -d '\n' ./mach lint -l stylelint rules: - if: $CI_PIPELINE_SOURCE == 'merge_request_event' changes: @@ -81,7 +67,7 @@ stylelint: py-black: extends: .base script: - - .gitlab/ci/scripts/helpers.py --get-changed-files | xargs -d '\n' ./mach lint -l black + - .gitlab/ci/jobs/lint/helpers.py --get-changed-files | xargs -d '\n' ./mach lint -l black rules: - if: $CI_PIPELINE_SOURCE == 'merge_request_event' changes: @@ -100,7 +86,7 @@ py-black: py-ruff: extends: .base script: - - .gitlab/ci/scripts/helpers.py --get-changed-files | xargs -d '\n' ./mach lint -l ruff + - .gitlab/ci/jobs/lint/helpers.py --get-changed-files | xargs -d '\n' ./mach lint -l ruff rules: - if: $CI_PIPELINE_SOURCE == 'merge_request_event' changes: @@ -119,7 +105,7 @@ py-ruff: yaml: extends: .base script: - - .gitlab/ci/scripts/helpers.py --get-changed-files | xargs -d '\n' ./mach lint -l yaml + - .gitlab/ci/jobs/lint/helpers.py --get-changed-files | xargs -d '\n' ./mach lint -l yaml rules: - if: $CI_PIPELINE_SOURCE == 'merge_request_event' changes: @@ -134,7 +120,7 @@ yaml: shellcheck: extends: .base script: - - .gitlab/ci/scripts/helpers.py --get-changed-files | xargs -d '\n' ./mach lint -l shellcheck + - .gitlab/ci/jobs/lint/helpers.py --get-changed-files | xargs -d '\n' ./mach lint -l shellcheck rules: - if: $CI_PIPELINE_SOURCE == 'merge_request_event' changes: @@ -149,7 +135,7 @@ clang-format: extends: .base script: - ./mach configure --without-wasm-sandboxed-libraries --with-base-browser-version=0.0.0 - - .gitlab/ci/scripts/helpers.py --get-changed-files | xargs -d '\n' ./mach lint -l clang-format + - .gitlab/ci/jobs/lint/helpers.py --get-changed-files | xargs -d '\n' ./mach lint -l clang-format rules: - if: $CI_PIPELINE_SOURCE == 'merge_request_event' changes: @@ -168,7 +154,7 @@ clang-format: rustfmt: extends: .base script: - - .gitlab/ci/scripts/helpers.py --get-changed-files | xargs -d '\n' ./mach lint -l rustfmt + - .gitlab/ci/jobs/lint/helpers.py --get-changed-files | xargs -d '\n' ./mach lint -l rustfmt rules: - if: $CI_PIPELINE_SOURCE == 'merge_request_event' changes: @@ -182,7 +168,7 @@ rustfmt: fluent-lint: extends: .base script: - - .gitlab/ci/scripts/helpers.py --get-changed-files | xargs -d '\n' ./mach lint -l fluent-lint + - .gitlab/ci/jobs/lint/helpers.py --get-changed-files | xargs -d '\n' ./mach lint -l fluent-lint rules: - if: $CI_PIPELINE_SOURCE == 'merge_request_event' changes: @@ -197,7 +183,7 @@ fluent-lint: localization: extends: .base script: - - .gitlab/ci/scripts/helpers.py --get-changed-files | xargs -d '\n' ./mach lint -l l10n + - .gitlab/ci/jobs/lint/helpers.py --get-changed-files | xargs -d '\n' ./mach lint -l l10n rules: - if: $CI_PIPELINE_SOURCE == 'merge_request_event' changes: @@ -214,7 +200,7 @@ localization: mingw-capitalization: extends: .base script: - - .gitlab/ci/scripts/helpers.py --get-changed-files | xargs -d '\n' ./mach lint -l mingw-capitalization + - .gitlab/ci/jobs/lint/helpers.py --get-changed-files | xargs -d '\n' ./mach lint -l mingw-capitalization rules: - if: $CI_PIPELINE_SOURCE == 'merge_request_event' changes: @@ -231,7 +217,7 @@ mingw-capitalization: mscom-init: extends: .base script: - - .gitlab/ci/scripts/helpers.py --get-changed-files | xargs -d '\n' ./mach lint -l mscom-init + - .gitlab/ci/jobs/lint/helpers.py --get-changed-files | xargs -d '\n' ./mach lint -l mscom-init rules: - if: $CI_PIPELINE_SOURCE == 'merge_request_event' changes: @@ -248,7 +234,7 @@ mscom-init: file-whitespace: extends: .base script: - - .gitlab/ci/scripts/helpers.py --get-changed-files | xargs -d '\n' ./mach lint -l file-whitespace + - .gitlab/ci/jobs/lint/helpers.py --get-changed-files | xargs -d '\n' ./mach lint -l file-whitespace rules: - if: $CI_PIPELINE_SOURCE == 'merge_request_event' changes: @@ -278,7 +264,7 @@ file-whitespace: test-manifest: extends: .base script: - - .gitlab/ci/scripts/helpers.py --get-changed-files | xargs -d '\n' ./mach lint -l test-manifest-alpha -l test-manifest-disable -l test-manifest-skip-if + - .gitlab/ci/jobs/lint/helpers.py --get-changed-files | xargs -d '\n' ./mach lint -l test-manifest-alpha -l test-manifest-disable -l test-manifest-skip-if rules: - if: $CI_PIPELINE_SOURCE == 'merge_request_event' changes: @@ -293,7 +279,7 @@ test-manifest: trojan-source: extends: .base script: - - .gitlab/ci/scripts/helpers.py --get-changed-files | xargs -d '\n' ./mach lint -l trojan-source + - .gitlab/ci/jobs/lint/helpers.py --get-changed-files | xargs -d '\n' ./mach lint -l trojan-source rules: - if: $CI_PIPELINE_SOURCE == 'merge_request_event' changes: ===================================== .gitlab/ci/jobs/startup-test/startup-test.py ===================================== @@ -0,0 +1,101 @@ +#!/usr/bin/env python3 + +import argparse +import subprocess +from datetime import datetime, timedelta + +PLATFORM_TO_ARCH = { + "linux": ["x86_64", "i686"], + "macos": ["x86_64", "aarch64"], + "windows": ["x86_64", "i686"], +} + + +class DynamicArchAction(argparse.Action): + def __call__(self, parser, namespace, values, option_string=None): + platform = getattr(namespace, "platform", None) + if not platform: + raise argparse.ArgumentError( + self, "The --platform argument must be provided before --arch." + ) + + valid_archs = PLATFORM_TO_ARCH.get(platform, []) + if values not in valid_archs: + raise argparse.ArgumentError( + self, + f"Invalid architecture '{values}' for platform '{platform}'. " + f"Valid options are: {', '.join(valid_archs)}", + ) + setattr(namespace, self.dest, values) + + +parser = argparse.ArgumentParser( + description="Downloads and executes yesterday's build of Tor or Mullvad browser nightly." +) + +parser.add_argument( + "--platform", + required=True, + help="Specify the platform (linux, macos or windows). Must be provided before --arch.", + choices=PLATFORM_TO_ARCH.keys(), +) +parser.add_argument( + "--arch", + required=True, + help="Specify the architecture (validated dynamically based on --platform).", + action=DynamicArchAction, +) +parser.add_argument( + "--browser", + required=True, + choices=["tor", "mullvad"], + help="Specify the browser (tor or mullvad)", +) + +args = parser.parse_args() +arch = f"-{args.arch}" +extra = "" + +if args.platform == "linux": + archive_extension = "tar.xz" + binary = f"Browser/start-{args.browser}-browser" +elif args.platform == "macos": + archive_extension = "dmg" + # The URL doesn't include the architecture for MacOS, + # because it's a universal build. + arch = "" + if args.browser == "tor": + binary = "Contents/MacOS/firefox" + else: + binary = "Contents/MacOS/mullvadbrowser" +elif args.platform == "windows": + archive_extension = "exe" + + if args.browser == "tor": + extra = "-portable" + binary = "Browser/firefox.exe" + else: + binary = "mullvadbrowser.exe" + +yesterday = (datetime.now() - timedelta(days=1)).strftime("%Y.%m.%d") + +download_url_base = ( + "https://nightlies.tbb.torproject.org/nightly-builds/tor-browser-builds" +) +if args.browser == "tor": + download_url = f"{download_url_base}/tbb-nightly.{yesterday}/nightly-{args.platform}{arch}/{args.browser}-browser-{args.platform}{arch}{extra}-tbb-nightly.{yesterday}.{archive_extension}" +else: + download_url = f"{download_url_base}/tbb-nightly.{yesterday}/mullvadbrowser-nightly-{args.platform}{arch}/{args.browser}-browser-{args.platform}{arch}-tbb-nightly.{yesterday}.{archive_extension}" + +subprocess.run( + [ + "python3", + "testing/mozharness/scripts/does_it_crash.py", + "--run-for", + "30", + "--thing-url", + download_url, + "--thing-to-run", + binary, + ] +) ===================================== .gitlab/ci/jobs/startup-test/startup-test.yml ===================================== @@ -0,0 +1,51 @@ +# startup-test-windows: +# extends: .with-local-repo-pwsh +# variables: +# LOCAL_REPO_PATH: "C:\\Users\\windoes\\tor-browser.git" +# stage: startup-test +# interruptible: true +# parallel: +# matrix: +# - BROWSER: ["tor", "mullvad"] +# tags: +# - x86-win11 +# script: +# - ./mach python testing/mozbase/setup_development.py +# - ./mach python .gitlab/ci/jobs/startup-test/startup-test.py --platform windows --arch x86_64 --browser tor +# rules: +# - if: $CI_PIPELINE_SOURCE == "schedule" + +# startup-test-macos: +# extends: .with-local-repo-bash +# variables: +# LOCAL_REPO_PATH: "/Users/gitlab-runner/tor-browser.git" +# stage: startup-test +# interruptible: true +# parallel: +# matrix: +# - BROWSER: ["tor", "mullvad"] +# tags: +# - x86-macos +# script: +# - ./mach python testing/mozbase/setup_development.py +# - ./mach python .gitlab/ci/jobs/startup-test/startup-test.py --platform macos --arch x86_64 --browser tor +# rules: +# - if: $CI_PIPELINE_SOURCE == "schedule" + +startup-test-linux: + extends: .with-local-repo-bash + image: $IMAGE_PATH + stage: startup-test + interruptible: true + parallel: + matrix: + - BROWSER: ["tor", "mullvad"] + tags: + - firefox + script: + - Xvfb :99 -screen 0 1400x900x24 & + - export DISPLAY=:99 + - ./mach python testing/mozbase/setup_development.py + - ./mach python .gitlab/ci/jobs/startup-test/startup-test.py --platform linux --arch x86_64 --browser tor + rules: + - if: $CI_PIPELINE_SOURCE == "schedule" ===================================== .gitlab/ci/update-containers.yml → .gitlab/ci/jobs/update-containers.yml ===================================== ===================================== .gitlab/ci/update-translations.yml → .gitlab/ci/jobs/update-translations.yml ===================================== ===================================== .gitlab/ci/mixins.yml ===================================== @@ -0,0 +1,37 @@ +.with-local-repo-bash: + variables: + GIT_STRATEGY: "none" + before_script: + - git init + - git remote add local "$LOCAL_REPO_PATH" + - git fetch --depth 500 local + - git remote add origin "$CI_REPOSITORY_URL" + - | + if [ -z "${CI_COMMIT_BRANCH:-$CI_MERGE_REQUEST_SOURCE_BRANCH_NAME}" ]; then + echo "No branch specified. Stopping the pipeline." + exit 1 + fi + - echo "Fetching from remote branch ${CI_COMMIT_BRANCH:-$CI_MERGE_REQUEST_SOURCE_BRANCH_NAME}" + - git fetch origin "${CI_COMMIT_BRANCH:-$CI_MERGE_REQUEST_SOURCE_BRANCH_NAME}" + - git checkout origin/${CI_COMMIT_BRANCH:-$CI_MERGE_REQUEST_SOURCE_BRANCH_NAME} + +.with-local-repo-pwsh: + variables: + GIT_STRATEGY: "none" + before_script: + - git init + - git remote add local $env:LOCAL_REPO_PATH + - git fetch --depth 500 local + - git remote add origin $env:CI_REPOSITORY_URL + - | + $branchName = $env:CI_COMMIT_BRANCH + if ([string]::IsNullOrEmpty($branchName)) { + $branchName = $env:CI_MERGE_REQUEST_SOURCE_BRANCH_NAME + } + if ([string]::IsNullOrEmpty($branchName)) { + Write-Output "No branch specified. Stopping the pipeline." + exit 1 + } + - Write-Output "Fetching from remote branch $branchName" + - git fetch origin $branchName + - git checkout origin/$branchName ===================================== testing/mozbase/mozinstall/mozinstall/mozinstall.py ===================================== @@ -352,7 +352,8 @@ def _install_exe(src, dest): # possibly gets around UAC in vista (still need to run as administrator) os.environ["__compat_layer"] = "RunAsInvoker" - cmd = '"%s" /extractdir=%s' % (src, os.path.realpath(dest)) + cmd = '"%s" /S /D=%s' % (src, os.path.realpath(dest)) + # cmd = '"%s" /extractdir=%s' % (src, os.path.realpath(dest)) subprocess.check_call(cmd) ===================================== testing/mozbase/setup_development.py ===================================== @@ -267,23 +267,28 @@ def main(args=sys.argv[1:]): os.environ.get("PATH", "").strip(os.path.pathsep), ) + current_file_path = os.path.abspath(__file__) + topobjdir = os.path.dirname(os.path.dirname(os.path.dirname(current_file_path))) + mach = str(os.path.join(topobjdir, "mach")) + # install non-mozbase dependencies # these need to be installed separately and the --no-deps flag # subsequently used due to a bug in setuptools; see # https://bugzilla.mozilla.org/show_bug.cgi?id=759836 pypi_deps = dict([(i, j) for i, j in alldeps.items() if i not in unrolled]) for package, version in pypi_deps.items(): - # easy_install should be available since we rely on setuptools - call(["easy_install", version]) + # Originally, Mozilla used easy_install here. + # That tool is deprecated, therefore we swich to pip. + call([sys.executable, mach, "python", "-m", "pip", "install", version]) # install packages required for unit testing for package in test_packages: - call(["easy_install", package]) + call([sys.executable, mach, "python", "-m", "pip", "install", package]) # install extra non-mozbase packages if desired if options.extra: for package in extra_packages: - call(["easy_install", package]) + call([sys.executable, mach, "python", "-m", "pip", "install", package]) if __name__ == "__main__": ===================================== testing/mozharness/scripts/does_it_crash.py ===================================== @@ -112,6 +112,13 @@ class DoesItCrash(BaseScript): for retry in range(3): if is_win: proc.send_signal(signal.CTRL_BREAK_EVENT) + + # Manually kill all processes we spawned, + # not sure why this is required, but without it we hang forever. + process_name = self.config["thing_to_run"].split("/")[-1] + subprocess.run( + ["taskkill", "/T", "/F", "/IM", process_name], check=True + ) else: os.killpg(proc.pid, signal.SIGKILL) try: View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser/-/compare/653c14… -- View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser/-/compare/653c14… You're receiving this email because of your account on gitlab.torproject.org.

1 0

[Git][tpo/applications/tor-browser][tor-browser-115.18.0esr-13.5-1] 3 commits: Bug 1842187 a=dmeehan
by ma1 (＠ma1) 11 Dec '24

11 Dec '24

ma1 pushed to branch tor-browser-115.18.0esr-13.5-1 at The Tor Project / Applications / Tor Browser Commits: f703416b by Gijs Kruitbosch at 2024-11-20T13:05:17+01:00 Bug 1842187 a=dmeehan Original Revision: https://phabricator.services.mozilla.com/D223948 Differential Revision: https://phabricator.services.mozilla.com/D228631 - - - - - 5e64483b by Edgar Chen at 2024-11-22T17:14:16+01:00 Bug 1909535 - Don't show select dropdown in background tabs; a=dmeehan Original Revision: https://phabricator.services.mozilla.com/D225706 Differential Revision: https://phabricator.services.mozilla.com/D227607 - - - - - 36cabf84 by Marco Bonardo at 2024-11-22T20:11:33+01:00 Bug 1921458. a=dmeehan Original Revision: https://phabricator.services.mozilla.com/D226775 Differential Revision: https://phabricator.services.mozilla.com/D227221 - - - - - 6 changed files: - toolkit/actors/SelectParent.sys.mjs - toolkit/components/downloads/DownloadUIHelper.sys.mjs - toolkit/components/reputationservice/ApplicationReputation.cpp - toolkit/components/reputationservice/test/gtest/TestExecutableLists.cpp - xpcom/io/nsLocalFileCommon.cpp - xpcom/io/nsLocalFileCommon.h Changes: ===================================== toolkit/actors/SelectParent.sys.mjs ===================================== @@ -278,6 +278,31 @@ export var SelectParentHelper = { }, open(browser, menulist, rect, isOpenedViaTouch, selectParentActor) { + const canOpen = (() => { + if (!menulist.ownerDocument.hasFocus()) { + // Don't open in inactive browser windows. + return false; + } + if (browser) { + if (!browser.browsingContext.isActive) { + // Don't open in inactive tabs. + return false; + } + let tabbrowser = browser.getTabBrowser(); + if (tabbrowser && tabbrowser.selectedBrowser != browser) { + // AsyncTabSwitcher might delay activating our browser, check + // explicitly for tabbrowser. + return false; + } + } + return true; + })(); + + if (!canOpen) { + selectParentActor.sendAsyncMessage("Forms:DismissedDropDown", {}); + return; + } + this._actor = selectParentActor; menulist.hidden = false; this._currentBrowser = browser; ===================================== toolkit/components/downloads/DownloadUIHelper.sys.mjs ===================================== @@ -175,7 +175,10 @@ DownloadPrompter.prototype = { "download-ui-file-executable-security-warning", { executable: PathUtils.filename(path) } ); - return this._prompter.confirm(title, message); + let flags = + Ci.nsIPrompt.BUTTON_DELAY_ENABLE | Ci.nsIPrompt.STD_OK_CANCEL_BUTTONS; + let nulls = Array(4).fill(null); + return 0 == this._prompter.confirmEx(title, message, flags, ...nulls, {}); }, /** ===================================== toolkit/components/reputationservice/ApplicationReputation.cpp ===================================== @@ -291,6 +291,7 @@ const char* const ApplicationReputationService::kBinaryFileExtensions[] = { //".jse", exec // JScript ".ksh", // Linux shell //".lha", + //".library-ms", exec // Windows //".lnk", exec // Windows ".local", // Windows //".lpaq1", ===================================== toolkit/components/reputationservice/test/gtest/TestExecutableLists.cpp ===================================== @@ -121,6 +121,7 @@ static const char* const kTestFileExtensions[] = { ".js", // JavaScript script ".jse", // JScript ".ksh", // Linux shell + ".library-ms", // Windows Library Files ".lnk", // Windows ".local", // Windows ".mad", // MS Access ===================================== xpcom/io/nsLocalFileCommon.cpp ===================================== @@ -72,6 +72,7 @@ const char* const sExecutableExts[] = { #endif ".js", ".jse", + ".library-ms", // Windows Library Files ".lnk", ".mad", // Access Module Shortcut ".maf", // Access ===================================== xpcom/io/nsLocalFileCommon.h ===================================== @@ -8,9 +8,9 @@ #define _NS_LOCAL_FILE_COMMON_H_ #ifdef MOZ_ESR -extern const char* const sExecutableExts[108]; -#else extern const char* const sExecutableExts[109]; +#else +extern const char* const sExecutableExts[110]; #endif #endif View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser/-/compare/75c944… -- View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser/-/compare/75c944… You're receiving this email because of your account on gitlab.torproject.org.

1 0

[Git][tpo/applications/mullvad-browser][mullvad-browser-128.5.0esr-14.0-1] Bug 1935621 - Fix virtual environment sysconfig path calculation...
by morgan (＠morgan) 10 Dec '24

10 Dec '24

morgan pushed to branch mullvad-browser-128.5.0esr-14.0-1 at The Tor Project / Applications / Mullvad Browser Commits: 94b5b3e9 by Filipe Laíns at 2024-12-10T18:14:53+00:00 Bug 1935621 - Fix virtual environment sysconfig path calculation r=firefox-build-system-reviewers,ahochheiden Signed-off-by: Filipe Laíns <lains(a)riseup.net> Signed-off-by: Filipe Laíns <lains(a)riseup.net> Signed-off-by: Filipe Laíns <lains(a)riseup.net> Differential Revision: https://phabricator.services.mozilla.com/D231480 - - - - - 1 changed file: - python/mach/mach/site.py Changes: ===================================== python/mach/mach/site.py ===================================== @@ -17,6 +17,7 @@ import subprocess import sys import sysconfig import tempfile +import warnings from contextlib import contextmanager from pathlib import Path from typing import Callable, Optional @@ -817,33 +818,75 @@ class PythonVirtualenv: """Calculates paths of interest for general python virtual environments""" def __init__(self, prefix): - if _is_windows: - self.bin_path = os.path.join(prefix, "Scripts") - self.python_path = os.path.join(self.bin_path, "python.exe") - else: - self.bin_path = os.path.join(prefix, "bin") - self.python_path = os.path.join(self.bin_path, "python") self.prefix = os.path.realpath(prefix) + self.paths = self._get_sysconfig_paths(self.prefix) - @functools.lru_cache(maxsize=None) - def resolve_sysconfig_packages_path(self, sysconfig_path): - # macOS uses a different default sysconfig scheme based on whether it's using the - # system Python or running in a virtualenv. - # Manually define the scheme (following the implementation in - # "sysconfig._get_default_scheme()") so that we're always following the - # code path for a virtualenv directory structure. - if os.name == "posix": - scheme = "posix_prefix" - else: - scheme = os.name + # Name of the Python executable to use in virtual environments. + # An executable with the same name as sys.executable might not exist in + # virtual environments. An executable with 'python' as the steam — + # without version numbers or ABI flags — will always be present in + # virtual environments, so we use that. + python_exe_name = "python" + sysconfig.get_config_var("EXE") + + self.bin_path = self.paths["scripts"] + self.python_path = os.path.join(self.bin_path, python_exe_name) - sysconfig_paths = sysconfig.get_paths(scheme) - data_path = Path(sysconfig_paths["data"]) - path = Path(sysconfig_paths[sysconfig_path]) - relative_path = path.relative_to(data_path) + @staticmethod + def _get_sysconfig_paths(prefix): + """Calculate the sysconfig paths of a virtual environment in the given prefix. - # Path to virtualenv's "site-packages" directory for provided sysconfig path - return os.path.normpath(os.path.normcase(Path(self.prefix) / relative_path)) + The virtual environment MUST be using the same Python distribution as us. + """ + # Determine the sysconfig scheme used in virtual environments + if "venv" in sysconfig.get_scheme_names(): + # A 'venv' scheme was added in Python 3.11 to allow users to + # calculate the paths for a virtual environment, since the default + # scheme may not always be the same as used on virtual environments. + # Some common examples are the system Python distributed by macOS, + # Debian, and Fedora. + # For more information, see https://github.com/python/cpython/issues/89576 + venv_scheme = "venv" + elif os.name == "nt": + # We know that before the 'venv' scheme was added, on Windows, + # the 'nt' scheme was used in virtual environments. + venv_scheme = "nt" + elif os.name == "posix": + # We know that before the 'venv' scheme was added, on POSIX, + # the 'posix_prefix' scheme was used in virtual environments. + venv_scheme = "posix_prefix" + else: + # This should never happen with upstream Python, as the 'venv' + # scheme should always be available on >=3.11, and no other + # platforms are supported by the upstream on older Python versions. + # + # Since the 'venv' scheme isn't available, and we have no knowledge + # of this platform/distribution, fallback to the default scheme. + # + # Hitting this will likely be the result of running a custom Python + # distribution targetting a platform that is not supported by the + # upstream. + # In this case, unless the Python vendor patched the Python + # distribution in such a way as the default scheme may not always be + # the same scheme, using the default scheme should be correct. + # If the vendor did patch Python as such, to work around this issue, + # I would recommend them to define a 'venv' scheme that matches + # the layout used on virtual environments in their Python distribution. + # (rec. signed Filipe Laíns — upstream sysconfig maintainer) + venv_scheme = sysconfig.get_default_scheme() + warnings.warn( + f"Unknown platform '{os.name}', using the default install scheme '{venv_scheme}'. " + "If this is incorrect, please ask your Python vendor to add a 'venv' sysconfig scheme " + "(see https://github.com/python/cpython/issues/89576, or check the code comment).", + stacklevel=2, + ) + # Build the sysconfig config_vars dictionary for the virtual environment. + venv_vars = sysconfig.get_config_vars().copy() + venv_vars["base"] = venv_vars["platbase"] = prefix + # Get sysconfig paths for the virtual environment. + return sysconfig.get_paths(venv_scheme, vars=venv_vars) + + def resolve_sysconfig_packages_path(self, sysconfig_path): + return self.paths[sysconfig_path] def site_packages_dirs(self): dirs = [] View it on GitLab: https://gitlab.torproject.org/tpo/applications/mullvad-browser/-/commit/94b… -- View it on GitLab: https://gitlab.torproject.org/tpo/applications/mullvad-browser/-/commit/94b… You're receiving this email because of your account on gitlab.torproject.org.

1 0

[Git][tpo/applications/tor-browser][base-browser-128.5.0esr-14.0-1] Bug 1935621 - Fix virtual environment sysconfig path calculation...
by morgan (＠morgan) 10 Dec '24

10 Dec '24

morgan pushed to branch base-browser-128.5.0esr-14.0-1 at The Tor Project / Applications / Tor Browser Commits: 83e5b5e2 by Filipe Laíns at 2024-12-10T18:12:04+00:00 Bug 1935621 - Fix virtual environment sysconfig path calculation r=firefox-build-system-reviewers,ahochheiden Signed-off-by: Filipe Laíns <lains(a)riseup.net> Signed-off-by: Filipe Laíns <lains(a)riseup.net> Signed-off-by: Filipe Laíns <lains(a)riseup.net> Differential Revision: https://phabricator.services.mozilla.com/D231480 - - - - - 1 changed file: - python/mach/mach/site.py Changes: ===================================== python/mach/mach/site.py ===================================== @@ -17,6 +17,7 @@ import subprocess import sys import sysconfig import tempfile +import warnings from contextlib import contextmanager from pathlib import Path from typing import Callable, Optional @@ -817,33 +818,75 @@ class PythonVirtualenv: """Calculates paths of interest for general python virtual environments""" def __init__(self, prefix): - if _is_windows: - self.bin_path = os.path.join(prefix, "Scripts") - self.python_path = os.path.join(self.bin_path, "python.exe") - else: - self.bin_path = os.path.join(prefix, "bin") - self.python_path = os.path.join(self.bin_path, "python") self.prefix = os.path.realpath(prefix) + self.paths = self._get_sysconfig_paths(self.prefix) - @functools.lru_cache(maxsize=None) - def resolve_sysconfig_packages_path(self, sysconfig_path): - # macOS uses a different default sysconfig scheme based on whether it's using the - # system Python or running in a virtualenv. - # Manually define the scheme (following the implementation in - # "sysconfig._get_default_scheme()") so that we're always following the - # code path for a virtualenv directory structure. - if os.name == "posix": - scheme = "posix_prefix" - else: - scheme = os.name + # Name of the Python executable to use in virtual environments. + # An executable with the same name as sys.executable might not exist in + # virtual environments. An executable with 'python' as the steam — + # without version numbers or ABI flags — will always be present in + # virtual environments, so we use that. + python_exe_name = "python" + sysconfig.get_config_var("EXE") + + self.bin_path = self.paths["scripts"] + self.python_path = os.path.join(self.bin_path, python_exe_name) - sysconfig_paths = sysconfig.get_paths(scheme) - data_path = Path(sysconfig_paths["data"]) - path = Path(sysconfig_paths[sysconfig_path]) - relative_path = path.relative_to(data_path) + @staticmethod + def _get_sysconfig_paths(prefix): + """Calculate the sysconfig paths of a virtual environment in the given prefix. - # Path to virtualenv's "site-packages" directory for provided sysconfig path - return os.path.normpath(os.path.normcase(Path(self.prefix) / relative_path)) + The virtual environment MUST be using the same Python distribution as us. + """ + # Determine the sysconfig scheme used in virtual environments + if "venv" in sysconfig.get_scheme_names(): + # A 'venv' scheme was added in Python 3.11 to allow users to + # calculate the paths for a virtual environment, since the default + # scheme may not always be the same as used on virtual environments. + # Some common examples are the system Python distributed by macOS, + # Debian, and Fedora. + # For more information, see https://github.com/python/cpython/issues/89576 + venv_scheme = "venv" + elif os.name == "nt": + # We know that before the 'venv' scheme was added, on Windows, + # the 'nt' scheme was used in virtual environments. + venv_scheme = "nt" + elif os.name == "posix": + # We know that before the 'venv' scheme was added, on POSIX, + # the 'posix_prefix' scheme was used in virtual environments. + venv_scheme = "posix_prefix" + else: + # This should never happen with upstream Python, as the 'venv' + # scheme should always be available on >=3.11, and no other + # platforms are supported by the upstream on older Python versions. + # + # Since the 'venv' scheme isn't available, and we have no knowledge + # of this platform/distribution, fallback to the default scheme. + # + # Hitting this will likely be the result of running a custom Python + # distribution targetting a platform that is not supported by the + # upstream. + # In this case, unless the Python vendor patched the Python + # distribution in such a way as the default scheme may not always be + # the same scheme, using the default scheme should be correct. + # If the vendor did patch Python as such, to work around this issue, + # I would recommend them to define a 'venv' scheme that matches + # the layout used on virtual environments in their Python distribution. + # (rec. signed Filipe Laíns — upstream sysconfig maintainer) + venv_scheme = sysconfig.get_default_scheme() + warnings.warn( + f"Unknown platform '{os.name}', using the default install scheme '{venv_scheme}'. " + "If this is incorrect, please ask your Python vendor to add a 'venv' sysconfig scheme " + "(see https://github.com/python/cpython/issues/89576, or check the code comment).", + stacklevel=2, + ) + # Build the sysconfig config_vars dictionary for the virtual environment. + venv_vars = sysconfig.get_config_vars().copy() + venv_vars["base"] = venv_vars["platbase"] = prefix + # Get sysconfig paths for the virtual environment. + return sysconfig.get_paths(venv_scheme, vars=venv_vars) + + def resolve_sysconfig_packages_path(self, sysconfig_path): + return self.paths[sysconfig_path] def site_packages_dirs(self): dirs = [] View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser/-/commit/83e5b5e… -- View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser/-/commit/83e5b5e… You're receiving this email because of your account on gitlab.torproject.org.

1 0