October 2020 - tbb-commits - lists.torproject.org

[tor-browser-build/master] Bug 40134: Support release train in android-components
by sysrqb＠torproject.org 27 Oct '20

27 Oct '20

commit 666785d9d539a23873f924986026a02fb32154a8 Author: Georg Koppen <gk(a)torproject.org> Date: Thu Oct 22 17:17:37 2020 +0000 Bug 40134: Support release train in android-components --- projects/android-components/build | 12 ++++++++---- projects/android-components/config | 1 + 2 files changed, 9 insertions(+), 4 deletions(-) diff --git a/projects/android-components/build b/projects/android-components/build index ace9455..622da23 100644 --- a/projects/android-components/build +++ b/projects/android-components/build @@ -54,9 +54,14 @@ patch -p1 < $rootdir/git.patch fi # We want to make sure only our code is built. Overwrite engine-gecko code we - # don't need. XXX: Take release into account. + # don't need. cd components/browser - [% IF c("var/alpha") || c("var/nightly") %] + [% IF c("var/release") %] + rm -rf engine-gecko-beta/src/* + rm -rf engine-gecko-nightly/src/* + cp -rf engine-gecko/src/* engine-gecko-beta/src/ + cp -rf engine-gecko/src/* engine-gecko-nightly/src/ + [% ELSE %] rm -rf engine-gecko/src/* rm -rf engine-gecko-nightly/src/* cp -rf engine-gecko-beta/src/* engine-gecko/src/ @@ -65,8 +70,7 @@ patch -p1 < $rootdir/git.patch cd ../../ gradle_args="--offline --no-daemon -Dmaven.repo.local=$gradle_repo" - # XXX: Take release into account - gradle $gradle_args assembleGeckoBeta -x lint + gradle $gradle_args assembleGecko[% c('variant') %] -x lint gradle $gradle_args publish # We only need the archives and .pom files for now. diff --git a/projects/android-components/config b/projects/android-components/config index adf0e11..b715d2e 100644 --- a/projects/android-components/config +++ b/projects/android-components/config @@ -5,6 +5,7 @@ git_hash: '[% project %]-[% c("var/android_components_version") %]-[% c("var/tor git_url: https://gitlab.torproject.org/tpo/applications/android-components.git tag_gpg_id: 1 gpg_keyring: torbutton.gpg +variant: '[% IF c("var/release") %]Release[% ELSE %]Beta[% END %]' var: android_components_version: 60.0.5

1 0

[Git][tpo/applications/fenix][tor-browser-82.1.1-10.0-1] 2 commits: fixup! Bug 40015: Modify Home menu
by Matthew Finkel 27 Oct '20

27 Oct '20

Matthew Finkel pushed to branch tor-browser-82.1.1-10.0-1 at The Tor Project / Applications / fenix Commits: 94888f5b by Matthew Finkel at 2020-10-27T22:12:23+00:00 fixup! Bug 40015: Modify Home menu Bug 40093: Enable Quit menu button - - - - - b30df1e9 by Matthew Finkel at 2020-10-27T22:12:49+00:00 fixup! Bug 40016: Modify Default toolbar menu Bug 40093: Enable Quit menu button - - - - - 2 changed files: - app/src/main/java/org/mozilla/fenix/components/toolbar/DefaultToolbarMenu.kt - app/src/main/java/org/mozilla/fenix/home/HomeMenu.kt Changes: ===================================== app/src/main/java/org/mozilla/fenix/components/toolbar/DefaultToolbarMenu.kt ===================================== @@ -175,16 +175,13 @@ class DefaultToolbarMenu( // Predicates that are called once, during screen init val shouldShowSaveToCollection = (context.asActivity() as? HomeActivity) ?.browsingModeManager?.mode == BrowsingMode.Normal - val shouldDeleteDataOnQuit = context.components.settings - .shouldDeleteBrowsingDataOnQuit && - !context.components.settings.shouldDisableNormalMode val menuItems = listOfNotNull( downloadsItem, // historyItem, bookmarksItem, settings, - if (shouldDeleteDataOnQuit) deleteDataOnQuit else null, + deleteDataOnQuit, BrowserMenuDivider(), findInPage, addToTopSites, ===================================== app/src/main/java/org/mozilla/fenix/home/HomeMenu.kt ===================================== @@ -160,12 +160,10 @@ class HomeMenu( null } - val settings = context.components.settings - val shouldDeleteBrowsingDataOnQuit = settings.shouldDeleteBrowsingDataOnQuit && - !settings.shouldDisableNormalMode + // val settings = context.components.settings val menuItems = listOfNotNull( - if (shouldDeleteBrowsingDataOnQuit) quitItem else null, + quitItem, settingsItem, BrowserMenuDivider(), // if (settings.syncedTabsInTabsTray) null else syncedTabsItem, View it on GitLab: https://gitlab.torproject.org/tpo/applications/fenix/-/compare/abfb81fec559… -- View it on GitLab: https://gitlab.torproject.org/tpo/applications/fenix/-/compare/abfb81fec559… You're receiving this email because of your account on gitlab.torproject.org.

1 0

[tor-browser-build/maint-10.0a9] Add missing ChangeLog entry and tag build1
by sysrqb＠torproject.org 26 Oct '20

26 Oct '20

commit 638b865f9c42400f44112e8e34e7cb51190620c1 Author: Matthew Finkel <sysrqb(a)torproject.org> Date: Mon Oct 26 13:40:04 2020 +0000 Add missing ChangeLog entry and tag build1 --- projects/tor-browser/Bundle-Data/Docs/ChangeLog.txt | 1 + 1 file changed, 1 insertion(+) diff --git a/projects/tor-browser/Bundle-Data/Docs/ChangeLog.txt b/projects/tor-browser/Bundle-Data/Docs/ChangeLog.txt index e759802..5f9e6ef 100644 --- a/projects/tor-browser/Bundle-Data/Docs/ChangeLog.txt +++ b/projects/tor-browser/Bundle-Data/Docs/ChangeLog.txt @@ -4,6 +4,7 @@ Tor Browser 10.0a9 -- October 26 2020 * Update NoScript to 11.1.3 * Update OpenSSL to 1.1.1h * Bug 30605: Honor privacy.spoof_english + * Bug 40003: Block starting Tor when setup is not complete [tor-android-service] * Bug 40004: "Tor Browser" string is used instead of "Alpha"/"Nightly" for non en-US locales [tor-android-service] * Bug 40016: Allow inheriting from AddonCollectionProvider [android-components] * Bug 40017: Rebase android-components patches to 60 [android-components]

1 0

[tor-browser/tor-browser-83.0b1-10.0-1] Bug 30605: Honor privacy.spoof_english in Android
by gk＠torproject.org 25 Oct '20

25 Oct '20

commit 244d7caea0f00f1b07f6a9a4a67daab7d0643cb3 Author: Alex Catarineu <acat(a)torproject.org> Date: Fri Oct 16 10:45:17 2020 +0200 Bug 30605: Honor privacy.spoof_english in Android This checks `privacy.spoof_english` whenever `setLocales` is called from Fenix side and sets `intl.accept_languages` accordingly. --- mobile/android/components/geckoview/GeckoViewStartup.jsm | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/mobile/android/components/geckoview/GeckoViewStartup.jsm b/mobile/android/components/geckoview/GeckoViewStartup.jsm index 6eaf47ad4440..4a53dcde55c7 100644 --- a/mobile/android/components/geckoview/GeckoViewStartup.jsm +++ b/mobile/android/components/geckoview/GeckoViewStartup.jsm @@ -18,6 +18,7 @@ XPCOMUtils.defineLazyModuleGetters(this, { Preferences: "resource://gre/modules/Preferences.jsm", SafeBrowsing: "resource://gre/modules/SafeBrowsing.jsm", Services: "resource://gre/modules/Services.jsm", + RFPHelper: "resource://gre/modules/RFPHelper.jsm", }); const { debug, warn } = GeckoViewUtils.initLogging("Startup"); @@ -257,6 +258,10 @@ class GeckoViewStartup { if (aData.requestedLocales) { Services.locale.requestedLocales = aData.requestedLocales; } + RFPHelper._handleSpoofEnglishChanged(); + if (Services.prefs.getIntPref("privacy.spoof_english", 0) === 2) { + break; + } const pls = Cc["@mozilla.org/pref-localizedstring;1"].createInstance( Ci.nsIPrefLocalizedString );

1 0

[tor-browser/tor-browser-83.0b1-10.0-1] Bug 40199: Avoid using system locale for intl.accept_languages in GeckoView
by gk＠torproject.org 25 Oct '20

25 Oct '20

commit 8f7ea57115cd1dc921ca3c8e1dc8c6098fc2b91e Author: Alex Catarineu <acat(a)torproject.org> Date: Tue Oct 20 17:44:36 2020 +0200 Bug 40199: Avoid using system locale for intl.accept_languages in GeckoView --- .../mozilla/geckoview/GeckoRuntimeSettings.java | 26 +++++++++++++--------- 1 file changed, 16 insertions(+), 10 deletions(-) diff --git a/mobile/android/geckoview/src/main/java/org/mozilla/geckoview/GeckoRuntimeSettings.java b/mobile/android/geckoview/src/main/java/org/mozilla/geckoview/GeckoRuntimeSettings.java index 85517821b59a..b7c080557470 100644 --- a/mobile/android/geckoview/src/main/java/org/mozilla/geckoview/GeckoRuntimeSettings.java +++ b/mobile/android/geckoview/src/main/java/org/mozilla/geckoview/GeckoRuntimeSettings.java @@ -783,19 +783,25 @@ public final class GeckoRuntimeSettings extends RuntimeSettings { private String computeAcceptLanguages() { ArrayList<String> locales = new ArrayList<String>(); - // Explicitly-set app prefs come first: - if (mRequestedLocales != null) { - for (String locale : mRequestedLocales) { - locales.add(locale.toLowerCase()); - } - } - // OS prefs come second: - for (String locale : getDefaultLocales()) { - locale = locale.toLowerCase(); - if (!locales.contains(locale)) { + // In Desktop, these are defined in the `intl.accept_languages` localized property. + // At some point we should probably use the same values here, but for now we use a simple + // strategy which will hopefully result in reasonable acceptLanguage values. + if (mRequestedLocales != null && mRequestedLocales.length > 0) { + String locale = mRequestedLocales[0].toLowerCase(); + // No need to include `en-us` twice. + if (!locale.equals("en-us")) { locales.add(locale); + if (locale.contains("-")) { + String lang = locale.split("-")[0]; + // No need to include `en` twice. + if (!lang.equals("en")) { + locales.add(lang); + } + } } } + locales.add("en-us"); + locales.add("en"); return TextUtils.join(",", locales); }

1 0

[tor-browser/tor-browser-83.0b1-10.0-1] Bug 40198: Expose privacy.spoof_english pref in GeckoView
by gk＠torproject.org 25 Oct '20

25 Oct '20

commit 024c389bebe7c4cbcf63efca911f9aea65ab7f18 Author: Alex Catarineu <acat(a)torproject.org> Date: Sun Oct 18 17:06:04 2020 +0200 Bug 40198: Expose privacy.spoof_english pref in GeckoView --- mobile/android/geckoview/api.txt | 3 ++ .../mozilla/geckoview/GeckoRuntimeSettings.java | 33 ++++++++++++++++++++++ 2 files changed, 36 insertions(+) diff --git a/mobile/android/geckoview/api.txt b/mobile/android/geckoview/api.txt index dc1349a96c26..0bfb65a2fb91 100644 --- a/mobile/android/geckoview/api.txt +++ b/mobile/android/geckoview/api.txt @@ -630,6 +630,7 @@ package org.mozilla.geckoview { method public boolean getRemoteDebuggingEnabled(); method @Nullable public GeckoRuntime getRuntime(); method @Nullable public Rect getScreenSizeOverride(); + method public boolean getSpoofEnglish(); method @Nullable public RuntimeTelemetry.Delegate getTelemetryDelegate(); method public int getTorSecurityLevel(); method public boolean getUseMaxScreenDepth(); @@ -650,6 +651,7 @@ package org.mozilla.geckoview { method @NonNull public GeckoRuntimeSettings setLoginAutofillEnabled(boolean); method @NonNull public GeckoRuntimeSettings setPreferredColorScheme(int); method @NonNull public GeckoRuntimeSettings setRemoteDebuggingEnabled(boolean); + method @NonNull public GeckoRuntimeSettings setSpoofEnglish(boolean); method @NonNull public GeckoRuntimeSettings setTorSecurityLevel(int); method @NonNull public GeckoRuntimeSettings setWebFontsEnabled(boolean); method @NonNull public GeckoRuntimeSettings setWebManifestEnabled(boolean); @@ -685,6 +687,7 @@ package org.mozilla.geckoview { method @NonNull public GeckoRuntimeSettings.Builder preferredColorScheme(int); method @NonNull public GeckoRuntimeSettings.Builder remoteDebuggingEnabled(boolean); method @NonNull public GeckoRuntimeSettings.Builder screenSizeOverride(int, int); + method @NonNull public GeckoRuntimeSettings.Builder spoofEnglish(boolean); method @NonNull public GeckoRuntimeSettings.Builder telemetryDelegate(@NonNull RuntimeTelemetry.Delegate); method @NonNull public GeckoRuntimeSettings.Builder torSecurityLevel(int); method @NonNull public GeckoRuntimeSettings.Builder useMaxScreenDepth(boolean); diff --git a/mobile/android/geckoview/src/main/java/org/mozilla/geckoview/GeckoRuntimeSettings.java b/mobile/android/geckoview/src/main/java/org/mozilla/geckoview/GeckoRuntimeSettings.java index b7c080557470..5bad77d4f880 100644 --- a/mobile/android/geckoview/src/main/java/org/mozilla/geckoview/GeckoRuntimeSettings.java +++ b/mobile/android/geckoview/src/main/java/org/mozilla/geckoview/GeckoRuntimeSettings.java @@ -468,6 +468,17 @@ public final class GeckoRuntimeSettings extends RuntimeSettings { getSettings().mTorSecurityLevel.set(level); return this; } + + /** + * Sets whether we should spoof locale to English for webpages. + * + * @param flag True if we should spoof locale to English for webpages, false otherwise. + * @return This Builder instance. + */ + public @NonNull Builder spoofEnglish(final boolean flag) { + getSettings().mSpoofEnglish.set(flag ? 2 : 1); + return this; + } } private GeckoRuntime mRuntime; @@ -520,6 +531,8 @@ public final class GeckoRuntimeSettings extends RuntimeSettings { "signon.autofillForms", true); /* package */ final Pref<Integer> mTorSecurityLevel = new Pref<>( "extensions.torbutton.security_slider", 4); + /* package */ final Pref<Integer> mSpoofEnglish = new Pref<>( + "privacy.spoof_english", 0); /* package */ int mPreferredColorScheme = COLOR_SCHEME_SYSTEM; @@ -1196,6 +1209,26 @@ public final class GeckoRuntimeSettings extends RuntimeSettings { return this; } + /** + * Get whether we should spoof locale to English for webpages. + * + * @return Whether we should spoof locale to English for webpages. + */ + public boolean getSpoofEnglish() { + return mSpoofEnglish.get() == 2; + } + + /** + * Set whether we should spoof locale to English for webpages. + * + * @param flag A flag determining whether we should locale to English for webpages. + * @return This GeckoRuntimeSettings instance. + */ + public @NonNull GeckoRuntimeSettings setSpoofEnglish(final boolean flag) { + mSpoofEnglish.commit(flag ? 2 : 1); + return this; + } + @Override // Parcelable public void writeToParcel(final Parcel out, final int flags) { super.writeToParcel(out, flags);

1 0

[tor-browser-build/master] Bug 25102: Add a script to sign and publish nightly updates
by gk＠torproject.org 25 Oct '20

25 Oct '20

commit 8d66414b7860751ffec6a83a6bc6dbfbd94f801a Author: Nicolas Vigier <boklm(a)torproject.org> Date: Mon Dec 16 19:04:39 2019 +0100 Bug 25102: Add a script to sign and publish nightly updates --- keyring/torbrowser-nightly.gpg | Bin 0 -> 2289 bytes tools/signing/README | 3 + tools/signing/nightly/.gitignore | 2 + tools/signing/nightly/config.yml | 18 ++ tools/signing/nightly/sign-nightly | 260 +++++++++++++++++++++ .../nightly/update-responses-base-config.yml | 32 +++ 6 files changed, 315 insertions(+) diff --git a/keyring/torbrowser-nightly.gpg b/keyring/torbrowser-nightly.gpg new file mode 100644 index 0000000..15d2b62 Binary files /dev/null and b/keyring/torbrowser-nightly.gpg differ diff --git a/tools/signing/README b/tools/signing/README index e18a761..799fdf8 100644 --- a/tools/signing/README +++ b/tools/signing/README @@ -5,3 +5,6 @@ omitting specific paths and credential information. Additionally, when starting to used them for an own signing setup don't forget to adapt the locale list if needed. The entitlement files, however, are kept up-to-date. + +The scripts in the nightly/ directory are used to sign and publish the +nightly updates. diff --git a/tools/signing/nightly/.gitignore b/tools/signing/nightly/.gitignore index ec07f9a..557bef7 100644 --- a/tools/signing/nightly/.gitignore +++ b/tools/signing/nightly/.gitignore @@ -1 +1,3 @@ +lock nssdb +martools-* diff --git a/tools/signing/nightly/config.yml b/tools/signing/nightly/config.yml new file mode 100644 index 0000000..36743a4 --- /dev/null +++ b/tools/signing/nightly/config.yml @@ -0,0 +1,18 @@ +--- +martools_version: 9.0.2 +martools_url: https://archive.torproject.org/tor-package-archive/torbrowser/ +martools_gpg_keyring: keyring/torbrowser.gpg +builds_url: http://f4amtbsowhix7rrf.onion/tor-browser-builds +builds_url_auth_basic_username: tor-guest +builds_url_auth_basic_password: tor-guest +publish_dirs: + - nightly-linux-x86_64 + - nightly-linux-i686 + - nightly-windows-x86_64 + - nightly-windows-i686 + - nightly-osx-x86_64 +nss_db_dir: nssdb +nss_certname: nightly-marsigner +gpg_keyring: keyring/torbrowser-nightly.gpg +rsync_dest: nightly-publish:nightly-updates/ +post_rsync_cmd: ssh nightly-publish-static-update true diff --git a/tools/signing/nightly/sign-nightly b/tools/signing/nightly/sign-nightly new file mode 100755 index 0000000..9292748 --- /dev/null +++ b/tools/signing/nightly/sign-nightly @@ -0,0 +1,260 @@ +#!/usr/bin/perl -w +use strict; +use FindBin; +use YAML::XS qw(LoadFile DumpFile); +use POSIX qw(setlocale LC_ALL); +use File::Path qw(make_path); +use File::Temp; +use File::Copy; +use File::Basename; +use LWP::Simple; +use Path::Tiny; +use DateTime; +use Digest::SHA qw(sha256_hex); +use JSON; + +# Set umask and locale to provide a consistent environment +umask(0022); +$ENV{"LC_ALL"} = "C"; +setlocale(LC_ALL, "C"); + +sub exit_error { + print STDERR "Error: ", $_[0], "\n"; + chdir '/'; + unlink "$FindBin::Bin/lock"; + exit (exists $_[1] ? $_[1] : 1); +} + +exit_error "Missing config file: $FindBin::Bin/config.yml" + unless -f "$FindBin::Bin/config.yml"; +my $config = LoadFile("$FindBin::Bin/config.yml"); +my $topdir = "$FindBin::Bin/../../.."; + +{ + no warnings 'redefine'; + sub LWP::UserAgent::get_basic_credentials { + if ($config->{builds_url_auth_basic_username} + && $config->{builds_url_auth_basic_password}) { + return ( $config->{builds_url_auth_basic_username}, + $config->{builds_url_auth_basic_password} ); + } + return (); + } +} + +sub run_alone { + my $pidfile = "$FindBin::Bin/lock"; + if (-f $pidfile) { + my $pid = path($pidfile)->slurp_utf8; + if (kill(0, $pid)) { + print STDERR "Script is already running ($pid). Exiting.\n"; + exit 0; + } + } + path($pidfile)->spew_utf8($$); +} + +END { + unlink "$FindBin::Bin/lock"; +} + +sub get_tmpdir { + my ($config) = @_; + return File::Temp->newdir($config->{tmp_dir} ? + (DIR => $config->{tmp_dir}) + : ()); +} + +sub basedir_path { + my ($path, $basedir) = @_; + return ( $path =~ m|^/| ) ? $path : "$basedir/$path"; +} + +sub get_last_build_version { + my ($config, $publish_dir) = @_; + my $today = 'tbb-nightly.' . DateTime->now->ymd('.'); + my $dt_yesterday = DateTime->now - DateTime::Duration->new(days => 1); + my $yesterday = 'tbb-nightly.' . $dt_yesterday->ymd('.'); + for my $version ($today, $yesterday) { + my $url = "$config->{builds_url}/$version/$publish_dir/sha256sums-unsigned-build.incrementals.txt"; + return $version if get($url); + } + return undef; +} + +sub get_current_version { + my ($publish_dir) = @_; + my $file = "$topdir/nightly/$publish_dir-current-version.txt"; + return undef unless -f $file; + return path($file)->slurp_utf8; +} + +sub set_current_version { + my ($publish_dir, $version) = @_; + my $file = "$topdir/nightly/$publish_dir-current-version.txt"; + path($file)->spew_utf8($version); +} + +sub get_new_version { + my ($config, $publish_dir) = @_; + my $today = 'tbb-nightly.' . DateTime->now->ymd('.'); + my $current_ver = get_current_version($publish_dir); + my $last_ver = get_last_build_version($config, $publish_dir); + return $last_ver unless defined($current_ver); + return undef if $current_ver eq $today; + return undef unless defined($last_ver); + return undef if $current_ver eq $last_ver; + return $last_ver; +} + +sub download_file { + my ($url, $file, $sha256sum) = @_; + my $retries = 5; + while ($retries > 0) { + $retries--; + print "Downloading $url\n"; + next unless getstore("$url", "$file.tmp") == 200; + next unless $sha256sum eq sha256_hex(path("$file.tmp")->slurp_raw); + move("$file.tmp", $file); + return 1; + } + exit_error "Error downloading $url"; +} + +sub fetch_version { + my ($config, $publish_dir, $version) = @_; + my $tmpdir = get_tmpdir($config); + my $urldir = "$config->{builds_url}/$version/$publish_dir"; + my $destdir = "$topdir/nightly/$publish_dir/$version"; + + return if -d $destdir; + my $gpg_keyring = basedir_path($config->{gpg_keyring}, $topdir); + for my $file (qw/sha256sums-unsigned-build.txt sha256sums-unsigned-build.incrementals.txt/) { + my $url = "$urldir/$file"; + exit_error "Error downloading $url" + unless getstore($url, "$tmpdir/$file") == 200; + exit_error "Error downloading $url.asc" + unless getstore("$url.asc", "$tmpdir/$file.asc") == 200; + exit_error "Error checking gpg signature for $url" + if system('gpg', '--no-default-keyring', '--keyring', $gpg_keyring, + '--verify', "$tmpdir/$file.asc", + "$tmpdir/$file"); + } + my %sums = map { chomp; reverse split ' ', $_ } + ( + path("$tmpdir/sha256sums-unsigned-build.txt")->lines_utf8, + path("$tmpdir/sha256sums-unsigned-build.incrementals.txt")->lines_utf8, + ); + my @build_infos_file = grep { $_ =~ m/build-infos-.*\.json/ } keys %sums; + exit_error "Missing build-infos.json in $urldir" unless @build_infos_file; + download_file("$urldir/$build_infos_file[0]", + "$tmpdir/build-infos.json", $sums{$build_infos_file[0]}); + foreach my $file (sort grep { $_ =~ m/\.mar$/ } keys %sums) { + download_file("$urldir/$file", "$tmpdir/$file", $sums{$file}); + } + make_path("$topdir/nightly/$publish_dir"); + move $tmpdir, $destdir; + chmod 0755, $destdir; +} + +sub setup_martools { + my ($config) = @_; + my $martools_dir = "$FindBin::Bin/mar-tools-$config->{martools_version}"; + if (! -d $martools_dir) { + my $file = "mar-tools-linux64.zip"; + my $url = "$config->{martools_url}/$config->{martools_version}/$file"; + my $tmpdir = get_tmpdir($config); + exit_error "Error downloading $url" + unless getstore($url, "$tmpdir/$file") == 200; + exit_error "Error downloading $url.asc" + unless getstore("$url.asc", "$tmpdir/$file.asc") == 200; + my $gpg_keyring = basedir_path($config->{martools_gpg_keyring}, $topdir); + exit_error "Error checking gpg signature for $url" + if system('gpg', '--no-default-keyring', '--keyring', $gpg_keyring, + '--verify', "$tmpdir/$file.asc", + "$tmpdir/$file"); + exit_error "Error extracting martools" + unless system('unzip', '-d', $martools_dir, '-x', + "$tmpdir/$file") == 0; + } + $ENV{PATH} = "$martools_dir/mar-tools:$ENV{PATH}"; + if ($ENV{LD_LIBRARY_PATH}) { + $ENV{LD_LIBRARY_PATH} = "$martools_dir/mar-tools:$ENV{LD_LIBRARY_PATH}"; + } else { + $ENV{LD_LIBRARY_PATH} = "$martools_dir/mar-tools"; + } +} + +sub sign_version { + my ($config, $publish_dir, $version) = @_; + setup_martools($config); + my $nss_db_dir = basedir_path($config->{nss_db_dir}, $FindBin::Bin); + for my $marfile (path("$topdir/nightly/$publish_dir/$version")->children(qr/\.mar$/)) { + print "Signing $marfile\n"; + exit_error "Error signing $marfile" + unless system('signmar', '-d', $nss_db_dir, '-n', + $config->{nss_certname}, '-s', $marfile, + "$marfile-signed") == 0; + move("$marfile-signed", $marfile); + } +} + +sub get_buildinfos { + my ($filename) = @_; + exit_error "$filename does not exist" unless -f $filename; + return decode_json(path($filename)->slurp_utf8); +} + +sub update_responses { + my ($config, $publish_dir, $version) = @_; + my $ur_config = LoadFile("$FindBin::Bin/update-responses-base-config.yml"); + $ur_config->{download}{mars_url} .= "/$publish_dir"; + $ur_config->{releases_dir} = "$topdir/nightly/$publish_dir"; + $ur_config->{channels}->{nightly} = $version; + $ur_config->{versions}->{$version} = $ur_config->{versions}->{nightly_version}; + my $buildinfos = get_buildinfos("$topdir/nightly/$publish_dir/$version/build-infos.json"); + $ur_config->{versions}->{$version}{platformVersion} = $buildinfos->{firefox_platform_version}; + $ur_config->{versions}->{$version}{buildID} = $buildinfos->{firefox_buildid}; + DumpFile("$topdir/tools/update-responses/config.yml", $ur_config); + my $htdocsdir = "$topdir/tools/update-responses/htdocs/nightly"; + path($htdocsdir)->remove_tree({ safe => 0 }); + exit_error "Error running update_responses for $publish_dir" unless + system("$topdir/tools/update-responses/update_responses") == 0; + path("$topdir/nightly/updates/$publish_dir/nightly")->remove_tree({ safe => 0 }); + make_path("$topdir/nightly/updates/$publish_dir"); + move($htdocsdir, "$topdir/nightly/updates/$publish_dir/nightly"); +} + +sub remove_oldversions { + my ($config, $publish_dir, $version) = @_; + for my $dir (path("$topdir/nightly/$publish_dir")->children) { + my ($filename) = fileparse($dir); + next if $filename eq $version; + path($dir)->remove_tree({ safe => 0 }); + } +} + +sub sync_dest { + my ($config) = @_; + exit_error "Error running rsync" + if system('rsync', '-aH', '--delete-after', + "$topdir/nightly/", "$config->{rsync_dest}/"); + if ($config->{post_rsync_cmd}) { + exit_error "Error running $config->{post_rsync_cmd}" + if system($config->{post_rsync_cmd}); + } +} + +run_alone; +my $some_updates = 0; +foreach my $publish_dir (@{$config->{publish_dirs}}) { + my $new_version = get_new_version($config, $publish_dir); + next unless $new_version; + fetch_version($config, $publish_dir, $new_version); + sign_version($config, $publish_dir, $new_version); + update_responses($config, $publish_dir, $new_version); + set_current_version($publish_dir, $new_version); + remove_oldversions($config, $publish_dir, $new_version); + $some_updates = 1; +} +sync_dest($config) if $some_updates; diff --git a/tools/signing/nightly/update-responses-base-config.yml b/tools/signing/nightly/update-responses-base-config.yml new file mode 100644 index 0000000..4d60565 --- /dev/null +++ b/tools/signing/nightly/update-responses-base-config.yml @@ -0,0 +1,32 @@ +--- +download: + mars_url: https://nightlies.tbb.torproject.org/nightly-updates +appname_marfile: tor-browser +build_targets: + linux32: Linux_x86-gcc3 + linux64: Linux_x86_64-gcc3 + win32: + - WINNT_x86-gcc3 + - WINNT_x86-gcc3-x86 + - WINNT_x86-gcc3-x64 + win64: WINNT_x86_64-gcc3-x64 + osx32: Darwin_x86-gcc3 + osx64: Darwin_x86_64-gcc3 +channels: + nightly: nightly_version +versions: + nightly_version: + platformVersion: 68.2.0 + detailsURL: http://f4amtbsowhix7rrf.onion/ + migrate_langs: + pt-PT: pt-BR + minSupportedInstructionSet: SSE2 + osx32: + minSupportedOSVersion: 13.0.0 + osx64: + minSupportedOSVersion: 13.0.0 + win32: + minSupportedOSVersion: 6.1 + win64: + minSupportedOSVersion: 6.1 +mar_compression: xz

1 0

[tor-browser-build/master] Bug 25102: Setup nightly signing
by gk＠torproject.org 25 Oct '20

25 Oct '20

commit 17804f5ada032276eabf7e33d41feef46ca511d7 Author: Nicolas Vigier <boklm(a)torproject.org> Date: Thu Dec 19 16:46:07 2019 +0100 Bug 25102: Setup nightly signing --- tools/ansible/Makefile | 3 ++ tools/ansible/README | 7 +++ tools/ansible/ansible.cfg | 6 +++ tools/ansible/inventory | 1 + .../roles/tbb-nightly-signing/defaults/main.yml | 7 +++ .../roles/tbb-nightly-signing/tasks/main.yml | 54 ++++++++++++++++++++++ tools/ansible/tbb-nightly-signing.yml | 6 +++ 7 files changed, 84 insertions(+) diff --git a/tools/ansible/Makefile b/tools/ansible/Makefile index ea63a44..97a63c1 100644 --- a/tools/ansible/Makefile +++ b/tools/ansible/Makefile @@ -6,3 +6,6 @@ fpcentral: boklm-tbb-nightly-build: ansible-playbook --vault-password-file=~/ansible-vault/boklm-tbb-nightly -i inventory boklm-tbb-nightly-build.yml + +tbb-nightly-signing: + ANSIBLE_CONFIG='$((a)D)/ansible.cfg' ansible-playbook -i inventory tbb-nightly-signing.yml diff --git a/tools/ansible/README b/tools/ansible/README index 6056372..5407a73 100644 --- a/tools/ansible/README +++ b/tools/ansible/README @@ -25,6 +25,13 @@ boklm-tbb-nightly-build: For more details, see also this ticket: https://trac.torproject.org/projects/tor/ticket/33948 +tbb-nightly-signing: + This target is used to deploy a nightly signing machine. The + configuration of nightly signing is done in the following files: + tools/ansible/roles/tbb-nightly-signing/defaults/main.yml + tools/signing/nightly/config.yml + tools/signing/nightly/update-responses-base-config.yml + Adding, removing, updating users on the Tor Browser team build machine ====================================================================== diff --git a/tools/ansible/ansible.cfg b/tools/ansible/ansible.cfg new file mode 100644 index 0000000..0663746 --- /dev/null +++ b/tools/ansible/ansible.cfg @@ -0,0 +1,6 @@ +[ssh_connection] +; When connecting to a v3 onion, we get the error: +; "unix_listener: [...] too long for Unix domain socket" +; We solve this by using %n (The original remote hostname, as given on +; the command line) instead of %h (The remote hostname) in the control path. +control_path=%(directory)s/%%r-%%n-%%r diff --git a/tools/ansible/inventory b/tools/ansible/inventory index fc25842..47fda66 100644 --- a/tools/ansible/inventory +++ b/tools/ansible/inventory @@ -1,6 +1,7 @@ build-sunet-a ansible_ssh_user=root ansible_ssh_host=build-sunet-a.torproject.net fpcentral ansible_become=True ansible_become_method=sudo ansible_become_user=fpcentral ansible_ssh_host=forrestii.torproject.org allow_world_readable_tmpfiles=True boklm-tbb-nightly-build ansible_ssh_user=root ansible_become_method=su +tbb-nightly-signing ansible_ssh_user=root ansible_become_method=su [tbb-build] build-sunet-a diff --git a/tools/ansible/roles/tbb-nightly-signing/defaults/main.yml b/tools/ansible/roles/tbb-nightly-signing/defaults/main.yml new file mode 100644 index 0000000..cbe3b82 --- /dev/null +++ b/tools/ansible/roles/tbb-nightly-signing/defaults/main.yml @@ -0,0 +1,7 @@ +--- +nightly_signing_user: nightly-signing +nightly_signing_cron_hour: '*' +nightly_signing_cron_minute: '0,30' +tor_browser_build_dir: "/home/{{ nightly_signing_user }}/tor-browser-build" +tor_browser_build_git_url: https://git.torproject.org/builders/tor-browser-build.git +tor_browser_build_commit: 8d66414b7860751ffec6a83a6bc6dbfbd94f801a diff --git a/tools/ansible/roles/tbb-nightly-signing/tasks/main.yml b/tools/ansible/roles/tbb-nightly-signing/tasks/main.yml new file mode 100644 index 0000000..3cc96ba --- /dev/null +++ b/tools/ansible/roles/tbb-nightly-signing/tasks/main.yml @@ -0,0 +1,54 @@ +--- +- name: Install dependencies + apt: + name: "{{ item }}" + state: present + with_items: + - git + - libdatetime-perl + - libfindbin-libs-perl + - libfile-slurp-perl + - libxml-writer-perl + - libio-captureoutput-perl + - libparallel-forkmanager-perl + - libxml-libxml-perl + - libwww-perl + - libjson-perl + - libyaml-libyaml-perl + - libyaml-perl + - libtemplate-perl + - libio-handle-util-perl + - libio-all-perl + - libio-captureoutput-perl + - libpath-tiny-perl + - libstring-shellquote-perl + - libsort-versions-perl + - libdigest-sha-perl + - libdata-uuid-perl + - libdata-dump-perl + - libfile-copy-recursive-perl + - libnss3-tools + - rsync + +- name: create nightly-signing user + user: + name: "{{ nightly_signing_user }}" + comment: "Tor Browser Nightly Signing" + createhome: yes + home: "/home/{{ nightly_signing_user }}" + +- name: clone tor-browser-build + become: yes + become_user: "{{ nightly_signing_user }}" + git: + repo: "{{ tor_browser_build_git_url }}" + dest: "{{ tor_browser_build_dir }}" + version: "{{ tor_browser_build_commit }}" + +- name: add cron to sign nighly build + cron: + name: tbb-sign-nightly-build + user: "{{ nightly_signing_user }}" + hour: "{{ nightly_signing_cron_hour }}" + minute: "{{ nightly_signing_cron_minute }}" + job: "torsocks /home/{{ nightly_signing_user }}/tor-browser-build/tools/signing/nightly/sign-nightly" diff --git a/tools/ansible/tbb-nightly-signing.yml b/tools/ansible/tbb-nightly-signing.yml new file mode 100644 index 0000000..bc0c681 --- /dev/null +++ b/tools/ansible/tbb-nightly-signing.yml @@ -0,0 +1,6 @@ +--- + +- hosts: tbb-nightly-signing + roles: + - role: tbb-nightly-signing + - role: unattended-upgrades

1 0

[tor-browser-build/master] Merge remote-tracking branch 'boklm-gitlab/bug_25102_v13' into master
by gk＠torproject.org 25 Oct '20

25 Oct '20

commit 49ec15d1edc69fc5e3e1c04e398337d8ad24f7f7 Merge: 653f38e 17804f5 Author: Georg Koppen <gk(a)torproject.org> Date: Sun Oct 25 12:46:24 2020 +0000 Merge remote-tracking branch 'boklm-gitlab/bug_25102_v13' into master keyring/torbrowser-nightly.gpg | Bin 0 -> 2289 bytes tools/ansible/Makefile | 3 + tools/ansible/README | 7 + tools/ansible/ansible.cfg | 6 + tools/ansible/inventory | 1 + .../roles/tbb-nightly-signing/defaults/main.yml | 7 + .../roles/tbb-nightly-signing/tasks/main.yml | 54 +++++ tools/ansible/tbb-nightly-signing.yml | 6 + tools/signing/README | 3 + tools/signing/nightly/.gitignore | 2 + tools/signing/nightly/config.yml | 18 ++ tools/signing/nightly/sign-nightly | 260 +++++++++++++++++++++ .../nightly/update-responses-base-config.yml | 32 +++ 13 files changed, 399 insertions(+)

1 0

[tor-browser-bundle-testsuite/master] Bug 40008: Adapt dom-objects-enumeration test for 82
by gk＠torproject.org 25 Oct '20

25 Oct '20

commit 3f122e5fa7d7f8cd11230805de1b45c8dc4f06df Author: Alex Catarineu <acat(a)torproject.org> Date: Thu Oct 22 21:18:56 2020 +0200 Bug 40008: Adapt dom-objects-enumeration test for 82 --- .../tor_browser_tests/test_dom-objects-enumeration-worker.py | 8 +++----- marionette/tor_browser_tests/test_dom-objects-enumeration.py | 12 ++++++------ 2 files changed, 9 insertions(+), 11 deletions(-) diff --git a/marionette/tor_browser_tests/test_dom-objects-enumeration-worker.py b/marionette/tor_browser_tests/test_dom-objects-enumeration-worker.py index 5d21d1b..e03cdad 100644 --- a/marionette/tor_browser_tests/test_dom-objects-enumeration-worker.py +++ b/marionette/tor_browser_tests/test_dom-objects-enumeration-worker.py @@ -8,7 +8,7 @@ class Test(testsuite.TorBrowserTest): self.marionette.set_pref("network.proxy.allow_hijacking_localhost", False) self.test_page_file_url = self.marionette.absolute_url("dom-objects-enumeration.html?testType=worker") - self.expectedObjects = [ + self.expectedObjects = { "AbortController", "AbortSignal", "addEventListener", @@ -195,14 +195,12 @@ class Test(testsuite.TorBrowserTest): "XMLHttpRequest", "XMLHttpRequestEventTarget", "XMLHttpRequestUpload", - ] - self.expectedObjects80 = self.expectedObjects + ["AggregateError", "FinalizationRegistry", "WeakRef"] - self.expectedObjects80.sort() + } def test_dom_objects_enumeration_workers(self): expectedObjects = self.expectedObjects if self.get_version() >= 80: - expectedObjects = self.expectedObjects80 + expectedObjects = expectedObjects.union({"AggregateError", "FinalizationRegistry", "WeakRef"}) with self.marionette.using_context('content'): self.marionette.navigate(self.test_page_file_url) diff --git a/marionette/tor_browser_tests/test_dom-objects-enumeration.py b/marionette/tor_browser_tests/test_dom-objects-enumeration.py index 143a554..cdc2a1e 100644 --- a/marionette/tor_browser_tests/test_dom-objects-enumeration.py +++ b/marionette/tor_browser_tests/test_dom-objects-enumeration.py @@ -12,7 +12,7 @@ class Test(testsuite.TorBrowserTest): self.marionette.set_pref("network.proxy.allow_hijacking_localhost", False) self.test_page_file_url = self.marionette.absolute_url("dom-objects-enumeration.html?testType=window") # The list of expected DOM objects - self.expectedObjects = [ + self.expectedObjects = { "AbortController", "AbortSignal", "AbstractRange", @@ -749,15 +749,15 @@ class Test(testsuite.TorBrowserTest): "XPathExpression", "XPathResult", "XSLTProcessor", - ] - self.expectedObjects80 = self.expectedObjects + ["AggregateError", "FinalizationRegistry", "WeakRef"] - self.expectedObjects80.remove("content") - self.expectedObjects80.sort() + } def test_dom_objects_enumeration(self): expectedObjects = self.expectedObjects if self.get_version() >= 80: - expectedObjects = self.expectedObjects80 + expectedObjects = expectedObjects.union({"AggregateError", "FinalizationRegistry", "WeakRef"}) + expectedObjects.remove("content") + if self.get_version() >= 82: + expectedObjects = expectedObjects.union({"MediaMetadata","MediaSession","Sanitizer"}) with self.marionette.using_context('content'): self.marionette.navigate(self.test_page_file_url)

1 0