- tbb-commits - lists.torproject.org

[tor-browser-build/master] Bug 40139: Pick up rbm commit for bug 40008
by sysrqb＠torproject.org 28 Oct '20

28 Oct '20

commit 20d0226341581f8dbc61ffbb12cae102ad7470bd Author: Matthew Finkel <sysrqb(a)torproject.org> Date: Wed Oct 28 22:08:13 2020 +0000 Bug 40139: Pick up rbm commit for bug 40008 --- rbm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rbm b/rbm index a32239d..15c8476 160000 --- a/rbm +++ b/rbm @@ -1 +1 @@ -Subproject commit a32239ddb1f465115cb4a8bb9f4b853c149506f3 +Subproject commit 15c84760ebe3f2677d1212aa00ef6f7485446a69

1 0

[rbm/master] Bug 40008: Fix caching of the exec template function
by sysrqb＠torproject.org 28 Oct '20

28 Oct '20

commit 15c84760ebe3f2677d1212aa00ef6f7485446a69 Author: Nicolas Vigier <boklm(a)torproject.org> Date: Wed Oct 28 22:21:16 2020 +0100 Bug 40008: Fix caching of the exec template function In ce6b9136fc9e006536b we added caching of the exec template function, however we didn't take the project into account. --- lib/RBM.pm | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/lib/RBM.pm b/lib/RBM.pm index 825502a..074e385 100644 --- a/lib/RBM.pm +++ b/lib/RBM.pm @@ -483,7 +483,7 @@ sub execute { if (project_config($project, 'git_url', $options)) { my $git_hash = project_config($project, 'git_hash', $options) || exit_error "No git_hash specified for project $project"; - $res_name = "git-$git_hash-$cmd"; + $res_name = "git-$project-/-$git_hash-/-$cmd"; return $cache{$res_name} if exists $cache{$res_name}; git_clone_fetch_chdir($project, $options); my ($stdout, $stderr, $success, $exit_code) @@ -500,7 +500,7 @@ sub execute { } elsif (project_config($project, 'hg_url', $options)) { my $hg_hash = project_config($project, 'hg_hash', $options) || exit_error "No hg_hash specified for project $project"; - $res_name = "hg-$hg_hash-$cmd"; + $res_name = "hg-$project-/-$hg_hash-/-$cmd"; return $cache{$res_name} if exists $cache{$res_name}; hg_clone_fetch_chdir($project, $options); my ($stdout, $stderr, $success, $exit_code)

1 0

[tor-browser-build/master] Fold in 10.0a9 changelog
by sysrqb＠torproject.org 27 Oct '20

27 Oct '20

commit e82520103023dca58361811a6d96043a87550986 Author: Matthew Finkel <sysrqb(a)torproject.org> Date: Tue Oct 27 22:30:55 2020 +0000 Fold in 10.0a9 changelog --- .../tor-browser/Bundle-Data/Docs/ChangeLog.txt | 50 ++++++++++++++++++++++ 1 file changed, 50 insertions(+) diff --git a/projects/tor-browser/Bundle-Data/Docs/ChangeLog.txt b/projects/tor-browser/Bundle-Data/Docs/ChangeLog.txt index a95a67d..5f9e6ef 100644 --- a/projects/tor-browser/Bundle-Data/Docs/ChangeLog.txt +++ b/projects/tor-browser/Bundle-Data/Docs/ChangeLog.txt @@ -1,3 +1,53 @@ +Tor Browser 10.0a9 -- October 26 2020 + * Android + * Update Fenix to 82.1.1 + * Update NoScript to 11.1.3 + * Update OpenSSL to 1.1.1h + * Bug 30605: Honor privacy.spoof_english + * Bug 40003: Block starting Tor when setup is not complete [tor-android-service] + * Bug 40004: "Tor Browser" string is used instead of "Alpha"/"Nightly" for non en-US locales [tor-android-service] + * Bug 40016: Allow inheriting from AddonCollectionProvider [android-components] + * Bug 40017: Rebase android-components patches to 60 [android-components] + * Bug 40019: Expose spoofEnglish pref [android-components] + * Bug 40020: Disable third-party cookies [android-components] + * Bug 40021: Force telemetry=false in Fennec settings migration [android-components] + * Bug 40022: Migrate tor security settings [android-components] + * Bug 40023: Stop Private Notification Service [android-components] + * Bug 40024: Disable tracking protection by default [android-components] + * Bug 40050: Rebase Fenix patches to Fenix 82 [fenix] + * Bug 40053: Select your security settings panel on start page is confusing [fenix] + * Bug 40058: Disabling/Enabling addon still shows option to disallow in private mode [fenix] + * Bug 40062: HTTPS Everywhere is not shown as installed [fenix] + * Bug 40068: Tor Service closes when changing theme [fenix] + * Bug 40071: Show only supported locales [fenix] + * Bug 40073: Use correct branding on About page [fenix] + * Bug 40076: "Explore privately" not visible [fenix] + * Bug 40078: Crash at Android startup from background service [fenix] + * Bug 40082: Security level is reset when the app is killed [fenix] + * Bug 40083: Locale ordering in BuildConfig is non-deterministic [fenix] + * Bug 40087: Implement a switch for english locale spoofing [fenix] + * Bug 40088: Use Tor Browser logo in migration screen [fenix] + * Bug 40094: Do not use MasterPasswordTipProvider in HomeFragment [fenix] + * Bug 40095: Hide "Sign in to sync" in bookmarks [fenix] + * Bug 40097: Bump allowed_addons.json [fenix] + * Bug 40133: Rebase tor-browser patches to 82.0b1 [tor-browser] + * Bug 40166: Disable security.certerrors.mitm.auto_enable_enterprise_roots [tor-browser] + * Bug 40198: Expose privacy.spoof_english pref [tor-browser] + * Bug 40199: Avoid using system locale for intl.accept_languages [tor-browser] + * Translations update + * Build System + * Android + * Update Go to 1.14.10 + * Bug 34360: Bump binutils version to 2.35.1 + * Bug 40097: Update toolchain for Fenix 82 [tor-browser-build] + * Bug 40108: Package tooling-glean-gradle archive, too [tor-browser-build] + * Bug 40115: Update components for switch to mozilla82-based Fenix [tor-browser-build] + * Bug 40121: Use updated glean_parser for application-services as well [tor-browser-build] + * Bug 40124: Remove unused torbrowser-android-all (and related) targets [tor-browser-build] + * Bug 40125: Remove fenix-* projects [tor-browser-build] + * Bug 40129: application-services is missing rustc in PATH [tor-browser-build] + * Bug 40130: More mobile clean-up [tor-browser-build] + Tor Browser 10.5a2 -- October 20 2020 * Windows + OS X + Linux * Update Firefox to 78.4.0esr

1 0

[tor-browser-build/master] Bug 40134: Support release train in android-components
by sysrqb＠torproject.org 27 Oct '20

27 Oct '20

commit 666785d9d539a23873f924986026a02fb32154a8 Author: Georg Koppen <gk(a)torproject.org> Date: Thu Oct 22 17:17:37 2020 +0000 Bug 40134: Support release train in android-components --- projects/android-components/build | 12 ++++++++---- projects/android-components/config | 1 + 2 files changed, 9 insertions(+), 4 deletions(-) diff --git a/projects/android-components/build b/projects/android-components/build index ace9455..622da23 100644 --- a/projects/android-components/build +++ b/projects/android-components/build @@ -54,9 +54,14 @@ patch -p1 < $rootdir/git.patch fi # We want to make sure only our code is built. Overwrite engine-gecko code we - # don't need. XXX: Take release into account. + # don't need. cd components/browser - [% IF c("var/alpha") || c("var/nightly") %] + [% IF c("var/release") %] + rm -rf engine-gecko-beta/src/* + rm -rf engine-gecko-nightly/src/* + cp -rf engine-gecko/src/* engine-gecko-beta/src/ + cp -rf engine-gecko/src/* engine-gecko-nightly/src/ + [% ELSE %] rm -rf engine-gecko/src/* rm -rf engine-gecko-nightly/src/* cp -rf engine-gecko-beta/src/* engine-gecko/src/ @@ -65,8 +70,7 @@ patch -p1 < $rootdir/git.patch cd ../../ gradle_args="--offline --no-daemon -Dmaven.repo.local=$gradle_repo" - # XXX: Take release into account - gradle $gradle_args assembleGeckoBeta -x lint + gradle $gradle_args assembleGecko[% c('variant') %] -x lint gradle $gradle_args publish # We only need the archives and .pom files for now. diff --git a/projects/android-components/config b/projects/android-components/config index adf0e11..b715d2e 100644 --- a/projects/android-components/config +++ b/projects/android-components/config @@ -5,6 +5,7 @@ git_hash: '[% project %]-[% c("var/android_components_version") %]-[% c("var/tor git_url: https://gitlab.torproject.org/tpo/applications/android-components.git tag_gpg_id: 1 gpg_keyring: torbutton.gpg +variant: '[% IF c("var/release") %]Release[% ELSE %]Beta[% END %]' var: android_components_version: 60.0.5

1 0

[Git][tpo/applications/fenix][tor-browser-82.1.1-10.0-1] 2 commits: fixup! Bug 40015: Modify Home menu
by Matthew Finkel 27 Oct '20

27 Oct '20

Matthew Finkel pushed to branch tor-browser-82.1.1-10.0-1 at The Tor Project / Applications / fenix Commits: 94888f5b by Matthew Finkel at 2020-10-27T22:12:23+00:00 fixup! Bug 40015: Modify Home menu Bug 40093: Enable Quit menu button - - - - - b30df1e9 by Matthew Finkel at 2020-10-27T22:12:49+00:00 fixup! Bug 40016: Modify Default toolbar menu Bug 40093: Enable Quit menu button - - - - - 2 changed files: - app/src/main/java/org/mozilla/fenix/components/toolbar/DefaultToolbarMenu.kt - app/src/main/java/org/mozilla/fenix/home/HomeMenu.kt Changes: ===================================== app/src/main/java/org/mozilla/fenix/components/toolbar/DefaultToolbarMenu.kt ===================================== @@ -175,16 +175,13 @@ class DefaultToolbarMenu( // Predicates that are called once, during screen init val shouldShowSaveToCollection = (context.asActivity() as? HomeActivity) ?.browsingModeManager?.mode == BrowsingMode.Normal - val shouldDeleteDataOnQuit = context.components.settings - .shouldDeleteBrowsingDataOnQuit && - !context.components.settings.shouldDisableNormalMode val menuItems = listOfNotNull( downloadsItem, // historyItem, bookmarksItem, settings, - if (shouldDeleteDataOnQuit) deleteDataOnQuit else null, + deleteDataOnQuit, BrowserMenuDivider(), findInPage, addToTopSites, ===================================== app/src/main/java/org/mozilla/fenix/home/HomeMenu.kt ===================================== @@ -160,12 +160,10 @@ class HomeMenu( null } - val settings = context.components.settings - val shouldDeleteBrowsingDataOnQuit = settings.shouldDeleteBrowsingDataOnQuit && - !settings.shouldDisableNormalMode + // val settings = context.components.settings val menuItems = listOfNotNull( - if (shouldDeleteBrowsingDataOnQuit) quitItem else null, + quitItem, settingsItem, BrowserMenuDivider(), // if (settings.syncedTabsInTabsTray) null else syncedTabsItem, View it on GitLab: https://gitlab.torproject.org/tpo/applications/fenix/-/compare/abfb81fec559… -- View it on GitLab: https://gitlab.torproject.org/tpo/applications/fenix/-/compare/abfb81fec559… You're receiving this email because of your account on gitlab.torproject.org.

1 0

[tor-browser-build/maint-10.0a9] Add missing ChangeLog entry and tag build1
by sysrqb＠torproject.org 26 Oct '20

26 Oct '20

commit 638b865f9c42400f44112e8e34e7cb51190620c1 Author: Matthew Finkel <sysrqb(a)torproject.org> Date: Mon Oct 26 13:40:04 2020 +0000 Add missing ChangeLog entry and tag build1 --- projects/tor-browser/Bundle-Data/Docs/ChangeLog.txt | 1 + 1 file changed, 1 insertion(+) diff --git a/projects/tor-browser/Bundle-Data/Docs/ChangeLog.txt b/projects/tor-browser/Bundle-Data/Docs/ChangeLog.txt index e759802..5f9e6ef 100644 --- a/projects/tor-browser/Bundle-Data/Docs/ChangeLog.txt +++ b/projects/tor-browser/Bundle-Data/Docs/ChangeLog.txt @@ -4,6 +4,7 @@ Tor Browser 10.0a9 -- October 26 2020 * Update NoScript to 11.1.3 * Update OpenSSL to 1.1.1h * Bug 30605: Honor privacy.spoof_english + * Bug 40003: Block starting Tor when setup is not complete [tor-android-service] * Bug 40004: "Tor Browser" string is used instead of "Alpha"/"Nightly" for non en-US locales [tor-android-service] * Bug 40016: Allow inheriting from AddonCollectionProvider [android-components] * Bug 40017: Rebase android-components patches to 60 [android-components]

1 0

[tor-browser/tor-browser-83.0b1-10.0-1] Bug 30605: Honor privacy.spoof_english in Android
by gk＠torproject.org 25 Oct '20

25 Oct '20

commit 244d7caea0f00f1b07f6a9a4a67daab7d0643cb3 Author: Alex Catarineu <acat(a)torproject.org> Date: Fri Oct 16 10:45:17 2020 +0200 Bug 30605: Honor privacy.spoof_english in Android This checks `privacy.spoof_english` whenever `setLocales` is called from Fenix side and sets `intl.accept_languages` accordingly. --- mobile/android/components/geckoview/GeckoViewStartup.jsm | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/mobile/android/components/geckoview/GeckoViewStartup.jsm b/mobile/android/components/geckoview/GeckoViewStartup.jsm index 6eaf47ad4440..4a53dcde55c7 100644 --- a/mobile/android/components/geckoview/GeckoViewStartup.jsm +++ b/mobile/android/components/geckoview/GeckoViewStartup.jsm @@ -18,6 +18,7 @@ XPCOMUtils.defineLazyModuleGetters(this, { Preferences: "resource://gre/modules/Preferences.jsm", SafeBrowsing: "resource://gre/modules/SafeBrowsing.jsm", Services: "resource://gre/modules/Services.jsm", + RFPHelper: "resource://gre/modules/RFPHelper.jsm", }); const { debug, warn } = GeckoViewUtils.initLogging("Startup"); @@ -257,6 +258,10 @@ class GeckoViewStartup { if (aData.requestedLocales) { Services.locale.requestedLocales = aData.requestedLocales; } + RFPHelper._handleSpoofEnglishChanged(); + if (Services.prefs.getIntPref("privacy.spoof_english", 0) === 2) { + break; + } const pls = Cc["@mozilla.org/pref-localizedstring;1"].createInstance( Ci.nsIPrefLocalizedString );

1 0

[tor-browser/tor-browser-83.0b1-10.0-1] Bug 40199: Avoid using system locale for intl.accept_languages in GeckoView
by gk＠torproject.org 25 Oct '20

25 Oct '20

commit 8f7ea57115cd1dc921ca3c8e1dc8c6098fc2b91e Author: Alex Catarineu <acat(a)torproject.org> Date: Tue Oct 20 17:44:36 2020 +0200 Bug 40199: Avoid using system locale for intl.accept_languages in GeckoView --- .../mozilla/geckoview/GeckoRuntimeSettings.java | 26 +++++++++++++--------- 1 file changed, 16 insertions(+), 10 deletions(-) diff --git a/mobile/android/geckoview/src/main/java/org/mozilla/geckoview/GeckoRuntimeSettings.java b/mobile/android/geckoview/src/main/java/org/mozilla/geckoview/GeckoRuntimeSettings.java index 85517821b59a..b7c080557470 100644 --- a/mobile/android/geckoview/src/main/java/org/mozilla/geckoview/GeckoRuntimeSettings.java +++ b/mobile/android/geckoview/src/main/java/org/mozilla/geckoview/GeckoRuntimeSettings.java @@ -783,19 +783,25 @@ public final class GeckoRuntimeSettings extends RuntimeSettings { private String computeAcceptLanguages() { ArrayList<String> locales = new ArrayList<String>(); - // Explicitly-set app prefs come first: - if (mRequestedLocales != null) { - for (String locale : mRequestedLocales) { - locales.add(locale.toLowerCase()); - } - } - // OS prefs come second: - for (String locale : getDefaultLocales()) { - locale = locale.toLowerCase(); - if (!locales.contains(locale)) { + // In Desktop, these are defined in the `intl.accept_languages` localized property. + // At some point we should probably use the same values here, but for now we use a simple + // strategy which will hopefully result in reasonable acceptLanguage values. + if (mRequestedLocales != null && mRequestedLocales.length > 0) { + String locale = mRequestedLocales[0].toLowerCase(); + // No need to include `en-us` twice. + if (!locale.equals("en-us")) { locales.add(locale); + if (locale.contains("-")) { + String lang = locale.split("-")[0]; + // No need to include `en` twice. + if (!lang.equals("en")) { + locales.add(lang); + } + } } } + locales.add("en-us"); + locales.add("en"); return TextUtils.join(",", locales); }

1 0

[tor-browser/tor-browser-83.0b1-10.0-1] Bug 40198: Expose privacy.spoof_english pref in GeckoView
by gk＠torproject.org 25 Oct '20

25 Oct '20

commit 024c389bebe7c4cbcf63efca911f9aea65ab7f18 Author: Alex Catarineu <acat(a)torproject.org> Date: Sun Oct 18 17:06:04 2020 +0200 Bug 40198: Expose privacy.spoof_english pref in GeckoView --- mobile/android/geckoview/api.txt | 3 ++ .../mozilla/geckoview/GeckoRuntimeSettings.java | 33 ++++++++++++++++++++++ 2 files changed, 36 insertions(+) diff --git a/mobile/android/geckoview/api.txt b/mobile/android/geckoview/api.txt index dc1349a96c26..0bfb65a2fb91 100644 --- a/mobile/android/geckoview/api.txt +++ b/mobile/android/geckoview/api.txt @@ -630,6 +630,7 @@ package org.mozilla.geckoview { method public boolean getRemoteDebuggingEnabled(); method @Nullable public GeckoRuntime getRuntime(); method @Nullable public Rect getScreenSizeOverride(); + method public boolean getSpoofEnglish(); method @Nullable public RuntimeTelemetry.Delegate getTelemetryDelegate(); method public int getTorSecurityLevel(); method public boolean getUseMaxScreenDepth(); @@ -650,6 +651,7 @@ package org.mozilla.geckoview { method @NonNull public GeckoRuntimeSettings setLoginAutofillEnabled(boolean); method @NonNull public GeckoRuntimeSettings setPreferredColorScheme(int); method @NonNull public GeckoRuntimeSettings setRemoteDebuggingEnabled(boolean); + method @NonNull public GeckoRuntimeSettings setSpoofEnglish(boolean); method @NonNull public GeckoRuntimeSettings setTorSecurityLevel(int); method @NonNull public GeckoRuntimeSettings setWebFontsEnabled(boolean); method @NonNull public GeckoRuntimeSettings setWebManifestEnabled(boolean); @@ -685,6 +687,7 @@ package org.mozilla.geckoview { method @NonNull public GeckoRuntimeSettings.Builder preferredColorScheme(int); method @NonNull public GeckoRuntimeSettings.Builder remoteDebuggingEnabled(boolean); method @NonNull public GeckoRuntimeSettings.Builder screenSizeOverride(int, int); + method @NonNull public GeckoRuntimeSettings.Builder spoofEnglish(boolean); method @NonNull public GeckoRuntimeSettings.Builder telemetryDelegate(@NonNull RuntimeTelemetry.Delegate); method @NonNull public GeckoRuntimeSettings.Builder torSecurityLevel(int); method @NonNull public GeckoRuntimeSettings.Builder useMaxScreenDepth(boolean); diff --git a/mobile/android/geckoview/src/main/java/org/mozilla/geckoview/GeckoRuntimeSettings.java b/mobile/android/geckoview/src/main/java/org/mozilla/geckoview/GeckoRuntimeSettings.java index b7c080557470..5bad77d4f880 100644 --- a/mobile/android/geckoview/src/main/java/org/mozilla/geckoview/GeckoRuntimeSettings.java +++ b/mobile/android/geckoview/src/main/java/org/mozilla/geckoview/GeckoRuntimeSettings.java @@ -468,6 +468,17 @@ public final class GeckoRuntimeSettings extends RuntimeSettings { getSettings().mTorSecurityLevel.set(level); return this; } + + /** + * Sets whether we should spoof locale to English for webpages. + * + * @param flag True if we should spoof locale to English for webpages, false otherwise. + * @return This Builder instance. + */ + public @NonNull Builder spoofEnglish(final boolean flag) { + getSettings().mSpoofEnglish.set(flag ? 2 : 1); + return this; + } } private GeckoRuntime mRuntime; @@ -520,6 +531,8 @@ public final class GeckoRuntimeSettings extends RuntimeSettings { "signon.autofillForms", true); /* package */ final Pref<Integer> mTorSecurityLevel = new Pref<>( "extensions.torbutton.security_slider", 4); + /* package */ final Pref<Integer> mSpoofEnglish = new Pref<>( + "privacy.spoof_english", 0); /* package */ int mPreferredColorScheme = COLOR_SCHEME_SYSTEM; @@ -1196,6 +1209,26 @@ public final class GeckoRuntimeSettings extends RuntimeSettings { return this; } + /** + * Get whether we should spoof locale to English for webpages. + * + * @return Whether we should spoof locale to English for webpages. + */ + public boolean getSpoofEnglish() { + return mSpoofEnglish.get() == 2; + } + + /** + * Set whether we should spoof locale to English for webpages. + * + * @param flag A flag determining whether we should locale to English for webpages. + * @return This GeckoRuntimeSettings instance. + */ + public @NonNull GeckoRuntimeSettings setSpoofEnglish(final boolean flag) { + mSpoofEnglish.commit(flag ? 2 : 1); + return this; + } + @Override // Parcelable public void writeToParcel(final Parcel out, final int flags) { super.writeToParcel(out, flags);

1 0

[tor-browser-build/master] Bug 25102: Add a script to sign and publish nightly updates
by gk＠torproject.org 25 Oct '20

25 Oct '20

commit 8d66414b7860751ffec6a83a6bc6dbfbd94f801a Author: Nicolas Vigier <boklm(a)torproject.org> Date: Mon Dec 16 19:04:39 2019 +0100 Bug 25102: Add a script to sign and publish nightly updates --- keyring/torbrowser-nightly.gpg | Bin 0 -> 2289 bytes tools/signing/README | 3 + tools/signing/nightly/.gitignore | 2 + tools/signing/nightly/config.yml | 18 ++ tools/signing/nightly/sign-nightly | 260 +++++++++++++++++++++ .../nightly/update-responses-base-config.yml | 32 +++ 6 files changed, 315 insertions(+) diff --git a/keyring/torbrowser-nightly.gpg b/keyring/torbrowser-nightly.gpg new file mode 100644 index 0000000..15d2b62 Binary files /dev/null and b/keyring/torbrowser-nightly.gpg differ diff --git a/tools/signing/README b/tools/signing/README index e18a761..799fdf8 100644 --- a/tools/signing/README +++ b/tools/signing/README @@ -5,3 +5,6 @@ omitting specific paths and credential information. Additionally, when starting to used them for an own signing setup don't forget to adapt the locale list if needed. The entitlement files, however, are kept up-to-date. + +The scripts in the nightly/ directory are used to sign and publish the +nightly updates. diff --git a/tools/signing/nightly/.gitignore b/tools/signing/nightly/.gitignore index ec07f9a..557bef7 100644 --- a/tools/signing/nightly/.gitignore +++ b/tools/signing/nightly/.gitignore @@ -1 +1,3 @@ +lock nssdb +martools-* diff --git a/tools/signing/nightly/config.yml b/tools/signing/nightly/config.yml new file mode 100644 index 0000000..36743a4 --- /dev/null +++ b/tools/signing/nightly/config.yml @@ -0,0 +1,18 @@ +--- +martools_version: 9.0.2 +martools_url: https://archive.torproject.org/tor-package-archive/torbrowser/ +martools_gpg_keyring: keyring/torbrowser.gpg +builds_url: http://f4amtbsowhix7rrf.onion/tor-browser-builds +builds_url_auth_basic_username: tor-guest +builds_url_auth_basic_password: tor-guest +publish_dirs: + - nightly-linux-x86_64 + - nightly-linux-i686 + - nightly-windows-x86_64 + - nightly-windows-i686 + - nightly-osx-x86_64 +nss_db_dir: nssdb +nss_certname: nightly-marsigner +gpg_keyring: keyring/torbrowser-nightly.gpg +rsync_dest: nightly-publish:nightly-updates/ +post_rsync_cmd: ssh nightly-publish-static-update true diff --git a/tools/signing/nightly/sign-nightly b/tools/signing/nightly/sign-nightly new file mode 100755 index 0000000..9292748 --- /dev/null +++ b/tools/signing/nightly/sign-nightly @@ -0,0 +1,260 @@ +#!/usr/bin/perl -w +use strict; +use FindBin; +use YAML::XS qw(LoadFile DumpFile); +use POSIX qw(setlocale LC_ALL); +use File::Path qw(make_path); +use File::Temp; +use File::Copy; +use File::Basename; +use LWP::Simple; +use Path::Tiny; +use DateTime; +use Digest::SHA qw(sha256_hex); +use JSON; + +# Set umask and locale to provide a consistent environment +umask(0022); +$ENV{"LC_ALL"} = "C"; +setlocale(LC_ALL, "C"); + +sub exit_error { + print STDERR "Error: ", $_[0], "\n"; + chdir '/'; + unlink "$FindBin::Bin/lock"; + exit (exists $_[1] ? $_[1] : 1); +} + +exit_error "Missing config file: $FindBin::Bin/config.yml" + unless -f "$FindBin::Bin/config.yml"; +my $config = LoadFile("$FindBin::Bin/config.yml"); +my $topdir = "$FindBin::Bin/../../.."; + +{ + no warnings 'redefine'; + sub LWP::UserAgent::get_basic_credentials { + if ($config->{builds_url_auth_basic_username} + && $config->{builds_url_auth_basic_password}) { + return ( $config->{builds_url_auth_basic_username}, + $config->{builds_url_auth_basic_password} ); + } + return (); + } +} + +sub run_alone { + my $pidfile = "$FindBin::Bin/lock"; + if (-f $pidfile) { + my $pid = path($pidfile)->slurp_utf8; + if (kill(0, $pid)) { + print STDERR "Script is already running ($pid). Exiting.\n"; + exit 0; + } + } + path($pidfile)->spew_utf8($$); +} + +END { + unlink "$FindBin::Bin/lock"; +} + +sub get_tmpdir { + my ($config) = @_; + return File::Temp->newdir($config->{tmp_dir} ? + (DIR => $config->{tmp_dir}) + : ()); +} + +sub basedir_path { + my ($path, $basedir) = @_; + return ( $path =~ m|^/| ) ? $path : "$basedir/$path"; +} + +sub get_last_build_version { + my ($config, $publish_dir) = @_; + my $today = 'tbb-nightly.' . DateTime->now->ymd('.'); + my $dt_yesterday = DateTime->now - DateTime::Duration->new(days => 1); + my $yesterday = 'tbb-nightly.' . $dt_yesterday->ymd('.'); + for my $version ($today, $yesterday) { + my $url = "$config->{builds_url}/$version/$publish_dir/sha256sums-unsigned-build.incrementals.txt"; + return $version if get($url); + } + return undef; +} + +sub get_current_version { + my ($publish_dir) = @_; + my $file = "$topdir/nightly/$publish_dir-current-version.txt"; + return undef unless -f $file; + return path($file)->slurp_utf8; +} + +sub set_current_version { + my ($publish_dir, $version) = @_; + my $file = "$topdir/nightly/$publish_dir-current-version.txt"; + path($file)->spew_utf8($version); +} + +sub get_new_version { + my ($config, $publish_dir) = @_; + my $today = 'tbb-nightly.' . DateTime->now->ymd('.'); + my $current_ver = get_current_version($publish_dir); + my $last_ver = get_last_build_version($config, $publish_dir); + return $last_ver unless defined($current_ver); + return undef if $current_ver eq $today; + return undef unless defined($last_ver); + return undef if $current_ver eq $last_ver; + return $last_ver; +} + +sub download_file { + my ($url, $file, $sha256sum) = @_; + my $retries = 5; + while ($retries > 0) { + $retries--; + print "Downloading $url\n"; + next unless getstore("$url", "$file.tmp") == 200; + next unless $sha256sum eq sha256_hex(path("$file.tmp")->slurp_raw); + move("$file.tmp", $file); + return 1; + } + exit_error "Error downloading $url"; +} + +sub fetch_version { + my ($config, $publish_dir, $version) = @_; + my $tmpdir = get_tmpdir($config); + my $urldir = "$config->{builds_url}/$version/$publish_dir"; + my $destdir = "$topdir/nightly/$publish_dir/$version"; + + return if -d $destdir; + my $gpg_keyring = basedir_path($config->{gpg_keyring}, $topdir); + for my $file (qw/sha256sums-unsigned-build.txt sha256sums-unsigned-build.incrementals.txt/) { + my $url = "$urldir/$file"; + exit_error "Error downloading $url" + unless getstore($url, "$tmpdir/$file") == 200; + exit_error "Error downloading $url.asc" + unless getstore("$url.asc", "$tmpdir/$file.asc") == 200; + exit_error "Error checking gpg signature for $url" + if system('gpg', '--no-default-keyring', '--keyring', $gpg_keyring, + '--verify', "$tmpdir/$file.asc", + "$tmpdir/$file"); + } + my %sums = map { chomp; reverse split ' ', $_ } + ( + path("$tmpdir/sha256sums-unsigned-build.txt")->lines_utf8, + path("$tmpdir/sha256sums-unsigned-build.incrementals.txt")->lines_utf8, + ); + my @build_infos_file = grep { $_ =~ m/build-infos-.*\.json/ } keys %sums; + exit_error "Missing build-infos.json in $urldir" unless @build_infos_file; + download_file("$urldir/$build_infos_file[0]", + "$tmpdir/build-infos.json", $sums{$build_infos_file[0]}); + foreach my $file (sort grep { $_ =~ m/\.mar$/ } keys %sums) { + download_file("$urldir/$file", "$tmpdir/$file", $sums{$file}); + } + make_path("$topdir/nightly/$publish_dir"); + move $tmpdir, $destdir; + chmod 0755, $destdir; +} + +sub setup_martools { + my ($config) = @_; + my $martools_dir = "$FindBin::Bin/mar-tools-$config->{martools_version}"; + if (! -d $martools_dir) { + my $file = "mar-tools-linux64.zip"; + my $url = "$config->{martools_url}/$config->{martools_version}/$file"; + my $tmpdir = get_tmpdir($config); + exit_error "Error downloading $url" + unless getstore($url, "$tmpdir/$file") == 200; + exit_error "Error downloading $url.asc" + unless getstore("$url.asc", "$tmpdir/$file.asc") == 200; + my $gpg_keyring = basedir_path($config->{martools_gpg_keyring}, $topdir); + exit_error "Error checking gpg signature for $url" + if system('gpg', '--no-default-keyring', '--keyring', $gpg_keyring, + '--verify', "$tmpdir/$file.asc", + "$tmpdir/$file"); + exit_error "Error extracting martools" + unless system('unzip', '-d', $martools_dir, '-x', + "$tmpdir/$file") == 0; + } + $ENV{PATH} = "$martools_dir/mar-tools:$ENV{PATH}"; + if ($ENV{LD_LIBRARY_PATH}) { + $ENV{LD_LIBRARY_PATH} = "$martools_dir/mar-tools:$ENV{LD_LIBRARY_PATH}"; + } else { + $ENV{LD_LIBRARY_PATH} = "$martools_dir/mar-tools"; + } +} + +sub sign_version { + my ($config, $publish_dir, $version) = @_; + setup_martools($config); + my $nss_db_dir = basedir_path($config->{nss_db_dir}, $FindBin::Bin); + for my $marfile (path("$topdir/nightly/$publish_dir/$version")->children(qr/\.mar$/)) { + print "Signing $marfile\n"; + exit_error "Error signing $marfile" + unless system('signmar', '-d', $nss_db_dir, '-n', + $config->{nss_certname}, '-s', $marfile, + "$marfile-signed") == 0; + move("$marfile-signed", $marfile); + } +} + +sub get_buildinfos { + my ($filename) = @_; + exit_error "$filename does not exist" unless -f $filename; + return decode_json(path($filename)->slurp_utf8); +} + +sub update_responses { + my ($config, $publish_dir, $version) = @_; + my $ur_config = LoadFile("$FindBin::Bin/update-responses-base-config.yml"); + $ur_config->{download}{mars_url} .= "/$publish_dir"; + $ur_config->{releases_dir} = "$topdir/nightly/$publish_dir"; + $ur_config->{channels}->{nightly} = $version; + $ur_config->{versions}->{$version} = $ur_config->{versions}->{nightly_version}; + my $buildinfos = get_buildinfos("$topdir/nightly/$publish_dir/$version/build-infos.json"); + $ur_config->{versions}->{$version}{platformVersion} = $buildinfos->{firefox_platform_version}; + $ur_config->{versions}->{$version}{buildID} = $buildinfos->{firefox_buildid}; + DumpFile("$topdir/tools/update-responses/config.yml", $ur_config); + my $htdocsdir = "$topdir/tools/update-responses/htdocs/nightly"; + path($htdocsdir)->remove_tree({ safe => 0 }); + exit_error "Error running update_responses for $publish_dir" unless + system("$topdir/tools/update-responses/update_responses") == 0; + path("$topdir/nightly/updates/$publish_dir/nightly")->remove_tree({ safe => 0 }); + make_path("$topdir/nightly/updates/$publish_dir"); + move($htdocsdir, "$topdir/nightly/updates/$publish_dir/nightly"); +} + +sub remove_oldversions { + my ($config, $publish_dir, $version) = @_; + for my $dir (path("$topdir/nightly/$publish_dir")->children) { + my ($filename) = fileparse($dir); + next if $filename eq $version; + path($dir)->remove_tree({ safe => 0 }); + } +} + +sub sync_dest { + my ($config) = @_; + exit_error "Error running rsync" + if system('rsync', '-aH', '--delete-after', + "$topdir/nightly/", "$config->{rsync_dest}/"); + if ($config->{post_rsync_cmd}) { + exit_error "Error running $config->{post_rsync_cmd}" + if system($config->{post_rsync_cmd}); + } +} + +run_alone; +my $some_updates = 0; +foreach my $publish_dir (@{$config->{publish_dirs}}) { + my $new_version = get_new_version($config, $publish_dir); + next unless $new_version; + fetch_version($config, $publish_dir, $new_version); + sign_version($config, $publish_dir, $new_version); + update_responses($config, $publish_dir, $new_version); + set_current_version($publish_dir, $new_version); + remove_oldversions($config, $publish_dir, $new_version); + $some_updates = 1; +} +sync_dest($config) if $some_updates; diff --git a/tools/signing/nightly/update-responses-base-config.yml b/tools/signing/nightly/update-responses-base-config.yml new file mode 100644 index 0000000..4d60565 --- /dev/null +++ b/tools/signing/nightly/update-responses-base-config.yml @@ -0,0 +1,32 @@ +--- +download: + mars_url: https://nightlies.tbb.torproject.org/nightly-updates +appname_marfile: tor-browser +build_targets: + linux32: Linux_x86-gcc3 + linux64: Linux_x86_64-gcc3 + win32: + - WINNT_x86-gcc3 + - WINNT_x86-gcc3-x86 + - WINNT_x86-gcc3-x64 + win64: WINNT_x86_64-gcc3-x64 + osx32: Darwin_x86-gcc3 + osx64: Darwin_x86_64-gcc3 +channels: + nightly: nightly_version +versions: + nightly_version: + platformVersion: 68.2.0 + detailsURL: http://f4amtbsowhix7rrf.onion/ + migrate_langs: + pt-PT: pt-BR + minSupportedInstructionSet: SSE2 + osx32: + minSupportedOSVersion: 13.0.0 + osx64: + minSupportedOSVersion: 13.0.0 + win32: + minSupportedOSVersion: 6.1 + win64: + minSupportedOSVersion: 6.1 +mar_compression: xz

1 0