March 2024 - tbb-commits - lists.torproject.org

[Git][tpo/applications/tor-browser] Pushed new tag FIREFOX_115_9_0esr_BUILD1
by Pier Angelo Vendrame (＠pierov) 11 Mar '24

11 Mar '24

Pier Angelo Vendrame pushed new tag FIREFOX_115_9_0esr_BUILD1 at The Tor Project / Applications / Tor Browser -- View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser/-/tree/FIREFOX_1… You're receiving this email because of your account on gitlab.torproject.org.

1 0

[Git][tpo/applications/firefox-android][firefox-android-115.2.1-13.0-1] fixup! Disable features and functionality
by Pier Angelo Vendrame (＠pierov) 11 Mar '24

11 Mar '24

Pier Angelo Vendrame pushed to branch firefox-android-115.2.1-13.0-1 at The Tor Project / Applications / firefox-android Commits: 56f2b71c by Dan Ballard at 2024-03-11T17:43:00+01:00 fixup! Disable features and functionality bug 42407: lock task affinity of HomeActivity to block potential on app phishing - - - - - 1 changed file: - fenix/app/src/main/AndroidManifest.xml Changes: ===================================== fenix/app/src/main/AndroidManifest.xml ===================================== @@ -96,6 +96,7 @@ android:exported="true" android:configChanges="keyboard|keyboardHidden|mcc|mnc|orientation|screenSize|layoutDirection|smallestScreenSize|screenLayout" android:launchMode="singleTask" + android:taskAffinity="" android:resizeableActivity="true" android:supportsPictureInPicture="true" android:windowSoftInputMode="adjustResize"> View it on GitLab: https://gitlab.torproject.org/tpo/applications/firefox-android/-/commit/56f… -- View it on GitLab: https://gitlab.torproject.org/tpo/applications/firefox-android/-/commit/56f… You're receiving this email because of your account on gitlab.torproject.org.

1 0

[Git][tpo/applications/tor-browser][tor-browser-115.8.0esr-13.5-1] 5 commits: fixup! Bug 40933: Add tor-launcher functionality
by Pier Angelo Vendrame (＠pierov) 11 Mar '24

11 Mar '24

Pier Angelo Vendrame pushed to branch tor-browser-115.8.0esr-13.5-1 at The Tor Project / Applications / Tor Browser Commits: 9cecc61f by Pier Angelo Vendrame at 2024-03-11T09:33:51+01:00 fixup! Bug 40933: Add tor-launcher functionality Bug 42336: Rework the relationship between TorSettings and TorProvider. - - - - - 138e8d6b by Pier Angelo Vendrame at 2024-03-11T16:19:47+01:00 fixup! Bug 40597: Implement TorSettings module Bug 42336: Rework the relationship between TorSettings and TorProvider. - - - - - 6265cc34 by Pier Angelo Vendrame at 2024-03-11T16:19:52+01:00 fixup! Bug 42247: Android helpers for the TorProvider Bug 42336: Rework the relationship between TorSettings and TorProvider. - - - - - e409c32a by Pier Angelo Vendrame at 2024-03-11T16:19:52+01:00 fixup! Bug 31286: Implementation of bridge, proxy, and firewall settings in about:preferences#connection Bug 42336: Rework the relationship between TorSettings and TorProvider. - - - - - f8451ae0 by Pier Angelo Vendrame at 2024-03-11T16:19:53+01:00 fixup! Bug 40597: Implement TorSettings module Use a logger instead of console in TorConnect. - - - - - 10 changed files: - browser/components/torpreferences/content/connectionPane.js - browser/components/torpreferences/content/connectionSettingsDialog.js - toolkit/components/tor-launcher/TorControlPort.sys.mjs - toolkit/components/tor-launcher/TorProvider.sys.mjs - toolkit/components/tor-launcher/TorProviderBuilder.sys.mjs - toolkit/components/tor-launcher/TorStartupService.sys.mjs - toolkit/modules/Moat.sys.mjs - toolkit/modules/TorAndroidIntegration.sys.mjs - toolkit/modules/TorConnect.sys.mjs - toolkit/modules/TorSettings.sys.mjs Changes: ===================================== browser/components/torpreferences/content/connectionPane.js ===================================== @@ -114,10 +114,9 @@ async function setTorSettings(changes) { // This will trigger TorSettings.#cleanupSettings() TorSettings.saveToPrefs(); try { - // May throw. await TorSettings.applySettings(); } catch (e) { - console.error("Failed to save Tor settings", e); + console.error("Failed to apply Tor settings", e); } } finally { TorSettings.thawNotifications(); ===================================== browser/components/torpreferences/content/connectionSettingsDialog.js ===================================== @@ -362,6 +362,8 @@ const gConnectionSettingsDialog = { } TorSettings.saveToPrefs(); + // FIXME: What if this fails? Should we prevent the dialog to close and show + // an error? TorSettings.applySettings(); }, }; ===================================== toolkit/components/tor-launcher/TorControlPort.sys.mjs ===================================== @@ -838,10 +838,12 @@ export class TorController { /** * Send multiple configuration values to tor. * - * @param {object} values The values to set + * @param {Array} values The values to set. It should be an array of + * [key, value] pairs to pass to SETCONF. Keys can be repeated, and array + * values will be automatically unrolled. */ async setConf(values) { - const args = Object.entries(values) + const args = values .flatMap(([key, value]) => { if (value === undefined || value === null) { return [key]; @@ -871,7 +873,7 @@ export class TorController { * @param {boolean} enabled Tell whether the network should be enabled */ async setNetworkEnabled(enabled) { - return this.setConf({ DisableNetwork: !enabled }); + return this.setConf([["DisableNetwork", !enabled]]); } /** ===================================== toolkit/components/tor-launcher/TorProvider.sys.mjs ===================================== @@ -15,6 +15,8 @@ ChromeUtils.defineESModuleGetters(lazy, { TorController: "resource://gre/modules/TorControlPort.sys.mjs", TorProcess: "resource://gre/modules/TorProcess.sys.mjs", TorProcessAndroid: "resource://gre/modules/TorProcessAndroid.sys.mjs", + TorProxyType: "resource://gre/modules/TorSettings.sys.mjs", + TorSettings: "resource://gre/modules/TorSettings.sys.mjs", }); const logger = new ConsoleAPI({ @@ -73,6 +75,20 @@ const Preferences = Object.freeze({ PromptAtStartup: "extensions.torlauncher.prompt_at_startup", }); +/* Config Keys used to configure tor daemon */ +const TorConfigKeys = Object.freeze({ + useBridges: "UseBridges", + bridgeList: "Bridge", + socks4Proxy: "Socks4Proxy", + socks5Proxy: "Socks5Proxy", + socks5ProxyUsername: "Socks5ProxyUsername", + socks5ProxyPassword: "Socks5ProxyPassword", + httpsProxy: "HTTPSProxy", + httpsProxyAuthenticator: "HTTPSProxyAuthenticator", + reachableAddresses: "ReachableAddresses", + clientTransportPlugin: "ClientTransportPlugin", +}); + /** * This is a Tor provider for the C Tor daemon. * @@ -166,15 +182,6 @@ export class TorProvider { */ #currentBridge = null; - /** - * Maintain a map of tor settings set by Tor Browser so that we don't - * repeatedly set the same key/values over and over. - * This map contains string keys to primitives or array values. - * - * @type {Map<string, any>} - */ - #settingsCache = new Map(); - /** * Starts a new tor process and connect to its control port, or connect to the * control port of an existing tor daemon. @@ -219,13 +226,22 @@ export class TorProvider { throw e; } + try { + await lazy.TorSettings.initializedPromise; + await this.writeSettings(lazy.TorSettings.getSettings()); + } catch (e) { + logger.warn( + "Failed to initialize TorSettings or to write our settings, so uninitializing.", + e + ); + this.uninit(); + throw e; + } + TorLauncherUtil.setProxyConfiguration(this.#socksSettings); logger.info("The Tor provider is ready."); - logger.debug(`Notifying ${TorProviderTopics.ProcessIsReady}`); - Services.obs.notifyObservers(null, TorProviderTopics.ProcessIsReady); - // If we are using an external Tor daemon, we might need to fetch circuits // already, in case streams use them. Do not await because we do not want to // block the intialization on this (it should not fail anyway...). @@ -252,42 +268,74 @@ export class TorProvider { // Provider API - async writeSettings(settingsObj) { - // TODO: Move the translation from settings object to settings understood by - // tor here. - const entries = - settingsObj instanceof Map - ? Array.from(settingsObj.entries()) - : Object.entries(settingsObj); - // only write settings that have changed - const newSettings = entries.filter(([setting, value]) => { - if (!this.#settingsCache.has(setting)) { - // no cached setting, so write - return true; - } + /** + * Send settings to the tor daemon. + * + * @param {object} settings A settings object, as returned by + * TorSettings.getSettings(). This allow to try settings without passing + * through TorSettings. + */ + async writeSettings(settings) { + logger.debug("TorProvider.writeSettings", settings); + const torSettings = new Map(); + + // Bridges + const haveBridges = + settings.bridges?.enabled && !!settings.bridges.bridge_strings.length; + torSettings.set(TorConfigKeys.useBridges, haveBridges); + if (haveBridges) { + torSettings.set( + TorConfigKeys.bridgeList, + settings.bridges.bridge_strings + ); + } else { + torSettings.set(TorConfigKeys.bridgeList, null); + } - const cachedValue = this.#settingsCache.get(setting); - // Arrays are the only special case for which === could fail. - // The other values we accept (strings, booleans, numbers, null and - // undefined) work correctly with ===. - if (Array.isArray(value) && Array.isArray(cachedValue)) { - return ( - value.length !== cachedValue.length || - value.some((val, idx) => val !== cachedValue[idx]) + // Proxy + torSettings.set(TorConfigKeys.socks4Proxy, null); + torSettings.set(TorConfigKeys.socks5Proxy, null); + torSettings.set(TorConfigKeys.socks5ProxyUsername, null); + torSettings.set(TorConfigKeys.socks5ProxyPassword, null); + torSettings.set(TorConfigKeys.httpsProxy, null); + torSettings.set(TorConfigKeys.httpsProxyAuthenticator, null); + if (settings.proxy && !settings.proxy.enabled) { + settings.proxy.type = null; + } + const address = settings.proxy?.address; + const port = settings.proxy?.port; + const username = settings.proxy?.username; + const password = settings.proxy?.password; + switch (settings.proxy?.type) { + case lazy.TorProxyType.Socks4: + torSettings.set(TorConfigKeys.socks4Proxy, `${address}:${port}`); + break; + case lazy.TorProxyType.Socks5: + torSettings.set(TorConfigKeys.socks5Proxy, `${address}:${port}`); + torSettings.set(TorConfigKeys.socks5ProxyUsername, username); + torSettings.set(TorConfigKeys.socks5ProxyPassword, password); + break; + case lazy.TorProxyType.HTTPS: + torSettings.set(TorConfigKeys.httpsProxy, `${address}:${port}`); + torSettings.set( + TorConfigKeys.httpsProxyAuthenticator, + `${username}:${password}` ); - } - return value !== cachedValue; - }); - - // only write if new setting to save - if (newSettings.length) { - await this.#controller.setConf(Object.fromEntries(newSettings)); + break; + } - // save settings to cache after successfully writing to Tor - for (const [setting, value] of newSettings) { - this.#settingsCache.set(setting, value); - } + // Firewall + if (settings.firewall?.enabled) { + const reachableAddresses = settings.firewall.allowed_ports + .map(port => `*:${port}`) + .join(","); + torSettings.set(TorConfigKeys.reachableAddresses, reachableAddresses); + } else { + torSettings.set(TorConfigKeys.reachableAddresses, null); } + + logger.debug("Mapped settings object", settings, torSettings); + await this.#controller.setConf(Array.from(torSettings)); } async flushSettings() { ===================================== toolkit/components/tor-launcher/TorProviderBuilder.sys.mjs ===================================== @@ -9,7 +9,6 @@ ChromeUtils.defineESModuleGetters(lazy, { }); export const TorProviderTopics = Object.freeze({ - ProcessIsReady: "TorProcessIsReady", ProcessExited: "TorProcessExited", BootstrapStatus: "TorBootstrapStatus", BootstrapError: "TorBootstrapError", ===================================== toolkit/components/tor-launcher/TorStartupService.sys.mjs ===================================== @@ -31,15 +31,16 @@ export class TorStartupService { } } - async #init() { + #init() { Services.obs.addObserver(this, BrowserTopics.QuitApplicationGranted); - // Do not await on this init. build() is expected to await the - // initialization, so anything that should need the Tor Provider should - // block there, instead. + lazy.TorSettings.init(); + + // Theoretically, build() is expected to await the initialization of the + // provider, and anything needing the Tor Provider should be able to just + // await on TorProviderBuilder.build(). lazy.TorProviderBuilder.init(); - await lazy.TorSettings.init(); lazy.TorConnect.init(); lazy.TorDomainIsolator.init(); ===================================== toolkit/modules/Moat.sys.mjs ===================================== @@ -2,13 +2,13 @@ * License, v. 2.0. If a copy of the MPL was not distributed with this * file, You can obtain one at http://mozilla.org/MPL/2.0/. */ -import { TorBridgeSource } from "resource://gre/modules/TorSettings.sys.mjs"; - const lazy = {}; ChromeUtils.defineESModuleGetters(lazy, { DomainFrontRequestBuilder: "resource://gre/modules/DomainFrontedRequests.sys.mjs", + TorBridgeSource: "resource://gre/modules/TorSettings.sys.mjs", + TorSettings: "resource://gre/modules/TorSettings.sys.mjs", }); const TorLauncherPrefs = Object.freeze({ @@ -211,13 +211,23 @@ export class MoatRPC { }; switch (settings.bridges.source) { case "builtin": - retval.bridges.source = TorBridgeSource.BuiltIn; + retval.bridges.source = lazy.TorBridgeSource.BuiltIn; retval.bridges.builtin_type = settings.bridges.type; // TorSettings will ignore strings for built-in bridges, and use the - // ones it already knows, instead. + // ones it already knows, instead. However, when we try these settings + // in the connect assist, we skip TorSettings. Therefore, we set the + // lines also here (the ones we already known, not the ones we receive + // from Moat). This needs TorSettings to be initialized, which by now + // should have already happened (this method is used only by TorConnect, + // that needs TorSettings to be initialized). + // In any case, getBuiltinBridges will throw if the data is not ready, + // yet. + retval.bridges.bridge_strings = lazy.TorSettings.getBuiltinBridges( + settings.bridges.type + ); break; case "bridgedb": - retval.bridges.source = TorBridgeSource.BridgeDB; + retval.bridges.source = lazy.TorBridgeSource.BridgeDB; if (settings.bridges.bridge_strings) { retval.bridges.bridge_strings = settings.bridges.bridge_strings; } else { ===================================== toolkit/modules/TorAndroidIntegration.sys.mjs ===================================== @@ -150,7 +150,7 @@ class TorAndroidIntegrationImpl { lazy.TorSettings.saveToPrefs(); } if (data.apply) { - lazy.TorSettings.applySettings(); + await lazy.TorSettings.applySettings(); } break; case ListenedEvents.settingsApply: ===================================== toolkit/modules/TorConnect.sys.mjs ===================================== @@ -2,14 +2,17 @@ * License, v. 2.0. If a copy of the MPL was not distributed with this * file, You can obtain one at http://mozilla.org/MPL/2.0/. */ +import { XPCOMUtils } from "resource://gre/modules/XPCOMUtils.sys.mjs"; import { setTimeout, clearTimeout } from "resource://gre/modules/Timer.sys.mjs"; const lazy = {}; ChromeUtils.defineESModuleGetters(lazy, { + ConsoleAPI: "resource://gre/modules/Console.sys.mjs", EventDispatcher: "resource://gre/modules/Messaging.sys.mjs", MoatRPC: "resource://gre/modules/Moat.sys.mjs", TorBootstrapRequest: "resource://gre/modules/TorBootstrapRequest.sys.mjs", + TorProviderBuilder: "resource://gre/modules/TorProviderBuilder.sys.mjs", }); // TODO: Should we move this to the about:torconnect actor? @@ -20,10 +23,7 @@ ChromeUtils.defineModuleGetter( ); import { TorLauncherUtil } from "resource://gre/modules/TorLauncherUtil.sys.mjs"; -import { - TorSettings, - TorSettingsTopics, -} from "resource://gre/modules/TorSettings.sys.mjs"; +import { TorSettings } from "resource://gre/modules/TorSettings.sys.mjs"; import { TorStrings } from "resource://gre/modules/TorStrings.sys.mjs"; @@ -40,6 +40,7 @@ const TorLauncherPrefs = Object.freeze({ const TorConnectPrefs = Object.freeze({ censorship_level: "torbrowser.debug.censorship_level", allow_internet_test: "torbrowser.bootstrap.allow_internet_test", + log_level: "torbrowser.bootstrap.log_level", }); export const TorConnectState = Object.freeze({ @@ -59,6 +60,17 @@ export const TorConnectState = Object.freeze({ Disabled: "Disabled", }); +XPCOMUtils.defineLazyGetter( + lazy, + "logger", + () => + new lazy.ConsoleAPI({ + maxLogLevel: "info", + maxLogLevelPref: TorConnectPrefs.log_level, + prefix: "TorConnect", + }) +); + /* TorConnect State Transitions @@ -194,12 +206,12 @@ class StateCallback { } async begin(...args) { - console.log(`TorConnect: Entering ${this._state} state`); + lazy.logger.trace(`Entering ${this._state} state`); this._init(); try { // this Promise will block until this StateCallback has completed its work await Promise.resolve(this._callback.call(this._context, ...args)); - console.log(`TorConnect: Exited ${this._state} state`); + lazy.logger.info(`Exited ${this._state} state`); // handled state transition Services.obs.notifyObservers( @@ -267,16 +279,14 @@ class InternetTest { this.cancel(); this._pending = true; - console.log("TorConnect: starting the Internet test"); + lazy.logger.info("Starting the Internet test"); this._testAsync() .then(status => { this._pending = false; this._status = status.successful ? InternetStatus.Online : InternetStatus.Offline; - console.log( - `TorConnect: performed Internet test, outcome ${this._status}` - ); + lazy.logger.info(`Performed Internet test, outcome ${this._status}`); this.onResult(this.status, status.date); }) .catch(error => { @@ -305,7 +315,7 @@ class InternetTest { await mrpc.init(); status = await mrpc.testInternetConnection(); } catch (err) { - console.error("Error while checking the Internet connection", err); + lazy.logger.error("Error while checking the Internet connection", err); error = err; } finally { mrpc.uninit(); @@ -523,8 +533,8 @@ export const TorConnect = (() => { // get "Building circuits: Establishing a Tor circuit failed". // TODO: Maybe move this logic deeper in the process to know // when to filter out such errors triggered by cancelling. - console.log( - `TorConnect: Post-cancel error => ${message}; ${details}` + lazy.logger.warn( + `Post-cancel error => ${message}; ${details}` ); return; } @@ -628,7 +638,7 @@ export const TorConnect = (() => { "vanilla", ]); } catch (err) { - console.error( + lazy.logger.error( "We did not get localized settings, and default settings failed as well", err ); @@ -651,10 +661,19 @@ export const TorConnect = (() => { } } + const restoreOriginalSettings = async () => { + try { + await TorSettings.applySettings(); + } catch (e) { + // We cannot do much if the original settings were bad or + // if the connection closed, so just report it in the + // console. + lazy.logger.warn("Failed to restore original settings.", e); + } + }; + // apply each of our settings and try to bootstrap with each try { - this.originalSettings = TorSettings.getSettings(); - for (const [ index, currentSetting, @@ -664,14 +683,32 @@ export const TorConnect = (() => { break; } - console.log( - `TorConnect: Attempting Bootstrap with configuration ${ - index + 1 - }/${this.settings.length}` + lazy.logger.info( + `Attempting Bootstrap with configuration ${index + 1}/${ + this.settings.length + }` ); - TorSettings.setSettings(currentSetting); - await TorSettings.applySettings(); + // Send the new settings directly to the provider. We will + // save them only if the bootstrap succeeds. + // FIXME: We should somehow signal TorSettings users that we + // have set custom settings, and they should not apply + // theirs until we are done with trying ours. + // Otherwise, the new settings provided by the user while we + // were bootstrapping could be the ones that cause the + // bootstrap to succeed, but we overwrite them (unless we + // backup the original settings, and then save our new + // settings only if they have not changed). + // Another idea (maybe easier to implement) is to disable + // the settings UI while *any* bootstrap is going on. + // This is also documented in tor-browser#41921. + const provider = await lazy.TorProviderBuilder.build(); + // We need to merge with old settings, in case the user is + // using a proxy or is behind a firewall. + await provider.writeSettings({ + ...TorSettings.getSettings(), + ...currentSetting, + }); // build out our bootstrap request const tbr = new lazy.TorBootstrapRequest(); @@ -679,8 +716,8 @@ export const TorConnect = (() => { TorConnect._updateBootstrapStatus(progress, status); }; tbr.onbootstraperror = (message, details) => { - console.log( - `TorConnect: Auto-Bootstrap error => ${message}; ${details}` + lazy.logger.error( + `Auto-Bootstrap error => ${message}; ${details}` ); }; @@ -688,6 +725,7 @@ export const TorConnect = (() => { this.on_transition = async nextState => { if (nextState === TorConnectState.Configuring) { await tbr.cancel(); + await restoreOriginalSettings(); } resolve(); }; @@ -695,23 +733,20 @@ export const TorConnect = (() => { // begin bootstrap if (await tbr.bootstrap()) { // persist the current settings to preferences + TorSettings.setSettings(currentSetting); TorSettings.saveToPrefs(); + await TorSettings.applySettings(); TorConnect._changeState(TorConnectState.Bootstrapped); return; } } - // bootstrapped failed for all potential settings, so reset daemon to use original - TorSettings.setSettings(this.originalSettings); - // The original settings should be good, so we save them to - // preferences before trying to apply them, as it might fail - // if the actual problem is with the connection to the control - // port. - // FIXME: We should handle this case in a better way. - TorSettings.saveToPrefs(); - await TorSettings.applySettings(); - - // only explicitly change state here if something else has not transitioned us + // Bootstrap failed for all potential settings, so restore the + // original settings the provider. + await restoreOriginalSettings(); + + // Only explicitly change state here if something else has not + // transitioned us. if (!this.transitioning) { throw_error( TorStrings.torConnect.autoBootstrappingFailed, @@ -720,18 +755,8 @@ export const TorConnect = (() => { } return; } catch (err) { - // restore original settings in case of error - try { - TorSettings.setSettings(this.originalSettings); - // As above - TorSettings.saveToPrefs(); - await TorSettings.applySettings(); - } catch (errRestore) { - console.log( - `TorConnect: Failed to restore original settings => ${errRestore}` - ); - } - // throw to outer catch to transition us + await restoreOriginalSettings(); + // throw to outer catch to transition us. throw err; } } catch (err) { @@ -748,8 +773,8 @@ export const TorConnect = (() => { true ); } else { - console.error( - "TorConnect: Received AutoBootstrapping error after transitioning", + lazy.logger.error( + "Received AutoBootstrapping error after transitioning", err ); } @@ -793,8 +818,8 @@ export const TorConnect = (() => { TorConnect._errorMessage = errorMessage; TorConnect._errorDetails = errorDetails; - console.error( - `[TorConnect] Entering error state (${errorMessage}, ${errorDetails})` + lazy.logger.error( + `Entering error state (${errorMessage}, ${errorDetails})` ); Services.obs.notifyObservers( @@ -835,9 +860,7 @@ export const TorConnect = (() => { ); } - console.log( - `TorConnect: Try transitioning from ${prevState} to ${newState}` - ); + lazy.logger.trace(`Try transitioning from ${prevState} to ${newState}`); // set our new state first so that state transitions can themselves trigger // a state transition @@ -851,8 +874,8 @@ export const TorConnect = (() => { this._bootstrapProgress = progress; this._bootstrapStatus = status; - console.log( - `TorConnect: Bootstrapping ${this._bootstrapProgress}% complete (${this._bootstrapStatus})` + lazy.logger.info( + `Bootstrapping ${this._bootstrapProgress}% complete (${this._bootstrapStatus})` ); Services.obs.notifyObservers( { @@ -866,7 +889,7 @@ export const TorConnect = (() => { // init should be called by TorStartupService init() { - console.log("TorConnect: init()"); + lazy.logger.debug("TorConnect.init()"); this._callback(TorConnectState.Initial).begin(); if (!this.enabled) { @@ -875,9 +898,17 @@ export const TorConnect = (() => { } else { let observeTopic = addTopic => { Services.obs.addObserver(this, addTopic); - console.log(`TorConnect: Observing topic '${addTopic}'`); + lazy.logger.debug(`Observing topic '${addTopic}'`); }; + // Wait for TorSettings, as we will need it. + // We will wait for a TorProvider only after TorSettings is ready, + // because the TorProviderBuilder initialization might not have finished + // at this point, and TorSettings initialization is a prerequisite for + // having a provider. + // So, we prefer initializing TorConnect as soon as possible, so that + // the UI will be able to detect it is in the Initializing state and act + // consequently. TorSettings.initializedPromise.then(() => this._settingsInitialized()); // register the Tor topics we always care about @@ -887,7 +918,7 @@ export const TorConnect = (() => { }, async observe(subject, topic, data) { - console.log(`TorConnect: Observed ${topic}`); + lazy.logger.debug(`Observed ${topic}`); switch (topic) { case TorTopics.LogHasWarnOrErr: { @@ -919,19 +950,25 @@ export const TorConnect = (() => { } }, - _settingsInitialized() { + async _settingsInitialized() { + // TODO: Handle failures here, instead of the prompt to restart the + // daemon when it exits (tor-browser#21053, tor-browser#41921). + await lazy.TorProviderBuilder.build(); + // tor-browser#41907: This is only a workaround to avoid users being // bounced back to the initial panel without any explanation. // Longer term we should disable the clickable elements, or find a UX // to prevent this from happening (e.g., allow buttons to be clicked, // but show an intermediate starting state, or a message that tor is // starting while the butons are disabled, etc...). + // See also tor-browser#41921. if (this.state !== TorConnectState.Initial) { - console.warn( - "TorConnect: Seen the torsettings:ready after the state has already changed, ignoring the notification." + lazy.logger.warn( + "The TorProvider was built after the state had already changed." ); return; } + lazy.logger.debug("The TorProvider is ready, changing state."); if (this.shouldQuickStart) { // Quickstart this._changeState(TorConnectState.Bootstrapping); @@ -1074,17 +1111,17 @@ export const TorConnect = (() => { */ beginBootstrap() { - console.log("TorConnect: beginBootstrap()"); + lazy.logger.debug("TorConnect.beginBootstrap()"); this._changeState(TorConnectState.Bootstrapping); }, cancelBootstrap() { - console.log("TorConnect: cancelBootstrap()"); + lazy.logger.debug("TorConnect.cancelBootstrap()"); this._changeState(TorConnectState.Configuring); }, beginAutoBootstrap(countryCode) { - console.log("TorConnect: beginAutoBootstrap()"); + lazy.logger.debug("TorConnect.beginAutoBootstrap()"); this._changeState(TorConnectState.AutoBootstrapping, countryCode); }, @@ -1154,7 +1191,10 @@ export const TorConnect = (() => { await mrpc.init(); this._countryCodes = await mrpc.circumvention_countries(); } catch (err) { - console.log("An error occurred while fetching country codes", err); + lazy.logger.error( + "An error occurred while fetching country codes", + err + ); } finally { mrpc.uninit(); } @@ -1187,8 +1227,8 @@ export const TorConnect = (() => { uriArray = uriVariant; } else { // about:tor as safe fallback - console.error( - `TorConnect: received unknown variant '${JSON.stringify(uriVariant)}'` + lazy.logger.error( + `Received unknown variant '${JSON.stringify(uriVariant)}'` ); uriArray = ["about:tor"]; } @@ -1209,9 +1249,7 @@ export const TorConnect = (() => { // which redirect after bootstrapping getURIsToLoad(uriVariant) { const uris = this.fixupURIs(uriVariant); - console.log( - `TorConnect: Will load after bootstrap => [${uris.join(", ")}]` - ); + lazy.logger.debug(`Will load after bootstrap => [${uris.join(", ")}]`); return uris.map(uri => this.getRedirectURL(uri)); }, }; ===================================== toolkit/modules/TorSettings.sys.mjs ===================================== @@ -6,10 +6,9 @@ const lazy = {}; ChromeUtils.defineESModuleGetters(lazy, { TorLauncherUtil: "resource://gre/modules/TorLauncherUtil.sys.mjs", - TorProviderBuilder: "resource://gre/modules/TorProviderBuilder.sys.mjs", - TorProviderTopics: "resource://gre/modules/TorProviderBuilder.sys.mjs", Lox: "resource://gre/modules/Lox.sys.mjs", TorParsers: "resource://gre/modules/TorParsers.sys.mjs", + TorProviderBuilder: "resource://gre/modules/TorProviderBuilder.sys.mjs", }); ChromeUtils.defineLazyGetter(lazy, "logger", () => { @@ -71,20 +70,6 @@ const TorSettingsPrefs = Object.freeze({ }, }); -/* Config Keys used to configure tor daemon */ -const TorConfigKeys = Object.freeze({ - useBridges: "UseBridges", - bridgeList: "Bridge", - socks4Proxy: "Socks4Proxy", - socks5Proxy: "Socks5Proxy", - socks5ProxyUsername: "Socks5ProxyUsername", - socks5ProxyPassword: "Socks5ProxyPassword", - httpsProxy: "HTTPSProxy", - httpsProxyAuthenticator: "HTTPSProxyAuthenticator", - reachableAddresses: "ReachableAddresses", - clientTransportPlugin: "ClientTransportPlugin", -}); - export const TorBridgeSource = Object.freeze({ Invalid: -1, BuiltIn: 0, @@ -322,7 +307,7 @@ class TorSettingsImpl { if (!val) { return; } - const bridgeStrings = this.#getBuiltinBridges(val); + const bridgeStrings = this.getBuiltinBridges(val); if (bridgeStrings.length) { this.bridges.bridge_strings = bridgeStrings; return; @@ -659,14 +644,17 @@ class TorSettingsImpl { * @param {string} pt The pluggable transport to return the lines for * @returns {string[]} The bridge lines in random order */ - #getBuiltinBridges(pt) { + getBuiltinBridges(pt) { + if (!this.#allowUninitialized) { + this.#checkIfInitialized(); + } // Shuffle so that Tor Browser users do not all try the built-in bridges in // the same order. return arrayShuffle(this.#builtinBridges[pt] ?? []); } /** - * Load or init our settings, and register observers. + * Load or init our settings. */ async init() { if (this.#initialized) { @@ -677,6 +665,7 @@ class TorSettingsImpl { await this.#initInternal(); this.#initialized = true; this.#initComplete(); + Services.obs.notifyObservers(null, TorSettingsTopics.Ready); } catch (e) { this.#initFailed(e); throw e; @@ -698,45 +687,35 @@ class TorSettingsImpl { lazy.logger.error("Could not load the built-in PT config.", e); } - // Initialize this before loading from prefs because we need Lox initialized before - // any calls to Lox.getBridges() + // Initialize this before loading from prefs because we need Lox initialized + // before any calls to Lox.getBridges(). try { await lazy.Lox.init(); } catch (e) { lazy.logger.error("Could not initialize Lox.", e.type); } - // TODO: We could use a shared promise, and wait for it to be fullfilled - // instead of Service.obs. - if (lazy.TorLauncherUtil.shouldStartAndOwnTor) { - // if the settings branch exists, load settings from prefs - if (Services.prefs.getBoolPref(TorSettingsPrefs.enabled, false)) { - // Do not want notifications for initially loaded prefs. - this.freezeNotifications(); - try { - this.#allowUninitialized = true; - this.#loadFromPrefs(); - } finally { - this.#allowUninitialized = false; - this.#notificationQueue.clear(); - this.thawNotifications(); - } - } + if ( + lazy.TorLauncherUtil.shouldStartAndOwnTor && + Services.prefs.getBoolPref(TorSettingsPrefs.enabled, false) + ) { + // Do not want notifications for initially loaded prefs. + this.freezeNotifications(); try { - const provider = await lazy.TorProviderBuilder.build(); - if (provider.isRunning) { - this.#handleProcessReady(); - // No need to add an observer to call this again. - return; - } - } catch {} - - Services.obs.addObserver(this, lazy.TorProviderTopics.ProcessIsReady); + this.#allowUninitialized = true; + this.#loadFromPrefs(); + } finally { + this.#allowUninitialized = false; + this.#notificationQueue.clear(); + this.thawNotifications(); + } } + + lazy.logger.info("Ready"); } /** - * Unload or uninit our settings, and unregister observers. + * Unload or uninit our settings. */ async uninit() { await lazy.Lox.uninit(); @@ -764,34 +743,6 @@ class TorSettingsImpl { return this.#initialized; } - /** - * Wait for relevant life-cycle events to apply saved settings. - */ - async observe(subject, topic, data) { - lazy.logger.debug(`Observed ${topic}`); - - switch (topic) { - case lazy.TorProviderTopics.ProcessIsReady: - Services.obs.removeObserver( - this, - lazy.TorProviderTopics.ProcessIsReady - ); - await this.#handleProcessReady(); - break; - } - } - - /** - * Apply the settings once the tor provider is ready and notify any observer - * that the settings can be used. - */ - async #handleProcessReady() { - // push down settings to tor - await this.#applySettings(true); - lazy.logger.info("Ready"); - Services.obs.notifyObservers(null, TorSettingsTopics.Ready); - } - /** * Load our settings from prefs. */ @@ -972,85 +923,14 @@ class TorSettingsImpl { /** * Push our settings down to the tor provider. + * + * Even though this introduces a circular depdency, it makes the API nicer for + * frontend consumers. */ async applySettings() { this.#checkIfInitialized(); - return this.#applySettings(false); - } - - /** - * Internal implementation of applySettings that does not check if we are - * initialized. - */ - async #applySettings(allowUninitialized) { - lazy.logger.debug("#applySettings()"); - - this.#cleanupSettings(); - - const settingsMap = new Map(); - - // #applySettings can be called only when #allowUninitialized is false - this.#allowUninitialized = allowUninitialized; - - try { - /* Bridges */ - const haveBridges = - this.bridges.enabled && !!this.bridges.bridge_strings.length; - settingsMap.set(TorConfigKeys.useBridges, haveBridges); - if (haveBridges) { - settingsMap.set(TorConfigKeys.bridgeList, this.bridges.bridge_strings); - } else { - settingsMap.set(TorConfigKeys.bridgeList, null); - } - - /* Proxy */ - settingsMap.set(TorConfigKeys.socks4Proxy, null); - settingsMap.set(TorConfigKeys.socks5Proxy, null); - settingsMap.set(TorConfigKeys.socks5ProxyUsername, null); - settingsMap.set(TorConfigKeys.socks5ProxyPassword, null); - settingsMap.set(TorConfigKeys.httpsProxy, null); - settingsMap.set(TorConfigKeys.httpsProxyAuthenticator, null); - if (this.proxy.enabled) { - const address = this.proxy.address; - const port = this.proxy.port; - const username = this.proxy.username; - const password = this.proxy.password; - - switch (this.proxy.type) { - case TorProxyType.Socks4: - settingsMap.set(TorConfigKeys.socks4Proxy, `${address}:${port}`); - break; - case TorProxyType.Socks5: - settingsMap.set(TorConfigKeys.socks5Proxy, `${address}:${port}`); - settingsMap.set(TorConfigKeys.socks5ProxyUsername, username); - settingsMap.set(TorConfigKeys.socks5ProxyPassword, password); - break; - case TorProxyType.HTTPS: - settingsMap.set(TorConfigKeys.httpsProxy, `${address}:${port}`); - settingsMap.set( - TorConfigKeys.httpsProxyAuthenticator, - `${username}:${password}` - ); - break; - } - } - - /* Firewall */ - if (this.firewall.enabled) { - const reachableAddresses = this.firewall.allowed_ports - .map(port => `*:${port}`) - .join(","); - settingsMap.set(TorConfigKeys.reachableAddresses, reachableAddresses); - } else { - settingsMap.set(TorConfigKeys.reachableAddresses, null); - } - } finally { - this.#allowUninitialized = false; - } - - /* Push to Tor */ const provider = await lazy.TorProviderBuilder.build(); - await provider.writeSettings(settingsMap); + await provider.writeSettings(this.getSettings()); } /** View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser/-/compare/f6c7b7… -- View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser/-/compare/f6c7b7… You're receiving this email because of your account on gitlab.torproject.org.

1 0

[Git][tpo/applications/mullvad-browser][mullvad-browser-115.8.0esr-13.0-1] 2 commits: fixup! Firefox preference overrides.
by Pier Angelo Vendrame (＠pierov) 11 Mar '24

11 Mar '24

Pier Angelo Vendrame pushed to branch mullvad-browser-115.8.0esr-13.0-1 at The Tor Project / Applications / Mullvad Browser Commits: eaa8c377 by Pier Angelo Vendrame at 2024-03-11T15:53:55+01:00 fixup! Firefox preference overrides. Bug 42377: Hidden fonts should obey the allow list. Remove .Helvetica Neue DeskInterface from the allow list. This font might not be actually available, and in any case fonts starting with period will not be displayed by the browser. - - - - - 63219c0c by Pier Angelo Vendrame at 2024-03-11T15:53:56+01:00 Bug 42377: Hidden fonts should obey the allow list. Hidden font families were automatically added to the `font.system.whitelist`, which is a behavior that conflicts with our font picking. - - - - - 2 changed files: - browser/app/profile/001-base-profile.js - gfx/thebes/gfxPlatformFontList.cpp Changes: ===================================== browser/app/profile/001-base-profile.js ===================================== @@ -641,7 +641,7 @@ pref("toolkit.winRegisterApplicationRestart", false); pref("gfx.bundled-fonts.activate", 1); #ifdef XP_MACOSX -pref("font.system.whitelist", "AppleGothic, Apple Color Emoji, Arial, Courier, Courier New, Geneva, Georgia, Heiti TC, Helvetica, Helvetica Neue, .Helvetica Neue DeskInterface, Hiragino Kaku Gothic ProN, Kailasa, Lucida Grande, Menlo, Monaco, PingFang HK, PingFang SC, PingFang TC, Songti SC, Songti TC, Tahoma, Thonburi, Times, Times New Roman, Verdana, STIX Two Math, Noto Sans Adlam, Noto Sans Armenian, Noto Sans Balinese, Noto Sans Bamum, Noto Sans Bassa Vah, Noto Sans Batak, Noto Sans Bengali, Noto Sans Buginese, Noto Sans Buhid, Noto Sans Canadian Aboriginal, Noto Sans Chakma, Noto Sans Cham, Noto Sans Cherokee, Noto Sans Coptic, Noto Sans Deseret, Noto Sans Devanagari, Noto Sans Elbasan, Noto Sans Ethiopic, Noto Sans Georgian, Noto Sans Grantha, Noto Sans Gujarati, Noto Sans Gunjala Gondi, Noto Sans Gurmukhi, Noto Sans Hanifi Rohingya, Noto Sans Hanunoo, Noto Sans Hebrew, Noto Sans Kannada, Noto Sans Khmer, Noto Sans Javanese, Noto Sans Kayah Li, Noto Sans Khojki, Noto Sans Khudawadi, Noto Sans Lao, Noto Sans Lepcha, Noto Sans Limbu, Noto Sans Lisu, Noto Sans Mahajani, Noto Sans Malayalam, Noto Sans Mandaic, Noto Sans Masaram Gondi, Noto Sans Medefaidrin, Noto Sans Meetei Mayek, Noto Sans Mende Kikakui, Noto Sans Miao, Noto Sans Modi, Noto Sans Mongolian, Noto Sans Mro, Noto Sans Myanmar, Noto Sans Multani, Noto Sans Newa, Noto Sans New Tai Lue, Noto Sans NKo, Noto Sans Ol Chiki, Noto Sans Oriya, Noto Sans Osage, Noto Sans Osmanya, Noto Sans Pahawh Hmong, Noto Sans Pau Cin Hau, Noto Sans Rejang, Noto Sans Runic, Noto Sans Samaritan, Noto Sans Saurashtra, Noto Sans Sharada, Noto Sans Shavian, Noto Sans Sinhala, Noto Sans Sora Sompeng, Noto Sans Soyombo, Noto Sans Sundanese, Noto Sans Syloti Nagri, Noto Sans Symbols 2, Noto Sans Symbols, Noto Sans Syriac, Noto Sans Tagalog, Noto Sans Tagbanwa, Noto Sans Tai Le, Noto Sans Tai Tham, Noto Sans Tai Viet, Noto Sans Takri, Noto Sans Tamil, Noto Sans Telugu, Noto Sans Thaana, Noto Sans Tifinagh Adrar, Noto Sans Tifinagh Agraw Imazighen, Noto Sans Tifinagh Ahaggar, Noto Sans Tifinagh Air, Noto Sans Tifinagh APT, Noto Sans Tifinagh Azawagh, Noto Sans Tifinagh Ghat, Noto Sans Tifinagh Hawad, Noto Sans Tifinagh, Noto Sans Tifinagh Rhissa Ixa, Noto Sans Tifinagh SIL, Noto Sans Tifinagh Tawellemmet, Noto Sans Tirhuta, Noto Sans Vai, Noto Sans Wancho, Noto Sans Warang Citi, Noto Sans Yi, Noto Sans Zanabazar Square, Noto Serif Armenian, Noto Serif Balinese, Noto Serif Bengali, Noto Serif Devanagari, Noto Serif Dogra, Noto Serif Ethiopic, Noto Serif Georgian, Noto Serif Grantha, Noto Serif Gujarati, Noto Serif Gurmukhi, Noto Serif Hebrew, Noto Serif Kannada, Noto Serif Khmer, Noto Serif Khojki, Noto Serif Lao, Noto Serif Malayalam, Noto Serif Myanmar, Noto Serif NP Hmong, Noto Serif Sinhala, Noto Serif Tamil, Noto Serif Telugu, Noto Serif Tibetan, Noto Serif Yezidi"); +pref("font.system.whitelist", "AppleGothic, Apple Color Emoji, Arial, Courier, Courier New, Geneva, Georgia, Heiti TC, Helvetica, Helvetica Neue, Hiragino Kaku Gothic ProN, Kailasa, Lucida Grande, Menlo, Monaco, PingFang HK, PingFang SC, PingFang TC, Songti SC, Songti TC, Tahoma, Thonburi, Times, Times New Roman, Verdana, STIX Two Math, Noto Sans Adlam, Noto Sans Armenian, Noto Sans Balinese, Noto Sans Bamum, Noto Sans Bassa Vah, Noto Sans Batak, Noto Sans Bengali, Noto Sans Buginese, Noto Sans Buhid, Noto Sans Canadian Aboriginal, Noto Sans Chakma, Noto Sans Cham, Noto Sans Cherokee, Noto Sans Coptic, Noto Sans Deseret, Noto Sans Devanagari, Noto Sans Elbasan, Noto Sans Ethiopic, Noto Sans Georgian, Noto Sans Grantha, Noto Sans Gujarati, Noto Sans Gunjala Gondi, Noto Sans Gurmukhi, Noto Sans Hanifi Rohingya, Noto Sans Hanunoo, Noto Sans Hebrew, Noto Sans Kannada, Noto Sans Khmer, Noto Sans Javanese, Noto Sans Kayah Li, Noto Sans Khojki, Noto Sans Khudawadi, Noto Sans Lao, Noto Sans Lepcha, Noto Sans Limbu, Noto Sans Lisu, Noto Sans Mahajani, Noto Sans Malayalam, Noto Sans Mandaic, Noto Sans Masaram Gondi, Noto Sans Medefaidrin, Noto Sans Meetei Mayek, Noto Sans Mende Kikakui, Noto Sans Miao, Noto Sans Modi, Noto Sans Mongolian, Noto Sans Mro, Noto Sans Myanmar, Noto Sans Multani, Noto Sans Newa, Noto Sans New Tai Lue, Noto Sans NKo, Noto Sans Ol Chiki, Noto Sans Oriya, Noto Sans Osage, Noto Sans Osmanya, Noto Sans Pahawh Hmong, Noto Sans Pau Cin Hau, Noto Sans Rejang, Noto Sans Runic, Noto Sans Samaritan, Noto Sans Saurashtra, Noto Sans Sharada, Noto Sans Shavian, Noto Sans Sinhala, Noto Sans Sora Sompeng, Noto Sans Soyombo, Noto Sans Sundanese, Noto Sans Syloti Nagri, Noto Sans Symbols 2, Noto Sans Symbols, Noto Sans Syriac, Noto Sans Tagalog, Noto Sans Tagbanwa, Noto Sans Tai Le, Noto Sans Tai Tham, Noto Sans Tai Viet, Noto Sans Takri, Noto Sans Tamil, Noto Sans Telugu, Noto Sans Thaana, Noto Sans Tifinagh Adrar, Noto Sans Tifinagh Agraw Imazighen, Noto Sans Tifinagh Ahaggar, Noto Sans Tifinagh Air, Noto Sans Tifinagh APT, Noto Sans Tifinagh Azawagh, Noto Sans Tifinagh Ghat, Noto Sans Tifinagh Hawad, Noto Sans Tifinagh, Noto Sans Tifinagh Rhissa Ixa, Noto Sans Tifinagh SIL, Noto Sans Tifinagh Tawellemmet, Noto Sans Tirhuta, Noto Sans Vai, Noto Sans Wancho, Noto Sans Warang Citi, Noto Sans Yi, Noto Sans Zanabazar Square, Noto Serif Armenian, Noto Serif Balinese, Noto Serif Bengali, Noto Serif Devanagari, Noto Serif Dogra, Noto Serif Ethiopic, Noto Serif Georgian, Noto Serif Grantha, Noto Serif Gujarati, Noto Serif Gurmukhi, Noto Serif Hebrew, Noto Serif Kannada, Noto Serif Khmer, Noto Serif Khojki, Noto Serif Lao, Noto Serif Malayalam, Noto Serif Myanmar, Noto Serif NP Hmong, Noto Serif Sinhala, Noto Serif Tamil, Noto Serif Telugu, Noto Serif Tibetan, Noto Serif Yezidi"); // Armenian pref("font.name-list.serif.x-armn", "Noto Serif Armenian, Times, Times New Roman"); ===================================== gfx/thebes/gfxPlatformFontList.cpp ===================================== @@ -401,12 +401,6 @@ void gfxPlatformFontList::ApplyWhitelist() { AutoTArray<RefPtr<gfxFontFamily>, 128> accepted; bool whitelistedFontFound = false; for (const auto& entry : mFontFamilies) { - if (entry.GetData()->IsHidden()) { - // Hidden system fonts are exempt from whitelisting, but don't count - // towards determining whether we "kept" any (user-visible) fonts - accepted.AppendElement(entry.GetData()); - continue; - } nsAutoCString fontFamilyName(entry.GetKey()); ToLowerCase(fontFamilyName); if (familyNamesWhitelist.Contains(fontFamilyName)) { @@ -442,8 +436,7 @@ void gfxPlatformFontList::ApplyWhitelist( AutoTArray<fontlist::Family::InitData, 128> accepted; bool keptNonHidden = false; for (auto& f : aFamilies) { - if (f.mVisibility == FontVisibility::Hidden || - familyNamesWhitelist.Contains(f.mKey)) { + if (familyNamesWhitelist.Contains(f.mKey)) { accepted.AppendElement(f); if (f.mVisibility != FontVisibility::Hidden) { keptNonHidden = true; View it on GitLab: https://gitlab.torproject.org/tpo/applications/mullvad-browser/-/compare/a1… -- View it on GitLab: https://gitlab.torproject.org/tpo/applications/mullvad-browser/-/compare/a1… You're receiving this email because of your account on gitlab.torproject.org.

1 0

[Git][tpo/applications/tor-browser][base-browser-115.8.0esr-13.0-1] 2 commits: fixup! Firefox preference overrides.
by Pier Angelo Vendrame (＠pierov) 11 Mar '24

11 Mar '24

Pier Angelo Vendrame pushed to branch base-browser-115.8.0esr-13.0-1 at The Tor Project / Applications / Tor Browser Commits: f251f057 by Pier Angelo Vendrame at 2024-03-11T15:53:27+01:00 fixup! Firefox preference overrides. Bug 42377: Hidden fonts should obey the allow list. Remove .Helvetica Neue DeskInterface from the allow list. This font might not be actually available, and in any case fonts starting with period will not be displayed by the browser. - - - - - 479be760 by Pier Angelo Vendrame at 2024-03-11T15:53:29+01:00 Bug 42377: Hidden fonts should obey the allow list. Hidden font families were automatically added to the `font.system.whitelist`, which is a behavior that conflicts with our font picking. - - - - - 2 changed files: - browser/app/profile/001-base-profile.js - gfx/thebes/gfxPlatformFontList.cpp Changes: ===================================== browser/app/profile/001-base-profile.js ===================================== @@ -643,7 +643,7 @@ pref("toolkit.winRegisterApplicationRestart", false); pref("gfx.bundled-fonts.activate", 1); #ifdef XP_MACOSX -pref("font.system.whitelist", "AppleGothic, Apple Color Emoji, Arial, Courier, Courier New, Geneva, Georgia, Heiti TC, Helvetica, Helvetica Neue, .Helvetica Neue DeskInterface, Hiragino Kaku Gothic ProN, Kailasa, Lucida Grande, Menlo, Monaco, PingFang HK, PingFang SC, PingFang TC, Songti SC, Songti TC, Tahoma, Thonburi, Times, Times New Roman, Verdana, STIX Two Math, Noto Sans Adlam, Noto Sans Armenian, Noto Sans Balinese, Noto Sans Bamum, Noto Sans Bassa Vah, Noto Sans Batak, Noto Sans Bengali, Noto Sans Buginese, Noto Sans Buhid, Noto Sans Canadian Aboriginal, Noto Sans Chakma, Noto Sans Cham, Noto Sans Cherokee, Noto Sans Coptic, Noto Sans Deseret, Noto Sans Devanagari, Noto Sans Elbasan, Noto Sans Ethiopic, Noto Sans Georgian, Noto Sans Grantha, Noto Sans Gujarati, Noto Sans Gunjala Gondi, Noto Sans Gurmukhi, Noto Sans Hanifi Rohingya, Noto Sans Hanunoo, Noto Sans Hebrew, Noto Sans Kannada, Noto Sans Khmer, Noto Sans Javanese, Noto Sans Kayah Li, Noto Sans Khojki, Noto Sans Khudawadi, Noto Sans Lao, Noto Sans Lepcha, Noto Sans Limbu, Noto Sans Lisu, Noto Sans Mahajani, Noto Sans Malayalam, Noto Sans Mandaic, Noto Sans Masaram Gondi, Noto Sans Medefaidrin, Noto Sans Meetei Mayek, Noto Sans Mende Kikakui, Noto Sans Miao, Noto Sans Modi, Noto Sans Mongolian, Noto Sans Mro, Noto Sans Myanmar, Noto Sans Multani, Noto Sans Newa, Noto Sans New Tai Lue, Noto Sans NKo, Noto Sans Ol Chiki, Noto Sans Oriya, Noto Sans Osage, Noto Sans Osmanya, Noto Sans Pahawh Hmong, Noto Sans Pau Cin Hau, Noto Sans Rejang, Noto Sans Runic, Noto Sans Samaritan, Noto Sans Saurashtra, Noto Sans Sharada, Noto Sans Shavian, Noto Sans Sinhala, Noto Sans Sora Sompeng, Noto Sans Soyombo, Noto Sans Sundanese, Noto Sans Syloti Nagri, Noto Sans Symbols 2, Noto Sans Symbols, Noto Sans Syriac, Noto Sans Tagalog, Noto Sans Tagbanwa, Noto Sans Tai Le, Noto Sans Tai Tham, Noto Sans Tai Viet, Noto Sans Takri, Noto Sans Tamil, Noto Sans Telugu, Noto Sans Thaana, Noto Sans Tifinagh Adrar, Noto Sans Tifinagh Agraw Imazighen, Noto Sans Tifinagh Ahaggar, Noto Sans Tifinagh Air, Noto Sans Tifinagh APT, Noto Sans Tifinagh Azawagh, Noto Sans Tifinagh Ghat, Noto Sans Tifinagh Hawad, Noto Sans Tifinagh, Noto Sans Tifinagh Rhissa Ixa, Noto Sans Tifinagh SIL, Noto Sans Tifinagh Tawellemmet, Noto Sans Tirhuta, Noto Sans Vai, Noto Sans Wancho, Noto Sans Warang Citi, Noto Sans Yi, Noto Sans Zanabazar Square, Noto Serif Armenian, Noto Serif Balinese, Noto Serif Bengali, Noto Serif Devanagari, Noto Serif Dogra, Noto Serif Ethiopic, Noto Serif Georgian, Noto Serif Grantha, Noto Serif Gujarati, Noto Serif Gurmukhi, Noto Serif Hebrew, Noto Serif Kannada, Noto Serif Khmer, Noto Serif Khojki, Noto Serif Lao, Noto Serif Malayalam, Noto Serif Myanmar, Noto Serif NP Hmong, Noto Serif Sinhala, Noto Serif Tamil, Noto Serif Telugu, Noto Serif Tibetan, Noto Serif Yezidi"); +pref("font.system.whitelist", "AppleGothic, Apple Color Emoji, Arial, Courier, Courier New, Geneva, Georgia, Heiti TC, Helvetica, Helvetica Neue, Hiragino Kaku Gothic ProN, Kailasa, Lucida Grande, Menlo, Monaco, PingFang HK, PingFang SC, PingFang TC, Songti SC, Songti TC, Tahoma, Thonburi, Times, Times New Roman, Verdana, STIX Two Math, Noto Sans Adlam, Noto Sans Armenian, Noto Sans Balinese, Noto Sans Bamum, Noto Sans Bassa Vah, Noto Sans Batak, Noto Sans Bengali, Noto Sans Buginese, Noto Sans Buhid, Noto Sans Canadian Aboriginal, Noto Sans Chakma, Noto Sans Cham, Noto Sans Cherokee, Noto Sans Coptic, Noto Sans Deseret, Noto Sans Devanagari, Noto Sans Elbasan, Noto Sans Ethiopic, Noto Sans Georgian, Noto Sans Grantha, Noto Sans Gujarati, Noto Sans Gunjala Gondi, Noto Sans Gurmukhi, Noto Sans Hanifi Rohingya, Noto Sans Hanunoo, Noto Sans Hebrew, Noto Sans Kannada, Noto Sans Khmer, Noto Sans Javanese, Noto Sans Kayah Li, Noto Sans Khojki, Noto Sans Khudawadi, Noto Sans Lao, Noto Sans Lepcha, Noto Sans Limbu, Noto Sans Lisu, Noto Sans Mahajani, Noto Sans Malayalam, Noto Sans Mandaic, Noto Sans Masaram Gondi, Noto Sans Medefaidrin, Noto Sans Meetei Mayek, Noto Sans Mende Kikakui, Noto Sans Miao, Noto Sans Modi, Noto Sans Mongolian, Noto Sans Mro, Noto Sans Myanmar, Noto Sans Multani, Noto Sans Newa, Noto Sans New Tai Lue, Noto Sans NKo, Noto Sans Ol Chiki, Noto Sans Oriya, Noto Sans Osage, Noto Sans Osmanya, Noto Sans Pahawh Hmong, Noto Sans Pau Cin Hau, Noto Sans Rejang, Noto Sans Runic, Noto Sans Samaritan, Noto Sans Saurashtra, Noto Sans Sharada, Noto Sans Shavian, Noto Sans Sinhala, Noto Sans Sora Sompeng, Noto Sans Soyombo, Noto Sans Sundanese, Noto Sans Syloti Nagri, Noto Sans Symbols 2, Noto Sans Symbols, Noto Sans Syriac, Noto Sans Tagalog, Noto Sans Tagbanwa, Noto Sans Tai Le, Noto Sans Tai Tham, Noto Sans Tai Viet, Noto Sans Takri, Noto Sans Tamil, Noto Sans Telugu, Noto Sans Thaana, Noto Sans Tifinagh Adrar, Noto Sans Tifinagh Agraw Imazighen, Noto Sans Tifinagh Ahaggar, Noto Sans Tifinagh Air, Noto Sans Tifinagh APT, Noto Sans Tifinagh Azawagh, Noto Sans Tifinagh Ghat, Noto Sans Tifinagh Hawad, Noto Sans Tifinagh, Noto Sans Tifinagh Rhissa Ixa, Noto Sans Tifinagh SIL, Noto Sans Tifinagh Tawellemmet, Noto Sans Tirhuta, Noto Sans Vai, Noto Sans Wancho, Noto Sans Warang Citi, Noto Sans Yi, Noto Sans Zanabazar Square, Noto Serif Armenian, Noto Serif Balinese, Noto Serif Bengali, Noto Serif Devanagari, Noto Serif Dogra, Noto Serif Ethiopic, Noto Serif Georgian, Noto Serif Grantha, Noto Serif Gujarati, Noto Serif Gurmukhi, Noto Serif Hebrew, Noto Serif Kannada, Noto Serif Khmer, Noto Serif Khojki, Noto Serif Lao, Noto Serif Malayalam, Noto Serif Myanmar, Noto Serif NP Hmong, Noto Serif Sinhala, Noto Serif Tamil, Noto Serif Telugu, Noto Serif Tibetan, Noto Serif Yezidi"); // Armenian pref("font.name-list.serif.x-armn", "Noto Serif Armenian, Times, Times New Roman"); ===================================== gfx/thebes/gfxPlatformFontList.cpp ===================================== @@ -401,12 +401,6 @@ void gfxPlatformFontList::ApplyWhitelist() { AutoTArray<RefPtr<gfxFontFamily>, 128> accepted; bool whitelistedFontFound = false; for (const auto& entry : mFontFamilies) { - if (entry.GetData()->IsHidden()) { - // Hidden system fonts are exempt from whitelisting, but don't count - // towards determining whether we "kept" any (user-visible) fonts - accepted.AppendElement(entry.GetData()); - continue; - } nsAutoCString fontFamilyName(entry.GetKey()); ToLowerCase(fontFamilyName); if (familyNamesWhitelist.Contains(fontFamilyName)) { @@ -442,8 +436,7 @@ void gfxPlatformFontList::ApplyWhitelist( AutoTArray<fontlist::Family::InitData, 128> accepted; bool keptNonHidden = false; for (auto& f : aFamilies) { - if (f.mVisibility == FontVisibility::Hidden || - familyNamesWhitelist.Contains(f.mKey)) { + if (familyNamesWhitelist.Contains(f.mKey)) { accepted.AppendElement(f); if (f.mVisibility != FontVisibility::Hidden) { keptNonHidden = true; View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser/-/compare/63c89d… -- View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser/-/compare/63c89d… You're receiving this email because of your account on gitlab.torproject.org.

1 0

[Git][tpo/applications/tor-browser][tor-browser-115.8.0esr-13.0-1] 2 commits: fixup! Firefox preference overrides.
by Pier Angelo Vendrame (＠pierov) 11 Mar '24

11 Mar '24

Pier Angelo Vendrame pushed to branch tor-browser-115.8.0esr-13.0-1 at The Tor Project / Applications / Tor Browser Commits: 9a246151 by Pier Angelo Vendrame at 2024-03-11T15:52:04+01:00 fixup! Firefox preference overrides. Bug 42377: Hidden fonts should obey the allow list. Remove .Helvetica Neue DeskInterface from the allow list. This font might not be actually available, and in any case fonts starting with period will not be displayed by the browser. - - - - - 6bbacc00 by Pier Angelo Vendrame at 2024-03-11T15:52:13+01:00 Bug 42377: Hidden fonts should obey the allow list. Hidden font families were automatically added to the `font.system.whitelist`, which is a behavior that conflicts with our font picking. - - - - - 2 changed files: - browser/app/profile/001-base-profile.js - gfx/thebes/gfxPlatformFontList.cpp Changes: ===================================== browser/app/profile/001-base-profile.js ===================================== @@ -643,7 +643,7 @@ pref("toolkit.winRegisterApplicationRestart", false); pref("gfx.bundled-fonts.activate", 1); #ifdef XP_MACOSX -pref("font.system.whitelist", "AppleGothic, Apple Color Emoji, Arial, Courier, Courier New, Geneva, Georgia, Heiti TC, Helvetica, Helvetica Neue, .Helvetica Neue DeskInterface, Hiragino Kaku Gothic ProN, Kailasa, Lucida Grande, Menlo, Monaco, PingFang HK, PingFang SC, PingFang TC, Songti SC, Songti TC, Tahoma, Thonburi, Times, Times New Roman, Verdana, STIX Two Math, Noto Sans Adlam, Noto Sans Armenian, Noto Sans Balinese, Noto Sans Bamum, Noto Sans Bassa Vah, Noto Sans Batak, Noto Sans Bengali, Noto Sans Buginese, Noto Sans Buhid, Noto Sans Canadian Aboriginal, Noto Sans Chakma, Noto Sans Cham, Noto Sans Cherokee, Noto Sans Coptic, Noto Sans Deseret, Noto Sans Devanagari, Noto Sans Elbasan, Noto Sans Ethiopic, Noto Sans Georgian, Noto Sans Grantha, Noto Sans Gujarati, Noto Sans Gunjala Gondi, Noto Sans Gurmukhi, Noto Sans Hanifi Rohingya, Noto Sans Hanunoo, Noto Sans Hebrew, Noto Sans Kannada, Noto Sans Khmer, Noto Sans Javanese, Noto Sans Kayah Li, Noto Sans Khojki, Noto Sans Khudawadi, Noto Sans Lao, Noto Sans Lepcha, Noto Sans Limbu, Noto Sans Lisu, Noto Sans Mahajani, Noto Sans Malayalam, Noto Sans Mandaic, Noto Sans Masaram Gondi, Noto Sans Medefaidrin, Noto Sans Meetei Mayek, Noto Sans Mende Kikakui, Noto Sans Miao, Noto Sans Modi, Noto Sans Mongolian, Noto Sans Mro, Noto Sans Myanmar, Noto Sans Multani, Noto Sans Newa, Noto Sans New Tai Lue, Noto Sans NKo, Noto Sans Ol Chiki, Noto Sans Oriya, Noto Sans Osage, Noto Sans Osmanya, Noto Sans Pahawh Hmong, Noto Sans Pau Cin Hau, Noto Sans Rejang, Noto Sans Runic, Noto Sans Samaritan, Noto Sans Saurashtra, Noto Sans Sharada, Noto Sans Shavian, Noto Sans Sinhala, Noto Sans Sora Sompeng, Noto Sans Soyombo, Noto Sans Sundanese, Noto Sans Syloti Nagri, Noto Sans Symbols 2, Noto Sans Symbols, Noto Sans Syriac, Noto Sans Tagalog, Noto Sans Tagbanwa, Noto Sans Tai Le, Noto Sans Tai Tham, Noto Sans Tai Viet, Noto Sans Takri, Noto Sans Tamil, Noto Sans Telugu, Noto Sans Thaana, Noto Sans Tifinagh Adrar, Noto Sans Tifinagh Agraw Imazighen, Noto Sans Tifinagh Ahaggar, Noto Sans Tifinagh Air, Noto Sans Tifinagh APT, Noto Sans Tifinagh Azawagh, Noto Sans Tifinagh Ghat, Noto Sans Tifinagh Hawad, Noto Sans Tifinagh, Noto Sans Tifinagh Rhissa Ixa, Noto Sans Tifinagh SIL, Noto Sans Tifinagh Tawellemmet, Noto Sans Tirhuta, Noto Sans Vai, Noto Sans Wancho, Noto Sans Warang Citi, Noto Sans Yi, Noto Sans Zanabazar Square, Noto Serif Armenian, Noto Serif Balinese, Noto Serif Bengali, Noto Serif Devanagari, Noto Serif Dogra, Noto Serif Ethiopic, Noto Serif Georgian, Noto Serif Grantha, Noto Serif Gujarati, Noto Serif Gurmukhi, Noto Serif Hebrew, Noto Serif Kannada, Noto Serif Khmer, Noto Serif Khojki, Noto Serif Lao, Noto Serif Malayalam, Noto Serif Myanmar, Noto Serif NP Hmong, Noto Serif Sinhala, Noto Serif Tamil, Noto Serif Telugu, Noto Serif Tibetan, Noto Serif Yezidi"); +pref("font.system.whitelist", "AppleGothic, Apple Color Emoji, Arial, Courier, Courier New, Geneva, Georgia, Heiti TC, Helvetica, Helvetica Neue, Hiragino Kaku Gothic ProN, Kailasa, Lucida Grande, Menlo, Monaco, PingFang HK, PingFang SC, PingFang TC, Songti SC, Songti TC, Tahoma, Thonburi, Times, Times New Roman, Verdana, STIX Two Math, Noto Sans Adlam, Noto Sans Armenian, Noto Sans Balinese, Noto Sans Bamum, Noto Sans Bassa Vah, Noto Sans Batak, Noto Sans Bengali, Noto Sans Buginese, Noto Sans Buhid, Noto Sans Canadian Aboriginal, Noto Sans Chakma, Noto Sans Cham, Noto Sans Cherokee, Noto Sans Coptic, Noto Sans Deseret, Noto Sans Devanagari, Noto Sans Elbasan, Noto Sans Ethiopic, Noto Sans Georgian, Noto Sans Grantha, Noto Sans Gujarati, Noto Sans Gunjala Gondi, Noto Sans Gurmukhi, Noto Sans Hanifi Rohingya, Noto Sans Hanunoo, Noto Sans Hebrew, Noto Sans Kannada, Noto Sans Khmer, Noto Sans Javanese, Noto Sans Kayah Li, Noto Sans Khojki, Noto Sans Khudawadi, Noto Sans Lao, Noto Sans Lepcha, Noto Sans Limbu, Noto Sans Lisu, Noto Sans Mahajani, Noto Sans Malayalam, Noto Sans Mandaic, Noto Sans Masaram Gondi, Noto Sans Medefaidrin, Noto Sans Meetei Mayek, Noto Sans Mende Kikakui, Noto Sans Miao, Noto Sans Modi, Noto Sans Mongolian, Noto Sans Mro, Noto Sans Myanmar, Noto Sans Multani, Noto Sans Newa, Noto Sans New Tai Lue, Noto Sans NKo, Noto Sans Ol Chiki, Noto Sans Oriya, Noto Sans Osage, Noto Sans Osmanya, Noto Sans Pahawh Hmong, Noto Sans Pau Cin Hau, Noto Sans Rejang, Noto Sans Runic, Noto Sans Samaritan, Noto Sans Saurashtra, Noto Sans Sharada, Noto Sans Shavian, Noto Sans Sinhala, Noto Sans Sora Sompeng, Noto Sans Soyombo, Noto Sans Sundanese, Noto Sans Syloti Nagri, Noto Sans Symbols 2, Noto Sans Symbols, Noto Sans Syriac, Noto Sans Tagalog, Noto Sans Tagbanwa, Noto Sans Tai Le, Noto Sans Tai Tham, Noto Sans Tai Viet, Noto Sans Takri, Noto Sans Tamil, Noto Sans Telugu, Noto Sans Thaana, Noto Sans Tifinagh Adrar, Noto Sans Tifinagh Agraw Imazighen, Noto Sans Tifinagh Ahaggar, Noto Sans Tifinagh Air, Noto Sans Tifinagh APT, Noto Sans Tifinagh Azawagh, Noto Sans Tifinagh Ghat, Noto Sans Tifinagh Hawad, Noto Sans Tifinagh, Noto Sans Tifinagh Rhissa Ixa, Noto Sans Tifinagh SIL, Noto Sans Tifinagh Tawellemmet, Noto Sans Tirhuta, Noto Sans Vai, Noto Sans Wancho, Noto Sans Warang Citi, Noto Sans Yi, Noto Sans Zanabazar Square, Noto Serif Armenian, Noto Serif Balinese, Noto Serif Bengali, Noto Serif Devanagari, Noto Serif Dogra, Noto Serif Ethiopic, Noto Serif Georgian, Noto Serif Grantha, Noto Serif Gujarati, Noto Serif Gurmukhi, Noto Serif Hebrew, Noto Serif Kannada, Noto Serif Khmer, Noto Serif Khojki, Noto Serif Lao, Noto Serif Malayalam, Noto Serif Myanmar, Noto Serif NP Hmong, Noto Serif Sinhala, Noto Serif Tamil, Noto Serif Telugu, Noto Serif Tibetan, Noto Serif Yezidi"); // Armenian pref("font.name-list.serif.x-armn", "Noto Serif Armenian, Times, Times New Roman"); ===================================== gfx/thebes/gfxPlatformFontList.cpp ===================================== @@ -401,12 +401,6 @@ void gfxPlatformFontList::ApplyWhitelist() { AutoTArray<RefPtr<gfxFontFamily>, 128> accepted; bool whitelistedFontFound = false; for (const auto& entry : mFontFamilies) { - if (entry.GetData()->IsHidden()) { - // Hidden system fonts are exempt from whitelisting, but don't count - // towards determining whether we "kept" any (user-visible) fonts - accepted.AppendElement(entry.GetData()); - continue; - } nsAutoCString fontFamilyName(entry.GetKey()); ToLowerCase(fontFamilyName); if (familyNamesWhitelist.Contains(fontFamilyName)) { @@ -442,8 +436,7 @@ void gfxPlatformFontList::ApplyWhitelist( AutoTArray<fontlist::Family::InitData, 128> accepted; bool keptNonHidden = false; for (auto& f : aFamilies) { - if (f.mVisibility == FontVisibility::Hidden || - familyNamesWhitelist.Contains(f.mKey)) { + if (familyNamesWhitelist.Contains(f.mKey)) { accepted.AppendElement(f); if (f.mVisibility != FontVisibility::Hidden) { keptNonHidden = true; View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser/-/compare/572c0b… -- View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser/-/compare/572c0b… You're receiving this email because of your account on gitlab.torproject.org.

1 0

[Git][tpo/applications/tor-browser-build][maint-13.0] Bug 41102: Compress firefox source tarball in a container
by boklm (＠boklm) 11 Mar '24

11 Mar '24

boklm pushed to branch maint-13.0 at The Tor Project / Applications / tor-browser-build Commits: c73503ae by Nicolas Vigier at 2024-03-11T15:48:23+01:00 Bug 41102: Compress firefox source tarball in a container As the xz version on the host can be different and produce different output, we compress the firefox source tarball inside a container. At the same time we use xz's --threads option to make compression faster by using multiple threads. We also move the src-tarballs step definition from rbm.conf to projects/firefox/config since it's not used in any other project. - - - - - 3 changed files: - projects/firefox/config - projects/release/config - rbm.conf Changes: ===================================== projects/firefox/config ===================================== @@ -58,9 +58,25 @@ steps: src-tarballs: filename: 'src-[% project %]-[% c("version") %].tar.xz' version: '[% c("git_hash") %]' - input_files: [] + input_files: + - project: container-image + pkg_type: build + compress_tar: '' container: - use_container: 0 + use_container: 1 + var: + # single-thread and multi-thread xz will generate a different result, + # se we use at least 2 threads + xz_threads: '[% c("num_procs") == "1" ? "2" : c("num_procs") %]' + src-tarballs: | + #!/bin/bash + set -e + mkdir -p '[% dest_dir %]' + # Files copied to the container are owned by group root (rbm#40074), + # and it seems xz doesn't like that and exits with an error + chgrp rbm '[% project %]-[% c("version") %].tar' + xz --threads=[% c("var/xz_threads") %] -f '[% project %]-[% c("version") %].tar' + mv -vf '[% project %]-[% c("version") %].tar.xz' '[% dest_dir %]/[% c("filename") %]' targets: nightly: version: '[% c("abbrev") %]' ===================================== projects/release/config ===================================== @@ -229,8 +229,7 @@ input_files: pkg_type: src-tarballs target: - '[% c("var/build_target") %]' - - browser-src - - '[% c("var/browser_type") %]' + - '[% c("var/browser_type") %]-macos' steps: signtag: ===================================== rbm.conf ===================================== @@ -14,14 +14,6 @@ container: build: 1 steps: - src-tarballs: - compress_tar: xz - src-tarballs: | - #!/bin/bash - set -e - mkdir -p '[% dest_dir %]' - mv -vf '[% project %]-[% c("version") %].tar.xz' '[% dest_dir %]/[% c("filename") %]' - list_toolchain_updates: build_log: '-' list_toolchain_updates: '[% INCLUDE list_toolchain_updates %]' View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/commit/c… -- View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/commit/c… You're receiving this email because of your account on gitlab.torproject.org.

1 0

[Git][tpo/applications/tor-browser-build][main] Bug 41102: Compress firefox source tarball in a container
by boklm (＠boklm) 11 Mar '24

11 Mar '24

boklm pushed to branch main at The Tor Project / Applications / tor-browser-build Commits: 5e9f8f0c by Nicolas Vigier at 2024-03-11T15:47:20+01:00 Bug 41102: Compress firefox source tarball in a container As the xz version on the host can be different and produce different output, we compress the firefox source tarball inside a container. At the same time we use xz's --threads option to make compression faster by using multiple threads. We also move the src-tarballs step definition from rbm.conf to projects/firefox/config since it's not used in any other project. - - - - - 3 changed files: - projects/firefox/config - projects/release/config - rbm.conf Changes: ===================================== projects/firefox/config ===================================== @@ -58,9 +58,25 @@ steps: src-tarballs: filename: 'src-[% project %]-[% c("version") %].tar.xz' version: '[% c("git_hash") %]' - input_files: [] + input_files: + - project: container-image + pkg_type: build + compress_tar: '' container: - use_container: 0 + use_container: 1 + var: + # single-thread and multi-thread xz will generate a different result, + # se we use at least 2 threads + xz_threads: '[% c("num_procs") == "1" ? "2" : c("num_procs") %]' + src-tarballs: | + #!/bin/bash + set -e + mkdir -p '[% dest_dir %]' + # Files copied to the container are owned by group root (rbm#40074), + # and it seems xz doesn't like that and exits with an error + chgrp rbm '[% project %]-[% c("version") %].tar' + xz --threads=[% c("var/xz_threads") %] -f '[% project %]-[% c("version") %].tar' + mv -vf '[% project %]-[% c("version") %].tar.xz' '[% dest_dir %]/[% c("filename") %]' targets: nightly: version: '[% c("abbrev") %]' ===================================== projects/release/config ===================================== @@ -229,8 +229,7 @@ input_files: pkg_type: src-tarballs target: - '[% c("var/build_target") %]' - - browser-src - - '[% c("var/browser_type") %]' + - '[% c("var/browser_type") %]-macos' steps: signtag: ===================================== rbm.conf ===================================== @@ -14,14 +14,6 @@ container: build: 1 steps: - src-tarballs: - compress_tar: xz - src-tarballs: | - #!/bin/bash - set -e - mkdir -p '[% dest_dir %]' - mv -vf '[% project %]-[% c("version") %].tar.xz' '[% dest_dir %]/[% c("filename") %]' - list_toolchain_updates: build_log: '-' list_toolchain_updates: '[% INCLUDE list_toolchain_updates %]' View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/commit/5… -- View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/commit/5… You're receiving this email because of your account on gitlab.torproject.org.

1 0

[Git][tpo/applications/tor-browser][tor-browser-115.8.0esr-13.5-1] fixup! Add TorStrings module for localization
by Pier Angelo Vendrame (＠pierov) 07 Mar '24

07 Mar '24

Pier Angelo Vendrame pushed to branch tor-browser-115.8.0esr-13.5-1 at The Tor Project / Applications / Tor Browser Commits: f6c7b7f2 by Henry Wilkes at 2024-03-07T11:46:58+00:00 fixup! Add TorStrings module for localization Bug 42442: Remove unused bridge settings strings. - - - - - 1 changed file: - toolkit/torbutton/chrome/locale/en-US/settings.properties Changes: ===================================== toolkit/torbutton/chrome/locale/en-US/settings.properties ===================================== @@ -96,32 +96,3 @@ settings.allowedPortsPlaceholder=Comma-separated values # Log dialog settings.torLogDialogTitle=Tor Logs settings.copyLog=Copy Tor Log to Clipboard - - -# TODO: Remove - -settings.bridgeCurrent=Your Current Bridges -settings.bridgeCurrentDescription=You can keep one or more bridges saved, and Tor will choose which one to use when you connect. Tor will automatically switch to use another bridge when needed. -# Translation note: %1$S = bridge type; %2$S = bridge emoji id -settings.bridgeId=%1$S bridge: %2$S -settings.connectedBridge=Connected -settings.bridgeShare=Share this bridge using the QR code or by copying its address: -settings.whatAreThese=What are these? -settings.bridgeCopy=Copy Bridge Address -settings.bridgeShowAll=Show All Bridges -settings.bridgeShowFewer=Show Fewer Bridges -settings.allBridgesEnabled=Use current bridges -settings.bridgeRemoveAll=Remove All Bridges -settings.bridgeAdd=Add a New Bridge -settings.bridgeSelectBrowserBuiltin=Choose from one of Tor Browser’s built-in bridges -settings.bridgeSelectBuiltin=Select a Built-In Bridge… -settings.bridgeRequestFromTorProject=Request a bridge from torproject.org -settings.bridgeRequest=Request a Bridge… -settings.bridgeEnterKnown=Enter a bridge address you already know -settings.bridgeAddManually=Add a Bridge Manually… - -# Provide bridge dialog -settings.provideBridgeTitleAdd=Add a Bridge Manually -# Translation note: %S is a Learn more link. -settings.provideBridgeDescription=Add a bridge provided by a trusted organization or someone you know. If you don’t have a bridge, you can request one from the Tor Project. %S -settings.provideBridgePlaceholder=type address:port (one per line) View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser/-/commit/f6c7b7f… -- View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser/-/commit/f6c7b7f… You're receiving this email because of your account on gitlab.torproject.org.

1 0

[Git][tpo/applications/tor-browser][tor-browser-115.8.0esr-13.5-1] 3 commits: fixup! Tor Browser localization migration scripts.
by Pier Angelo Vendrame (＠pierov) 07 Mar '24

07 Mar '24

Pier Angelo Vendrame pushed to branch tor-browser-115.8.0esr-13.5-1 at The Tor Project / Applications / Tor Browser Commits: 708a2c47 by Henry Wilkes at 2024-03-07T10:37:42+00:00 fixup! Tor Browser localization migration scripts. Bug 42305: Move localization scripts into new folder. - - - - - ad71ddb2 by Henry Wilkes at 2024-03-07T10:37:42+00:00 Bug 42305: Add script to combine translation files across versions. - - - - - f13a9d22 by Henry Wilkes at 2024-03-07T10:37:42+00:00 Add CI for Tor Browser - - - - - 13 changed files: - + .gitlab-ci.yml - + tools/torbrowser/l10n/combine-translation-versions.py - + tools/torbrowser/l10n/combine/__init__.py - + tools/torbrowser/l10n/combine/combine.py - + tools/torbrowser/l10n/combine/tests/README - tools/torbrowser/l10n_migrations/__init__.py → tools/torbrowser/l10n/combine/tests/__init__.py - + tools/torbrowser/l10n/combine/tests/test_android.py - + tools/torbrowser/l10n/combine/tests/test_dtd.py - + tools/torbrowser/l10n/combine/tests/test_fluent.py - + tools/torbrowser/l10n/combine/tests/test_properties.py - tools/torbrowser/migrate_l10n.py → tools/torbrowser/l10n/migrate.py - + tools/torbrowser/l10n/migrations/__init__.py - tools/torbrowser/l10n_migrations/bug-41333-new-about-tor.py → tools/torbrowser/l10n/migrations/bug-41333-new-about-tor.py Changes: ===================================== .gitlab-ci.yml ===================================== @@ -0,0 +1,69 @@ +stages: + - update-translations + +.update-translation-base: + stage: update-translations + rules: + - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH + changes: + - "**/*.ftl" + - "**/*.properties" + - "**/*.dtd" + - if: $FORCE_UPDATE_TRANSLATIONS == "true" + variables: + TOR_BROWSER_COMBINED_FILES_JSON: "combined-translation-files.json" + + +combine-en-US-translations: + extends: .update-translation-base + image: python + variables: + PIP_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pip" + TRANSLATION_FILES: ' + tor-browser:tor-browser.ftl + tor-browser:aboutDialog.dtd + tor-browser:aboutTBUpdate.dtd + tor-browser:aboutTor.dtd + tor-browser:torbutton.dtd + tor-browser:browserOnboarding.properties + tor-browser:cryptoSafetyPrompt.properties + tor-browser:onboarding.properties + tor-browser:onionLocation.properties + tor-browser:rulesets.properties + tor-browser:settings.properties + tor-browser:torbutton.properties + tor-browser:torConnect.properties + tor-browser:torlauncher.properties + base-browser:base-browser.ftl + base-browser:newIdentity.properties + base-browser:securityLevel.properties + ' + cache: + paths: + - .cache/pip + # Artifact is for translation project job + artifacts: + paths: + - "$TOR_BROWSER_COMBINED_FILES_JSON" + expire_in: "60 min" + reports: + dotenv: job_id.env + # Don't load artifacts for this job. + dependencies: [] + script: + # Save this CI_JOB_ID to the dotenv file to be used in the variables for the + # push-en-US-translations job. + - echo 'COMBINE_TRANSLATIONS_JOB_ID='"$CI_JOB_ID" >job_id.env + - pip install compare_locales + - python ./tools/torbrowser/l10n/combine-translation-versions.py "$CI_COMMIT_BRANCH" "$TRANSLATION_FILES" "$TOR_BROWSER_COMBINED_FILES_JSON" + +push-en-US-translations: + extends: .update-translation-base + needs: + - job: combine-en-US-translations + variables: + TOR_BROWSER_COMBINED_FILES_JSON_URL: "${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/jobs/${COMBINE_TRANSLATIONS_JOB_ID}/artifacts/${TOR_BROWSER_COMBINED_FILES_JSON}" + trigger: + strategy: depend + project: tor-browser-translation-bot/translation + branch: tor-browser-ci ===================================== tools/torbrowser/l10n/combine-translation-versions.py ===================================== @@ -0,0 +1,208 @@ +import argparse +import json +import logging +import os +import re +import subprocess + +from combine import combine_files + +arg_parser = argparse.ArgumentParser( + description="Combine a translation file across two different versions" +) + +arg_parser.add_argument( + "current_branch", metavar="<current-branch>", help="branch for the newest version" +) +arg_parser.add_argument( + "filenames", metavar="<filenames>", help="name of the translation files" +) +arg_parser.add_argument("outname", metavar="<json>", help="name of the json output") + +args = arg_parser.parse_args() + +logging.basicConfig() +logger = logging.getLogger("combine-translation-versions") +logger.setLevel(logging.INFO) + + +def in_pink(msg: str) -> str: + """Present a message as pink in the terminal output. + + :param msg: The message to wrap in pink. + :returns: The message to print to terminal. + """ + # Pink and bold. + return f"\x1b[1;38;5;212m{msg}\x1b[0m" + + +def git_run(git_args: list[str]) -> None: + """Run a git command. + + :param git_args: The arguments that should follow "git". + """ + # Add some text to give context to git's stderr appearing in log. + logger.info("Running: " + in_pink("git " + " ".join(git_args))) + subprocess.run(["git", *git_args], check=True) + + +def git_text(git_args: list[str]) -> str: + """Get the text output for a git command. + + :param git_args: The arguments that should follow "git". + :returns: The stdout of the command. + """ + logger.info("Running: " + in_pink("git " + " ".join(git_args))) + return subprocess.run( + ["git", *git_args], text=True, check=True, stdout=subprocess.PIPE + ).stdout + + +def git_lines(git_args: list[str]) -> list[str]: + """Get the lines from a git command. + + :param git_args: The arguments that should follow "git". + :returns: The non-empty lines from stdout of the command. + """ + return [line for line in git_text(git_args).split("\n") if line] + + +def git_file_paths(git_ref: str) -> list[str]: + """Get the full list of file paths found under the given tree. + + :param git_ref: The git reference for the tree to search. + :returns: The found file paths. + """ + return git_lines(["ls-tree", "-r", "--format=%(path)", git_ref]) + + +def matching_path(search_paths: list[str], filename: str) -> str | None: + """Get the matching file path with the given filename, if it exists. + + :param search_paths: The file paths to search through. + :param filename: The file name to match. + :returns: The unique file path with the matching name, or None if no such + match was found. + :throws Exception: If multiple paths shared the same file name. + """ + matching = [path for path in search_paths if os.path.basename(path) == filename] + if not matching: + return None + if len(matching) > 1: + raise Exception("Multiple occurrences of {filename}") + return matching[0] + + +def git_file_content(git_ref: str, path: str | None) -> str | None: + """Get the file content of the specified git blob object. + + :param git_ref: The reference for the tree to find the file under. + :param path: The file path for the object, or None if there is no path. + :returns: The file content, or None if no path was given. + """ + if path is None: + return None + return git_text(["cat-file", "blob", f"{git_ref}:{path}"]) + + +def get_stable_branch(branch_prefix: str) -> str: + """Find the most recent stable branch in the origin repository. + + :param branch_prefix: The prefix that the stable branch should have. + :returns: The branch name. + """ + tag_glob = f"{branch_prefix}-*-build1" + # To speed up, only fetch the tags without blobs. + git_run( + ["fetch", "--depth=1", "--filter=object:type=tag", "origin", "tag", tag_glob] + ) + # Get most recent stable tag. + for build_tag, annotation in ( + line.split(" ", 1) + for line in git_lines(["tag", "-n1", "--list", tag_glob, "--sort=-taggerdate"]) + ): + if "stable" in annotation: + # Branch name is the same as the tag, minus "-build1". + return re.sub(r"-build1$", "", build_tag) + raise Exception("No stable build1 tag found") + + +def get_version_from_branch_name(branch_name: str) -> tuple[str, float]: + """Get the branch prefix and version from its name. + + :param branch_name: The branch to extract from. + :returns: The branch prefix and its version number. + """ + version_match = re.match( + r"([a-z-]+)-[^-]*-([0-9]+\.[05])-", + branch_name, + ) + + if not version_match: + raise ValueError(f"Unable to parse the version from the branch {branch_name}") + + return (version_match.group(1), float(version_match.group(2))) + + +branch_prefix, current_version = get_version_from_branch_name(args.current_branch) + +stable_branch = get_stable_branch(branch_prefix) +_, stable_version = get_version_from_branch_name(stable_branch) + +if stable_version > current_version or stable_version < current_version - 0.5: + raise Exception( + f"Version of stable branch {stable_branch} is not within 0.5 of the " + f"current branch {args.current_branch}" + ) + +# Minimal fetch of stable_branch. +# Individual file blobs will be downloaded as needed. +git_run(["fetch", "--depth=1", "--filter=blob:none", "origin", stable_branch]) + +current_file_paths = git_file_paths("HEAD") +old_file_paths = git_file_paths(f"origin/{stable_branch}") + +ci_commit = os.environ.get("CI_COMMIT_SHA", "") +ci_url_base = os.environ.get("CI_PROJECT_URL", "") + +json_data = { + "commit": ci_commit, + "commit-url": f"{ci_url_base}/-/commit/{ci_commit}" + if (ci_commit and ci_url_base) + else "", + "project-path": os.environ.get("CI_PROJECT_PATH", ""), + "current-branch": args.current_branch, + "stable-branch": stable_branch, + "files": [], +} + +for translation_branch, name in ( + part.strip().split(":", 1) for part in args.filenames.split(" ") if part.strip() +): + current_path = matching_path(current_file_paths, name) + old_path = matching_path(old_file_paths, name) + + if current_path is None and old_path is None: + # No file in either branch. + logger.warning(f"{name} does not exist in either the current or stable branch") + elif current_path is None: + logger.warning(f"{name} deleted in the current branch") + elif old_path is None: + logger.warning(f"{name} does not exist in the stable branch") + + content = combine_files( + name, + git_file_content("HEAD", current_path), + git_file_content(f"origin/{stable_branch}", old_path), + f"Will be unused in Tor Browser {current_version}!", + ) + json_data["files"].append( + { + "name": name, + "branch": translation_branch, + "content": content, + } + ) + +with open(args.outname, "w") as file: + json.dump(json_data, file) ===================================== tools/torbrowser/l10n/combine/__init__.py ===================================== @@ -0,0 +1,3 @@ +# flake8: noqa + +from .combine import combine_files ===================================== tools/torbrowser/l10n/combine/combine.py ===================================== @@ -0,0 +1,181 @@ +import re +from typing import TYPE_CHECKING, Any + +from compare_locales.parser import getParser +from compare_locales.parser.android import AndroidEntity, DocumentWrapper +from compare_locales.parser.base import Comment, Entity, Junk, Whitespace +from compare_locales.parser.dtd import DTDEntity +from compare_locales.parser.fluent import FluentComment, FluentEntity +from compare_locales.parser.properties import PropertiesEntity + +if TYPE_CHECKING: + from collections.abc import Iterable + + +def combine_files( + filename: str, + new_content: str | None, + old_content: str | None, + comment_prefix: str, +) -> str | None: + """Combine two translation files into one to include all strings from both. + The new content is presented first, and any strings only found in the old + content are placed at the end with an additional comment. + + :param filename: The filename for the file, determines the format. + :param new_content: The new content for the file, or None if it has been + deleted. + :param old_content: The old content for the file, or None if it did not + exist before. + :comment_prefix: A comment to include for any strings that are only found in + the old content. This will be placed before any other comments for the + string. + + :returns: The combined content, or None if both given contents are None. + """ + if new_content is None and old_content is None: + return None + + # getParser from compare_locale returns the same instance for the same file + # extension. + parser = getParser(filename) + + is_android = filename.endswith(".xml") + if new_content is None: + if is_android: + # File was deleted, add some document parts. + content_start = ( + '<?xml version="1.0" encoding="utf-8" standalone="yes"?>\n<resources>\n' + ) + content_end = "</resources>\n" + else: + # Treat as an empty file. + content_start = "" + content_end = "" + existing_keys = [] + else: + parser.readUnicode(new_content) + + # Start with the same content as the current file. + # For android strings, we want to keep the final "</resources>" until after. + if is_android: + closing_match = re.match( + r"^(.*)(</resources>\s*)$", parser.ctx.contents, re.DOTALL + ) + if not closing_match: + raise ValueError("Missing a final </resources>") + content_start = closing_match.group(1) + content_end = closing_match.group(2) + else: + content_start = parser.ctx.contents + content_end = "" + existing_keys = [entry.key for entry in parser.walk(only_localizable=True)] + + # For Fluent, we want to prefix the strings using GroupComments. + # On weblate this will cause all the strings that fall under the GroupComment's + # scope to have the prefix added to their "notes". + # We set up an initial GroupComment for the first string we find. This will also + # end the scope of the last GroupComment in the new translation file. + # This will be replaced with a the next GroupComment when it is found. + fluent_group_comment_prefix = f"\n## {comment_prefix}\n" + fluent_group_comment: str | None = fluent_group_comment_prefix + + # For other formats, we want to keep all the comment lines that come directly + # before the string. + # In compare_locales.parser, only the comment line directly before an Entity + # counts as the pre_comment for that Entity. I.e. only this line will be + # included in Entity.all + # However, in weblate every comment line that comes before the Entity is + # included as a comment. So we also want to keep these additional comments to + # preserve them for weblate. + # We gather these extra comments in stacked_comments, and clear them whenever we + # reach an Entity or a blank line (Whitespace is more than "\n"). + stacked_comments: list[str] = [] + + additions: list[str] = [] + + entry_iter: Iterable[Any] = () + # If the file does not exist in the old branch, don't make any additions. + if old_content is not None: + parser.readUnicode(old_content) + entry_iter = parser.walk(only_localizable=False) + for entry in entry_iter: + if isinstance(entry, Junk): + raise ValueError(f"Unexpected Junk: {entry.all}") + if isinstance(entry, Whitespace): + # Clear stacked comments if more than one empty line. + if entry.all != "\n": + stacked_comments.clear() + continue + if isinstance(entry, Comment): + if isinstance(entry, FluentComment): + # Don't stack Fluent comments. + # Only the comments included in Entity.pre_comment count towards + # that Entity's comment. + if entry.all.startswith("##"): + # A Fluent GroupComment + if entry.all == "##": + # Empty GroupComment. Used to end the scope of a previous + # GroupComment. + # Replace this with our prefix comment. + fluent_group_comment = fluent_group_comment_prefix + else: + # Prefix the group comment. + fluent_group_comment = ( + f"{fluent_group_comment_prefix}{entry.all}\n" + ) + else: + stacked_comments.append(entry.all) + continue + if isinstance(entry, DocumentWrapper): + # Not needed. + continue + + if not isinstance(entry, Entity): + raise ValueError(f"Unexpected type: {entry.__class__.__name__}") + + if entry.key in existing_keys: + # Already included this string in the new translation file. + # Drop the gathered comments for this Entity. + stacked_comments.clear() + continue + + if isinstance(entry, FluentEntity): + if fluent_group_comment is not None: + # We have a found GroupComment which has not been included yet. + # All following Entity's will be under its scope, until the next + # GroupComment. + additions.append(fluent_group_comment) + # Added GroupComment, so don't need to add again. + fluent_group_comment = None + elif isinstance(entry, DTDEntity): + # Include our additional comment before we print the rest for this + # Entity. + additions.append(f"") + elif isinstance(entry, PropertiesEntity): + additions.append(f"# {comment_prefix}") + elif isinstance(entry, AndroidEntity): + additions.append(f"") + else: + raise ValueError(f"Unexpected Entity type: {entry.__class__.__name__}") + + # Add any other comment lines that came directly before this Entity. + additions.extend(stacked_comments) + stacked_comments.clear() + additions.append(entry.all) + + content_middle = "" + + if additions: + # New line before and after the additions + additions.insert(0, "") + additions.append("") + if is_android: + content_middle = "\n ".join(additions) + else: + content_middle = "\n".join(additions) + + # Remove " " in otherwise blank lines. + content_middle = re.sub("^ +$", "", content_middle, flags=re.MULTILINE) + + return content_start + content_middle + content_end ===================================== tools/torbrowser/l10n/combine/tests/README ===================================== @@ -0,0 +1,2 @@ +python tests to be run with pytest. +Requires the compare-locales package. ===================================== tools/torbrowser/l10n_migrations/__init__.py → tools/torbrowser/l10n/combine/tests/__init__.py ===================================== ===================================== tools/torbrowser/l10n/combine/tests/test_android.py ===================================== @@ -0,0 +1,330 @@ +import textwrap + +from combine import combine_files + + +def wrap_in_xml(content): + if content is None: + return None + # Allow for indents to make the tests more readable. + content = textwrap.dedent(content) + return f"""\ +<?xml version="1.0" encoding="utf-8" standalone="yes"?> +<resources> +{textwrap.indent(content, " ")}</resources> +""" + + +def assert_result(new_content, old_content, expect): + new_content = wrap_in_xml(new_content) + old_content = wrap_in_xml(old_content) + expect = wrap_in_xml(expect) + assert expect == combine_files( + "test_strings.xml", new_content, old_content, "REMOVED STRING" + ) + + +def test_combine_empty(): + assert_result(None, None, None) + + +def test_combine_new_file(): + # New file with no old content. + assert_result( + """\ + <string name="string_1">First</string> + <string name="string_2">Second</string> + """, + None, + """\ + <string name="string_1">First</string> + <string name="string_2">Second</string> + """, + ) + + +def test_combine_removed_file(): + # Entire file was removed. + assert_result( + None, + """\ + <string name="string_1">First</string> + <string name="string_2">Second</string> + """, + """\ + +  + <string name="string_1">First</string> +  + <string name="string_2">Second</string> + """, + ) + + +def test_no_change(): + content = """\ + <string name="string_1">First</string> + <string name="string_2">Second</string> + """ + assert_result(content, content, content) + + +def test_added_string(): + assert_result( + """\ + <string name="string_1">First</string> + <string name="string_new">NEW</string> + <string name="string_2">Second</string> + """, + """\ + <string name="string_1">First</string> + <string name="string_2">Second</string> + """, + """\ + <string name="string_1">First</string> + <string name="string_new">NEW</string> + <string name="string_2">Second</string> + """, + ) + + +def test_removed_string(): + assert_result( + """\ + <string name="string_1">First</string> + <string name="string_2">Second</string> + """, + """\ + <string name="string_1">First</string> + <string name="removed">REMOVED</string> + <string name="string_2">Second</string> + """, + """\ + <string name="string_1">First</string> + <string name="string_2">Second</string> + +  + <string name="removed">REMOVED</string> + """, + ) + + +def test_removed_and_added(): + assert_result( + """\ + <string name="new_1">New string</string> + <string name="string_1">First</string> + <string name="string_2">Second</string> + <string name="new_2">New string 2</string> + """, + """\ + <string name="string_1">First</string> + <string name="removed_1">First removed</string> + <string name="removed_2">Second removed</string> + <string name="string_2">Second</string> + <string name="removed_3">Third removed</string> + """, + """\ + <string name="new_1">New string</string> + <string name="string_1">First</string> + <string name="string_2">Second</string> + <string name="new_2">New string 2</string> + +  + <string name="removed_1">First removed</string> +  + <string name="removed_2">Second removed</string> +  + <string name="removed_3">Third removed</string> + """, + ) + + +def test_updated(): + # String content was updated. + assert_result( + """\ + <string name="changed_string">NEW</string> + """, + """\ + <string name="changed_string">OLD</string> + """, + """\ + <string name="changed_string">NEW</string> + """, + ) + + +def test_updated_comment(): + # String comment was updated. + assert_result( + """\ +  + <string name="changed_string">string</string> + """, + """\ +  + <string name="changed_string">string</string> + """, + """\ +  + <string name="changed_string">string</string> + """, + ) + # Comment added. + assert_result( + """\ +  + <string name="changed_string">string</string> + """, + """\ + <string name="changed_string">string</string> + """, + """\ +  + <string name="changed_string">string</string> + """, + ) + # Comment removed. + assert_result( + """\ + <string name="changed_string">string</string> + """, + """\ +  + <string name="changed_string">string</string> + """, + """\ + <string name="changed_string">string</string> + """, + ) + + # With file comments + assert_result( + """\ +  + +  + <string name="changed_string">string</string> + """, + """\ +  + +  + <string name="changed_string">string</string> + """, + """\ +  + +  + <string name="changed_string">string</string> + """, + ) + + +def test_reordered(): + # String was re_ordered. + assert_result( + """\ + <string name="string_1">value</string> + <string name="moved_string">move</string> + """, + """\ + <string name="moved_string">move</string> + <string name="string_1">value</string> + """, + """\ + <string name="string_1">value</string> + <string name="moved_string">move</string> + """, + ) + + +def test_removed_string_with_comment(): + assert_result( + """\ +  + <string name="string_1">First</string> + <string name="string_2">Second</string> + """, + """\ +  + <string name="string_1">First</string> +  + <string name="removed">REMOVED</string> + <string name="string_2">Second</string> + """, + """\ +  + <string name="string_1">First</string> + <string name="string_2">Second</string> + +  +  + <string name="removed">REMOVED</string> + """, + ) + + # With file comments and multi-line. + # All comments prior to a removed string are moved with it, until another + # entity or blank line is reached. + assert_result( + """\ +  + +  +  + <string name="string_1">First</string> + +  +  + + <string name="string_2">Second</string> + """, + """\ +  + +  +  + <string name="string_1">First</string> + <string name="removed_1">First removed</string> +  + <string name="removed_2">Second removed</string> + +  + +  +  + <string name="removed_3">Third removed</string> + +  +  + + <string name="removed_4">Fourth removed</string> + <string name="string_2">Second</string> + """, + """\ +  + +  +  + <string name="string_1">First</string> + +  +  + + <string name="string_2">Second</string> + +  + <string name="removed_1">First removed</string> +  +  + <string name="removed_2">Second removed</string> +  +  +  + <string name="removed_3">Third removed</string> +  + <string name="removed_4">Fourth removed</string> + """, + ) ===================================== tools/torbrowser/l10n/combine/tests/test_dtd.py ===================================== @@ -0,0 +1,325 @@ +import textwrap + +from combine import combine_files + + +def assert_result(new_content, old_content, expect): + # Allow for indents to make the tests more readable. + if new_content is not None: + new_content = textwrap.dedent(new_content) + if old_content is not None: + old_content = textwrap.dedent(old_content) + if expect is not None: + expect = textwrap.dedent(expect) + assert expect == combine_files( + "test.dtd", new_content, old_content, "REMOVED STRING" + ) + + +def test_combine_empty(): + assert_result(None, None, None) + + +def test_combine_new_file(): + # New file with no old content. + assert_result( + """\ + <!ENTITY string.1 "First"> + <!ENTITY string.2 "Second"> + """, + None, + """\ + <!ENTITY string.1 "First"> + <!ENTITY string.2 "Second"> + """, + ) + + +def test_combine_removed_file(): + # Entire file was removed. + assert_result( + None, + """\ + <!ENTITY string.1 "First"> + <!ENTITY string.2 "Second"> + """, + """\ + +  + <!ENTITY string.1 "First"> +  + <!ENTITY string.2 "Second"> + """, + ) + + +def test_no_change(): + content = """\ + <!ENTITY string.1 "First"> + <!ENTITY string.2 "Second"> + """ + assert_result(content, content, content) + + +def test_added_string(): + assert_result( + """\ + <!ENTITY string.1 "First"> + <!ENTITY string.new "NEW"> + <!ENTITY string.2 "Second"> + """, + """\ + <!ENTITY string.1 "First"> + <!ENTITY string.2 "Second"> + """, + """\ + <!ENTITY string.1 "First"> + <!ENTITY string.new "NEW"> + <!ENTITY string.2 "Second"> + """, + ) + + +def test_removed_string(): + assert_result( + """\ + <!ENTITY string.1 "First"> + <!ENTITY string.2 "Second"> + """, + """\ + <!ENTITY string.1 "First"> + <!ENTITY removed "REMOVED"> + <!ENTITY string.2 "Second"> + """, + """\ + <!ENTITY string.1 "First"> + <!ENTITY string.2 "Second"> + +  + <!ENTITY removed "REMOVED"> + """, + ) + + +def test_removed_and_added(): + assert_result( + """\ + <!ENTITY new.1 "New string"> + <!ENTITY string.1 "First"> + <!ENTITY string.2 "Second"> + <!ENTITY new.2 "New string 2"> + """, + """\ + <!ENTITY string.1 "First"> + <!ENTITY removed.1 "First removed"> + <!ENTITY removed.2 "Second removed"> + <!ENTITY string.2 "Second"> + <!ENTITY removed.3 "Third removed"> + """, + """\ + <!ENTITY new.1 "New string"> + <!ENTITY string.1 "First"> + <!ENTITY string.2 "Second"> + <!ENTITY new.2 "New string 2"> + +  + <!ENTITY removed.1 "First removed"> +  + <!ENTITY removed.2 "Second removed"> +  + <!ENTITY removed.3 "Third removed"> + """, + ) + + +def test_updated(): + # String content was updated. + assert_result( + """\ + <!ENTITY changed.string "NEW"> + """, + """\ + <!ENTITY changed.string "OLD"> + """, + """\ + <!ENTITY changed.string "NEW"> + """, + ) + + +def test_updated_comment(): + # String comment was updated. + assert_result( + """\ +  + <!ENTITY changed.string "string"> + """, + """\ +  + <!ENTITY changed.string "string"> + """, + """\ +  + <!ENTITY changed.string "string"> + """, + ) + # Comment added. + assert_result( + """\ +  + <!ENTITY changed.string "string"> + """, + """\ + <!ENTITY changed.string "string"> + """, + """\ +  + <!ENTITY changed.string "string"> + """, + ) + # Comment removed. + assert_result( + """\ + <!ENTITY changed.string "string"> + """, + """\ +  + <!ENTITY changed.string "string"> + """, + """\ + <!ENTITY changed.string "string"> + """, + ) + + # With multiple comments + assert_result( + """\ +  + +  + <!ENTITY changed.string "string"> + """, + """\ +  + +  + <!ENTITY changed.string "string"> + """, + """\ +  + +  + <!ENTITY changed.string "string"> + """, + ) + + +def test_reordered(): + # String was re.ordered. + assert_result( + """\ + <!ENTITY string.1 "value"> + <!ENTITY moved.string "move"> + """, + """\ + <!ENTITY moved.string "move"> + <!ENTITY string.1 "value"> + """, + """\ + <!ENTITY string.1 "value"> + <!ENTITY moved.string "move"> + """, + ) + + +def test_removed_string_with_comment(): + assert_result( + """\ +  + <!ENTITY string.1 "First"> + <!ENTITY string.2 "Second"> + """, + """\ +  + <!ENTITY string.1 "First"> +  + <!ENTITY removed "REMOVED"> + <!ENTITY string.2 "Second"> + """, + """\ +  + <!ENTITY string.1 "First"> + <!ENTITY string.2 "Second"> + +  +  + <!ENTITY removed "REMOVED"> + """, + ) + + # With multiple lines of comments. + + assert_result( + """\ +  + +  +  + <!ENTITY string.1 "First"> + +  + + <!ENTITY string.2 "Second"> + """, + """\ +  + +  + <!ENTITY string.1 "First"> + <!ENTITY removed.1 "First removed"> +  + <!ENTITY removed.2 "Second removed"> + +  + +  +  + <!ENTITY removed.3 "Third removed"> + +  + + <!ENTITY removed.4 "Fourth removed"> + <!ENTITY string.2 "Second"> + """, + """\ +  + +  +  + <!ENTITY string.1 "First"> + +  + + <!ENTITY string.2 "Second"> + +  + <!ENTITY removed.1 "First removed"> +  +  + <!ENTITY removed.2 "Second removed"> +  +  +  + <!ENTITY removed.3 "Third removed"> +  + <!ENTITY removed.4 "Fourth removed"> + """, + ) ===================================== tools/torbrowser/l10n/combine/tests/test_fluent.py ===================================== @@ -0,0 +1,344 @@ +import textwrap + +from combine import combine_files + + +def assert_result(new_content, old_content, expect): + # Allow for indents to make the tests more readable. + if new_content is not None: + new_content = textwrap.dedent(new_content) + if old_content is not None: + old_content = textwrap.dedent(old_content) + if expect is not None: + expect = textwrap.dedent(expect) + assert expect == combine_files( + "test.ftl", new_content, old_content, "REMOVED STRING" + ) + + +def test_combine_empty(): + assert_result(None, None, None) + + +def test_combine_new_file(): + # New file with no old content. + assert_result( + """\ + string-1 = First + string-2 = Second + """, + None, + """\ + string-1 = First + string-2 = Second + """, + ) + + +def test_combine_removed_file(): + # Entire file was removed. + assert_result( + None, + """\ + string-1 = First + string-2 = Second + """, + """\ + + + ## REMOVED STRING + + string-1 = First + string-2 = Second + """, + ) + + +def test_no_change(): + content = """\ + string-1 = First + string-2 = Second + """ + assert_result(content, content, content) + + +def test_added_string(): + assert_result( + """\ + string-1 = First + string-new = NEW + string-2 = Second + """, + """\ + string-1 = First + string-2 = Second + """, + """\ + string-1 = First + string-new = NEW + string-2 = Second + """, + ) + + +def test_removed_string(): + assert_result( + """\ + string-1 = First + string-2 = Second + """, + """\ + string-1 = First + removed = REMOVED + string-2 = Second + """, + """\ + string-1 = First + string-2 = Second + + + ## REMOVED STRING + + removed = REMOVED + """, + ) + + +def test_removed_and_added(): + assert_result( + """\ + new-1 = New string + string-1 = + .attr = First + string-2 = Second + new-2 = + .title = New string 2 + """, + """\ + string-1 = + .attr = First + removed-1 = First removed + removed-2 = + .attr = Second removed + string-2 = Second + removed-3 = Third removed + """, + """\ + new-1 = New string + string-1 = + .attr = First + string-2 = Second + new-2 = + .title = New string 2 + + + ## REMOVED STRING + + removed-1 = First removed + removed-2 = + .attr = Second removed + removed-3 = Third removed + """, + ) + + +def test_updated(): + # String content was updated. + assert_result( + """\ + changed-string = NEW + """, + """\ + changed-string = OLD + """, + """\ + changed-string = NEW + """, + ) + + +def test_updated_comment(): + # String comment was updated. + assert_result( + """\ + # NEW + changed-string = string + """, + """\ + # OLD + changed-string = string + """, + """\ + # NEW + changed-string = string + """, + ) + # Comment added. + assert_result( + """\ + # NEW + changed-string = string + """, + """\ + changed-string = string + """, + """\ + # NEW + changed-string = string + """, + ) + # Comment removed. + assert_result( + """\ + changed-string = string + """, + """\ + # OLD + changed-string = string + """, + """\ + changed-string = string + """, + ) + + # With group comments. + assert_result( + """\ + ## GROUP NEW + + # NEW + changed-string = string + """, + """\ + ## GROUP OLD + + # OLD + changed-string = string + """, + """\ + ## GROUP NEW + + # NEW + changed-string = string + """, + ) + + +def test_reordered(): + # String was re-ordered. + assert_result( + """\ + string-1 = value + moved-string = move + """, + """\ + moved-string = move + string-1 = value + """, + """\ + string-1 = value + moved-string = move + """, + ) + + +def test_removed_string_with_comment(): + assert_result( + """\ + # Comment for first. + string-1 = First + string-2 = Second + """, + """\ + # Comment for first. + string-1 = First + # Comment for removed. + removed = REMOVED + string-2 = Second + """, + """\ + # Comment for first. + string-1 = First + string-2 = Second + + + ## REMOVED STRING + + # Comment for removed. + removed = REMOVED + """, + ) + + # Group comments are combined with the "REMOVED STRING" comments. + # If strings have no group comment, then a single "REMOVED STRING" is + # included for them. + assert_result( + """\ + ## First Group comment + + # Comment for first. + string-1 = First + + ## + + no-group = No group comment + + ## Second + ## Group comment + + string-2 = Second + """, + """\ + ## First Group comment + + # Comment for first. + string-1 = First + removed-1 = First removed + # Comment for second removed. + removed-2 = Second removed + + ## + + no-group = No group comment + removed-3 = Third removed + + ## Second + ## Group comment + + removed-4 = Fourth removed + string-2 = Second + """, + """\ + ## First Group comment + + # Comment for first. + string-1 = First + + ## + + no-group = No group comment + + ## Second + ## Group comment + + string-2 = Second + + + ## REMOVED STRING + ## First Group comment + + removed-1 = First removed + # Comment for second removed. + removed-2 = Second removed + + ## REMOVED STRING + + removed-3 = Third removed + + ## REMOVED STRING + ## Second + ## Group comment + + removed-4 = Fourth removed + """, + ) ===================================== tools/torbrowser/l10n/combine/tests/test_properties.py ===================================== @@ -0,0 +1,322 @@ +import textwrap + +from combine import combine_files + + +def assert_result(new_content, old_content, expect): + # Allow for indents to make the tests more readable. + if new_content is not None: + new_content = textwrap.dedent(new_content) + if old_content is not None: + old_content = textwrap.dedent(old_content) + if expect is not None: + expect = textwrap.dedent(expect) + assert expect == combine_files( + "test.properties", new_content, old_content, "REMOVED STRING" + ) + + +def test_combine_empty(): + assert_result(None, None, None) + + +def test_combine_new_file(): + # New file with no old content. + assert_result( + """\ + string.1 = First + string.2 = Second + """, + None, + """\ + string.1 = First + string.2 = Second + """, + ) + + +def test_combine_removed_file(): + # Entire file was removed. + assert_result( + None, + """\ + string.1 = First + string.2 = Second + """, + """\ + + # REMOVED STRING + string.1 = First + # REMOVED STRING + string.2 = Second + """, + ) + + +def test_no_change(): + content = """\ + string.1 = First + string.2 = Second + """ + assert_result(content, content, content) + + +def test_added_string(): + assert_result( + """\ + string.1 = First + string.new = NEW + string.2 = Second + """, + """\ + string.1 = First + string.2 = Second + """, + """\ + string.1 = First + string.new = NEW + string.2 = Second + """, + ) + + +def test_removed_string(): + assert_result( + """\ + string.1 = First + string.2 = Second + """, + """\ + string.1 = First + removed = REMOVED + string.2 = Second + """, + """\ + string.1 = First + string.2 = Second + + # REMOVED STRING + removed = REMOVED + """, + ) + + +def test_removed_and_added(): + assert_result( + """\ + new.1 = New string + string.1 = First + string.2 = Second + new.2 = New string 2 + """, + """\ + string.1 = First + removed.1 = First removed + removed.2 = Second removed + string.2 = Second + removed.3 = Third removed + """, + """\ + new.1 = New string + string.1 = First + string.2 = Second + new.2 = New string 2 + + # REMOVED STRING + removed.1 = First removed + # REMOVED STRING + removed.2 = Second removed + # REMOVED STRING + removed.3 = Third removed + """, + ) + + +def test_updated(): + # String content was updated. + assert_result( + """\ + changed.string = NEW + """, + """\ + changed.string = OLD + """, + """\ + changed.string = NEW + """, + ) + + +def test_updated_comment(): + # String comment was updated. + assert_result( + """\ + # NEW + changed.string = string + """, + """\ + # OLD + changed.string = string + """, + """\ + # NEW + changed.string = string + """, + ) + # Comment added. + assert_result( + """\ + # NEW + changed.string = string + """, + """\ + changed.string = string + """, + """\ + # NEW + changed.string = string + """, + ) + # Comment removed. + assert_result( + """\ + changed.string = string + """, + """\ + # OLD + changed.string = string + """, + """\ + changed.string = string + """, + ) + + # With file comments + assert_result( + """\ + # NEW file comment + + # NEW + changed.string = string + """, + """\ + # OLD file comment + + # OLD + changed.string = string + """, + """\ + # NEW file comment + + # NEW + changed.string = string + """, + ) + + +def test_reordered(): + # String was re.ordered. + assert_result( + """\ + string.1 = value + moved.string = move + """, + """\ + moved.string = move + string.1 = value + """, + """\ + string.1 = value + moved.string = move + """, + ) + + +def test_removed_string_with_comment(): + assert_result( + """\ + # Comment for first. + string.1 = First + string.2 = Second + """, + """\ + # Comment for first. + string.1 = First + # Comment for removed. + removed = REMOVED + string.2 = Second + """, + """\ + # Comment for first. + string.1 = First + string.2 = Second + + # REMOVED STRING + # Comment for removed. + removed = REMOVED + """, + ) + + # With file comments and multi-line. + # All comments prior to a removed string are moved with it, until another + # entity or blank line is reached. + assert_result( + """\ + # First File comment + + # Comment for first. + # Comment 2 for first. + string.1 = First + + # Second + # File comment + + string.2 = Second + """, + """\ + # First File comment + + # Comment for first. + # Comment 2 for first. + string.1 = First + removed.1 = First removed + # Comment for second removed. + removed.2 = Second removed + + # Removed file comment + + # Comment 1 for third removed + # Comment 2 for third removed + removed.3 = Third removed + + # Second + # File comment + + removed.4 = Fourth removed + string.2 = Second + """, + """\ + # First File comment + + # Comment for first. + # Comment 2 for first. + string.1 = First + + # Second + # File comment + + string.2 = Second + + # REMOVED STRING + removed.1 = First removed + # REMOVED STRING + # Comment for second removed. + removed.2 = Second removed + # REMOVED STRING + # Comment 1 for third removed + # Comment 2 for third removed + removed.3 = Third removed + # REMOVED STRING + removed.4 = Fourth removed + """, + ) ===================================== tools/torbrowser/migrate_l10n.py → tools/torbrowser/l10n/migrate.py ===================================== ===================================== tools/torbrowser/l10n/migrations/__init__.py ===================================== ===================================== tools/torbrowser/l10n_migrations/bug-41333-new-about-tor.py → tools/torbrowser/l10n/migrations/bug-41333-new-about-tor.py ===================================== View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser/-/compare/a014a0… -- View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser/-/compare/a014a0… You're receiving this email because of your account on gitlab.torproject.org.

1 0