- tor-commits - lists.torproject.org

[Git][tpo/applications/mullvad-browser][mullvad-browser-153.7.0esr-16.0-1] 3 commits: fixup! Mullvad Browser Strings
by morgan (＠morgan) 13 Nov '26

13 Nov '26

morgan pushed to branch mullvad-browser-153.7.0esr-16.0-1 at The Tor Project / Applications / Mullvad Browser Commits: 80298bc8 by june wilde at 2026-11-13T09:22:33+00:00 fixup! Mullvad Browser Strings MB 594: Update relevant neterror page when Quad9 DoH resolution fails - - - - - 16654999 by june wilde at 2026-11-13T09:22:34+00:00 fixup! MB 38: Mullvad Browser configuration MB 593: Re-Add Cloudflare as available DoH provider in about:settings - - - - - 1b4b5ec7 by june wilde at 2026-11-13T09:22:34+00:00 MB 594: Add Quad9 resolution guidance to about:neterror - - - - - 3 changed files: - browser/app/profile/000-mullvad-browser.js - toolkit/content/aboutNetError.html - toolkit/locales/en-US/toolkit/global/mullvad-browser.ftl Changes: ===================================== browser/app/profile/000-mullvad-browser.js ===================================== @@ -14,7 +14,7 @@ pref("browser.toolbars.bookmarks.visibility", "never"); pref("network.trr.uri", "https://doh-mullvad.quad9.net/dns-query"); pref("network.trr.default_provider_uri", "https://doh-mullvad.quad9.net/dns-query"); pref("network.trr.mode", 3); -pref("doh-rollout.provider-list", "[{\"uri\":\"https://doh-mullvad.quad9.net/dns-query\",\"UIName\":\"Quad9 - no threat blocking\",\"schema\":0,\"autoDefault\":false,\"canonicalName\":\"\",\"id\":\"quad9-no-threat-blocking\",\"last_modified\":0},{\"uri\":\"https://dns.quad9.net/dns-query\",\"UIName\":\"Quad9 - secure: threat blocking\",\"schema\":0,\"autoDefault\":true,\"canonicalName\":\"\",\"id\":\"quad9-threat-blocking\",\"last_modified\":0}]"); +pref("doh-rollout.provider-list", "[{\"uri\":\"https://doh-mullvad.quad9.net/dns-query\",\"UIName\":\"Quad9 - no threat blocking\",\"schema\":0,\"autoDefault\":false,\"canonicalName\":\"\",\"id\":\"quad9-no-threat-blocking\",\"last_modified\":0},{\"uri\":\"https://dns.quad9.net/dns-query\",\"UIName\":\"Quad9 - secure: threat blocking\",\"schema\":0,\"autoDefault\":true,\"canonicalName\":\"\",\"id\":\"quad9-threat-blocking\",\"last_modified\":0},{\"uri\":\"https://cloudflare-dns.com/dns-query\",\"UIName\":\"Cloudflare\",\"schema\":0,\"autoDefault\":false,\"canonicalName\":\"\",\"id\":\"cloudflare\",\"last_modified\":0}]"); // mullvad-browser#122: Audit DoH heuristics pref("doh-rollout.disable-heuristics", true); // mullvad-browser#537: migrate users from Mullvad DoH service. ===================================== toolkit/content/aboutNetError.html ===================================== @@ -21,6 +21,7 @@ <link rel="localization" href="toolkit/neterror/certError.ftl" /> <link rel="localization" href="toolkit/neterror/netError.ftl" /> <link rel="localization" href="toolkit/neterror/nsserrors.ftl" /> + <link rel="localization" href="toolkit/global/mullvad-browser.ftl" /> </head> <body> <div class="container"> @@ -60,6 +61,7 @@ </div> + </div> ===================================== toolkit/locales/en-US/toolkit/global/mullvad-browser.ftl ===================================== @@ -55,3 +55,8 @@ mullvad-preferences-doh-radio-off = mullvad-doh-notification-body = Your default DNS Over HTTPS settings have been changed. Mullvad secure DNS service is being discontinued. mullvad-doh-notification-settings-button = .label = Review settings + +## DNS over HTTPS migration notification for french quad9 users + +# The first sentence should be wrapped in '' and ''. +mullvad-quad9-france-dns-error-warning = 🇫🇷 Are you in France? You might need to change your DNS provider to Cloudflare in the DNS settings. View it on GitLab: https://gitlab.torproject.org/tpo/applications/mullvad-browser/-/compare/7c… -- View it on GitLab: https://gitlab.torproject.org/tpo/applications/mullvad-browser/-/compare/7c… You're receiving this email because of your account on gitlab.torproject.org. Manage all notifications: https://gitlab.torproject.org/-/profile/notifications | Help: https://gitlab.torproject.org/help

1 0

[Git][tpo/applications/mullvad-browser][mullvad-browser-156.0a1-17.0-2] 3 commits: fixup! Mullvad Browser Strings
by morgan (＠morgan) 13 Nov '26

13 Nov '26

morgan pushed to branch mullvad-browser-156.0a1-17.0-2 at The Tor Project / Applications / Mullvad Browser Commits: 58bb9a61 by june wilde at 2026-11-13T09:20:32+00:00 fixup! Mullvad Browser Strings MB 594: Update relevant neterror page when Quad9 DoH resolution fails - - - - - 4370706f by june wilde at 2026-11-13T09:20:33+00:00 fixup! MB 38: Mullvad Browser configuration MB 593: Re-Add Cloudflare as available DoH provider in about:settings - - - - - 11c3f7dc by june wilde at 2026-11-13T09:20:33+00:00 MB 594: Add Quad9 resolution guidance to about:neterror - - - - - 3 changed files: - browser/app/profile/000-mullvad-browser.js - toolkit/content/aboutNetError.html - toolkit/locales/en-US/toolkit/global/mullvad-browser.ftl Changes: ===================================== browser/app/profile/000-mullvad-browser.js ===================================== @@ -14,7 +14,7 @@ pref("browser.toolbars.bookmarks.visibility", "never"); pref("network.trr.uri", "https://doh-mullvad.quad9.net/dns-query"); pref("network.trr.default_provider_uri", "https://doh-mullvad.quad9.net/dns-query"); pref("network.trr.mode", 3); -pref("doh-rollout.provider-list", "[{\"uri\":\"https://doh-mullvad.quad9.net/dns-query\",\"UIName\":\"Quad9 - no threat blocking\",\"schema\":0,\"autoDefault\":false,\"canonicalName\":\"\",\"id\":\"quad9-no-threat-blocking\",\"last_modified\":0},{\"uri\":\"https://dns.quad9.net/dns-query\",\"UIName\":\"Quad9 - secure: threat blocking\",\"schema\":0,\"autoDefault\":true,\"canonicalName\":\"\",\"id\":\"quad9-threat-blocking\",\"last_modified\":0}]"); +pref("doh-rollout.provider-list", "[{\"uri\":\"https://doh-mullvad.quad9.net/dns-query\",\"UIName\":\"Quad9 - no threat blocking\",\"schema\":0,\"autoDefault\":false,\"canonicalName\":\"\",\"id\":\"quad9-no-threat-blocking\",\"last_modified\":0},{\"uri\":\"https://dns.quad9.net/dns-query\",\"UIName\":\"Quad9 - secure: threat blocking\",\"schema\":0,\"autoDefault\":true,\"canonicalName\":\"\",\"id\":\"quad9-threat-blocking\",\"last_modified\":0},{\"uri\":\"https://cloudflare-dns.com/dns-query\",\"UIName\":\"Cloudflare\",\"schema\":0,\"autoDefault\":false,\"canonicalName\":\"\",\"id\":\"cloudflare\",\"last_modified\":0}]"); // mullvad-browser#122: Audit DoH heuristics pref("doh-rollout.disable-heuristics", true); // mullvad-browser#537: migrate users from Mullvad DoH service. ===================================== toolkit/content/aboutNetError.html ===================================== @@ -23,6 +23,7 @@ <link rel="localization" href="toolkit/global/mozMessageBar.ftl" /> <link rel="localization" href="toolkit/global/mozSupportLink.ftl" /> <link rel="localization" href="toolkit/neterror/nsserrors.ftl" /> + <link rel="localization" href="toolkit/global/mullvad-browser.ftl" /> </head> <body> <moz-message-bar @@ -70,6 +71,7 @@ </div> + </div> ===================================== toolkit/locales/en-US/toolkit/global/mullvad-browser.ftl ===================================== @@ -55,3 +55,8 @@ mullvad-preferences-doh-radio-off = mullvad-doh-notification-body = Your default DNS Over HTTPS settings have been changed. Mullvad secure DNS service is being discontinued. mullvad-doh-notification-settings-button = .label = Review settings + +## DNS over HTTPS migration notification for french quad9 users + +# The first sentence should be wrapped in '' and ''. +mullvad-quad9-france-dns-error-warning = 🇫🇷 Are you in France? You might need to change your DNS provider to Cloudflare in the DNS settings. View it on GitLab: https://gitlab.torproject.org/tpo/applications/mullvad-browser/-/compare/00… -- View it on GitLab: https://gitlab.torproject.org/tpo/applications/mullvad-browser/-/compare/00… You're receiving this email because of your account on gitlab.torproject.org. Manage all notifications: https://gitlab.torproject.org/-/profile/notifications | Help: https://gitlab.torproject.org/help

1 0

[Git][tpo/applications/mullvad-browser][mullvad-browser-155.0a1-17.0-1] 3 commits: fixup! Mullvad Browser Strings
by morgan (＠morgan) 13 Nov '26

13 Nov '26

morgan pushed to branch mullvad-browser-155.0a1-17.0-1 at The Tor Project / Applications / Mullvad Browser Commits: 554ad1c6 by june wilde at 2026-11-12T10:14:11-08:00 fixup! Mullvad Browser Strings MB 594: Update relevant neterror page when Quad9 DoH resolution fails - - - - - c4605284 by june wilde at 2026-11-12T10:19:06-08:00 fixup! MB 38: Mullvad Browser configuration MB 593: Re-Add Cloudflare as available DoH provider in about:settings - - - - - bd83bda7 by june wilde at 2026-11-12T10:22:07-08:00 MB 594: Add Quad9 resolution guidance to about:neterror - - - - - 3 changed files: - browser/app/profile/000-mullvad-browser.js - toolkit/content/aboutNetError.html - toolkit/locales/en-US/toolkit/global/mullvad-browser.ftl Changes: ===================================== browser/app/profile/000-mullvad-browser.js ===================================== @@ -14,7 +14,7 @@ pref("browser.toolbars.bookmarks.visibility", "never"); pref("network.trr.uri", "https://doh-mullvad.quad9.net/dns-query"); pref("network.trr.default_provider_uri", "https://doh-mullvad.quad9.net/dns-query"); pref("network.trr.mode", 3); -pref("doh-rollout.provider-list", "[{\"uri\":\"https://doh-mullvad.quad9.net/dns-query\",\"UIName\":\"Quad9 - no threat blocking\",\"schema\":0,\"autoDefault\":false,\"canonicalName\":\"\",\"id\":\"quad9-no-threat-blocking\",\"last_modified\":0},{\"uri\":\"https://dns.quad9.net/dns-query\",\"UIName\":\"Quad9 - secure: threat blocking\",\"schema\":0,\"autoDefault\":true,\"canonicalName\":\"\",\"id\":\"quad9-threat-blocking\",\"last_modified\":0}]"); +pref("doh-rollout.provider-list", "[{\"uri\":\"https://doh-mullvad.quad9.net/dns-query\",\"UIName\":\"Quad9 - no threat blocking\",\"schema\":0,\"autoDefault\":false,\"canonicalName\":\"\",\"id\":\"quad9-no-threat-blocking\",\"last_modified\":0},{\"uri\":\"https://dns.quad9.net/dns-query\",\"UIName\":\"Quad9 - secure: threat blocking\",\"schema\":0,\"autoDefault\":true,\"canonicalName\":\"\",\"id\":\"quad9-threat-blocking\",\"last_modified\":0},{\"uri\":\"https://cloudflare-dns.com/dns-query\",\"UIName\":\"Cloudflare\",\"schema\":0,\"autoDefault\":false,\"canonicalName\":\"\",\"id\":\"cloudflare\",\"last_modified\":0}]"); // mullvad-browser#122: Audit DoH heuristics pref("doh-rollout.disable-heuristics", true); // mullvad-browser#537: migrate users from Mullvad DoH service. ===================================== toolkit/content/aboutNetError.html ===================================== @@ -23,6 +23,7 @@ <link rel="localization" href="toolkit/global/mozMessageBar.ftl" /> <link rel="localization" href="toolkit/global/mozSupportLink.ftl" /> <link rel="localization" href="toolkit/neterror/nsserrors.ftl" /> + <link rel="localization" href="toolkit/global/mullvad-browser.ftl" /> </head> <body> <moz-message-bar @@ -70,6 +71,7 @@ </div> + </div> ===================================== toolkit/locales/en-US/toolkit/global/mullvad-browser.ftl ===================================== @@ -55,3 +55,8 @@ mullvad-preferences-doh-radio-off = mullvad-doh-notification-body = Your default DNS Over HTTPS settings have been changed. Mullvad secure DNS service is being discontinued. mullvad-doh-notification-settings-button = .label = Review settings + +## DNS over HTTPS migration notification for french quad9 users + +# The first sentence should be wrapped in '' and ''. +mullvad-quad9-france-dns-error-warning = 🇫🇷 Are you in France? You might need to change your DNS provider to Cloudflare in the DNS settings. View it on GitLab: https://gitlab.torproject.org/tpo/applications/mullvad-browser/-/compare/ab… -- View it on GitLab: https://gitlab.torproject.org/tpo/applications/mullvad-browser/-/compare/ab… You're receiving this email because of your account on gitlab.torproject.org. Manage all notifications: https://gitlab.torproject.org/-/profile/notifications | Help: https://gitlab.torproject.org/help

1 0

[Git][tpo/applications/tor-browser][tor-browser-156.0a1-17.0-2] 6 commits: fixup! TB 44806: Implement the tor integration in Rust.
by Pier Angelo Vendrame (＠pierov) 11 Nov '26

11 Nov '26

Pier Angelo Vendrame pushed to branch tor-browser-156.0a1-17.0-2 at The Tor Project / Applications / Tor Browser Commits: 0b737404 by Pier Angelo Vendrame at 2026-11-11T20:25:11+01:00 fixup! TB 44806: Implement the tor integration in Rust. TB 44930: Implement the commands on the Rust control port Simplify how we handle compulsory group in regexes. - - - - - 8c10dd07 by Pier Angelo Vendrame at 2026-11-11T20:25:11+01:00 fixup! TB 44806: Implement the tor integration in Rust. TB 44930: Implement the commands on the Rust control port Implement Debug for Command to be able to use .unwrap_err() in tests. - - - - - 7788d5fa by Pier Angelo Vendrame at 2026-11-11T20:25:12+01:00 fixup! TB 44806: Implement the tor integration in Rust. TB 44930: Implement the commands on the Rust control port Added docstrings to code that was previously merged. - - - - - 136d1f61 by Elena at 2026-11-11T20:25:12+01:00 fixup! TB 44806: Implement the tor integration in Rust. TB 44930: Implement the commands on the Rust control port Implemented parsers for some GETINFO-based commands. - - - - - 938c4cb0 by Elena at 2026-11-11T20:25:12+01:00 fixup! TB 44806: Implement the tor integration in Rust. TB 44930: Implement the commands on the Rust control port Implemented GETINFO commands. - - - - - 64100cf5 by Elena at 2026-11-11T20:25:13+01:00 fixup! TB 44806: Implement the tor integration in Rust. TB 44930: Implement the commands on the Rust control port Implemented Onion auth parsers and commands. - - - - - 23 changed files: - Cargo.lock - toolkit/components/tor-integration/tor_provider/Cargo.toml - toolkit/components/tor-integration/tor_provider/src/ctor/controller/commands/command.rs - + toolkit/components/tor-integration/tor_provider/src/ctor/controller/commands/get_info.rs - toolkit/components/tor-integration/tor_provider/src/ctor/controller/commands/mod.rs - + toolkit/components/tor-integration/tor_provider/src/ctor/controller/commands/onion_auth_add.rs - + toolkit/components/tor-integration/tor_provider/src/ctor/controller/commands/onion_auth_remove.rs - + toolkit/components/tor-integration/tor_provider/src/ctor/controller/commands/onion_auth_view_keys.rs - toolkit/components/tor-integration/tor_provider/src/ctor/controller/controller.rs - toolkit/components/tor-integration/tor_provider/src/ctor/controller/error.rs - + toolkit/components/tor-integration/tor_provider/src/ctor/controller/parsers/bootstrap_status.rs - toolkit/components/tor-integration/tor_provider/src/ctor/controller/parsers/bridge_line.rs - + toolkit/components/tor-integration/tor_provider/src/ctor/controller/parsers/circuit_status.rs - toolkit/components/tor-integration/tor_provider/src/ctor/controller/parsers/client_transport_plugin.rs - + toolkit/components/tor-integration/tor_provider/src/ctor/controller/parsers/country_code.rs - + toolkit/components/tor-integration/tor_provider/src/ctor/controller/parsers/key_value_pairs.rs - toolkit/components/tor-integration/tor_provider/src/ctor/controller/parsers/mod.rs - + toolkit/components/tor-integration/tor_provider/src/ctor/controller/parsers/node_address.rs - + toolkit/components/tor-integration/tor_provider/src/ctor/controller/parsers/onion_client_auth_view.rs - + toolkit/components/tor-integration/tor_provider/src/ctor/controller/parsers/socks_listeners.rs - + toolkit/components/tor-integration/tor_provider/src/ctor/controller/parsers/stream_status.rs - toolkit/components/tor-integration/tor_provider/src/ctor/controller/parsers/unescape.rs - toolkit/components/tor-integration/tor_provider/src/ctor/controller/types.rs Changes: ===================================== Cargo.lock ===================================== @@ -8124,6 +8124,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "5d4f6d1145dcb577acf783d4e601bc1d76a13337bb54e6233add580b07344c8b" dependencies = [ "displaydoc", + "serde", "zerovec", ] @@ -8262,6 +8263,7 @@ name = "tor_provider" version = "0.1.0" dependencies = [ "bytes", + "data-encoding", "hex", "itoa", "lazy_static", @@ -8269,7 +8271,10 @@ dependencies = [ "memchr", "regex", "serde", + "sha3", + "smallvec", "thiserror 2.0.12", + "tinystr", ] [[package]] ===================================== toolkit/components/tor-integration/tor_provider/Cargo.toml ===================================== @@ -6,6 +6,7 @@ edition = "2021" [dependencies] bytes = "1.4.0" +data-encoding = "2.3.3" hex = "0.4.3" itoa = "1.0.15" lazy_static = "1" @@ -13,4 +14,7 @@ log = "0.4" memchr = "2.7.4" regex = "1" serde = { version = "1.0", features = ["derive"] } +sha3 = "0.10.8" +smallvec = { version = "1.15.1", features = ["serde"] } thiserror = "2" +tinystr = { version = "0.8.1", features = ["serde"] } ===================================== toolkit/components/tor-integration/tor_provider/src/ctor/controller/commands/command.rs ===================================== @@ -3,9 +3,66 @@ // <http://opensource.org/licenses/MIT>, at your option. This file may not be // copied, modified, or distributed except according to those terms. +use std::fmt; + use crate::ctor::{controller::ControllerError, reply_parser::Reply}; pub struct Command<T> { pub command: String, pub handler: Box<dyn Fn(Reply) -> Result<T, ControllerError>>, } + +impl<T> fmt::Debug for Command<T> { + /// Custom formatter that redacts the arguments, as they might contain + /// sensitive information, such as passwords or keys. + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + let mut split = self.command.splitn(2, ' '); + let mut command = split.next().unwrap_or_default().to_string(); + if split.next().is_some() { + command.push_str(" ..."); + } + if command.is_empty() { + command = String::from("(empty)"); + } + + f.debug_struct("Command") + .field("command", &command) + .field("handler", &"<handler>") + .finish() + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn command_fmt_is_redacted() { + { + let cmd = Command { + command: String::from("QUIT"), + handler: Box::new(|_| Ok(())), + }; + let cmd_fmt = format!("{:?}", cmd); + assert!(cmd_fmt.contains("command: \"QUIT\"")); + } + { + let password = "p4ssw0rd"; + let cmd = Command { + command: format!("SETCONF Socks5ProxyPassword={}", password), + handler: Box::new(|_| Ok(())), + }; + let cmd_fmt = format!("{:?}", cmd); + assert!(cmd_fmt.contains("command: \"SETCONF ...\"")); + assert!(!cmd_fmt.contains(password)); + } + { + let cmd = Command { + command: String::from(""), + handler: Box::new(|_| Ok(())), + }; + let cmd_fmt = format!("{:?}", cmd); + assert!(cmd_fmt.contains("command: \"(empty)\"")); + } + } +} ===================================== toolkit/components/tor-integration/tor_provider/src/ctor/controller/commands/get_info.rs ===================================== @@ -0,0 +1,266 @@ +// Licensed under the Apache License, Version 2.0, +// <http://apache.org/licenses/LICENSE-2.0> or the MIT license +// <http://opensource.org/licenses/MIT>, at your option. This file may not be +// copied, modified, or distributed except according to those terms. + +use std::net::IpAddr; +use tinystr::TinyAsciiStr; + +use super::Command; +use crate::ctor::{ + controller::{parsers::*, types::*, ControllerError}, + reply_parser::{DetailReplyLine, Reply}, +}; + +pub fn bootstrap_phase() -> Result<Command<BootstrapStatus>, ControllerError> { + get_info_single( + String::from("status/bootstrap-phase"), + parse_bootstrap_status, + ) +} + +pub fn circuits() -> Result<Command<Vec<Circuit>>, ControllerError> { + get_info(String::from("circuit-status"), |lines| { + lines + .iter() + .filter_map(|line| parse_circuit_built(*line).transpose()) + .collect() + }) +} + +pub fn net_listeners_socks() -> Result<Command<Vec<SocksListener>>, ControllerError> { + get_info_single(String::from("net/listeners/socks"), parse_socks_listeners) +} + +pub fn node_addresses(node: NodeFingerprint) -> Result<Command<NodeAddresses>, ControllerError> { + get_info( + format!("ns/id/{}", hex::encode(node)), + addresses_from_status_info, + ) +} + +pub fn ip_to_country(ip: IpAddr) -> Result<Command<TinyAsciiStr<2>>, ControllerError> { + get_info_single(format!("ip-to-country/{}", ip), parse_country_code) +} + +fn get_info<F, T>(key: String, converter: F) -> Result<Command<T>, ControllerError> +where + F: Fn(&[&[u8]]) -> Result<T, ControllerError> + 'static, +{ + Ok(Command { + command: format!("GETINFO {}\r\n", key), + handler: Box::new(move |reply| parse_get_info(&key, &converter, reply)), + }) +} + +fn get_info_single<F, T>(key: String, converter: F) -> Result<Command<T>, ControllerError> +where + F: Fn(&[u8]) -> Result<T, ControllerError> + 'static, +{ + get_info(key, move |lines| { + if lines.len() != 1 { + Err(ControllerError::MalformedReply(String::from( + "Expected a single line on the reply.", + ))) + } else { + converter(lines[0]) + } + }) +} + +fn parse_get_info<S, F, T>(key: S, converter: &F, reply: Reply) -> Result<T, ControllerError> +where + S: AsRef<str>, + F: Fn(&[&[u8]]) -> Result<T, ControllerError> + 'static, +{ + if let Some(e) = ControllerError::from_reply(&reply) { + return Err(e); + } + + let k = key.as_ref().as_bytes(); + + for line in reply.details().iter() { + match line { + DetailReplyLine::MidReplyLine { code, line } => { + if *code != 250 { + continue; + } + if line.len() < k.len() + 1 { + continue; + } + if line[0..k.len()].eq_ignore_ascii_case(k) && line[k.len()] == b'=' { + return converter(&[&line[(k.len() + 1)..]]); + } + } + DetailReplyLine::DataReplyLine { code, lines } => { + if *code != 250 { + continue; + } + let first = &lines[0]; + if first.len() == k.len() + 1 + && first[0..k.len()].eq_ignore_ascii_case(k) + && first[k.len()] == b'=' + { + let lines: Vec<&[u8]> = lines[1..].iter().map(|b| &b[..]).collect(); + return converter(&lines); + } + } + } + } + + // We should never arrive here. Instead, tor should send 552. + Err(ControllerError::KeyNotFound(key.as_ref().into())) +} + +#[cfg(test)] +mod tests { + use std::assert_matches; + + use super::*; + use crate::ctor::reply_parser::make_reply; + + fn identity_converter(v: &[&[u8]]) -> Result<Vec<Vec<u8>>, ControllerError> { + Ok(v.iter().map(|s| Vec::from(*s)).collect()) + } + + #[test] + fn mid_line() { + assert_eq!( + parse_get_info( + "version", + &identity_converter, + make_reply(b"250-version=0.5.0.0-alpha-dev\r\n250 OK\r\n") + ) + .unwrap(), + &[b"0.5.0.0-alpha-dev"] + ); + } + + #[test] + fn data_line() { + assert_eq!( + parse_get_info( + "test", + &identity_converter, + make_reply(b"250+test=\r\n123\r\n456\r\n.\r\n250 OK\r\n") + ) + .unwrap(), + vec![b"123", b"456"] + ); + } + + #[test] + fn data_line_empty() { + assert_eq!( + parse_get_info( + "test", + &identity_converter, + make_reply(b"250+test=\r\n.\r\n250 OK\r\n") + ) + .unwrap(), + Vec::<&[u8]>::new() + ); + } + + #[test] + fn mid_line_prefix() { + assert_matches!( + parse_get_info( + "version", + &identity_converter, + make_reply(b"250-version__=0.5.0.0-alpha-dev\r\n250 OK\r\n") + ) + .unwrap_err(), + ControllerError::KeyNotFound(_) + ); + } + + #[test] + fn data_line_prefix() { + assert_matches!( + parse_get_info( + "test", + &identity_converter, + make_reply(b"250+test__=\r\n123\r\n456\r\n.\r\n250 OK\r\n") + ) + .unwrap_err(), + ControllerError::KeyNotFound(_) + ); + } + + #[test] + fn short_line_long_key() { + assert_matches!( + parse_get_info( + "averyverylongkey", + &identity_converter, + make_reply(b"250+short=\r\n123\r\n456\r\n.\r\n250 OK\r\n") + ) + .unwrap_err(), + ControllerError::KeyNotFound(_) + ); + assert_matches!( + parse_get_info( + "averyverylooooooooooongkey", + &identity_converter, + make_reply(b"250-short=\r\n250 OK\r\n") + ) + .unwrap_err(), + ControllerError::KeyNotFound(_) + ); + } + + #[test] + fn tor_error() { + assert_eq!( + parse_get_info( + "test", + &identity_converter, + make_reply(b"552 Unrecognized key \"test\"\r\n") + ) + .unwrap_err(), + ControllerError::TorError { + code: 552, + message: String::from("Unrecognized key \"test\"") + } + ); + } + + #[test] + fn error_in_success() { + // This case should not be possible in GETINFO. + // But we want to be sure we can handle it (as a key not found), just in + // case. + assert_matches!( + parse_get_info( + "test", + &identity_converter, + make_reply(b"552-test=0.5.0.0-alpha-dev\r\n250 OK\r\n") + ) + .unwrap_err(), + ControllerError::KeyNotFound(_) + ); + assert_matches!( + parse_get_info( + "test", + &identity_converter, + make_reply(b"552-test=\r\n250 OK\r\n") + ) + .unwrap_err(), + ControllerError::KeyNotFound(_) + ); + } + + #[test] + fn only_end_line() { + assert_matches!( + parse_get_info( + "test", + &identity_converter, + make_reply(b"250 OK\r\n") + ) + .unwrap_err(), + ControllerError::KeyNotFound(_) + ); + } +} ===================================== toolkit/components/tor-integration/tor_provider/src/ctor/controller/commands/mod.rs ===================================== @@ -6,6 +6,10 @@ mod authenticate; mod command; pub mod get_conf; +pub mod get_info; +mod onion_auth_add; +mod onion_auth_remove; +mod onion_auth_view_keys; mod reset_conf; mod save_conf; mod set_conf; @@ -15,6 +19,9 @@ mod take_ownership; pub use authenticate::authenticate; pub use command::Command; +pub use onion_auth_add::onion_auth_add; +pub use onion_auth_remove::onion_auth_remove; +pub use onion_auth_view_keys::onion_auth_view_keys; pub use reset_conf::reset_owning_controller_process; pub use save_conf::save_conf; pub use set_conf::set_conf; ===================================== toolkit/components/tor-integration/tor_provider/src/ctor/controller/commands/onion_auth_add.rs ===================================== @@ -0,0 +1,106 @@ +// Licensed under the Apache License, Version 2.0, +// <http://apache.org/licenses/LICENSE-2.0> or the MIT license +// <http://opensource.org/licenses/MIT>, at your option. This file may not be +// copied, modified, or distributed except according to those terms. + +use lazy_static::lazy_static; +use regex::bytes::Regex; + +use super::Command; +use crate::ctor::controller::{parsers::parse_ack, types::*, ControllerError}; + +lazy_static! { + // This is not a strict base64 regex, but we are intrested only in making + // sure user input does not allow command injection. + static ref BASE64ISH_REGEX: Regex = + Regex::new(r"^[A-Za-z0-9+/]*={0,3}$").expect("The regex is hardcoded, it should be good."); +} + +pub fn onion_auth_add( + address: HSAddress, + key: &[u8], + permanent: bool, +) -> Result<Command<u16>, ControllerError> { + if !BASE64ISH_REGEX.is_match(key) { + return Err(ControllerError::InvalidArgument(String::from( + "the key is not in base64", + ))); + } + let key = str::from_utf8(key).map_err(|e| ControllerError::InvalidArgument(e.to_string()))?; + let flags = if permanent { " Flags=Permanent" } else { "" }; + let command = format!( + "ONION_CLIENT_AUTH_ADD {} x25519:{}{}\r\n", + address.as_ref(), + key, + flags + ); + Ok(Command { + command, + handler: Box::new(parse_ack), + }) +} + +#[cfg(test)] +mod tests { + use std::assert_matches; + + use super::*; + + #[test] + fn permanent() { + let command = onion_auth_add( + HSAddress::from_str("pg6mmjiyjmcrsslvykfwnntlaru7p5svn6y2ymmju6nubxndf4pscryd") + .unwrap(), + b"theKEYinbase64", + true, + ) + .unwrap(); + assert!(command.command.to_lowercase().contains("flags=permanent")); + } + + #[test] + fn not_permanent() { + let command = onion_auth_add( + HSAddress::from_str("pg6mmjiyjmcrsslvykfwnntlaru7p5svn6y2ymmju6nubxndf4pscryd") + .unwrap(), + b"theKEYinbase64", + false, + ) + .unwrap(); + assert!(!command.command.to_lowercase().contains("permanent")); + } + + #[test] + fn pad_and_unpadded() { + // Tor is liberal about the necessity of padding, so make sure that + // padded strings are accepted by us even when not padded. + assert!(onion_auth_add( + HSAddress::from_str("pg6mmjiyjmcrsslvykfwnntlaru7p5svn6y2ymmju6nubxndf4pscryd") + .unwrap(), + b"SGVsbG8sIHdvcmxkIQo=", + false, + ) + .is_ok()); + assert!(onion_auth_add( + HSAddress::from_str("pg6mmjiyjmcrsslvykfwnntlaru7p5svn6y2ymmju6nubxndf4pscryd") + .unwrap(), + b"SGVsbG8sIHdvcmxkIQo", + false, + ) + .is_ok()); + } + + #[test] + fn reject_invalid_base64() { + assert_matches!( + onion_auth_add( + HSAddress::from_str("pg6mmjiyjmcrsslvykfwnntlaru7p5svn6y2ymmju6nubxndf4pscryd") + .unwrap(), + b"NotBase64!!!", + true, + ) + .unwrap_err(), + ControllerError::InvalidArgument(_) + ); + } +} ===================================== toolkit/components/tor-integration/tor_provider/src/ctor/controller/commands/onion_auth_remove.rs ===================================== @@ -0,0 +1,14 @@ +// Licensed under the Apache License, Version 2.0, +// <http://apache.org/licenses/LICENSE-2.0> or the MIT license +// <http://opensource.org/licenses/MIT>, at your option. This file may not be +// copied, modified, or distributed except according to those terms. + +use super::Command; +use crate::ctor::controller::{parsers::parse_ack, types::*, ControllerError}; + +pub fn onion_auth_remove(address: HSAddress) -> Result<Command<u16>, ControllerError> { + Ok(Command { + command: format!("ONION_CLIENT_AUTH_REMOVE {}\r\n", address.as_ref()), + handler: Box::new(parse_ack), + }) +} ===================================== toolkit/components/tor-integration/tor_provider/src/ctor/controller/commands/onion_auth_view_keys.rs ===================================== @@ -0,0 +1,14 @@ +// Licensed under the Apache License, Version 2.0, +// <http://apache.org/licenses/LICENSE-2.0> or the MIT license +// <http://opensource.org/licenses/MIT>, at your option. This file may not be +// copied, modified, or distributed except according to those terms. + +use super::Command; +use crate::ctor::controller::{parsers::parse_onion_client_auth_view, types::*, ControllerError}; + +pub fn onion_auth_view_keys() -> Result<Command<Vec<OnionClientAuth>>, ControllerError> { + Ok(Command { + command: String::from("ONION_CLIENT_AUTH_VIEW\r\n"), + handler: Box::new(parse_onion_client_auth_view), + }) +} ===================================== toolkit/components/tor-integration/tor_provider/src/ctor/controller/controller.rs ===================================== @@ -4,6 +4,8 @@ // copied, modified, or distributed except according to those terms. use bytes::Bytes; +use std::net::IpAddr; +use tinystr::TinyAsciiStr; use super::{ commands::{self, Command}, @@ -115,14 +117,73 @@ impl<CP: ControlPortInterface> TorController<CP> { self.send_command(commands::save_conf(), handler); } - pub fn get_bridges(&self, handler: Box<dyn FnOnce(Result<Vec<Bridge>, ControllerError>)>) { - self.send_command(commands::get_conf::bridges(), handler); + pub fn get_bootstrap_phase( + &self, + handler: Box<dyn FnOnce(Result<BootstrapStatus, ControllerError>)>, + ) { + self.send_command(commands::get_info::bootstrap_phase(), handler); + } + + pub fn get_socks_listeners( + &self, + handler: Box<dyn FnOnce(Result<Vec<SocksListener>, ControllerError>)>, + ) { + self.send_command(commands::get_info::net_listeners_socks(), handler); } // Circuit display + pub fn get_circuits(&self, handler: Box<dyn FnOnce(Result<Vec<Circuit>, ControllerError>)>) { + self.send_command(commands::get_info::circuits(), handler); + } + + pub fn get_node_addresses( + &self, + node: NodeFingerprint, + handler: Box<dyn FnOnce(Result<NodeAddresses, ControllerError>)>, + ) { + self.send_command(commands::get_info::node_addresses(node), handler); + } + + pub fn get_ip_country( + &self, + ip: IpAddr, + handler: Box<dyn FnOnce(Result<TinyAsciiStr<2>, ControllerError>)>, + ) { + self.send_command(commands::get_info::ip_to_country(ip), handler); + } + + pub fn get_bridges(&self, handler: Box<dyn FnOnce(Result<Vec<Bridge>, ControllerError>)>) { + self.send_command(commands::get_conf::bridges(), handler); + } + // Onion authentication + pub fn onion_auth_view_keys( + &self, + handler: Box<dyn FnOnce(Result<Vec<OnionClientAuth>, ControllerError>)>, + ) { + self.send_command(commands::onion_auth_view_keys(), handler); + } + + pub fn onion_auth_add( + &self, + address: HSAddress, + key: &[u8], + permanent: bool, + handler: Box<dyn FnOnce(Result<u16, ControllerError>)>, + ) { + self.send_command(commands::onion_auth_add(address, key, permanent), handler); + } + + pub fn onion_auth_remove( + &self, + address: HSAddress, + handler: Box<dyn FnOnce(Result<u16, ControllerError>)>, + ) { + self.send_command(commands::onion_auth_remove(address), handler); + } + // Miscellaneous pub fn get_pluggable_transports( ===================================== toolkit/components/tor-integration/tor_provider/src/ctor/controller/error.rs ===================================== @@ -5,6 +5,7 @@ use thiserror::Error; +use super::OnionValidationError; use crate::ctor::{ control_port::ControlPortError, reply_parser::{Reply, ReplyError}, @@ -34,6 +35,8 @@ pub enum ControllerError { MalformedReply(String), #[error("the argument passed to the command was not valid: {0}")] InvalidArgument(String), + #[error("invalid onion address: {0}")] + InvalidOnionAddress(#[from] OnionValidationError), } impl ControllerError { ===================================== toolkit/components/tor-integration/tor_provider/src/ctor/controller/parsers/bootstrap_status.rs ===================================== @@ -0,0 +1,267 @@ +// Licensed under the Apache License, Version 2.0, +// <http://apache.org/licenses/LICENSE-2.0> or the MIT license +// <http://opensource.org/licenses/MIT>, at your option. This file may not be +// copied, modified, or distributed except according to those terms. + +use lazy_static::lazy_static; +use regex::bytes::{Regex, RegexBuilder}; + +use super::{parse_key_value_pairs, tor_unescape}; +use crate::ctor::controller::{types::*, ControllerError}; + +lazy_static! { + static ref STATUS_REGEX: Regex = RegexBuilder::new(r"^(NOTICE|WARN) BOOTSTRAP (.+)$") + .case_insensitive(true) + .build() + .expect("The regex is hardcoded, it should be good."); +} + +const STATUS_KEYWORDS: &[&'static str] = + &["PROGRESS", "TAG", "SUMMARY", "RECOMMENDATION", "REASON"]; + +/// Parse a STATUS_CLIENT event. +/// +/// We handle events that have this format: +/// NOTICE BOOTSTRAP PROGRESS=n TAG=keyword SUMMARY="Human readable string" REASON=keyword +/// WARN BOOTSTRAP PROGRESS=n TAG=keyword SUMMARY="Human readable string" REASON=keyword +/// WARN BOOTSTRAP PROGRESS=n TAG=keyword SUMMARY="Human readable string" REASON=keyword RECOMMENDATION=warn +pub fn parse_bootstrap_status(value: &[u8]) -> Result<BootstrapStatus, ControllerError> { + let groups = STATUS_REGEX + .captures(value) + .ok_or_else(|| ControllerError::WrongFormat(String::from_utf8_lossy(value).into_owned()))?; + + let severity = groups[1].to_ascii_uppercase(); + let severity = match severity.as_slice() { + b"NOTICE" => StatusSeverity::Notice, + b"WARN" => StatusSeverity::Warn, + _ => { + // Notice: the regex should make this case impossible... + return Err(ControllerError::MalformedReply(format!( + "unknown severity: {}", + String::from_utf8_lossy(&severity) + ))); + } + }; + + let arguments = parse_key_value_pairs( + groups + .get(2) + .ok_or_else(|| { + ControllerError::MalformedReply(String::from( + "the bootstrap status does not contain the compulsory values", + )) + })? + .as_bytes(), + STATUS_KEYWORDS, + )?; + + let progress = arguments + .get("PROGRESS") + .ok_or_else(|| ControllerError::KeyNotFound(String::from("progress"))) + .and_then(|v| str::from_utf8(v).map_err(|e| ControllerError::MalformedReply(e.to_string()))) + .and_then(|v| { + v.parse().map_err(|e| { + ControllerError::MalformedReply(format!("progress is not an integer: {}", e)) + }) + })?; + if progress > 100 { + return Err(ControllerError::MalformedReply(format!( + "progress exceeds 100: {}", + progress + ))); + } + + // For robustness, we do not check this value. + let tag = String::from_utf8_lossy( + arguments + .get("TAG") + .ok_or_else(|| ControllerError::KeyNotFound(String::from("tag")))?, + ) + .into_owned(); + let summary = String::from_utf8_lossy( + &tor_unescape( + arguments + .get("SUMMARY") + .ok_or_else(|| ControllerError::KeyNotFound(String::from("summary")))?, + ) + .map_err(|e| ControllerError::MalformedReply(e.to_string()))?, + ) + .into_owned(); + let reason = arguments + .get("REASON") + .map(|r| String::from_utf8_lossy(r).into_owned()); + let should_warn = arguments + .get("RECOMMENDATION") + // Recommendation can be either "warn" or "ignore". + // We chose to match warn as we want to be sure before issuing a warn, + // as connection assist will interrupt the bootstrap. + .map_or(false, |r| r.eq_ignore_ascii_case(b"warn")); + + Ok(BootstrapStatus { + severity, + progress, + tag, + summary, + reason, + should_warn, + }) +} + +#[cfg(test)] +mod tests { + use super::*; + + use std::assert_matches; + + #[test] + fn without_recommendation() { + let line = b"NOTICE BOOTSTRAP PROGRESS=10 TAG=conn_done SUMMARY=\"Connected to a relay\""; + let status = parse_bootstrap_status(line).unwrap(); + assert_eq!(status.severity, StatusSeverity::Notice); + assert_eq!(status.progress, 10); + assert_eq!(status.tag, "conn_done"); + assert_eq!(status.summary, "Connected to a relay"); + assert!(status.reason.is_none()); + assert!(!status.should_warn); + } + + #[test] + fn with_recommendation() { + const PREFIX: &[u8] = + b"WARN BOOTSTRAP PROGRESS=14 TAG=handshake SUMMARY=\"Handshaking with a relay\" "; + { + let line = vec![PREFIX, b"RECOMMENDATION=warn"].concat(); + let status = parse_bootstrap_status(&line).unwrap(); + assert!(status.should_warn); + } + { + let line = vec![PREFIX, b"RECOMMENDATION=ignore"].concat(); + let status = parse_bootstrap_status(&line).unwrap(); + assert!(!status.should_warn); + } + { + let line = vec![PREFIX, b"RECOMMENDATION=invalid"].concat(); + let status = parse_bootstrap_status(&line).unwrap(); + assert!(!status.should_warn); + } + { + let line = vec![PREFIX, b"RECOMMENDATION="].concat(); + let status = parse_bootstrap_status(&line).unwrap(); + assert!(!status.should_warn); + } + } + + #[test] + fn with_reason() { + let line = b"WARN BOOTSTRAP PROGRESS=30 TAG=conn SUMMARY=\"Connecting to a relay\" REASON=CLOCK_SKEW"; + let status = parse_bootstrap_status(&line[..]).unwrap(); + assert_eq!(status.severity, StatusSeverity::Warn); + assert_eq!(status.progress, 30); + assert_eq!(status.tag, "conn"); + assert_eq!(status.summary, "Connecting to a relay"); + assert_eq!(status.reason.unwrap(), "CLOCK_SKEW"); + assert!(!status.should_warn); + } + + #[test] + fn case_insensitive() { + let line = b"wARn Bootstrap pRoGrEsS=10 tAg=conn_DONE sUmmaRY=\"connected TO a ReLaY\" reCOmmeNdATIOn=waRn"; + let status = parse_bootstrap_status(line).unwrap(); + assert_eq!(status.severity, StatusSeverity::Warn); + assert_eq!(status.progress, 10); + assert_eq!(status.tag, "conn_DONE"); + assert_eq!(status.summary, "connected TO a ReLaY"); + assert!(status.should_warn); + } + + #[test] + fn invalid_severity() { + let line = b"info bootstrap PROGRESS=10 TAG=test SUMMARY=\"Test\""; + assert_matches!( + parse_bootstrap_status(line).unwrap_err(), + ControllerError::WrongFormat(_) + ); + } + + #[test] + fn invalid_progress() { + // Sanity tests. + assert!( + parse_bootstrap_status(b"NOTICE bootstrap PROGRESS=0 TAG=test SUMMARY=\"Test\"") + .is_ok() + ); + assert!( + parse_bootstrap_status(b"NOTICE bootstrap PROGRESS=1 TAG=test SUMMARY=\"Test\"") + .is_ok() + ); + assert!( + parse_bootstrap_status(b"NOTICE bootstrap PROGRESS=10 TAG=test SUMMARY=\"Test\"") + .is_ok() + ); + assert!( + parse_bootstrap_status(b"NOTICE bootstrap PROGRESS=99 TAG=test SUMMARY=\"Test\"") + .is_ok() + ); + assert!( + parse_bootstrap_status(b"NOTICE bootstrap PROGRESS=100 TAG=test SUMMARY=\"Test\"") + .is_ok() + ); + + // Invalid lines. + assert_matches!( + parse_bootstrap_status(b"NOTICE bootstrap PROGRESS=aaa TAG=test SUMMARY=\"Test\"") + .unwrap_err(), + ControllerError::MalformedReply(_) + ); + assert_matches!( + parse_bootstrap_status(b"NOTICE bootstrap PROGRESS=101 TAG=test SUMMARY=\"Test\"") + .unwrap_err(), + ControllerError::MalformedReply(_) + ); + } + + #[test] + fn missing_args() { + assert_matches!( + parse_bootstrap_status(b"NOTICE bootstrap TAG=test SUMMARY=\"Test\"").unwrap_err(), + ControllerError::KeyNotFound(_) + ); + assert_matches!( + parse_bootstrap_status(b"NOTICE bootstrap PROGRESS=3 SUMMARY=\"Test\"").unwrap_err(), + ControllerError::KeyNotFound(_) + ); + assert_matches!( + parse_bootstrap_status(b"NOTICE bootstrap PROGRESS=3 TAG=test").unwrap_err(), + ControllerError::KeyNotFound(_) + ); + assert_matches!( + parse_bootstrap_status(b"NOTICE bootstrap PROGRESS=3").unwrap_err(), + ControllerError::KeyNotFound(_) + ); + assert_matches!( + parse_bootstrap_status(b"NOTICE bootstrap TAG=test").unwrap_err(), + ControllerError::KeyNotFound(_) + ); + assert_matches!( + parse_bootstrap_status(b"NOTICE bootstrap SUMMARY=\"Test\"").unwrap_err(), + ControllerError::KeyNotFound(_) + ); + } + + #[test] + fn reject_garbage() { + assert_matches!( + parse_bootstrap_status(b"UNALLOWED_PREFIX NOTICE BOOTSTRAP PROGRESS=10 TAG=conn_done SUMMARY=\"Connected to a relay\"").unwrap_err(), + ControllerError::WrongFormat(_) + ); + assert!(parse_bootstrap_status(b"NOTICE BOOTSTRAP PROGRESS=10 TAG=conn_done SUMMARY=\"Connected to a relay\" UNKNOWN_ARG=\"value\"").is_ok()); + assert!(parse_bootstrap_status(b"NOTICE BOOTSTRAP PROGRESS=10 UNKNOWN_ARG=\"value\" TAG=conn_done SUMMARY=\"Connected to a relay\"").is_ok()); + assert!(parse_bootstrap_status(b"NOTICE BOOTSTRAP PROGRESS=10 TAG=conn_done SUMMARY=\"Connected to a relay\" UNKNOWN_ARG=").is_ok()); + assert!(parse_bootstrap_status(b"NOTICE BOOTSTRAP PROGRESS=10 UNKNOWN_ARG= TAG=conn_done SUMMARY=\"Connected to a relay\"").is_ok()); + // Arguments that are not key=value pair (e.g., they are missing the + // equal sign) are rejected. + assert!(parse_bootstrap_status(b"NOTICE BOOTSTRAP INVALID_ARG PROGRESS=10 TAG=conn_done SUMMARY=\"Connected to a relay\"").is_err()); + assert!(parse_bootstrap_status(b"NOTICE BOOTSTRAP PROGRESS=10 INVALID_ARG TAG=conn_done SUMMARY=\"Connected to a relay\"").is_err()); + assert!(parse_bootstrap_status(b"NOTICE BOOTSTRAP PROGRESS=10 TAG=conn_done SUMMARY=\"Connected to a relay\" INVALID_ARG").is_err()); + } +} ===================================== toolkit/components/tor-integration/tor_provider/src/ctor/controller/parsers/bridge_line.rs ===================================== @@ -33,6 +33,9 @@ lazy_static! { .expect("The regex is hardcoded, it should be good."); } +/// Errors we might encounter when parsing the bridge line. +/// We have a specific type in case we want to re-use this parser for other +/// purposes. #[derive(Error, Debug, Clone, PartialEq, Eq)] pub enum BridgeParseError { #[error("the bridge line has a wrong format and could not be parsed")] @@ -47,6 +50,19 @@ pub enum BridgeParseError { ArgsWithoutTransport, } +/// Parse a bridge line. +/// +/// Supported lines are: +/// - <address> [<fingerprint>] +/// - transport <address> [<fingerprint>] [<optional-arguments>] +/// +/// There is not a spec for bridge lines. +/// The closest one is on the man page, but checking src/app/config/config.c of +/// c-tor reveals more information such as: +/// - "vanilla" bridges cannot have optional arguments (as they do not have a +/// pluggable transport) +/// - transports must be C identifiers (i.e., they can contain letters, digits +/// and '_', but cannot start with a number). pub fn parse_bridge_line(value: &[u8]) -> Result<Bridge, BridgeParseError> { let caps = BRIDGE_REGEX .captures(value) ===================================== toolkit/components/tor-integration/tor_provider/src/ctor/controller/parsers/circuit_status.rs ===================================== @@ -0,0 +1,327 @@ +// Licensed under the Apache License, Version 2.0, +// <http://apache.org/licenses/LICENSE-2.0> or the MIT license +// <http://opensource.org/licenses/MIT>, at your option. This file may not be +// copied, modified, or distributed except according to those terms. + +use lazy_static::lazy_static; +use regex::bytes::{Regex, RegexBuilder}; +use smallvec::SmallVec; + +use super::parse_key_value_pairs; +use crate::ctor::controller::{types::*, ControllerError}; + +lazy_static! { + // https://spec.torproject.org/control-spec/replies.html#CIRC + static ref CIRCUIT_BUILT_REGEX: Regex = RegexBuilder::new( + "^(?<CircuitID>[a-zA-Z0-9]{1,16}) BUILT (?<Path>[A-Za-z0-9,$~]+)(?: (?<Args>.*))?$", + ) + .case_insensitive(true) + .build() + .expect("The regex is hardcoded, it should be good."); + static ref NODE_REGEX: Regex = + Regex::new("^\\$(?<Fingerprint>[0-9a-fA-F]{40})(?:~[a-zA-Z0-9]+)?$") + .expect("The regex is hardcoded, it should be good."); +} + +/// The keywords we are interested in parsing for circuit statuses. +const CIRC_KEYWORDS: &[&'static str] = &["CONFLUX_ID"]; + +/// Parse CIRCUIT BUILT events. +/// We parse lines that look like `<circuitID> BUILT <path> <args>`. +/// We ignore circuits that do not match it, so that it can be used also to +/// parse the output of `GETINFO circuit-status`, as long as every line is +/// passed individually. +/// However, if a CIRCUIT looks like a BUILT but has some errors, we might +/// return an error instead (e.g., if one of the nodes in the path has a wrong +/// fingerprint length, or if the keyword-value pairs are encoded in a wrong +/// format). +/// +/// The only keyword argument we parse is the conflux ID. +pub fn parse_circuit_built(line: &[u8]) -> Result<Option<Circuit>, ControllerError> { + let Some(circuit) = CIRCUIT_BUILT_REGEX.captures(line) else { + // We might parse the status of all circuits together. + // Therefore we do not want to error when we are dealing with a status + // that is not "BUILT" but something else. + // Also, our regex forces a path to be available, even though the spec + // does not explicitly require it. However, we are already returning + // None, so we do not need to handle that case separately. + return Ok(None); + }; + + let id = CircuitID::new(&circuit["CircuitID"]).ok_or_else(|| { + ControllerError::MalformedReply(format!( + "invalid circuit id {}", + String::from_utf8_lossy(&circuit["CircuitID"]) + )) + })?; + + let path = circuit["Path"] + .split(|b| *b == b',') + .map(|node| { + let node = NODE_REGEX.captures(node).ok_or_else(|| { + ControllerError::MalformedReply(format!( + "Invalid path item: {}", + String::from_utf8_lossy(node) + )) + })?; + let mut fingerprint = NodeFingerprint::default(); + hex::decode_to_slice(&node["Fingerprint"], &mut fingerprint) + .map_err(|e| ControllerError::MalformedReply(e.to_string()))?; + Ok(fingerprint) + }) + .collect::<Result<SmallVec<_>, ControllerError>>()?; + debug_assert!( + !path.is_empty(), + "The path in the regex was not empty, if we arrived here we must have a path item." + ); + + let conflux_id = circuit + .name("Args") + .map(|args| parse_key_value_pairs(args.as_bytes(), CIRC_KEYWORDS)) + .transpose()? + .and_then(|args| { + let mut id = ConfluxID::default(); + args.get("CONFLUX_ID") + .and_then(|s| hex::decode_to_slice(s, &mut id).ok())?; + Some(id) + }); + + Ok(Some(Circuit { + id, + path, + conflux_id, + })) +} + +#[cfg(test)] +mod tests { + use super::*; + + fn fingerprint_from_hex(hex: &str) -> NodeFingerprint { + let mut fp = NodeFingerprint::default(); + hex::decode_to_slice(hex, &mut fp).unwrap(); + fp + } + + fn conflux_id_from_hex(hex: &str) -> ConfluxID { + let mut id = ConfluxID::default(); + hex::decode_to_slice(hex, &mut id).unwrap(); + id + } + + #[test] + fn parse_single_node_circuit() { + let input = b"1 BUILT $0000000000000000000000000000000000000001"; + let circuit = parse_circuit_built(input).unwrap().unwrap(); + + assert_eq!(circuit.id.as_str(), "1"); + assert_eq!(circuit.path.len(), 1); + assert_eq!( + circuit.path[0], + fingerprint_from_hex("0000000000000000000000000000000000000001") + ); + assert_eq!(circuit.conflux_id, None); + } + + #[test] + fn parse_three_node_circuit() { + let input = b"abc123 BUILT $aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa,$bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb,$cccccccccccccccccccccccccccccccccccccccc"; + let circuit = parse_circuit_built(input).unwrap().unwrap(); + + assert_eq!(circuit.id.as_str(), "abc123"); + assert_eq!(circuit.path.len(), 3); + assert_eq!( + circuit.path[0], + fingerprint_from_hex("aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa") + ); + assert_eq!( + circuit.path[1], + fingerprint_from_hex("bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb") + ); + assert_eq!( + circuit.path[2], + fingerprint_from_hex("cccccccccccccccccccccccccccccccccccccccc") + ); + } + + #[test] + fn parse_with_conflux_id() { + let input = b"qwerty999 BUILT $1111111111111111111111111111111111111111 CONFLUX_ID=0102030405060708090a0b0c0d0e0f10"; + let circuit = parse_circuit_built(input).unwrap().unwrap(); + + assert_eq!(circuit.id.as_str(), "qwerty999"); + assert_eq!(circuit.path.len(), 1); + assert_eq!( + circuit.conflux_id.unwrap(), + conflux_id_from_hex("0102030405060708090a0b0c0d0e0f10") + ); + } + + #[test] + fn parse_with_nicknames() { + let input = b"1234 BUILT $1111111111111111111111111111111111111111~relay1,$2222222222222222222222222222222222222222~relay2,$3333333333333333333333333333333333333333~relay3"; + let circuit = parse_circuit_built(input).unwrap().unwrap(); + + assert_eq!(circuit.id.as_str(), "1234"); + assert_eq!(circuit.path.len(), 3); + assert_eq!( + circuit.path[0], + fingerprint_from_hex("1111111111111111111111111111111111111111") + ); + assert_eq!( + circuit.path[1], + fingerprint_from_hex("2222222222222222222222222222222222222222") + ); + assert_eq!( + circuit.path[2], + fingerprint_from_hex("3333333333333333333333333333333333333333") + ); + } + + #[test] + fn parse_mixed_case() { + let input = b"test BuiLt $AaBbCcDdEeFf0123456789AaBbCcDdEeFf012345 conflux_ID=0102030405060708090a0b0c0d0E0F10"; + let circuit = parse_circuit_built(input).unwrap().unwrap(); + + assert_eq!(circuit.path.len(), 1); + assert_eq!( + circuit.path[0], + fingerprint_from_hex("aabbccddeeff0123456789aabbccddeeff012345") + ); + assert_eq!( + circuit.conflux_id.unwrap(), + conflux_id_from_hex("0102030405060708090a0b0c0d0e0f10") + ); + } + + #[test] + fn parse_multiple_args() { + let input = b"ID1 built $1111111111111111111111111111111111111111 PURPOSE=GENERAL CONFLUX_ID=aabbccddeeff00112233445566778899"; + let circuit = parse_circuit_built(input).unwrap().unwrap(); + + assert_eq!(circuit.id.as_str(), "ID1"); + assert_eq!( + circuit.conflux_id.unwrap(), + conflux_id_from_hex("aabbccddeeff00112233445566778899") + ); + } + + #[test] + fn long_circuit() { + let input = b"id1 BUILT $1111111111111111111111111111111111111111,$2222222222222222222222222222222222222222,$3333333333333333333333333333333333333333,$4444444444444444444444444444444444444444"; + let circuit = parse_circuit_built(input).unwrap().unwrap(); + assert_eq!(circuit.path.len(), 4); + } + + #[test] + fn not_built() { + { + let input = b"1000 EXTENDED $1111111111111111111111111111111111111111,$2222222222222222222222222222222222222222"; + assert!(parse_circuit_built(input).unwrap().is_none()); + } + { + let input = b"2000 CLOSED"; + assert!(parse_circuit_built(input).unwrap().is_none()); + } + { + let input = b"3000 GUARD_WAIT"; + assert!(parse_circuit_built(input).unwrap().is_none()); + } + } + + #[test] + fn missing_path() { + let input = b"circuit1 BUILT"; + assert!(parse_circuit_built(input).unwrap().is_none()); + } + + #[test] + fn missing_circuit_id() { + assert!( + parse_circuit_built(b"BUILT $1111111111111111111111111111111111111111") + .unwrap() + .is_none() + ); + assert!( + parse_circuit_built(b" BUILT $1111111111111111111111111111111111111111") + .unwrap() + .is_none() + ); + } + + #[test] + fn missing_status_keyword() { + let input = b"circuit1 $1111111111111111111111111111111111111111"; + assert!(parse_circuit_built(input).unwrap().is_none()); + } + + #[test] + fn invalid_circuit_id() { + { + // CircuitID must be alphanumeric only + let input = b"circuit+1 BUILT $1111111111111111111111111111111111111111"; + assert!(parse_circuit_built(input).unwrap().is_none()); + } + { + // CircuitID max length is 16 + let input = b"12345678901234567 BUILT $1111111111111111111111111111111111111111"; + assert!(parse_circuit_built(input).unwrap().is_none()); + } + } + + #[test] + fn invalid_fingerprint() { + assert!(parse_circuit_built(b"circuit1 BUILT $123456").is_err()); + assert!(parse_circuit_built( + b"circuit1 BUILT $11111111111111111111111111111111111111112222" + ) + .is_err()); + assert!( + parse_circuit_built(b"circuit1 BUILT $ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ") + .is_err() + ); + } + + #[test] + fn partially_invalid_path() { + assert!(parse_circuit_built(b"1 BUILT $1111111111111111111111111111111111111111,$222222222222222222222222222222222222222").is_err()); + assert!(parse_circuit_built(b"2 BUILT $1111111111111111111111111111111111111111,junk,$2222222222222222222222222222222222222222").is_err()); + assert!(parse_circuit_built(b"3 BUILT BUILT $1111111111111111111111111111111111111111$2222222222222222222222222222222222222222").is_err()); + assert!( + parse_circuit_built(b"4 BUILT $1111111111111111111111111111111111111111,").is_err() + ); + } + + #[test] + fn invalid_conflux_id() { + // Wrong conflux ID is not a fatal error. + { + let input = + b"circuit1 BUILT $1111111111111111111111111111111111111111 CONFLUX_ID=GGGGGGGGGGGGGGGG"; + let result = parse_circuit_built(input).unwrap().unwrap(); + assert!(result.conflux_id.is_none()); + } + { + let input = b"circuit1 BUILT $1111111111111111111111111111111111111111 CONFLUX_ID=0102"; + let result = parse_circuit_built(input).unwrap().unwrap(); + assert!(result.conflux_id.is_none()); + } + { + let input = b"circuit1 BUILT $1111111111111111111111111111111111111111 CONFLUX_ID=000000001111111122222222"; + let result = parse_circuit_built(input).unwrap().unwrap(); + assert!(result.conflux_id.is_none()); + } + } + + #[test] + fn invalid_path() { + let input = b"circuit1 BUILT $"; + assert!(parse_circuit_built(input).is_err()); + } + + #[test] + fn missing_dollar_sign() { + let input = b"circuit1 BUILT 1111111111111111111111111111111111111111"; + assert!(parse_circuit_built(input).is_err()); + } +} ===================================== toolkit/components/tor-integration/tor_provider/src/ctor/controller/parsers/client_transport_plugin.rs ===================================== @@ -19,6 +19,20 @@ lazy_static! { .expect("The regex is hardcoded, it should be good."); } +/// Parse the definition of a pluggable transport. +/// We handle these formats (from tor's man page): +/// - <transports> socks4 <ip:port> +/// - <transports> socks5 <ip:port> +/// - <transports> exec <path> [<options>] +/// +/// Note: +/// - each line can handle multiple transports (they are comma separated) +/// - IPv4 and IPv6 addresses are supported. Hostnames and Unix sockets are not +/// - we accept path with encoded strings, but tor might now. Also, the +/// conversion between bytes and Windows paths (UTF-16ish) is not well defined +/// - options can have spaces +/// - tor allows to configure multiple transport plugins. This function must be +/// called with one defition at the time pub fn parse_client_transport_plugin(line: &[u8]) -> Result<PluggableTransport, ControllerError> { if let Some(socks_line) = SOCKS_REGEX.captures(line) { parse_socks(&socks_line) @@ -31,11 +45,9 @@ pub fn parse_client_transport_plugin(line: &[u8]) -> Result<PluggableTransport, } } +/// Check and split the transports fn get_transports(c: &Captures) -> Result<Vec<String>, ControllerError> { - let transports = c - .name("transport") - .ok_or(ControllerError::KeyNotFound(String::from("transport")))? - .as_bytes() + let transports = c["transport"] .split(|b| *b == b',') .map(|t| String::from_utf8_lossy(t).into_owned()) .collect::<Vec<_>>(); @@ -47,21 +59,13 @@ fn get_transports(c: &Captures) -> Result<Vec<String>, ControllerError> { Ok(transports) } +/// Parse the socks4/socks5 cases fn parse_socks(socks_line: &Captures) -> Result<PluggableTransport, ControllerError> { - let address = str::from_utf8( - socks_line - .name("address") - .ok_or(ControllerError::KeyNotFound(String::from("address")))? - .as_bytes(), - ) - .expect("The regex matches only ASCII characters (hence valid UTF-8)") - .parse() - .map_err(|e| ControllerError::MalformedReply(format!("{}", e)))?; - let plugin = match socks_line - .name("protocol") - .ok_or(ControllerError::KeyNotFound(String::from("protocol")))? - .as_bytes() - { + let address = str::from_utf8(&socks_line["address"]) + .expect("The regex matches only ASCII characters (hence valid UTF-8)") + .parse() + .map_err(|e| ControllerError::MalformedReply(format!("{}", e)))?; + let plugin = match &socks_line["protocol"] { b"socks4" => ClientTransportPlugin::Socks4(address), b"socks5" => ClientTransportPlugin::Socks5(address), _ => unreachable!("We validated this with the regex"), @@ -72,19 +76,15 @@ fn parse_socks(socks_line: &Captures) -> Result<PluggableTransport, ControllerEr }) } +/// Parse the exec cases fn parse_exec(exec_line: &Captures) -> Result<PluggableTransport, ControllerError> { let options = exec_line .name("options") .map(|o| o.as_bytes()) .and_then(|s| if s.is_empty() { None } else { Some(s.into()) }); - let path = tor_unescape( - exec_line - .name("path") - .ok_or(ControllerError::KeyNotFound(String::from("path")))? - .as_bytes(), - ) - .map_err(|e| ControllerError::MalformedReply(e.to_string()))? - .into_owned(); + let path = tor_unescape(&exec_line["path"]) + .map_err(|e| ControllerError::MalformedReply(e.to_string()))? + .into_owned(); if path.is_empty() { return Err(ControllerError::MalformedReply(String::from("empty path"))); } ===================================== toolkit/components/tor-integration/tor_provider/src/ctor/controller/parsers/country_code.rs ===================================== @@ -0,0 +1,110 @@ +// Licensed under the Apache License, Version 2.0, +// <http://apache.org/licenses/LICENSE-2.0> or the MIT license +// <http://opensource.org/licenses/MIT>, at your option. This file may not be +// copied, modified, or distributed except according to those terms. + +use tinystr::TinyAsciiStr; + +use crate::ctor::ControllerError; + +/// Parse the country codes as passed by the +/// `GETINFO ip-to-country/<ip-address>` command. +/// The function checks that the country code is composed by two letters, or it +/// is ??. Anything else will be rejected. +/// Cases will be preserved. +pub fn parse_country_code(line: &[u8]) -> Result<TinyAsciiStr<2>, ControllerError> { + if line.len() != 2 { + return Err(ControllerError::MalformedReply(format!( + "expected two bytes (got {})", + line.len(), + ))); + } + let code = TinyAsciiStr::try_from_utf8(line) + .map_err(|e| ControllerError::MalformedReply(e.to_string()))?; + if code != "??" && !code.is_ascii_alphabetic() { + return Err(ControllerError::MalformedReply(String::from( + "country codes must be 2 letters or be ??", + ))); + } + Ok(code) +} + +#[cfg(test)] +mod tests { + use super::*; + use std::assert_matches; + + #[test] + fn country_code() { + assert_eq!(parse_country_code(b"IT").unwrap(), "IT"); + } + + #[test] + fn unknown() { + assert_eq!(parse_country_code(b"??").unwrap(), "??"); + } + + #[test] + fn case_preserved() { + assert_eq!(parse_country_code(b"aa").unwrap(), "aa"); + assert_eq!(parse_country_code(b"aB").unwrap(), "aB"); + assert_eq!(parse_country_code(b"Ba").unwrap(), "Ba"); + assert_eq!(parse_country_code(b"BB").unwrap(), "BB"); + } + + #[test] + fn too_short() { + assert_matches!( + parse_country_code(b"I").unwrap_err(), + ControllerError::MalformedReply(_) + ); + } + + #[test] + fn too_long() { + assert_matches!( + parse_country_code(b"ITA").unwrap_err(), + ControllerError::MalformedReply(_) + ); + } + + #[test] + fn not_ascii() { + assert_matches!( + parse_country_code(b"\xF0\xF0").unwrap_err(), + ControllerError::MalformedReply(_) + ); + } + + #[test] + fn empty() { + assert_matches!( + parse_country_code(b"").unwrap_err(), + ControllerError::MalformedReply(_) + ); + } + + #[test] + fn not_alphabetic() { + assert_matches!( + parse_country_code(b"12").unwrap_err(), + ControllerError::MalformedReply(_) + ); + assert_matches!( + parse_country_code(b"A2").unwrap_err(), + ControllerError::MalformedReply(_) + ); + assert_matches!( + parse_country_code(b"1B").unwrap_err(), + ControllerError::MalformedReply(_) + ); + assert_matches!( + parse_country_code(b"?c").unwrap_err(), + ControllerError::MalformedReply(_) + ); + assert_matches!( + parse_country_code(b"d?").unwrap_err(), + ControllerError::MalformedReply(_) + ); + } +} ===================================== toolkit/components/tor-integration/tor_provider/src/ctor/controller/parsers/key_value_pairs.rs ===================================== @@ -0,0 +1,189 @@ +// Licensed under the Apache License, Version 2.0, +// <http://apache.org/licenses/LICENSE-2.0> or the MIT license +// <http://opensource.org/licenses/MIT>, at your option. This file may not be +// copied, modified, or distributed except according to those terms. + +use lazy_static::lazy_static; +use regex::bytes::Regex; +use std::collections::HashMap; + +use crate::ctor::controller::ControllerError; + +lazy_static! { + // Matching the final group will assure we reject quotes in any unquoted + // string. + static ref KEY_VALUE_PAIR_REGEX: Regex = Regex::new(r#"^ *([A-Za-z0-9_]+)=("(?:\\.|[^"])*"|[^ "]*)(?: +|$)"#) + .expect("The regex is hardcoded, it should be good."); +} + +/// Get multiple key-value pairs from a line. +/// +/// Often the commands expect values to follow a certain format for the values. +/// Validation needs to be performed by the caller. +/// +/// Also, usually keywords are defined with double quotes, therefore they should +/// be treated as case-insensitive. +/// This function compares the keyword it finds with the keywords from the list +/// in a case-insensitive fashion, and uses the list's string as a key. +/// +/// All arguments are processed regardless of them being in the list, and the +/// order in allowed_keys does not influence how this function works. +/// In other words, this function does not validate the order in which the +/// key/value pairs appear in the line. +/// +/// Repeated entries are silently tolerated, and the last matching element will +/// be assigned to the returned map. However, callers should not depend on this +/// behavior. +pub(super) fn parse_key_value_pairs<'a>( + line: &'a [u8], + allowed_keys: &[&'static str], +) -> Result<HashMap<&'static str, &'a [u8]>, ControllerError> { + let mut values = HashMap::new(); + let mut pos = 0; + while pos < line.len() { + let Some(kv) = KEY_VALUE_PAIR_REGEX.captures(&line[pos..]) else { + return Err(ControllerError::MalformedReply(format!( + "invalid key/value pair: {}", + String::from_utf8_lossy(&line[pos..]) + ))); + }; + pos += kv[0].len(); + + let key = match allowed_keys + .iter() + .find(|k| kv[1].eq_ignore_ascii_case(k.as_bytes())) + { + Some(k) => *k, + None => { + continue; + } + }; + // Notice that kv[2] does not work here as it has explicitly the same + // lifetime as the Captures, rather than the original buffer's. + let value = kv.get(2).expect("Group 2 is compulsory.").as_bytes(); + values.insert(key, value); + } + Ok(values) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn single_pair() { + { + let values = parse_key_value_pairs(b"KEY=Value", &["KEY"]).unwrap(); + assert_eq!(values.len(), 1); + assert_eq!(values.get("KEY").unwrap(), b"Value"); + } + { + let values = parse_key_value_pairs(b"KEY=\"Value with quotes\"", &["KEY"]).unwrap(); + assert_eq!(values.len(), 1); + assert_eq!(values.get("KEY").unwrap(), b"\"Value with quotes\""); + } + { + let values = parse_key_value_pairs(b"KEY=\"Value\"", &["KEY"]).unwrap(); + assert_eq!(values.len(), 1); + assert_eq!(values.get("KEY").unwrap(), b"\"Value\""); + } + } + + #[test] + fn empty_value() { + let values = parse_key_value_pairs(b"KEY=", &["KEY"]).unwrap(); + assert_eq!(values.len(), 1); + assert_eq!(values.get("KEY").unwrap(), b""); + } + + #[test] + fn quotes() { + let values = parse_key_value_pairs(b"KEY=\"Value with \\\" esccape\"", &["KEY"]).unwrap(); + assert_eq!(values.len(), 1); + assert_eq!(values.get("KEY").unwrap(), b"\"Value with \\\" esccape\""); + } + + #[test] + fn test_multiple() { + let values = parse_key_value_pairs( + b"GOOD=yes QUOTED=\"Quoted value\" ANOTHER=1", + &["GOOD", "NOT_FOUND", "QUOTED", "ANOTHER"], + ) + .unwrap(); + assert_eq!(values.len(), 3); + assert_eq!(values.get("GOOD").unwrap(), b"yes"); + assert_eq!(values.get("QUOTED").unwrap(), b"\"Quoted value\""); + assert_eq!(values.get("ANOTHER").unwrap(), b"1"); + assert!(values.get("NOT_FOUND").is_none()); + } + + #[test] + fn ignore_if_unallowed() { + let values = parse_key_value_pairs(b"A=1 B=2 C=\"Quoted\"", &["A", "C"]).unwrap(); + assert_eq!(values.len(), 2); + assert_eq!(values.get("A").unwrap(), b"1"); + assert!(values.get("B").is_none()); + assert_eq!(values.get("C").unwrap(), b"\"Quoted\""); + } + + #[test] + fn only_in_allow_list() { + let values = parse_key_value_pairs(b"A=1 C=\"Quoted\"", &["A", "B", "C"]).unwrap(); + assert_eq!(values.len(), 2); + assert_eq!(values.get("A").unwrap(), b"1"); + assert!(values.get("B").is_none()); + assert_eq!(values.get("C").unwrap(), b"\"Quoted\""); + } + + #[test] + fn case_insensitive_keys() { + let values = + parse_key_value_pairs(b"A=123 tESt=value KEY=val", &["a", "TesT", "kEy"]).unwrap(); + assert_eq!(values.len(), 3); + assert_eq!(values.get("a").unwrap(), b"123"); + assert_eq!(values.get("TesT").unwrap(), b"value"); + assert_eq!(values.get("kEy").unwrap(), b"val"); + } + + #[test] + fn missing_equal_sign() { + assert!(parse_key_value_pairs(b"KEY", &["KEY"]).is_err()); + assert!(parse_key_value_pairs( + b"VALUE=something MISSING COUNT=3 QUOTED=\"Quoted value\"", + &["VALUE", "MISSING", "COUNT", "QUOTED"] + ) + .is_err()); + assert!(parse_key_value_pairs( + b"MISSING VALUE=something COUNT=3 QUOTED=\"Quoted value\"", + &["VALUE", "MISSING", "COUNT", "QUOTED"] + ) + .is_err()); + assert!(parse_key_value_pairs( + b"VALUE=something COUNT=3 QUOTED=\"Quoted value\" MISSING", + &["VALUE", "MISSING", "COUNT", "QUOTED"] + ) + .is_err()); + } + + #[test] + fn problematic_quotes() { + assert!(parse_key_value_pairs(b"KEY=\"Unbalanced quote", &["KEY"]).is_err()); + assert!(parse_key_value_pairs(b"KEY=value\"", &["KEY"]).is_err()); + assert!(parse_key_value_pairs(b"KEY=val\"ue", &["KEY"]).is_err()); + assert!(parse_key_value_pairs( + b"GOOD=yes INVALID=\"Unbalanced QUOTED=\"Quoted value\" ANOTHER=1", + &["GOOD", "INVALID", "QUOTED", "ANOTHER"] + ) + .is_err()); + assert!(parse_key_value_pairs( + b"GOOD=yes QUOTED=\"Quoted value\" INVALID=\"Unbalanced ANOTHER=1", + &["GOOD", "INVALID", "QUOTED", "ANOTHER"] + ) + .is_err()); + assert!(parse_key_value_pairs( + b"GOOD=yes QUOTED=\"Quoted value\" ANOTHER=1 INVALID=\"Unbalanced ", + &["GOOD", "INVALID", "QUOTED", "ANOTHER"] + ) + .is_err()); + } +} ===================================== toolkit/components/tor-integration/tor_provider/src/ctor/controller/parsers/mod.rs ===================================== @@ -4,16 +4,32 @@ // copied, modified, or distributed except according to those terms. mod ack; +mod bootstrap_status; mod bridge_line; +mod circuit_status; mod client_transport_plugin; +mod country_code; mod escape; +mod key_value_pairs; +mod node_address; +mod onion_client_auth_view; +mod socks_listeners; +mod stream_status; mod unescape; #[cfg(test)] mod tests; pub use ack::parse_ack; +pub use bootstrap_status::parse_bootstrap_status; pub use bridge_line::parse_bridge_line; +pub use circuit_status::parse_circuit_built; pub use client_transport_plugin::parse_client_transport_plugin; +pub use country_code::parse_country_code; pub use escape::tor_escape_into; +use key_value_pairs::parse_key_value_pairs; +pub use node_address::addresses_from_status_info; +pub use onion_client_auth_view::parse_onion_client_auth_view; +pub use socks_listeners::parse_socks_listeners; +pub use stream_status::parse_stream_sentconnect; pub use unescape::tor_unescape; ===================================== toolkit/components/tor-integration/tor_provider/src/ctor/controller/parsers/node_address.rs ===================================== @@ -0,0 +1,403 @@ +// Licensed under the Apache License, Version 2.0, +// <http://apache.org/licenses/LICENSE-2.0> or the MIT license +// <http://opensource.org/licenses/MIT>, at your option. This file may not be +// copied, modified, or distributed except according to those terms. + +use lazy_static::lazy_static; +use regex::bytes::Regex; +use std::net::{Ipv4Addr, SocketAddrV6}; + +use crate::ctor::controller::{types::*, ControllerError}; + +lazy_static! { + // First line is always: + // `r nickname identity digest PubDate PubTime IP ORPort DirPort` + // https://spec.torproject.org/dir-spec/consensus-formats.html#section:router-… + static ref R_LINE_REGEX: Regex = Regex::new( + r"^r [A-Za-z0-9]+ [A-Za-z0-9+/]+ [A-Za-z0-9+/=]+ \d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2} ([0-9\.]{7,15}) \d{1,5} \d{1,5}$", + ) + .expect("The regex is hardcoded, it should be good."); +} + +/// Parse a router status and return its IPv4 (and IPv6, if available). +/// We follow the specs from: +/// https://spec.torproject.org/dir-spec/consensus-formats.html#section:router-… +/// +/// We expect the first line to have this format: +/// `r nickname identity digest PubDate PubTime IP ORPort DirPort` +/// We only validate the allowed characters, we do not validate all the fields. +/// +/// We ignore any other lines, except the ones for IPv6 addresses, that have +/// this format: `a [<IPv6-address>]:port`. +/// Only the first valid "a" line will be picked up. +/// +/// This function parses one router at the time. +pub fn addresses_from_status_info(lines: &[&[u8]]) -> Result<NodeAddresses, ControllerError> { + if lines.is_empty() { + return Err(ControllerError::MalformedReply(String::from( + "empty status info", + ))); + } + + let Some(r_line) = R_LINE_REGEX.captures(lines[0]) else { + return Err(ControllerError::MalformedReply(format!( + "the first line cannot be parsed: {}", + String::from_utf8_lossy(lines[0]) + ))); + }; + let ipv4: Ipv4Addr = str::from_utf8(&r_line[1]) + .map_err(|e| ControllerError::MalformedReply(e.to_string()))? + .parse() + .map_err(|e| ControllerError::MalformedReply(format!("{}", e)))?; + + let ipv6 = lines[1..].iter().find_map(|line| { + if !line.starts_with(b"a ") { + return None; + } + let s = str::from_utf8(&line[2..]).ok()?; + let addrv6: SocketAddrV6 = s.parse().ok()?; + Some(*addrv6.ip()) + }); + + Ok(NodeAddresses { ipv4, ipv6 }) +} + +#[cfg(test)] +mod tests { + use super::*; + use std::{assert_matches, net::Ipv6Addr}; + + #[test] + fn happy_both() { + assert_eq!( + addresses_from_status_info(&[ + b"r nickname abAB123+/xyzXYZaaaaaaaaaaaa bbbbbbbbabAB123+/xyzXYZbbbb 2038-01-01 00:00:00 192.168.1.3 9001 0", + b"a [fdc8:08d6:f860:d6c::3100]:9001", + b"s Fast Guard HSDir Running Stable V2Dir Valid", + b"w Bandwidth=5000", + ]).unwrap(), + NodeAddresses { + ipv4: Ipv4Addr::new(192, 168, 1, 3), + ipv6: Some(Ipv6Addr::new( + 0xfdc8, 0x08d6, 0xf860, 0xd6c, 0, 0, 0, 0x3100 + )), + } + ); + assert_eq!( + addresses_from_status_info(&[ + b"r nickname aaaaaaaaaaaaaaaaaaaaaaaaaaa bbbbbbbbbbbbbbbbbbbbbbbbbbb 2038-01-01 00:00:00 192.168.1.3 9001 0", + b"s Fast Guard HSDir Running Stable V2Dir Valid", + b"w Bandwidth=5000", + b"a [fdc8:08d6:f860:d6c::3100]:9001", + ]).unwrap(), + NodeAddresses { + ipv4: Ipv4Addr::new(192, 168, 1, 3), + ipv6: Some(Ipv6Addr::new( + 0xfdc8, 0x08d6, 0xf860, 0xd6c, 0, 0, 0, 0x3100 + )), + } + ); + } + + #[test] + fn only_v4() { + assert_eq!( + addresses_from_status_info(&[b"r nickname aaaaaaaaaaaaaaaaaaaaaaaaaaa bbbbbbbbbbbbbbbbbbbbbbbbbbb 2038-01-01 00:00:00 192.168.1.3 9001 0"]).unwrap(), + NodeAddresses { + ipv4: Ipv4Addr::new(192, 168, 1, 3), + ipv6: None + } + ); + } + + #[test] + fn invalid_v6_ignored() { + assert_eq!( + addresses_from_status_info(&[ + b"r nickname aaaaaaaaaaaaaaaaaaaaaaaaaaa bbbbbbbbbbbbbbbbbbbbbbbbbbb 2038-01-01 00:00:00 192.168.1.3 9001 0", + b"a [zzzz:08d6:f860:d6c::3100]:9001", + ]).unwrap(), + NodeAddresses { + ipv4: Ipv4Addr::new(192, 168, 1, 3), + ipv6: None + } + ); + + // Missing the port is also rejected for IPv6 + assert_eq!( + addresses_from_status_info(&[ + b"r nickname aaaaaaaaaaaaaaaaaaaaaaaaaaa bbbbbbbbbbbbbbbbbbbbbbbbbbb 2038-01-01 00:00:00 192.168.1.3 9001 0", + b"a [fdc8:08d6:f860:d6c::3100]", + ]).unwrap(), + NodeAddresses { + ipv4: Ipv4Addr::new(192, 168, 1, 3), + ipv6: None + } + ); + + assert_eq!( + addresses_from_status_info(&[ + b"r nickname aaaaaaaaaaaaaaaaaaaaaaaaaaa bbbbbbbbbbbbbbbbbbbbbbbbbbb 2038-01-01 00:00:00 192.168.1.3 9001 0", + b"a", + ]).unwrap(), + NodeAddresses { + ipv4: Ipv4Addr::new(192, 168, 1, 3), + ipv6: None + } + ); + + assert_eq!( + addresses_from_status_info(&[ + b"r nickname aaaaaaaaaaaaaaaaaaaaaaaaaaa bbbbbbbbbbbbbbbbbbbbbbbbbbb 2038-01-01 00:00:00 192.168.1.3 9001 0", + b"a ", + ]).unwrap(), + NodeAddresses { + ipv4: Ipv4Addr::new(192, 168, 1, 3), + ipv6: None + } + ); + + assert_eq!( + addresses_from_status_info(&[ + b"r nickname aaaaaaaaaaaaaaaaaaaaaaaaaaa bbbbbbbbbbbbbbbbbbbbbbbbbbb 2038-01-01 00:00:00 192.168.1.3 9001 0", + b"abc [fdc8:dead:beef:d6c::3100]:9001", + ]).unwrap(), + NodeAddresses { + ipv4: Ipv4Addr::new(192, 168, 1, 3), + ipv6: None + } + ); + + assert_eq!( + addresses_from_status_info(&[ + b"r nickname aaaaaaaaaaaaaaaaaaaaaaaaaaa bbbbbbbbbbbbbbbbbbbbbbbbbbb 2038-01-01 00:00:00 192.168.1.3 9001 0", + b"a 127.0.0.1", + ]).unwrap(), + NodeAddresses { + ipv4: Ipv4Addr::new(192, 168, 1, 3), + ipv6: None + } + ); + } + + #[test] + fn multiple_a() { + assert_eq!( + addresses_from_status_info(&[ + b"r nickname aaaaaaaaaaaaaaaaaaaaaaaaaaa bbbbbbbbbbbbbbbbbbbbbbbbbbb 2038-01-01 00:00:00 192.168.1.3 9001 0", + b"a [cafe:dead:beef:0123::3100]:9001", + b"a [cafe:dead:beef:0123::3200]:9001", + ]).unwrap(), + NodeAddresses { + ipv4: Ipv4Addr::new(192, 168, 1, 3), + ipv6: Some(Ipv6Addr::new( + 0xcafe, 0xdead, 0xbeef, 0x0123, 0, 0, 0, 0x3100 + )) + } + ); + + assert_eq!( + addresses_from_status_info(&[ + b"r nickname aaaaaaaaaaaaaaaaaaaaaaaaaaa bbbbbbbbbbbbbbbbbbbbbbbbbbb 2038-01-01 00:00:00 192.168.1.3 9001 0", + b"a [cafe:dead:beef:0123::3100]:9001", + b"a [cafe:dead:xxxx:0123::3200]:9001", + ]).unwrap(), + NodeAddresses { + ipv4: Ipv4Addr::new(192, 168, 1, 3), + ipv6: Some(Ipv6Addr::new( + 0xcafe, 0xdead, 0xbeef, 0x0123, 0, 0, 0, 0x3100 + )) + } + ); + + assert_eq!( + addresses_from_status_info(&[ + b"r nickname aaaaaaaaaaaaaaaaaaaaaaaaaaa bbbbbbbbbbbbbbbbbbbbbbbbbbb 2038-01-01 00:00:00 192.168.1.3 9001 0", + b"a [cafe:dead:xxxx:0123::3200]:9001", + b"a [cafe:dead:beef:0123::3100]:9001", + ]).unwrap(), + NodeAddresses { + ipv4: Ipv4Addr::new(192, 168, 1, 3), + ipv6: Some(Ipv6Addr::new( + 0xcafe, 0xdead, 0xbeef, 0x0123, 0, 0, 0, 0x3100 + )) + } + ); + + assert_eq!( + addresses_from_status_info(&[ + b"r nickname aaaaaaaaaaaaaaaaaaaaaaaaaaa bbbbbbbbbbbbbbbbbbbbbbbbbbb 2038-01-01 00:00:00 192.168.1.3 9001 0", + b"a ", + b"a [cafe:dead:beef:0123::3100]:9001", + ]).unwrap(), + NodeAddresses { + ipv4: Ipv4Addr::new(192, 168, 1, 3), + ipv6: Some(Ipv6Addr::new( + 0xcafe, 0xdead, 0xbeef, 0x0123, 0, 0, 0, 0x3100 + )) + } + ); + + assert_eq!( + addresses_from_status_info(&[ + b"r nickname aaaaaaaaaaaaaaaaaaaaaaaaaaa bbbbbbbbbbbbbbbbbbbbbbbbbbb 2038-01-01 00:00:00 192.168.1.3 9001 0", + b"a", + b"a [cafe:dead:beef:0123::3100]:9001", + ]).unwrap(), + NodeAddresses { + ipv4: Ipv4Addr::new(192, 168, 1, 3), + ipv6: Some(Ipv6Addr::new( + 0xcafe, 0xdead, 0xbeef, 0x0123, 0, 0, 0, 0x3100 + )) + } + ); + } + + #[test] + fn invalid_r() { + assert_matches!( + addresses_from_status_info(&[]).unwrap_err(), + ControllerError::MalformedReply(_) + ); + + assert_matches!( + addresses_from_status_info(&[b"does not start with r"]).unwrap_err(), + ControllerError::MalformedReply(_) + ); + + assert_matches!( + addresses_from_status_info(&[ + b"r nick$$name aaaa bbbb 2038-01-01 00:00:00 127.0.0.1 9001 0" + ]) + .unwrap_err(), + ControllerError::MalformedReply(_) + ); + + assert_matches!( + addresses_from_status_info(&[ + b"r nickname aa!@aa bbbb 2038-01-01 00:00:00 127.0.0.1 9001 0" + ]) + .unwrap_err(), + ControllerError::MalformedReply(_) + ); + + assert_matches!( + addresses_from_status_info(&[ + b"r nickname aaaa bbbb 2038-1-01 00:00:00 127.0.0.1 9001 0" + ]) + .unwrap_err(), + ControllerError::MalformedReply(_) + ); + + assert_matches!( + addresses_from_status_info(&[ + b"r nickname aaaa bbbb 2038-01-01 00:00:00 127.0.0.1000 9001 0" + ]) + .unwrap_err(), + ControllerError::MalformedReply(_) + ); + + assert_matches!( + addresses_from_status_info(&[ + b"r nickname aaaa bbbb 2038-01-01 00:00:00 ....123 9001 0" + ]) + .unwrap_err(), + ControllerError::MalformedReply(_) + ); + + // Bad IPv4 (an octet is 3 digits but higher than 255) + assert_matches!( + addresses_from_status_info(&[ + b"r nickname aaaa bbbb 2038-01-01 00:00:00 127.0.0.300 9001 0" + ]) + .unwrap_err(), + ControllerError::MalformedReply(_) + ); + + assert_matches!( + addresses_from_status_info(&[ + b"r nickname aaaa bbbb 2000-06-08 15:10:00 1.2.3.4 100000 0" + ]) + .unwrap_err(), + ControllerError::MalformedReply(_) + ); + + assert_matches!( + addresses_from_status_info(&[ + b"rrr nickname aaaa bbbb 2000-06-08 15:10:00 1.2.3.4 1000 0" + ]) + .unwrap_err(), + ControllerError::MalformedReply(_) + ); + + assert_matches!( + addresses_from_status_info(&[ + b"garbage r nickname aaaa bbbb 2000-06-08 15:10:00 1.2.3.4 9000 0" + ]) + .unwrap_err(), + ControllerError::MalformedReply(_) + ); + + assert_matches!( + addresses_from_status_info(&[ + b"r nickname aaaa bbbb 2000-06-08 15:10:00 1.2.3.4 443 0 garbage" + ]) + .unwrap_err(), + ControllerError::MalformedReply(_) + ); + + assert_matches!( + addresses_from_status_info(&[ + b"R nickname aaaa bbbb 2000-06-08 15:10:00 1.2.3.4 443 0" + ]) + .unwrap_err(), + ControllerError::MalformedReply(_) + ); + + // Only IPv6 is not valid + assert_matches!( + addresses_from_status_info(&[b"a [fdc8:08d6:f860:d6c::3100]:9001"]).unwrap_err(), + ControllerError::MalformedReply(_) + ); + + // r must come first + assert_matches!( + addresses_from_status_info(&[ + b"a [fdc8:08d6:f860:d6c::3100]:9001", + b"r nickname aaaaaaaaaaaaaaaaaaaaaaaaaaa bbbbbbbbbbbbbbbbbbbbbbbbbbb 2038-01-01 00:00:00 192.168.1.3 9001 0", + ]).unwrap_err(), + ControllerError::MalformedReply(_) + ); + } + + #[test] + fn fields_not_validated() { + // Some fiels are not validated on purposes. + // We might change this in the future, but for now we accept them, so + // we create these tests just to remind us about this. + + // Digest is in base64. tor's convention is to use unpadded base64, but + // the specs are not explicit about it (while they are for identity). + // Let's check we accept a padded base64 for it. + assert!(addresses_from_status_info(&[ + b"r nickname aaaa bbbb= 2026-11-09 10:34:58 1.2.3.4 7777 8888" + ]) + .is_ok()); + + // No validation on date and time, except for the format. + assert!(addresses_from_status_info(&[ + b"r nickname aaaa bbbb 2038-25-50 00:00:00 1.2.3.4 7777 8888" + ]) + .is_ok()); + assert!(addresses_from_status_info(&[ + b"r nickname aaaa bbbb 2000-04-12 32:80:99 1.2.3.4 7777 8888" + ]) + .is_ok()); + + // Ports are not validated either, except for length. + assert!(addresses_from_status_info(&[ + b"r nickname aaaa bbbb 2000-10-15 12:34:56 1.2.3.4 99999 67890" + ]) + .is_ok()); + } +} ===================================== toolkit/components/tor-integration/tor_provider/src/ctor/controller/parsers/onion_client_auth_view.rs ===================================== @@ -0,0 +1,502 @@ +// Licensed under the Apache License, Version 2.0, +// <http://apache.org/licenses/LICENSE-2.0> or the MIT license +// <http://opensource.org/licenses/MIT>, at your option. This file may not be +// copied, modified, or distributed except according to those terms. + +use lazy_static::lazy_static; +use regex::Regex; + +use super::parse_key_value_pairs; +use crate::ctor::{ + controller::{types::*, ControllerError}, + reply_parser::{DetailReplyLine, Reply}, +}; + +lazy_static! { + static ref CLIENT_REGEX: Regex = Regex::new( + r"^(?i)CLIENT (?<HSAddress>[a-zA-Z2-7]{56}) (?<KeyType>[A-Za-z0-9]+):(?<PrivateKeyBlob>[A-Za-z0-9+/]+={0,3})(?: (?<Args>.+))?$" + ) + .expect("The regex is hardcoded, it should be good."); +} + +const AUTH_VIEW_HEADER: &[u8] = b"ONION_CLIENT_AUTH_VIEW"; + +/// Check the first line of the reply to ONION_CLIENT_AUTH_VIEW. +fn check_auth_view(line: Option<&DetailReplyLine>) -> Result<(), ControllerError> { + if let Some(DetailReplyLine::MidReplyLine { code, line }) = line { + if *code != 250 { + // The first thing we check is that we end with "250 OK", so this + // error code is okay. + return Err(ControllerError::MixedCodes); + } + if line.eq_ignore_ascii_case(AUTH_VIEW_HEADER) { + return Ok(()); + } + } + Err(ControllerError::MalformedReply(String::from( + "the first line must be 250-ONION_CLIENT_AUTH_VIEW.", + ))) +} + +/// Parse an individual 250-CLIENT line, which follows this format: +/// `250-CLIENT <hs-address> <key-type>:<key-blob> <kw-args>`. +/// +/// Note: +/// - hs-address is expected not to have the .onion suffix +/// - we will accept any key type, even though tor only supports x25519 +/// - key-blob is expected to be base64 (but we perform a light validation and +/// return the key as we received it) +/// - we only parse the Flags keyword argument, and only the permanent flag +/// (in a case-insensitive manner) +fn parse_client_line(data: &[u8]) -> Result<OnionClientAuth, ControllerError> { + // The reply for this command should not contain escaped strings, but only + // ASCII data. So, it must also be valid UTF-8. + let data = str::from_utf8(data).map_err(|e| ControllerError::MalformedReply(e.to_string()))?; + let groups = CLIENT_REGEX + .captures(data) + .ok_or_else(|| ControllerError::WrongFormat(data.to_string()))?; + + let address = + HSAddress::from_str(&groups["HSAddress"]).map_err(|e| ControllerError::from(e))?; + + let key_type = groups["KeyType"].into(); + let key_blob = groups["PrivateKeyBlob"].into(); + + let is_permanent = groups + .name("Args") + .map(|a| parse_key_value_pairs(a.as_str().as_bytes(), &["Flags"])) + .transpose()? + .and_then(|args| { + args.get("Flags").map(|flags| { + flags + .split(|b| *b == b',') + .any(|flag| flag.eq_ignore_ascii_case(b"Permanent")) + }) + }) + .unwrap_or(false); + + Ok(OnionClientAuth { + address, + key_type, + key_blob, + is_permanent, + }) +} + +/// Parse the full reply to a ONION_CLIENT_AUTH_VIEW command. +pub fn parse_onion_client_auth_view(reply: Reply) -> Result<Vec<OnionClientAuth>, ControllerError> { + if let Some(e) = ControllerError::from_reply(&reply) { + return Err(e); + } + if reply.end_line().code != 250 { + return Err(ControllerError::MalformedReply(format!( + "expected the end line to be a '250 OK', found '{}'", + String::from_utf8_lossy(reply.end_line().line.as_ref()) + ))); + } + + check_auth_view(reply.details().first())?; + + reply.details()[1..] + .iter() + .map(|l| match l { + DetailReplyLine::MidReplyLine { code, line } => { + if *code == 250 { + parse_client_line(line.as_ref()) + } else { + Err(ControllerError::MixedCodes) + } + } + DetailReplyLine::DataReplyLine { .. } => Err(ControllerError::MalformedReply( + String::from("Datalines are not allowed in ONION_CLIENT_AUTH_VIEW."), + )), + }) + .collect() +} + +#[cfg(test)] +mod tests { + use std::assert_matches; + + use super::*; + use crate::ctor::reply_parser::make_reply; + + #[test] + fn single_no_permanent() { + let raw = concat!( + "250-ONION_CLIENT_AUTH_VIEW\r\n", + "250-CLIENT pg6mmjiyjmcrsslvykfwnntlaru7p5svn6y2ymmju6nubxndf4pscryd x25519:abcdef1234567890+/==\r\n", + "250 OK\r\n" + ); + let reply = make_reply(raw.as_bytes()); + let result = parse_onion_client_auth_view(reply).unwrap(); + assert_eq!( + result, + vec![OnionClientAuth { + address: HSAddress::new( + b"pg6mmjiyjmcrsslvykfwnntlaru7p5svn6y2ymmju6nubxndf4pscryd" + ) + .unwrap(), + key_type: String::from("x25519"), + key_blob: String::from("abcdef1234567890+/=="), + is_permanent: false, + }] + ) + } + + #[test] + fn single_permanent() { + let raw = concat!( + "250-ONION_CLIENT_AUTH_VIEW\r\n", + "250-CLIENT pg6mmjiyjmcrsslvykfwnntlaru7p5svn6y2ymmju6nubxndf4pscryd x25519:abcdef1234567890+/== Flags=Permanent\r\n", + "250 OK\r\n" + ); + let reply = make_reply(raw.as_bytes()); + let result = parse_onion_client_auth_view(reply).unwrap(); + assert_eq!( + result, + vec![OnionClientAuth { + address: HSAddress::new( + b"pg6mmjiyjmcrsslvykfwnntlaru7p5svn6y2ymmju6nubxndf4pscryd" + ) + .unwrap(), + key_type: String::from("x25519"), + key_blob: String::from("abcdef1234567890+/=="), + is_permanent: true, + }] + ) + } + + #[test] + fn empty() { + let reply = make_reply(b"250-ONION_CLIENT_AUTH_VIEW\r\n250 OK\r\n"); + let result = parse_onion_client_auth_view(reply).unwrap(); + assert_eq!(result, vec![]); + } + + #[test] + fn multiple() { + let raw = concat!( + "250-ONION_CLIENT_AUTH_VIEW\r\n", + "250-CLIENT pg6mmjiyjmcrsslvykfwnntlaru7p5svn6y2ymmju6nubxndf4pscryd x25519:abcdef1234567890+/== Flags=Permanent\r\n", + "250-CLIENT sp3k262uwy4r2k3ycr5awluarykdpag6a7y33jxop4cs2lu5uz5sseqd x25519:thekeyhere\r\n", + "250 OK\r\n" + ); + let reply = make_reply(raw.as_bytes()); + let result = parse_onion_client_auth_view(reply).unwrap(); + assert_eq!( + result, + vec![ + OnionClientAuth { + address: HSAddress::new( + b"pg6mmjiyjmcrsslvykfwnntlaru7p5svn6y2ymmju6nubxndf4pscryd" + ) + .unwrap(), + key_type: String::from("x25519"), + key_blob: String::from("abcdef1234567890+/=="), + is_permanent: true, + }, + OnionClientAuth { + address: HSAddress::new( + b"sp3k262uwy4r2k3ycr5awluarykdpag6a7y33jxop4cs2lu5uz5sseqd" + ) + .unwrap(), + key_type: String::from("x25519"), + key_blob: String::from("thekeyhere"), + is_permanent: false, + }, + ] + ) + } + + #[test] + fn case_insensitive() { + let raw = concat!( + "250-onioN_CLIENt_autH_VIEW\r\n", + "250-CLieNt Pg6MmjiYjMcRssLvyKFWNntLARu7P5SVn6y2YMMJu6NUbxNdf4PscRYD x25519:abcdef1234567890+/== fLaGs=permAnEnt\r\n", + "250 OK\r\n" + ); + let reply = make_reply(raw.as_bytes()); + let result = parse_onion_client_auth_view(reply).unwrap(); + assert_eq!( + result, + vec![OnionClientAuth { + address: HSAddress::new( + b"pg6mmjiyjmcrsslvykfwnntlaru7p5svn6y2ymmju6nubxndf4pscryd" + ) + .unwrap(), + key_type: String::from("x25519"), + key_blob: String::from("abcdef1234567890+/=="), + is_permanent: true, + }] + ) + } + + #[test] + fn unknown_flags() { + let raw = concat!( + "250-ONION_CLIENT_AUTH_VIEW\r\n", + "250-CLIENT pg6mmjiyjmcrsslvykfwnntlaru7p5svn6y2ymmju6nubxndf4pscryd x25519:abcdef1234567890+/== Flags=SomeFlag,AnotherFlag\r\n", + "250 OK\r\n" + ); + let reply = make_reply(raw.as_bytes()); + let result = parse_onion_client_auth_view(reply).unwrap(); + assert_eq!( + result, + vec![OnionClientAuth { + address: HSAddress::new( + b"pg6mmjiyjmcrsslvykfwnntlaru7p5svn6y2ymmju6nubxndf4pscryd" + ) + .unwrap(), + key_type: String::from("x25519"), + key_blob: String::from("abcdef1234567890+/=="), + is_permanent: false, + }] + ) + } + + #[test] + fn multiple_flags() { + let raw = concat!( + "250-ONION_CLIENT_AUTH_VIEW\r\n", + "250-CLIENT pg6mmjiyjmcrsslvykfwnntlaru7p5svn6y2ymmju6nubxndf4pscryd x25519:abcdef1234567890+/== Flags=SomeFlag,Permanent,AnotherFlag\r\n", + "250 OK\r\n" + ); + let reply = make_reply(raw.as_bytes()); + let result = parse_onion_client_auth_view(reply).unwrap(); + assert_eq!( + result, + vec![OnionClientAuth { + address: HSAddress::new( + b"pg6mmjiyjmcrsslvykfwnntlaru7p5svn6y2ymmju6nubxndf4pscryd" + ) + .unwrap(), + key_type: String::from("x25519"), + key_blob: String::from("abcdef1234567890+/=="), + is_permanent: true, + }] + ) + } + + #[test] + fn client_name_ignored() { + { + let raw = concat!( + "250-ONION_CLIENT_AUTH_VIEW\r\n", + "250-CLIENT pg6mmjiyjmcrsslvykfwnntlaru7p5svn6y2ymmju6nubxndf4pscryd x25519:abcdef1234567890+/== ClientName=test\r\n", + "250 OK\r\n" + ); + let reply = make_reply(raw.as_bytes()); + assert!(parse_onion_client_auth_view(reply).is_ok()); + } + + { + let raw = concat!( + "250-ONION_CLIENT_AUTH_VIEW\r\n", + "250-CLIENT pg6mmjiyjmcrsslvykfwnntlaru7p5svn6y2ymmju6nubxndf4pscryd x25519:abcdef1234567890+/== ClientName=test Flags=Permanent\r\n", + "250 OK\r\n" + ); + let reply = make_reply(raw.as_bytes()); + let result = parse_onion_client_auth_view(reply).unwrap(); + assert!(result[0].is_permanent); + } + { + let raw = concat!( + "250-ONION_CLIENT_AUTH_VIEW\r\n", + "250-CLIENT pg6mmjiyjmcrsslvykfwnntlaru7p5svn6y2ymmju6nubxndf4pscryd x25519:abcdef1234567890+/== ClientName=test Flags=\r\n", + "250 OK\r\n" + ); + let reply = make_reply(raw.as_bytes()); + let result = parse_onion_client_auth_view(reply).unwrap(); + assert!(!result[0].is_permanent); + } + + // In the specs, Flags is listed after ClientName, but we accept any + // order. + { + let raw = concat!( + "250-ONION_CLIENT_AUTH_VIEW\r\n", + "250-CLIENT pg6mmjiyjmcrsslvykfwnntlaru7p5svn6y2ymmju6nubxndf4pscryd x25519:abcdef1234567890+/== Flags=Permanent ClientName=test\r\n", + "250 OK\r\n" + ); + let reply = make_reply(raw.as_bytes()); + let result = parse_onion_client_auth_view(reply).unwrap(); + assert!(result[0].is_permanent); + } + { + let raw = concat!( + "250-ONION_CLIENT_AUTH_VIEW\r\n", + "250-CLIENT pg6mmjiyjmcrsslvykfwnntlaru7p5svn6y2ymmju6nubxndf4pscryd x25519:abcdef1234567890+/== Flags= ClientName=test\r\n", + "250 OK\r\n" + ); + let reply = make_reply(raw.as_bytes()); + let result = parse_onion_client_auth_view(reply).unwrap(); + assert!(!result[0].is_permanent); + } + } + + #[test] + fn unsuccessful_end_code() { + { + let reply = make_reply(b"512 Syntax error\r\n"); + let result = parse_onion_client_auth_view(reply); + assert_matches!( + result.unwrap_err(), + ControllerError::TorError { code: 512, .. } + ); + } + { + let reply = make_reply(b"250-ONION_CLIENT_AUTH_VIEW\r\n512 Syntax error\r\n"); + let result = parse_onion_client_auth_view(reply); + assert_matches!( + result.unwrap_err(), + ControllerError::TorError { code: 512, .. } + ); + } + } + + #[test] + fn bad_header() { + { + let reply = make_reply(b"250 OK\r\n"); + let result = parse_onion_client_auth_view(reply); + assert_matches!(result.unwrap_err(), ControllerError::MalformedReply(_)); + } + { + let reply = make_reply(b"254-BAD HEADER\r\n250 OK\r\n"); + let result = parse_onion_client_auth_view(reply); + assert_eq!(result.unwrap_err(), ControllerError::MixedCodes); + } + { + let reply = make_reply(b"250-BAD HEADER\r\n250 OK\r\n"); + let result = parse_onion_client_auth_view(reply); + assert_matches!(result.unwrap_err(), ControllerError::MalformedReply(_)); + } + { + let reply = make_reply(b"250+DataReply Header\r\n.\r\n250 OK\r\n"); + let result = parse_onion_client_auth_view(reply); + assert_matches!(result.unwrap_err(), ControllerError::MalformedReply(_)); + } + } + + #[test] + fn datalines() { + { + let raw = concat!( + "250-ONION_CLIENT_AUTH_VIEW\r\n", + "250+CLIENT\r\n.\r\n", + "250 OK\r\n" + ); + let reply = make_reply(raw.as_bytes()); + let result = parse_onion_client_auth_view(reply); + assert_matches!(result.unwrap_err(), ControllerError::MalformedReply(_)); + } + { + let raw = concat!( + "250-ONION_CLIENT_AUTH_VIEW\r\n", + "250-CLIENT pg6mmjiyjmcrsslvykfwnntlaru7p5svn6y2ymmju6nubxndf4pscryd x25519:abcdef1234567890+/==\r\n", + "250+CLIENT\r\n.\r\n", + "250 OK\r\n" + ); + let reply = make_reply(raw.as_bytes()); + let result = parse_onion_client_auth_view(reply); + assert_matches!(result.unwrap_err(), ControllerError::MalformedReply(_)); + } + { + let raw = concat!( + "250-ONION_CLIENT_AUTH_VIEW\r\n", + "250+CLIENT\r\n.\r\n", + "250-CLIENT pg6mmjiyjmcrsslvykfwnntlaru7p5svn6y2ymmju6nubxndf4pscryd x25519:abcdef1234567890+/==\r\n", + "250 OK\r\n" + ); + let reply = make_reply(raw.as_bytes()); + let result = parse_onion_client_auth_view(reply); + assert_matches!(result.unwrap_err(), ControllerError::MalformedReply(_)); + } + } + + #[test] + fn no_250_codes() { + { + let raw = concat!( + "250-ONION_CLIENT_AUTH_VIEW\r\n", + "251-CLIENT pg6mmjiyjmcrsslvykfwnntlaru7p5svn6y2ymmju6nubxndf4pscryd x25519:abcdef1234567890+/==\r\n", + "250 OK\r\n" + ); + let reply = make_reply(raw.as_bytes()); + let result = parse_onion_client_auth_view(reply); + assert_eq!(result.unwrap_err(), ControllerError::MixedCodes); + } + { + let raw = concat!( + "250-ONION_CLIENT_AUTH_VIEW\r\n", + "250-CLIENT pg6mmjiyjmcrsslvykfwnntlaru7p5svn6y2ymmju6nubxndf4pscryd x25519:abcdef1234567890+/==\r\n", + "251-CLIENT sp3k262uwy4r2k3ycr5awluarykdpag6a7y33jxop4cs2lu5uz5sseqd x25519:abcdef1234567890+/==\r\n", + "250 OK\r\n" + ); + let reply = make_reply(raw.as_bytes()); + let result = parse_onion_client_auth_view(reply); + assert_eq!(result.unwrap_err(), ControllerError::MixedCodes); + } + { + let raw = concat!( + "250-ONION_CLIENT_AUTH_VIEW\r\n", + "251-CLIENT sp3k262uwy4r2k3ycr5awluarykdpag6a7y33jxop4cs2lu5uz5sseqd x25519:abcdef1234567890+/==\r\n", + "250-CLIENT pg6mmjiyjmcrsslvykfwnntlaru7p5svn6y2ymmju6nubxndf4pscryd x25519:abcdef1234567890+/==\r\n", + "250 OK\r\n" + ); + let reply = make_reply(raw.as_bytes()); + let result = parse_onion_client_auth_view(reply); + assert_eq!(result.unwrap_err(), ControllerError::MixedCodes); + } + { + let raw = concat!("251-ONION_CLIENT_AUTH_VIEW\r\n", "250 OK\r\n"); + let reply = make_reply(raw.as_bytes()); + let result = parse_onion_client_auth_view(reply); + assert_eq!(result.unwrap_err(), ControllerError::MixedCodes); + } + { + let raw = concat!("251-ONION_CLIENT_AUTH_VIEW\r\n", "251 OK\r\n"); + let reply = make_reply(raw.as_bytes()); + assert!(parse_onion_client_auth_view(reply).is_err()); + } + { + let raw = concat!("250-ONION_CLIENT_AUTH_VIEW\r\n", "251 OK\r\n"); + let reply = make_reply(raw.as_bytes()); + assert!(parse_onion_client_auth_view(reply).is_err()); + } + } + + #[test] + fn malformed_client_line() { + let make_error = + |r: &'static str| parse_onion_client_auth_view(make_reply(r.as_bytes())).unwrap_err(); + + assert_matches!( + make_error("250-ONION_CLIENT_AUTH_VIEW\r\n250-aaaaaaaaa\r\n250 OK\r\n"), + ControllerError::WrongFormat(_) + ); + assert_matches!( + make_error("250-ONION_CLIENT_AUTH_VIEW\r\n250-client aaaaaaaaa\r\n250 OK\r\n"), + ControllerError::WrongFormat(_) + ); + assert_matches!( + make_error( + "250-ONION_CLIENT_AUTH_VIEW\r\n250-client aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa x25519:aaaa\r\n250 OK\r\n" + ), + ControllerError::InvalidOnionAddress(_) + ); + assert_matches!( + make_error( + "250-ONION_CLIENT_AUTH_VIEW\r\n250-CLIENT sp3k262uwy4r2k3ycr5awluarykdpag6a7y33jxop4cs2lu5uz5sseqd x25519:not_base_64!!\r\n250 OK\r\n" + ), + ControllerError::WrongFormat(_) + ); + assert_matches!( + make_error( + "250-ONION_CLIENT_AUTH_VIEW\r\n250-CLIENT sp3k262uwy4r2k3ycr5awluarykdpag6a7y33jxop4cs2lu5uz5sseqd x25519:aaaa KeyWithoutEqual\r\n250 OK\r\n" + ), + ControllerError::MalformedReply(_) + ); + } +} ===================================== toolkit/components/tor-integration/tor_provider/src/ctor/controller/parsers/socks_listeners.rs ===================================== @@ -0,0 +1,215 @@ +// Licensed under the Apache License, Version 2.0, +// <http://apache.org/licenses/LICENSE-2.0> or the MIT license +// <http://opensource.org/licenses/MIT>, at your option. This file may not be +// copied, modified, or distributed except according to those terms. + +use lazy_static::lazy_static; +use regex::bytes::Regex; +use std::path::PathBuf; + +use super::tor_unescape; +use crate::ctor::controller::{types::*, ControllerError}; + +lazy_static! { + // This regular expression does not include $ on purpose, as we are looping + // on all values: we get the first quoted value and consume any spaces at + // end. Then we feed the rest of the string to the same regex, and if it + // fails it means there was garbage and we should reject the entire reply. + static ref LISTENERS_REGEX: Regex = Regex::new(r#"^ *("(?:[^"\\]|\\.)+")( +|$)"#) + .expect("The regex is hardcoded, it should be good."); +} + +/// Parse a Unix-domain socket path on Windows. +/// Since they are not supported on Windows, we return None. +#[cfg(target_os = "windows")] +fn make_path(_buf: &[u8]) -> Option<PathBuf> { + None +} + +/// Parse a Unix-domain socket on a non-Windows platform. +/// This function always returns a Some(<path>), even when the path is empty. +#[cfg(not(target_os = "windows"))] +fn make_path(buf: &[u8]) -> Option<PathBuf> { + use std::{ffi::OsStr, os::unix::ffi::OsStrExt}; + Some(PathBuf::from(OsStr::from_bytes(buf))) +} + +/// Parse a single SOCKS listener. +fn parse_listener(listener: &[u8]) -> Option<Result<SocksListener, ControllerError>> { + let addr = match tor_unescape(listener) { + Ok(a) => a, + Err(e) => { + return Some(Err(ControllerError::MalformedReply(e.to_string()))); + } + }; + if addr.starts_with(b"unix:/") { + make_path(&addr[5..]).map(|p| Ok(SocksListener::UnixSocket(p))) + } else { + Some( + str::from_utf8(&*addr) + .map_err(|e| ControllerError::MalformedReply(format!("{}", e))) + .and_then(|s| { + s.parse() + .map_err(|e| ControllerError::MalformedReply(format!("{}", e))) + }) + .map(|addr| SocksListener::Tcp(addr)), + ) + } +} + +/// Parse the output of a `GETINFO net/listeners/socks` command. +/// The reply is expected to be a single line which can contain multiple +/// listeners (each one quoted and separated by space). +/// +/// A reply without listeners is valid (it will return an empty list) and +/// expected before the bootstrap. +/// +/// We support IPv4 and IPv6 listeners on all platforms. +/// On non-Windows platforms, we support also Unix-domain sockets. +/// On Windows we parse them but we do not add them to the list. +pub fn parse_socks_listeners(line: &[u8]) -> Result<Vec<SocksListener>, ControllerError> { + let mut listeners = Vec::new(); + + // We use a manual loop because we want to make sure that items are head to + // tail and we are not simply ignoring invalid content. + let mut pos = 0; + while pos < line.len() { + let remaining = &line[pos..]; + match LISTENERS_REGEX.captures(remaining) { + Some(cap) => { + if let Some(l) = parse_listener(&cap[1]) { + listeners.push(l?); + } + pos += cap[0].len(); + } + None => { + return Err(ControllerError::WrongFormat( + String::from_utf8_lossy(line).into_owned(), + )); + } + } + } + + Ok(listeners) +} + +#[cfg(test)] +mod tests { + use super::*; + use std::net::{Ipv4Addr, SocketAddrV4, SocketAddrV6}; + + #[test] + fn single_ipv4() { + assert_eq!( + parse_socks_listeners(b"\"127.0.0.1:9150\"").unwrap(), + vec![SocksListener::Tcp( + SocketAddrV4::new(Ipv4Addr::new(127, 0, 0, 1), 9150).into() + )] + ); + } + + #[test] + fn single_ipv6() { + let addr_v6: SocketAddrV6 = "[2001:0db8:7000:0000:0000:dead:beef:1234]:9150" + .parse() + .unwrap(); + assert_eq!( + parse_socks_listeners(b"\"[2001:0db8:7000:0000:0000:dead:beef:1234]:9150\"").unwrap(), + vec![SocksListener::Tcp(addr_v6.into())] + ); + } + + #[test] + fn single_unix() { + let line = b"\"unix:/tmp/test.sock\""; + if cfg!(target_os = "windows") { + assert_eq!(parse_socks_listeners(line).unwrap(), vec![]); + } else { + assert_eq!( + parse_socks_listeners(line).unwrap(), + vec![SocksListener::UnixSocket(PathBuf::from("/tmp/test.sock"))] + ); + } + } + + #[test] + fn no_listeners() { + assert_eq!(parse_socks_listeners(b"").unwrap(), vec![]); + } + + #[test] + fn multiple_listeners() { + let mut listeners = vec![ + SocksListener::Tcp(SocketAddrV4::new(Ipv4Addr::new(127, 0, 0, 1), 9050).into()), + SocksListener::Tcp(SocketAddrV4::new(Ipv4Addr::new(127, 0, 0, 1), 9150).into()), + ]; + if cfg!(not(target_os = "windows")) { + listeners.push(SocksListener::UnixSocket(PathBuf::from("/tmp/test.sock"))); + } + assert_eq!( + parse_socks_listeners(b"\"127.0.0.1:9050\" \"127.0.0.1:9150\" \"unix:/tmp/test.sock\"") + .unwrap(), + listeners + ); + } + + #[test] + fn unbalanced_quotes() { + assert!(parse_socks_listeners(b"\"127.0.0.1:9050").is_err()); + assert!(parse_socks_listeners(b"127.0.0.1:9050\"").is_err()); + assert!(parse_socks_listeners(b"\"127.0.0.1:9050\" \"127.0.0.1:9051").is_err()); + assert!( + parse_socks_listeners(b"\"127.0.0.1:9050\" \"127.0.0.1:9051 \"127.0.0.1:9052\"") + .is_err() + ); + } + + #[test] + fn noise_in_reply() { + assert!(parse_socks_listeners(b"garbage \"127.0.0.1:9050\"").is_err()); + assert!(parse_socks_listeners(b"\"127.0.0.1:9050\" garbage").is_err()); + assert!(parse_socks_listeners(b"garbage \"127.0.0.1:9050\" garbage").is_err()); + assert!(parse_socks_listeners(b"\"127.0.0.1:9050\" garbage \"127.0.0.1:9051\"").is_err()); + assert!( + parse_socks_listeners(b"garbage \"127.0.0.1:9050\" garbage \"127.0.0.1:9051\"") + .is_err() + ); + assert!( + parse_socks_listeners(b"\"127.0.0.1:9050\" garbage \"127.0.0.1:9051\" garbage") + .is_err() + ); + assert!(parse_socks_listeners( + b"garbage \"127.0.0.1:9050\" garbage \"127.0.0.1:9051\" garbage" + ) + .is_err()); + assert!(parse_socks_listeners(b"\"127.0.0.1:9050\" 127.0.0.1:9150").is_err()); + } + + #[test] + fn spaces() { + assert!(parse_socks_listeners(b"\"127.0.0.1:9050\"\"127.0.0.1:9150\"").is_err()); + assert_eq!( + parse_socks_listeners(b"\"127.0.0.1:9050\" \"127.0.0.1:9150\"").unwrap(), + vec![ + SocksListener::Tcp(SocketAddrV4::new(Ipv4Addr::new(127, 0, 0, 1), 9050).into()), + SocksListener::Tcp(SocketAddrV4::new(Ipv4Addr::new(127, 0, 0, 1), 9150).into()) + ] + ); + assert_eq!( + parse_socks_listeners(b" \"127.0.0.1:9050\" \"127.0.0.1:9150\" ").unwrap(), + vec![ + SocksListener::Tcp(SocketAddrV4::new(Ipv4Addr::new(127, 0, 0, 1), 9050).into()), + SocksListener::Tcp(SocketAddrV4::new(Ipv4Addr::new(127, 0, 0, 1), 9150).into()) + ] + ); + } + + #[test] + fn invalid() { + assert!(parse_socks_listeners(b"\"127.0.0.1:9050\" \"127.0.0.1:100000\"").is_err()); + assert!(parse_socks_listeners(b"\"127.0.0.1:100000\" \"127.0.0.1:9050\"").is_err()); + assert!(parse_socks_listeners(b"\"127.0.0.1\"").is_err()); + assert!(parse_socks_listeners(b"\"localhost:9050\"").is_err()); + assert!(parse_socks_listeners(b"\"\"").is_err()); + } +} ===================================== toolkit/components/tor-integration/tor_provider/src/ctor/controller/parsers/stream_status.rs ===================================== @@ -0,0 +1,296 @@ +// Licensed under the Apache License, Version 2.0, +// <http://apache.org/licenses/LICENSE-2.0> or the MIT license +// <http://opensource.org/licenses/MIT>, at your option. This file may not be +// copied, modified, or distributed except according to those terms. + +use lazy_static::lazy_static; +use regex::bytes::Regex; + +use super::{parse_key_value_pairs, tor_unescape}; +use crate::ctor::controller::{CircuitID, ControllerError, Stream, StreamID}; + +const STREAM_EVENT: &[u8] = b"SENTCONNECT"; + +lazy_static! { + // https://spec.torproject.org/control-spec/replies.html#STREAM + static ref STATUS_REGEX: Regex = Regex::new( + "^(?<StreamID>[a-zA-Z0-9]{1,16}) (?<StreamStatus>[A-Za-z_]+) (?<CircuitID>[a-zA-Z0-9]{1,16}) (?:[^ ]+)(?: (?<Args>.*))?$" + ) + .expect("The regex is hardcoded, it should be good."); +} + +const USERNAME_KEYWORD: &'static str = "SOCKS_USERNAME"; +const PASSWORD_KEYWORD: &'static str = "SOCKS_PASSWORD"; +const STREAM_KEYWORDS: &[&'static str] = &[USERNAME_KEYWORD, PASSWORD_KEYWORD]; + +/// Parse a stream update event. +/// Only the SENTCONNECT events will be parsed, the other ones will be ignored. +pub fn parse_stream_sentconnect(line: &[u8]) -> Result<Option<Stream>, ControllerError> { + let stream = STATUS_REGEX + .captures(line) + .ok_or_else(|| ControllerError::WrongFormat(String::from_utf8_lossy(line).into_owned()))?; + + if !stream["StreamStatus"].eq_ignore_ascii_case(STREAM_EVENT) { + return Ok(None); + } + + let stream_id = StreamID::new(&stream["StreamID"]).ok_or_else(|| { + ControllerError::MalformedReply(format!( + "invalid stream ID '{}'", + String::from_utf8_lossy(&stream["StreamID"]) + )) + })?; + let circuit_id = CircuitID::new(&stream["CircuitID"]).ok_or_else(|| { + ControllerError::MalformedReply(format!( + "invalid circuit ID '{}'", + String::from_utf8_lossy(&stream["CircuitID"]) + )) + })?; + + let args = stream + .name("Args") + .map(|a| parse_key_value_pairs(a.as_bytes(), STREAM_KEYWORDS)) + .transpose()? + .unwrap_or_default(); + let get_credential = |name| { + args.get(name) + .map(|c| tor_unescape(c).map(|c| c.into_owned())) + .transpose() + .map_err(|e| ControllerError::MalformedReply(e.to_string())) + }; + let socks_username = get_credential(USERNAME_KEYWORD)?; + let socks_password = get_credential(PASSWORD_KEYWORD)?; + + Ok(Some(Stream { + stream_id, + circuit_id, + socks_username, + socks_password, + })) +} + +#[cfg(test)] +mod tests { + use std::assert_matches; + + use super::*; + + #[test] + fn simple() { + let line = b"1 SENTCONNECT 0 1.2.3.4:443"; + let result = parse_stream_sentconnect(line); + let stream = result.unwrap().unwrap(); + assert_eq!(stream.stream_id, StreamID::new(b"1").unwrap()); + assert_eq!(stream.circuit_id, CircuitID::new(b"0").unwrap()); + assert_eq!(stream.socks_username, None); + assert_eq!(stream.socks_password, None); + } + + #[test] + fn with_socks_credentials() { + let line = + b"42 SENTCONNECT 100 (Tor_internal):80 SOCKS_USERNAME=\"user\" SOCKS_PASSWORD=\"pass\""; + let result = parse_stream_sentconnect(line); + let stream = result.unwrap().unwrap(); + assert_eq!(stream.stream_id, StreamID::new(b"42").unwrap()); + assert_eq!(stream.circuit_id, CircuitID::new(b"100").unwrap()); + assert_eq!(stream.socks_username, Some(Vec::from("user"))); + assert_eq!(stream.socks_password, Some(Vec::from("pass"))); + } + + #[test] + fn only_username() { + let line = b"5 SENTCONNECT 10 10.20.30.40:443 SOCKS_USERNAME=\"alice\""; + let result = parse_stream_sentconnect(line); + let stream = result.unwrap().unwrap(); + assert_eq!(stream.stream_id, StreamID::new(b"5").unwrap()); + assert_eq!(stream.circuit_id, CircuitID::new(b"10").unwrap()); + assert_eq!(stream.socks_username, Some(Vec::from("alice"))); + assert_eq!(stream.socks_password, None); + } + + #[test] + fn only_password() { + let line = b"7 SENTCONNECT 20 192.168.1.2:22 SOCKS_PASSWORD=\"secret123\""; + let result = parse_stream_sentconnect(line); + let stream = result.unwrap().unwrap(); + assert_eq!(stream.stream_id, StreamID::new(b"7").unwrap()); + assert_eq!(stream.circuit_id, CircuitID::new(b"20").unwrap()); + assert_eq!(stream.socks_username, None); + assert_eq!(stream.socks_password, Some(Vec::from("secret123"))); + } + + #[test] + fn valid_but_other_status() { + let line = b"1 DETACHED 0 172.16.10.2:8080"; + let result = parse_stream_sentconnect(line); + assert!(result.unwrap().is_none()); + } + + #[test] + fn status_case_insensitive() { + let line = b"1 sentconnect 0 (Tor_internal):22 socks_username=USER socks_password=PASS"; + let result = parse_stream_sentconnect(line); + let stream = result.unwrap().unwrap(); + assert_eq!(stream.stream_id, StreamID::new(b"1").unwrap()); + assert_eq!(stream.circuit_id, CircuitID::new(b"0").unwrap()); + assert_eq!(stream.socks_username.unwrap(), b"USER"); + assert_eq!(stream.socks_password.unwrap(), b"PASS"); + } + + #[test] + fn status_mixed_case() { + let line = b"1 SentConnect 0 192.168.4.5:22 SOcKs_USeRNAMe=User soCKS_paSSWoRd=paSs"; + let result = parse_stream_sentconnect(line); + let stream = result.unwrap().unwrap(); + assert_eq!(stream.stream_id, StreamID::new(b"1").unwrap()); + assert_eq!(stream.circuit_id, CircuitID::new(b"0").unwrap()); + assert_eq!(stream.socks_username.unwrap(), b"User"); + assert_eq!(stream.socks_password.unwrap(), b"paSs"); + } + + #[test] + fn status_unescape_error() { + assert!(parse_stream_sentconnect( + b"5 SENTCONNECT 10 10.20.30.40:443 SOCKS_USERNAME=\"alice\\xyz\"" + ) + .is_err()); + } + + #[test] + fn large_ids() { + { + let line = b"9999999999999999 SENTCONNECT 1 5.6.7.8:80"; + let result = parse_stream_sentconnect(line); + let stream = result.unwrap().unwrap(); + assert_eq!( + stream.stream_id, + StreamID::new(b"9999999999999999").unwrap() + ); + assert_eq!(stream.circuit_id, CircuitID::new(b"1").unwrap()); + } + { + let line = b"2 SENTCONNECT 1234567890123456 5.6.7.8:80"; + let result = parse_stream_sentconnect(line); + let stream = result.unwrap().unwrap(); + assert_eq!(stream.stream_id, StreamID::new(b"2").unwrap()); + assert_eq!( + stream.circuit_id, + CircuitID::new(b"1234567890123456").unwrap() + ); + } + } + + #[test] + fn escaped_credentials() { + let line = + b"1 SENTCONNECT 0 (Tor_internal):80 SOCKS_USERNAME=\"user\\x20example\" SOCKS_PASSWORD=\"pass\\167ord\""; + let result = parse_stream_sentconnect(line); + let stream = result.unwrap().unwrap(); + assert_eq!(stream.socks_username.unwrap(), b"user example"); + assert_eq!(stream.socks_password.unwrap(), b"password"); + } + + #[test] + fn alphanumeric_ids() { + let line = b"abcX1y2Z3 SENTCONNECT ABCxyz 10.10.10.10:80"; + let result = parse_stream_sentconnect(line); + let stream = result.unwrap().unwrap(); + assert_eq!(stream.stream_id, StreamID::new(b"abcX1y2Z3").unwrap()); + assert_eq!(stream.circuit_id, CircuitID::new(b"ABCxyz").unwrap()); + } + + #[test] + fn stream_targets() { + // Valid: IPv4 + assert!( + parse_stream_sentconnect(b"1234 SENTCONNECT 5678 10.10.10.10:80") + .unwrap() + .is_some() + ); + // Valid: IPv6 + assert!( + parse_stream_sentconnect(b"1 SENTCONNECT 2 [cafe:dead:beef:0123::3200]:443") + .unwrap() + .is_some() + ); + // Valid: hostname + assert!(parse_stream_sentconnect(b"1 SENTCONNECT 2 example.org:443") + .unwrap() + .is_some()); + // Valid: (Tor_internal) literal + assert!( + parse_stream_sentconnect(b"1 SENTCONNECT 2 (Tor_internal):22") + .unwrap() + .is_some() + ); + + // Invalid (no port) but accepted anyway for now + assert!( + parse_stream_sentconnect(b"1234 SENTCONNECT 5678 12.34.56.78") + .unwrap() + .is_some() + ); + assert!( + parse_stream_sentconnect(b"1 SENTCONNECT 2 [cafe:dead:beef:0123::3200]") + .unwrap() + .is_some() + ); + assert!(parse_stream_sentconnect(b"1 SENTCONNECT 2 example.net") + .unwrap() + .is_some()); + assert!(parse_stream_sentconnect(b"1 SENTCONNECT 2 (Tor_internal)") + .unwrap() + .is_some()); + } + + #[test] + fn empty() { + let line = b""; + let result = parse_stream_sentconnect(line); + assert_matches!(result.unwrap_err(), ControllerError::WrongFormat(_)); + } + + #[test] + fn invalid() { + let line = b"invalid input"; + let result = parse_stream_sentconnect(line); + assert_matches!(result.unwrap_err(), ControllerError::WrongFormat(_)); + } + + #[test] + fn missing_status() { + let line = b"1 0 10.0.0.1:80"; + let result = parse_stream_sentconnect(line); + assert_matches!(result.unwrap_err(), ControllerError::WrongFormat(_)); + } + + #[test] + fn invalid_ids() { + { + let line = b"123+ SENTCONNECT 0 10.0.0.1:80"; + let result = parse_stream_sentconnect(line); + assert_matches!(result.unwrap_err(), ControllerError::WrongFormat(_)); + } + { + let line = b"+++ SENTCONNECT 0 10.0.0.1:80"; + let result = parse_stream_sentconnect(line); + assert_matches!(result.unwrap_err(), ControllerError::WrongFormat(_)); + } + { + let line = b"12345678901234567 SENTCONNECT 0 10.0.0.1:80"; + let result = parse_stream_sentconnect(line); + assert_matches!(result.unwrap_err(), ControllerError::WrongFormat(_)); + } + { + let line = b"1 SENTCONNECT 123+-+ 10.0.0.1:80"; + let result = parse_stream_sentconnect(line); + assert_matches!(result.unwrap_err(), ControllerError::WrongFormat(_)); + } + { + let line = b"1 SENTCONNECT +-+ 10.0.0.1:80"; + let result = parse_stream_sentconnect(line); + assert_matches!(result.unwrap_err(), ControllerError::WrongFormat(_)); + } + } +} ===================================== toolkit/components/tor-integration/tor_provider/src/ctor/controller/parsers/unescape.rs ===================================== @@ -21,7 +21,8 @@ pub enum UnescapeError { } /// Adaptation of tor's unescape_string (src/lib/encoding/cstring.c). -/// The main difference is that we allow unquoted strings. +/// The main difference is that we allow unquoted strings, as long as they do +/// not contain spaces. pub fn tor_unescape<'a>(buf: &'a [u8]) -> Result<Cow<'a, [u8]>, UnescapeError> { match buf.first() { Some(b'"') => { ===================================== toolkit/components/tor-integration/tor_provider/src/ctor/controller/types.rs ===================================== @@ -3,9 +3,49 @@ // <http://opensource.org/licenses/MIT>, at your option. This file may not be // copied, modified, or distributed except according to those terms. -use serde::Serialize; -use std::net::SocketAddr; +use data_encoding::{DecodeError, BASE32_NOPAD}; +use serde::{Serialize, Serializer}; +use sha3::{Digest, Sha3_256}; +use smallvec::SmallVec; +use std::{ + fmt, + net::{Ipv4Addr, Ipv6Addr, SocketAddr}, + path::PathBuf, +}; +use thiserror::Error; +use tinystr::TinyAsciiStr; +/// The severity of STATUS_CLIENT updates. It is the first token: +/// +/// NOTICE BOOTSTRAP PROGRESS=n ... +/// WARN BOOTSTRAP PROGRESS=n ... +#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize)] +pub enum StatusSeverity { + Notice, + Warn, +} + +impl ToString for StatusSeverity { + fn to_string(&self) -> String { + match self { + StatusSeverity::Notice => String::from("NOTICE"), + StatusSeverity::Warn => String::from("WARN"), + } + } +} + +/// Data structure with the information associated to a STATUS_CLIENT event. +#[derive(Debug, Clone, PartialEq, Eq, Serialize)] +pub struct BootstrapStatus { + pub severity: StatusSeverity, + pub progress: u8, + pub tag: String, + pub summary: String, + pub reason: Option<String>, + pub should_warn: bool, +} + +/// Information extracted from a bridge line. #[derive(Debug, Clone, PartialEq, Eq, Serialize)] pub struct Bridge { pub transport: Option<String>, @@ -14,6 +54,39 @@ pub struct Bridge { pub args: Option<Vec<u8>>, } +/// A circuit ID. It can be an alphanumeric string up to 16 characters. +/// At the moment, comparison between them is case-sensitive, but tor uses only +/// digits. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize)] +pub struct CircuitID(TinyAsciiStr<16>); + +impl CircuitID { + pub fn new(data: &[u8]) -> Option<Self> { + if data.is_empty() || data.iter().any(|c| !c.is_ascii_alphanumeric()) { + return None; + } + let s = str::from_utf8(data).ok()?.parse().ok()?; + Some(Self(s)) + } + + pub fn as_str(&self) -> &str { + self.0.as_str() + } +} + +/// A conflux cookie, used to group conflux circuits. +pub type ConfluxID = [u8; 16]; + +#[derive(Debug, Clone, PartialEq, Eq, Serialize)] +pub struct Circuit { + pub id: CircuitID, + pub path: SmallVec<[NodeFingerprint; 3]>, + pub conflux_id: Option<ConfluxID>, +} + +/// The definition of a handler for a pluggable transport. +/// The conversion between 8-bit characters and 16-bit characters on Windows is +/// not well defined. #[derive(Debug, Clone, PartialEq, Eq, Serialize)] pub enum ClientTransportPlugin { Executable { @@ -24,6 +97,10 @@ pub enum ClientTransportPlugin { Socks5(SocketAddr), } +/// A configuration value to be used with the SETCONF command. +/// Boolean will be converted to 1 and 0. +/// Null can be used to reset a certain value. When building the SETCONF +/// command, we will add `<key>`, rather than `<key>=<value>` or `<key>=`. #[derive(Debug, Clone)] pub enum ConfValue<'a> { Bool(bool), @@ -33,8 +110,243 @@ pub enum ConfValue<'a> { Null, } +/// Onion service address (in the canonical base32 representation, without the +/// .onion suffix). +/// The address is validated on construction, to make sure it adheres to the +/// spec: https://spec.torproject.org/rend-spec/encoding-onion-addresses.html. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct HSAddress([u8; 56]); + +impl HSAddress { + pub fn new(base32_address: &[u8; 56]) -> Result<Self, OnionValidationError> { + validate_onion_address(base32_address).map(Self) + } + + pub fn from_slice(base32_address: &[u8]) -> Result<Self, OnionValidationError> { + Self::new( + base32_address + .try_into() + .map_err(|_| OnionValidationError::InvalidLength)?, + ) + } + + pub fn from_str(base32_address: &str) -> Result<Self, OnionValidationError> { + Self::from_slice(base32_address.as_bytes()) + } +} + +impl AsRef<str> for HSAddress { + fn as_ref(&self) -> &str { + str::from_utf8(&self.0).expect( + "We validated the address on the constructor, therefore it must be valid UTF-8.", + ) + } +} + +impl Serialize for HSAddress { + fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error> + where + S: Serializer, + { + serializer.serialize_str(self.as_ref()) + } +} + +/// Possible errors that can happen when validating an onion address. +#[derive(Error, Debug, Clone, PartialEq, Eq)] +pub enum OnionValidationError { + #[error("Invalid length")] + InvalidLength, + #[error("cannot decode the address")] + CannotDecode(#[from] DecodeError), + #[error("unsupported version: {0}")] + UnsupportedVersion(u8), + #[error("bad checksum")] + BadChecksum, +} + +/// Check the validity of an onion address. +/// +/// https://spec.torproject.org/rend-spec/encoding-onion-addresses.html +fn validate_onion_address(base32_address: &[u8; 56]) -> Result<[u8; 56], OnionValidationError> { + let mut uppercased = *base32_address; + uppercased.make_ascii_uppercase(); + let decoded = BASE32_NOPAD + .decode(&uppercased) + .map_err(|e| OnionValidationError::from(e))?; + debug_assert_eq!( + decoded.len(), + 35, + "We are using fixed size, how can the length be wrong?!" + ); + + let pubkey = &decoded[0..32]; + let stored_checksum = &decoded[32..34]; + let version = decoded[34]; + + if version != 3 { + return Err(OnionValidationError::UnsupportedVersion(version)); + } + + let hash = { + let mut hasher = Sha3_256::new(); + hasher.update(b".onion checksum"); + hasher.update(pubkey); + hasher.update(&[version]); + hasher.finalize() + }; + + let calculated_checksum = &hash[0..2]; + if stored_checksum != calculated_checksum { + return Err(OnionValidationError::BadChecksum); + } + + Ok(uppercased) +} + +/// The information about the IP addresses of a relay. +/// At the moment, all realys must have an IPv4 address, and tor will use at +/// most one IPv6 address (which is optional). +#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize)] +pub struct NodeAddresses { + pub ipv4: Ipv4Addr, + pub ipv6: Option<Ipv6Addr>, +} + +/// The fingerprint of a node (as bytes, i.e., decoded from the 40 hex digits +/// format). +pub type NodeFingerprint = [u8; 20]; + +/// A private key for onion authentication. +/// The key_blob is a base64 string, rather than a binary blob. +#[derive(Clone, PartialEq, Eq, Serialize)] +pub struct OnionClientAuth { + pub address: HSAddress, + pub key_type: String, + pub key_blob: String, + pub is_permanent: bool, +} + +impl fmt::Debug for OnionClientAuth { + /// Explicit implementation of fmt::Debug for OnionClientAuth that redacts + /// the private key. + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.debug_struct("OnionClientAuth") + .field("address", &self.address) + .field("key_type", &self.key_type) + .field("key_blob", &"[redacted]") + .field("is_permanent", &self.is_permanent) + .finish() + } +} + +/// A pluggable transport, with its handler and the transport it can manages. #[derive(Debug, Clone, PartialEq, Eq, Serialize)] pub struct PluggableTransport { pub transports: Vec<String>, pub plugin: ClientTransportPlugin, } + +/// A listener for the SOCKS port. +/// On Windows, only TCP listeners are supported. +#[derive(Debug, PartialEq, Eq, Serialize)] +pub enum SocksListener { + Tcp(SocketAddr), + UnixSocket(PathBuf), +} + +/// The bare minimum information we need about a stream to associate some +/// isolation credentials to a circuit. +/// +/// Notice: since tor uses SOCKS passwords as isolation parameters rather than +/// as real passwords, we do not redact them, and we use the default Debug +/// implementation. +#[derive(Debug, Clone, PartialEq, Eq, Serialize)] +pub struct Stream { + pub stream_id: StreamID, + pub circuit_id: CircuitID, + pub socks_username: Option<Vec<u8>>, + pub socks_password: Option<Vec<u8>>, +} + +/// A stream ID. It can be an alphanumeric string up to 16 characters. +/// At the moment, comparison between them is case-sensitive, but tor uses only +/// digits. +/// Circuit IDs are defined in the same way, but we do not define them as alias +/// to prevent exchanging them by mistake. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize)] +pub struct StreamID(TinyAsciiStr<16>); + +impl StreamID { + pub fn new(data: &[u8]) -> Option<Self> { + if data.is_empty() || data.iter().any(|c| !c.is_ascii_alphanumeric()) { + return None; + } + let s = str::from_utf8(data).ok()?.parse().ok()?; + Some(Self(s)) + } + + pub fn as_str(&self) -> &str { + self.0.as_str() + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn test_onion_validation() { + assert!(validate_onion_address( + b"PG6MMJIYJMCRSSLVYKFWNNTLARU7P5SVN6Y2YMMJU6NUBXNDF4PSCRYD" + ) + .is_ok()); + assert!(validate_onion_address( + b"pg6mmjiyjmcrsslvykfwnntlaru7p5svn6y2ymmju6nubxndf4pscryd" + ) + .is_ok()); + assert!(validate_onion_address( + b"pG6mMjIyJmCrSsLvYkFwNnTlArU7p5SvN6y2YmMjU6nuBxNdF4pScRyD" + ) + .is_ok()); + + assert!(matches!( + validate_onion_address(b"+-6mmjiyjmcrsslvykfwnntlaru7p5svn6y2ymmju6nubxndf4pscryd") + .unwrap_err(), + OnionValidationError::CannotDecode(_) + )); + + assert_eq!( + validate_onion_address(b"pg6mmjiyjmcrsslvykfwnntlaru7p5svn6y2ymmju6nubxndf4pscrye") + .unwrap_err(), + OnionValidationError::UnsupportedVersion(4) + ); + + assert_eq!( + validate_onion_address(b"pg6mmjiyjmcrsslvykfwnntlaru7p5svn6y2ymmju6nubxndf4pscsyd") + .unwrap_err(), + OnionValidationError::BadChecksum + ); + assert_eq!( + validate_onion_address(b"aaammjiyjmcrsslvykfwnntlaru7p5svn6y2ymmju6nubxndf4pscsyd") + .unwrap_err(), + OnionValidationError::BadChecksum + ); + } + + #[test] + fn onion_private_key_redacted() { + let auth = OnionClientAuth { + address: HSAddress::from_str( + "PG6MMJIYJMCRSSLVYKFWNNTLARU7P5SVN6Y2YMMJU6NUBXNDF4PSCRYD", + ) + .unwrap(), + key_type: String::from("x25519"), + key_blob: String::from("abase64key"), + is_permanent: false, + }; + let auth_fmt = format!("{:?}", auth); + assert!(!auth_fmt.contains(&auth.key_blob)); + assert!(auth_fmt.contains("[redacted]")); + } +} View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser/-/compare/7adda3… -- View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser/-/compare/7adda3… You're receiving this email because of your account on gitlab.torproject.org. Manage all notifications: https://gitlab.torproject.org/-/profile/notifications | Help: https://gitlab.torproject.org/help

1 0

[Git][tpo/applications/tor-browser][tor-browser-156.0a1-17.0-2] fixup! TB 40933: Add tor-launcher functionality
by Pier Angelo Vendrame (＠pierov) 11 Nov '26

11 Nov '26

Pier Angelo Vendrame pushed to branch tor-browser-156.0a1-17.0-2 at The Tor Project / Applications / Tor Browser Commits: 7adda3d1 by Neel Chauhan at 2026-11-11T17:49:01+01:00 fixup! TB 40933: Add tor-launcher functionality TB 45344: Remove getBridges from TorProviderBase. - - - - - 3 changed files: - toolkit/components/tor-launcher/TorProvider.sys.mjs - toolkit/components/tor-launcher/TorProviderBase.sys.mjs - toolkit/components/tor-launcher/TorProviderMock.sys.mjs Changes: ===================================== toolkit/components/tor-launcher/TorProvider.sys.mjs ===================================== @@ -431,21 +431,6 @@ export class TorProvider extends TorProviderBase { await this.#controller.newnym(); } - /** - * Get the bridges Tor has been configured with. - * - * @returns {Bridge[]} The configured bridges - */ - async getBridges() { - // Ideally, we would not need this function, because we should be the one - // setting them with TorSettings. However, TorSettings is not notified of - // change of settings. So, asking tor directly with the control connection - // is the most reliable way of getting the configured bridges, at the - // moment. Also, we are using this for the circuit display, which should - // work also when we are not configuring the tor daemon, but just using it. - return this.#controller.getBridges(); - } - /** * Get the configured pluggable transports. * ===================================== toolkit/components/tor-launcher/TorProviderBase.sys.mjs ===================================== @@ -174,10 +174,6 @@ export class TorProviderBase { throw new Error("newnym not implemented."); } - async getBridges() { - throw new Error("getBridges not implemented."); - } - async getPluggableTransports() { throw new Error("getPluggableTransports not implemented."); } ===================================== toolkit/components/tor-launcher/TorProviderMock.sys.mjs ===================================== @@ -79,10 +79,6 @@ export class TorProviderMock extends TorProviderBase { async newnym() {} - async getBridges() { - return []; - } - async getPluggableTransports() { return []; } View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser/-/commit/7adda3d… -- View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser/-/commit/7adda3d… You're receiving this email because of your account on gitlab.torproject.org. Manage all notifications: https://gitlab.torproject.org/-/profile/notifications | Help: https://gitlab.torproject.org/help

1 0

[Git][tpo/applications/mullvad-browser] Pushed new tag mullvad-browser-156.0a1-17.0-2-build1
by Pier Angelo Vendrame (＠pierov) 11 Nov '26

11 Nov '26

Pier Angelo Vendrame pushed new tag mullvad-browser-156.0a1-17.0-2-build1 at The Tor Project / Applications / Mullvad Browser -- View it on GitLab: https://gitlab.torproject.org/tpo/applications/mullvad-browser/-/tree/mullv… You're receiving this email because of your account on gitlab.torproject.org. Manage all notifications: https://gitlab.torproject.org/-/profile/notifications | Help: https://gitlab.torproject.org/help

1 0

[Git][tpo/applications/mullvad-browser][mullvad-browser-156.0a1-17.0-2] 47 commits: Add CI for Mullvad Browser
by Pier Angelo Vendrame (＠pierov) 11 Nov '26

11 Nov '26

Pier Angelo Vendrame pushed to branch mullvad-browser-156.0a1-17.0-2 at The Tor Project / Applications / Mullvad Browser Commits: 5c09c6ae by Henry Wilkes at 2026-11-11T15:15:14+01:00 Add CI for Mullvad Browser - - - - - 2ccbbfcf by Pier Angelo Vendrame at 2026-11-11T15:16:17+01:00 MB 38: Mullvad Browser configuration - - - - - 865ec70c by Pier Angelo Vendrame at 2026-11-11T15:22:52+01:00 fixup! MB 38: Mullvad Browser configuration MB 537: Migrate from Mullvad to Quad9 DoH. - - - - - 3b981222 by Pier Angelo Vendrame at 2026-11-11T15:23:44+01:00 MB 1: Mullvad Browser branding See also: mullvad-browser#5: Product name and directory customization mullvad-browser#12: Create new branding directories and integrate Mullvad icons+branding mullvad-browser#14: Remove Default Built-in bookmarks mullvad-browser#35: Add custom PDF icons for Windows builds mullvad-browser#48: Replace Mozilla copyright and legal trademarks in mullvadbrowser.exe metadata mullvad-browser#51: Update trademark string mullvad-browser#104: Update shipped dll metadata copyright/licensing info mullvad-browser#107: Add alpha and nightly icons - - - - - 3895cdc5 by Pier Angelo Vendrame at 2026-11-11T15:23:44+01:00 fixup! MB 1: Mullvad Browser branding No issue: fix lint. - - - - - c9e49c64 by Pier Angelo Vendrame at 2026-11-11T15:23:44+01:00 fixup! MB 1: Mullvad Browser branding MB 575: Remove an unused customization. That block is removed from about:license in a BB commit. - - - - - 3f8776ed by Morgan at 2026-11-11T15:23:44+01:00 fixup! MB 1: Mullvad Browser branding MB 580: Migrate hiding feedbackPage menuitem to Mullvad Browser patchset - - - - - f2512213 by Beatriz Rizental at 2026-11-11T15:23:45+01:00 fixup! MB 1: Mullvad Browser branding Bug 569: Drop kitty image from default browser dialog - - - - - 2d5ab455 by Beatriz Rizental at 2026-11-11T15:23:45+01:00 fixup! MB 1: Mullvad Browser branding Bug 569: Use mullvad Browser logo in sidebar - - - - - 96e2a2ea by Morgan at 2026-11-11T15:23:45+01:00 fixup! MB 1: Mullvad Browser branding switch hard-coded fill to 'context-fill' so firefox can colour the icon based on theme - - - - - 4f488d56 by Beatriz Rizental at 2026-11-11T15:23:46+01:00 fixup! MB 1: Mullvad Browser branding Bug 576: Fix home page name - - - - - 70b69d8b by Beatriz Rizental at 2026-11-11T15:23:46+01:00 fixup! MB 1: Mullvad Browser branding Bug 576: Fix home page name _correctly_ - - - - - 3164f7cd by Beatriz Rizental at 2026-11-11T15:23:46+01:00 fixup! MB 1: Mullvad Browser branding This reverts commit ce60585f0d88eaef8d6eac4f063ae4032a6c8884. - - - - - ac4edabd by Henry Wilkes at 2026-11-11T15:23:46+01:00 Mullvad Browser strings This commit adds strings needed by the following Mullvad Browser patches. - - - - - 89158f5b by Henry Wilkes at 2026-11-11T15:23:47+01:00 fixup! Mullvad Browser strings MB 538: Strings for the DoH notification. - - - - - 74c07d9c by Pier Angelo Vendrame at 2026-11-11T15:23:47+01:00 fixup! Mullvad Browser strings MB 549: Customize the DoH preferences UI. - - - - - a49fc657 by Morgan at 2026-11-11T15:23:47+01:00 fixup! Mullvad Browser strings No issue: fix typo - - - - - d46572b9 by Pier Angelo Vendrame at 2026-11-11T15:23:47+01:00 MB 20: Allow packaged-addons in PBM. We install a few addons from the distribution directory, but they are not automatically enabled for PBM mode. This commit modifies the code that installs them to also add the PBM permission to the known ones. - - - - - e861b55a by Pier Angelo Vendrame at 2026-11-11T15:23:48+01:00 MB 63: Customize some about pages for Mullvad Browser Also: mullvad-browser#57: Purge unneeded about: pages - - - - - 816e944d by Pier Angelo Vendrame at 2026-11-11T15:23:48+01:00 MB 37: Customization for the about dialog - - - - - b487e65a by Henry Wilkes at 2026-11-11T15:23:48+01:00 MB 39: Add home page about:mullvad-browser - - - - - a7a4c6a9 by Beatriz Rizental at 2026-11-11T15:23:49+01:00 fixup! MB 39: Add home page about:mullvad-browser - - - - - a042fdce by hackademix at 2026-11-11T15:23:49+01:00 MB 97: Remove UI cues to install new extensions. - - - - - 0307665a by hackademix at 2026-11-11T15:23:49+01:00 MB 47: uBlock Origin customization - - - - - e14f30b0 by Pier Angelo Vendrame at 2026-11-11T15:23:49+01:00 MB 21: Disable the password manager This commit disables the about:login page and removes the "Login and Password" section of about:preferences. We do not do anything to the real password manager of Firefox, that is in toolkit: it contains C++ parts that make it difficult to actually prevent it from being built.. Finally, we modify the the function that opens about:login to report an error in the console so that we can quickly get a backtrace to the code that tries to use it. - - - - - 8ba06bbf by Pier Angelo Vendrame at 2026-11-11T15:23:50+01:00 MB 112: Updater customization for Mullvad Browser MB 71: Set the updater base URL to Mullvad domain - - - - - 5e87d6da by Pier Angelo Vendrame at 2026-11-11T15:25:04+01:00 fixup! MB 112: Updater customization for Mullvad Browser BB 45073: Part 1: Check tor-browser.js. We moved some common preferences that were defined both in 000-tor-browser.js and 000-mullvad-browser.js in the common file to reduce redundancy. - - - - - fccebaa8 by Nicolas Vigier at 2026-11-11T15:25:05+01:00 MB 79: Add Mullvad Browser MAR signing keys MB 256: Add mullvad-browser nightly mar signing key - - - - - dd3402f6 by Pier Angelo Vendrame at 2026-11-11T15:25:05+01:00 MB 34: Hide unwanted setting controls in Mullvad Browser. - - - - - c2bb3740 by hackademix at 2026-11-11T15:25:05+01:00 MB 163: Prevent uBlock Origin from being uninstalled/disabled - - - - - 8e3ce45c by Pier Angelo Vendrame at 2026-11-11T15:25:05+01:00 MB 160: Disable the cookie exceptions button Besides disabling the "Delete on close checkbox", disable also the "Manage Exceptions" button when always using PBM. - - - - - 1ab643e8 by Richard Pospesel at 2026-11-11T15:25:06+01:00 MB 188: Customize Gitlab Issue and Merge templates - - - - - dce88677 by rui hildt at 2026-11-11T15:25:06+01:00 MB 213: Customize the search engines list. - - - - - e19c7558 by hackademix at 2026-11-11T15:25:06+01:00 MB 214: Enable cross-tab identity leak protection in "quiet" mode - - - - - c6d40d8d by Pier Angelo Vendrame at 2026-11-11T15:25:06+01:00 MB 320: Temporarily disable WebRTC and WDBA on Windows. WebRTC should be re-enabled when tor-browser#42758 is resolved, and and the default browser agent when in general we make this feature work again. - - - - - 99aef5ea by Henry Wilkes at 2026-11-11T15:25:07+01:00 MB 329: Customize toolbar for mullvad-browser. - - - - - 61c8c2b3 by Henry Wilkes at 2026-11-11T15:25:07+01:00 MB 419: Mullvad Browser migration procedures. This commit implements the the Mullvad Browser's version of _migrateUI. - - - - - 8141aee2 by Pier Angelo Vendrame at 2026-11-11T15:25:07+01:00 fixup! MB 419: Mullvad Browser migration procedures. BB 45039: Make it easier to test ProfileDataUpgrader customization. - - - - - eb01cf44 by Pier Angelo Vendrame at 2026-11-11T15:25:08+01:00 fixup! MB 419: Mullvad Browser migration procedures. MB 537: Implement DoH migration. - - - - - 5acbd6b2 by Henry Wilkes at 2026-11-11T15:25:08+01:00 MB 488: Adjust search engine removal notification for Mullvad Leta. - - - - - 3a8b8a23 by Pier Angelo Vendrame at 2026-11-11T15:25:08+01:00 MB 80: Enable Mullvad Browser as a default browser - - - - - f1936197 by Beatriz Rizental at 2026-11-11T15:25:08+01:00 MB 43564: Modify ./mach bootstrap for Mullvad Browser - - - - - 23eaef3f by june wilde at 2026-11-11T15:25:09+01:00 MB 446: Selective Saving Mode Backend - - - - - b3ae18b1 by Henry Wilkes at 2026-11-11T15:25:09+01:00 MB 538: Mullvad DoH discontinued notification. - - - - - 9991ff05 by Morgan at 2026-11-11T15:25:09+01:00 fixup! MB 538: Mullvad DoH discontinued notification. MB 581: change Review settings button to move user to about:preferences#dnsOverHttps rather than about:preferences#privacy - - - - - 8ab22a5c by hackademix at 2026-11-11T15:25:10+01:00 fixup! MB 538: Mullvad DoH discontinued notification. MB 586: Stop leaking browser windows in mullvadDoHNotification.js - - - - - 00f3da81 by Pier Angelo Vendrame at 2026-11-11T15:25:10+01:00 MB 549: Customize the DoH preferences UI. - - - - - 283 changed files: - .gitlab/ci/jobs/update-translations.yml - .gitlab/issue_templates/000 Bug Report.md - .gitlab/issue_templates/010 Proposal.md - .gitlab/issue_templates/020 Web Compatibility.md - .gitlab/issue_templates/030 Test.md - .gitlab/issue_templates/040 Feature.md - .gitlab/issue_templates/041 Time-Gated Feature.md - .gitlab/issue_templates/051 Revert.md - + .gitlab/issue_templates/060 Rebase - Alpha.md - + .gitlab/issue_templates/061 Rebase - Stable.md - .gitlab/issue_templates/090 Emergency Security Issue.md - .gitlab/merge_request_templates/Default.md - + .gitlab/merge_request_templates/Rebase.md - browser/app/macbuild/Contents/Info.plist.in - browser/app/module.ver - browser/app/moz.build - browser/app/firefox.exe.manifest → browser/app/mullvadbrowser.exe.manifest - + browser/app/profile/000-mullvad-browser.js - browser/app/profile/001-base-profile.js - browser/base/content/aboutDialog.xhtml - browser/base/content/appmenu-viewcache.inc.xhtml - browser/base/content/browser-main.js - browser/base/content/browser-menubar.inc.xhtml - browser/base/content/browser-places.js - browser/base/content/browser.js - browser/base/content/browser.xhtml - browser/base/content/default-bookmarks.html - + browser/base/content/mullvadDoHNotification.js - browser/base/content/nsContextMenu.sys.mjs - browser/base/content/pageinfo/pageInfo.xhtml - browser/base/content/utilityOverlay.js - browser/base/jar.mn - browser/branding/branding-common.mozbuild - + browser/branding/mb-alpha/VisualElements_150.png - + browser/branding/mb-alpha/VisualElements_70.png - + browser/branding/mb-alpha/configure.sh - + browser/branding/mb-alpha/content/about-logo.png - + browser/branding/mb-alpha/content/about-logo.svg - + browser/branding/mb-alpha/content/about-logo(a)2x.png - + browser/branding/mb-alpha/content/about-wordmark.svg - + browser/branding/mb-alpha/content/about.png - + browser/branding/mb-alpha/content/aboutDialog.css - + browser/branding/mb-alpha/content/document_pdf.svg - + browser/branding/mb-alpha/content/firefox-wordmark.svg - + browser/branding/mb-alpha/content/icon128.png - + browser/branding/mb-alpha/content/icon16.png - + browser/branding/mb-alpha/content/icon256.png - + browser/branding/mb-alpha/content/icon32.png - + browser/branding/mb-alpha/content/icon48.png - + browser/branding/mb-alpha/content/icon64.png - + browser/branding/mb-alpha/content/jar.mn - + browser/branding/mb-alpha/content/moz.build - + browser/branding/mb-alpha/content/mullvad-branding.css - + browser/branding/mb-alpha/default128.png - + browser/branding/mb-alpha/default16.png - + browser/branding/mb-alpha/default22.png - + browser/branding/mb-alpha/default24.png - + browser/branding/mb-alpha/default256.png - + browser/branding/mb-alpha/default32.png - + browser/branding/mb-alpha/default48.png - + browser/branding/mb-alpha/default64.png - + browser/branding/mb-alpha/document.icns - + browser/branding/mb-alpha/document.ico - + browser/branding/mb-alpha/document_pdf.ico - + browser/branding/mb-alpha/firefox.icns - + browser/branding/mb-alpha/firefox.ico - + browser/branding/mb-alpha/firefox.svg - + browser/branding/mb-alpha/locales/en-US/brand.ftl - + browser/branding/mb-alpha/locales/en-US/brand.properties - + browser/branding/mb-alpha/locales/jar.mn - + browser/branding/mb-alpha/locales/moz.build - + browser/branding/mb-alpha/locales/mullvad-about-wordmark-en.ftl - + browser/branding/mb-alpha/moz.build - + browser/branding/mb-alpha/mullvadbrowser.VisualElementsManifest.xml - + browser/branding/mb-alpha/newtab.ico - + browser/branding/mb-alpha/newwindow.ico - + browser/branding/mb-alpha/pbmode.ico - + browser/branding/mb-alpha/pref/firefox-branding.js - + browser/branding/mb-nightly/VisualElements_150.png - + browser/branding/mb-nightly/VisualElements_70.png - + browser/branding/mb-nightly/configure.sh - + browser/branding/mb-nightly/content/about-logo.png - + browser/branding/mb-nightly/content/about-logo.svg - + browser/branding/mb-nightly/content/about-logo(a)2x.png - + browser/branding/mb-nightly/content/about-wordmark.svg - + browser/branding/mb-nightly/content/about.png - + browser/branding/mb-nightly/content/aboutDialog.css - + browser/branding/mb-nightly/content/document_pdf.svg - + browser/branding/mb-nightly/content/firefox-wordmark.svg - + browser/branding/mb-nightly/content/icon128.png - + browser/branding/mb-nightly/content/icon16.png - + browser/branding/mb-nightly/content/icon256.png - + browser/branding/mb-nightly/content/icon32.png - + browser/branding/mb-nightly/content/icon48.png - + browser/branding/mb-nightly/content/icon64.png - + browser/branding/mb-nightly/content/jar.mn - + browser/branding/mb-nightly/content/moz.build - + browser/branding/mb-nightly/content/mullvad-branding.css - + browser/branding/mb-nightly/default128.png - + browser/branding/mb-nightly/default16.png - + browser/branding/mb-nightly/default22.png - + browser/branding/mb-nightly/default24.png - + browser/branding/mb-nightly/default256.png - + browser/branding/mb-nightly/default32.png - + browser/branding/mb-nightly/default48.png - + browser/branding/mb-nightly/default64.png - + browser/branding/mb-nightly/document.icns - + browser/branding/mb-nightly/document.ico - + browser/branding/mb-nightly/document_pdf.ico - + browser/branding/mb-nightly/firefox.icns - + browser/branding/mb-nightly/firefox.ico - + browser/branding/mb-nightly/firefox.svg - + browser/branding/mb-nightly/locales/en-US/brand.ftl - + browser/branding/mb-nightly/locales/en-US/brand.properties - + browser/branding/mb-nightly/locales/jar.mn - + browser/branding/mb-nightly/locales/moz.build - + browser/branding/mb-nightly/locales/mullvad-about-wordmark-en.ftl - + browser/branding/mb-nightly/moz.build - + browser/branding/mb-nightly/mullvadbrowser.VisualElementsManifest.xml - + browser/branding/mb-nightly/newtab.ico - + browser/branding/mb-nightly/newwindow.ico - + browser/branding/mb-nightly/pbmode.ico - + browser/branding/mb-nightly/pref/firefox-branding.js - + browser/branding/mb-release/VisualElements_150.png - + browser/branding/mb-release/VisualElements_70.png - + browser/branding/mb-release/configure.sh - + browser/branding/mb-release/content/about-logo.png - + browser/branding/mb-release/content/about-logo.svg - + browser/branding/mb-release/content/about-logo(a)2x.png - + browser/branding/mb-release/content/about-wordmark.svg - + browser/branding/mb-release/content/about.png - + browser/branding/mb-release/content/aboutDialog.css - + browser/branding/mb-release/content/document_pdf.svg - + browser/branding/mb-release/content/firefox-wordmark.svg - + browser/branding/mb-release/content/icon128.png - + browser/branding/mb-release/content/icon16.png - + browser/branding/mb-release/content/icon256.png - + browser/branding/mb-release/content/icon32.png - + browser/branding/mb-release/content/icon48.png - + browser/branding/mb-release/content/icon64.png - + browser/branding/mb-release/content/jar.mn - + browser/branding/mb-release/content/moz.build - + browser/branding/mb-release/content/mullvad-branding.css - + browser/branding/mb-release/default128.png - + browser/branding/mb-release/default16.png - + browser/branding/mb-release/default22.png - + browser/branding/mb-release/default24.png - + browser/branding/mb-release/default256.png - + browser/branding/mb-release/default32.png - + browser/branding/mb-release/default48.png - + browser/branding/mb-release/default64.png - + browser/branding/mb-release/document.icns - + browser/branding/mb-release/document.ico - + browser/branding/mb-release/document_pdf.ico - + browser/branding/mb-release/firefox.icns - + browser/branding/mb-release/firefox.ico - + browser/branding/mb-release/firefox.svg - + browser/branding/mb-release/locales/en-US/brand.ftl - + browser/branding/mb-release/locales/en-US/brand.properties - + browser/branding/mb-release/locales/jar.mn - + browser/branding/mb-release/locales/moz.build - + browser/branding/mb-release/locales/mullvad-about-wordmark-en.ftl - + browser/branding/mb-release/moz.build - + browser/branding/mb-release/mullvadbrowser.VisualElementsManifest.xml - + browser/branding/mb-release/newtab.ico - + browser/branding/mb-release/newwindow.ico - + browser/branding/mb-release/pbmode.ico - + browser/branding/mb-release/pref/firefox-branding.js - browser/components/BrowserContentHandler.sys.mjs - browser/components/BrowserGlue.sys.mjs - browser/components/DesktopActorRegistry.sys.mjs - browser/components/ProfileDataUpgrader.sys.mjs - browser/components/about/AboutRedirector.cpp - browser/components/about/components.conf - browser/components/customizableui/CustomizableUI.sys.mjs - browser/components/moz.build - + browser/components/mullvad-browser/AboutMullvadBrowserChild.sys.mjs - + browser/components/mullvad-browser/AboutMullvadBrowserParent.sys.mjs - + browser/components/mullvad-browser/content/2728-sparkles.svg - + browser/components/mullvad-browser/content/aboutMullvadBrowser.css - + browser/components/mullvad-browser/content/aboutMullvadBrowser.html - + browser/components/mullvad-browser/content/aboutMullvadBrowser.js - + browser/components/mullvad-browser/jar.mn - + browser/components/mullvad-browser/moz.build - browser/components/preferences/config/passwords-autofill.mjs - browser/components/preferences/config/privacy.mjs - browser/components/preferences/config/search.mjs - browser/components/preferences/home.inc.xhtml - browser/components/preferences/main.js - browser/components/preferences/preferences.js - browser/components/preferences/preferences.xhtml - browser/components/preferences/privacy.inc.xhtml - browser/components/preferences/search.inc.xhtml - browser/components/search/SearchUIUtils.sys.mjs - browser/components/shell/ShellService.sys.mjs - browser/components/shell/WindowsDefaultBrowser.cpp - browser/components/shell/nsWindowsShellService.cpp - browser/components/tabbrowser/NewTabPagePreloading.sys.mjs - + browser/components/tests/unit/test_profileDataUpgrade_mullvad_doh.js - browser/components/tests/unit/xpcshell.toml - browser/config/mozconfigs/base-browser - + browser/config/mozconfigs/mullvad-browser - browser/extensions/newtab/lib/AboutPreferences.sys.mjs - browser/installer/package-manifest.in - browser/installer/windows/nsis/updater_append.ini - browser/locales/l10n.toml - browser/modules/HomePage.sys.mjs - browser/moz.build - browser/moz.configure - browser/themes/shared/sidebar/firefox.svg - build/moz.configure/basebrowser-resources.configure - build/moz.configure/bootstrap.configure - config/create_rc.py - devtools/client/aboutdebugging/src/actions/runtimes.js - devtools/client/aboutdebugging/src/components/sidebar/Sidebar.js - devtools/client/jar.mn - devtools/client/themes/images/aboutdebugging-firefox-aurora.svg - devtools/client/themes/images/aboutdebugging-firefox-beta.svg - devtools/client/themes/images/aboutdebugging-firefox-logo.svg - devtools/client/themes/images/aboutdebugging-firefox-nightly.svg - devtools/client/themes/images/aboutdebugging-firefox-release.svg - + devtools/client/themes/images/aboutdebugging-mullvadbrowser-logo.svg - docshell/base/nsAboutRedirector.cpp - docshell/build/components.conf - moz.configure - mozconfig-linux-aarch64 - mozconfig-linux-aarch64-dev - mozconfig-linux-x86_64 - mozconfig-linux-x86_64-asan - mozconfig-linux-x86_64-dev - mozconfig-macos - mozconfig-macos-dev - mozconfig-windows-x86_64 - + other-licenses/nsis/Contrib/ApplicationID/Makefile - other-licenses/nsis/Contrib/ApplicationID/Set.cpp - + other-licenses/nsis/Contrib/CityHash/Makefile - python/mozboot/mozboot/bootstrap.py - python/mozbuild/mozbuild/backend/base.py - python/mozbuild/mozbuild/tbbutils.py - toolkit/components/aboutpdf/content/aboutPDF.html - toolkit/components/extensions/child/ext-storage.js - toolkit/components/extensions/parent/ext-storage.js - toolkit/components/moz.build - toolkit/components/passwordmgr/LoginHelper.sys.mjs - toolkit/components/remote/nsDBusRemoteClient.cpp - toolkit/components/remote/nsDBusRemoteServer.cpp - toolkit/components/search/SearchService.sys.mjs - toolkit/components/search/content/base-browser-search-engine-icons.json - toolkit/components/search/content/base-browser-search-engines.json - + toolkit/components/search/content/brave.svg - + toolkit/components/search/content/mojeek.ico - toolkit/components/search/tests/xpcshell/test_base_browser.js - toolkit/components/securitylevel/SecurityLevel.sys.mjs - + toolkit/components/selectivesavingmode/SelectiveSavingMode.manifest - + toolkit/components/selectivesavingmode/SelectiveSavingMode.sys.mjs - + toolkit/components/selectivesavingmode/components.conf - + toolkit/components/selectivesavingmode/content/broom-clean.svg - + toolkit/components/selectivesavingmode/content/broom-saved.svg - + toolkit/components/selectivesavingmode/jar.mn - + toolkit/components/selectivesavingmode/moz.build - + toolkit/components/selectivesavingmode/tests/xpcshell/test_selective_saving_mode.js - + toolkit/components/selectivesavingmode/tests/xpcshell/xpcshell.toml - + toolkit/content/aboutTelemetryMullvad.xhtml - toolkit/content/jar.mn - toolkit/content/widgets/moz-promo/moz-promo.css - toolkit/content/widgets/moz-promo/moz-promo.tokens.css - + toolkit/locales/en-US/toolkit/global/mullvad-browser.ftl - toolkit/mozapps/defaultagent/EventLog.h - toolkit/mozapps/defaultagent/SetDefaultBrowser.cpp - toolkit/mozapps/extensions/AddonManager.sys.mjs - toolkit/mozapps/extensions/content/aboutaddons.css - toolkit/mozapps/extensions/internal/XPIDatabase.sys.mjs - toolkit/mozapps/extensions/internal/XPIProvider.sys.mjs - toolkit/mozapps/update/updater/nightly_aurora_level3_primary.der - toolkit/mozapps/update/updater/nightly_aurora_level3_secondary.der - toolkit/mozapps/update/updater/release_primary.der - toolkit/mozapps/update/updater/release_secondary.der - + toolkit/themes/shared/icons/mullvadbrowser.png - toolkit/themes/shared/minimal-toolkit.jar.inc.mn - toolkit/xre/nsAppRunner.cpp - tools/lint/fluent-lint/exclusions.yml - widget/windows/WinTaskbar.cpp - widget/windows/moz.build The diff was not included because it is too large. View it on GitLab: https://gitlab.torproject.org/tpo/applications/mullvad-browser/-/compare/ae… -- View it on GitLab: https://gitlab.torproject.org/tpo/applications/mullvad-browser/-/compare/ae… You're receiving this email because of your account on gitlab.torproject.org. Manage all notifications: https://gitlab.torproject.org/-/profile/notifications | Help: https://gitlab.torproject.org/help

1 0

[Git][tpo/applications/tor-browser-build][maint-16.0] Reset var/browser_platforms/is_desktop_release after 16.0.2
by boklm (＠boklm) 11 Nov '26

11 Nov '26

boklm pushed to branch maint-16.0 at The Tor Project / Applications / tor-browser-build Commits: 1df4ef29 by Nicolas Vigier at 2026-11-10T18:05:15+01:00 Reset var/browser_platforms/is_desktop_release after 16.0.2 - - - - - 1 changed file: - rbm.conf Changes: ===================================== rbm.conf ===================================== @@ -103,7 +103,7 @@ var: # check whether a release includes some android or desktop platforms # see signing_android and signing_desktop below. is_android_release: '[% c("var/tor-browser") || c("var/tor-vpn") %]' - is_desktop_release: '0' + is_desktop_release: '[% ! c("var/tor-vpn") %]' # signing_android is used in signing scripts to check if at least # one android platform is being signed/published View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/commit/1… -- View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/commit/1… You're receiving this email because of your account on gitlab.torproject.org. Manage all notifications: https://gitlab.torproject.org/-/profile/notifications | Help: https://gitlab.torproject.org/help

1 0

[Git][tpo/applications/tor-browser-build][main] Update gitlab templates for the new maint-16.0 branch
by boklm (＠boklm) 11 Nov '26

11 Nov '26

boklm pushed to branch main at The Tor Project / Applications / tor-browser-build Commits: 8fbc8589 by Nicolas Vigier at 2026-11-10T16:07:37+01:00 Update gitlab templates for the new maint-16.0 branch - - - - - 6 changed files: - .gitlab/issue_templates/010 Backport.md - .gitlab/issue_templates/020 Revert.md - .gitlab/issue_templates/041 Release Prep - Tor Browser Stable.md - .gitlab/issue_templates/051 Release Prep - Mullvad Browser Stable.md - .gitlab/merge_request_templates/default.md - rbm.local.conf.example Changes: ===================================== .gitlab/issue_templates/010 Backport.md ===================================== @@ -4,7 +4,7 @@ Title: Backport tor-browser-build-browser#12345: Title of Issue -This is an issue for tracking back-porting a patch-set (e.g. from main to maint-15.0) +This is an issue for tracking back-porting a patch-set (e.g. from main to maint-16.0) --> ## Book-keeping @@ -19,7 +19,7 @@ This is an issue for tracking back-porting a patch-set (e.g. from main to maint- ### Target Channels -- [ ] maint-15.0 +- [ ] maint-16.0 ## Notes ===================================== .gitlab/issue_templates/020 Revert.md ===================================== @@ -31,7 +31,7 @@ When should this patchset be reverted? ### Target Channels  - [ ] main -- [ ] maint-15.0 +- [ ] maint-16.0 ## Notes ===================================== .gitlab/issue_templates/041 Release Prep - Tor Browser Stable.md ===================================== @@ -118,7 +118,7 @@ Tor Browser Stable is on the `maint-${TOR_BROWSER_MAJOR}.${TOR_BROWSER_MINOR}` b - ESR updates correct - Component updates correct - [ ] Open MR with above changes, using the template for release preparations - - **NOTE**: target the `maint-15.0` branch + - **NOTE**: target the `maint-16.0` branch - [ ] Merge - [ ] Sign+Tag - **NOTE** this must be done by one of: @@ -209,7 +209,7 @@ Tor Browser Stable is on the `maint-${TOR_BROWSER_MAJOR}.${TOR_BROWSER_MINOR}` b # paste changelog as quote here ``` - [ ] On `${STAGING_SERVER}`, ensure updated: - - **NOTE** Having a local git branch with `maint-15.0` as the upstream branch with these values saved means you only need to periodically `git pull --rebase` + - **NOTE** Having a local git branch with `maint-16.0` as the upstream branch with these values saved means you only need to periodically `git pull --rebase` - [ ] `tor-browser-build` is on the right commit: `git tag -v tbb-${TOR_BROWSER_VERSION}-${TOR_BROWSER_BUILD_N} && git checkout tbb-${TOR_BROWSER_VERSION}-${TOR_BROWSER_BUILD_N}` - [ ] `tor-browser-build/tools/signing/set-config.hosts` - `ssh_host_builder`: ssh hostname of machine with unsigned builds ===================================== .gitlab/issue_templates/051 Release Prep - Mullvad Browser Stable.md ===================================== @@ -94,7 +94,7 @@ Mullvad Browser Stable is on the `maint-${MULLVAD_BROWSER_MAJOR}.${MULLVAD_BROWS - ESR updates correct - Component updates correct - [ ] Open MR with above changes, using the template for release preparations - - **NOTE**: target the `maint-15.0` branch + - **NOTE**: target the `maint-16.0` branch - [ ] Merge - [ ] Sign+Tag - **NOTE** this must be done by one of: @@ -135,7 +135,7 @@ Mullvad Browser Stable is on the `maint-${MULLVAD_BROWSER_MAJOR}.${MULLVAD_BROWS - pierov - [ ] Ensure all builders have matching builds - [ ] On `${STAGING_SERVER}`, ensure updated: - - **NOTE** Having a local git branch with `maint-15.0` as the upstream branch with these values saved means you only need to periodically `git pull --rebase` + - **NOTE** Having a local git branch with `maint-16.0` as the upstream branch with these values saved means you only need to periodically `git pull --rebase` - [ ] `tor-browser-build` is on the right commit: `git tag -v mb-${MULLVAD_BROWSER_VERSION}-${MULLVAD_BROWSER_BUILD_N} && git checkout mb-${MULLVAD_BROWSER_VERSION}-${MULLVAD_BROWSER_BUILD_N}` - [ ] `tor-browser-build/tools/signing/set-config.hosts` - `ssh_host_builder`: ssh hostname of machine with unsigned builds ===================================== .gitlab/merge_request_templates/default.md ===================================== @@ -19,8 +19,8 @@  #### Target Branches - - [ ] **`main`**: rapid release, 16.0 - - [ ] **`maint-15.0`**: esr140-15.0 + - [ ] **`main`**: rapid release, 17.0 + - [ ] **`maint-16.0`**: esr153-16.0 ### Backporting ===================================== rbm.local.conf.example ===================================== @@ -100,7 +100,7 @@ var: # - browser-all # - mullvadbrowser # - ### If you are doing Tor Browser 'release' builds in the maint-15.0 + ### If you are doing Tor Browser 'release' builds in the maint-16.0 ### branch and 'alpha' builds in the main branch, you can use the ### following configuration: #clean: @@ -110,7 +110,7 @@ var: # - alpha # - browser-all # - torbrowser - # maint-15.0: + # maint-16.0: # - project: release # target: # - release View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/commit/8… -- View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/commit/8… You're receiving this email because of your account on gitlab.torproject.org. Manage all notifications: https://gitlab.torproject.org/-/profile/notifications | Help: https://gitlab.torproject.org/help

1 0

[Git][tpo/applications/tor-browser][tor-browser-156.0a1-17.0-2] 12 commits: fixup! Tor Browser localization migration scripts.
by Pier Angelo Vendrame (＠pierov) 11 Nov '26

11 Nov '26

Pier Angelo Vendrame pushed to branch tor-browser-156.0a1-17.0-2 at The Tor Project / Applications / Tor Browser Commits: 60b70f9f by Henry Wilkes at 2026-11-11T08:59:14+01:00 fixup! Tor Browser localization migration scripts. TB 45433: Change the script to be more versatile for test runs. Instead of passing in a python module name for the recipe, you now pass in the file path to the recipe script. You can now change the reference directory with the --reference-directory argument, which is useful for testing scripts before the changes have reached the translation repository. - - - - - b7ea5196 by Henry Wilkes at 2026-11-11T08:59:15+01:00 fixup! Tor Browser localization migration scripts. TB 45433: Refactor the WeblateMetadata class. We are stricter about checking the return types from the weblate API, and making sure it doesn't make requests to unexpected URLs. We also limit the components to the browser components, and we slightly alter the `is_translated` public method. - - - - - 873fe4a3 by Henry Wilkes at 2026-11-11T08:59:15+01:00 fixup! Tor Browser localization migration scripts. TB 45433: Add type hints to the script. - - - - - 42d45b17 by Henry Wilkes at 2026-11-11T08:59:15+01:00 fixup! Tor Browser localization migration scripts. TB 45433: Update the migrate.py documentation. - - - - - 01dfa930 by hackademix at 2026-11-11T08:59:52+01:00 BB 45438: Use OneCRL dump from 153esr - - - - - ee6f9f1d by Dana Keeler at 2026-11-11T08:59:54+01:00 Bug 2084311 - ensure OneCRL bundle is loaded at init a=RyanVM Original Revision: https://phabricator.services.mozilla.com/D335790 Differential Revision: https://phabricator.services.mozilla.com/D335837 - - - - - a3805da7 by Elena at 2026-11-11T08:59:55+01:00 fixup! TB 40933: Add tor-launcher functionality TB 45425: Use an actual XPCOM type for status updates. - - - - - e77d5e6c by Elena at 2026-11-11T08:59:55+01:00 fixup! TB 31286: Implementation of bridge, proxy, and firewall settings in about:preferences#connection TB 45425: Use an actual XPCOM type for status updates. - - - - - 7ed1acfb by Elena at 2026-11-11T08:59:55+01:00 fixup! TB 42247: Android helpers for the TorProvider TB 45425: Use an actual XPCOM type for status updates. - - - - - cceebf2f by Elena at 2026-11-11T08:59:55+01:00 fixup! TB 3455: Add DomainIsolator, for isolating circuit by domain. TB 45425: Use an actual XPCOM type for provider topics. - - - - - 96898066 by Pier Angelo Vendrame at 2026-11-11T08:59:56+01:00 fixup! TB 41600: Add a tor circuit display panel. TB 45425: Use an explicit boolean for bridges in NodeData - - - - - bd6af55f by Dan Ballard at 2026-11-11T08:59:56+01:00 fixup! [android] Override settings TB 45365: Disable IPProtect for all in nimbus - - - - - 17 changed files: - browser/components/torcircuit/content/torCircuitPanel.js - browser/components/torpreferences/content/torLogDialog.js - mobile/android/fenix/app/nimbus.fml.yaml - security/manager/ssl/RemoteSecuritySettings.sys.mjs - + security/manager/ssl/tests/unit/test_onecrl_loads_bundle.js - security/manager/ssl/tests/unit/xpcshell.toml - services/settings/dumps/security-state/onecrl.json - toolkit/components/tor-launcher/TorBootstrapRequest.sys.mjs - toolkit/components/tor-launcher/TorDomainIsolator.sys.mjs - toolkit/components/tor-launcher/TorLauncherUtil.sys.mjs - toolkit/components/tor-launcher/TorProvider.sys.mjs - toolkit/components/tor-launcher/TorProviderBuilder.sys.mjs - toolkit/components/tor-launcher/TorProviderMock.sys.mjs - toolkit/components/tor-launcher/moz.build - + toolkit/components/tor-launcher/tor-launcher.idl - toolkit/modules/TorAndroidIntegration.sys.mjs - tools/torbrowser/l10n/migrate.py Changes: ===================================== browser/components/torcircuit/content/torCircuitPanel.js ===================================== @@ -547,9 +547,9 @@ var gTorCircuitPanel = { // bridge, or otherwise a guard. document.l10n.setAttributes( this._panelElements.newCircuitDescription, - nodes[0].bridgeType === null - ? "tor-circuit-panel-new-button-description-guard" - : "tor-circuit-panel-new-button-description-bridge" + nodes[0].isBridge + ? "tor-circuit-panel-new-button-description-bridge" + : "tor-circuit-panel-new-button-description-guard" ); }, @@ -565,7 +565,7 @@ var gTorCircuitPanel = { const nameEl = nodeItem.querySelector(".tor-circuit-node-name"); let flagSrc = null; - if (node.bridgeType === null) { + if (!node.isBridge) { const regionCode = node.regionCode; flagSrc = this._regionFlagSrc(regionCode); ===================================== browser/components/torpreferences/content/torLogDialog.js ===================================== @@ -87,7 +87,13 @@ const gTorLogDialog = { observe(subject, topic) { if (topic === TorProviderTopics.TorLog) { - this.addLogEntry(subject.wrappedJSObject, false); + let entry; + try { + entry = subject.QueryInterface(Ci.torITorLogEntry); + } catch { + return; + } + this.addLogEntry(entry, false); } }, @@ -96,7 +102,7 @@ const gTorLogDialog = { timeEl.textContent = logEntry.timestamp; timeEl.classList.add("time"); const messageEl = document.createElement("td"); - messageEl.textContent = `[${logEntry.type}] ${logEntry.msg}`; + messageEl.textContent = `[${logEntry.severity}] ${logEntry.message}`; messageEl.classList.add("message"); const row = document.createElement("tr"); ===================================== mobile/android/fenix/app/nimbus.fml.yaml ===================================== @@ -1082,14 +1082,13 @@ features: defaults: - channel: developer value: - enabled: true - allowed-regions: ["XX"] + enabled: false - channel: nightly value: - enabled: true + enabled: false - channel: beta value: - enabled: true + enabled: false native-share-sheet: description: Feature to enable the Native Share Sheet for users on Firefox on Android. ===================================== security/manager/ssl/RemoteSecuritySettings.sys.mjs ===================================== @@ -259,6 +259,28 @@ export var RemoteSecuritySettings = { signerName: SECURITY_STATE_SIGNER, }); this.OneCRLBlocklistClient.on("sync", updateCertBlocklist); + // Ensure the bundled list of revocations is loaded. + this.OneCRLBlocklistClient.get() + .then(records => { + return updateCertBlocklist({ + data: { + current: records, + created: records, + updated: [], + deleted: [], + }, + }); + }) + .catch(e => { + lazy.log.error(e); + }) + .finally(() => { + // Only used by tests, to know when the bundled list is in cert_storage. + Services.obs.notifyObservers( + null, + "remote-security-settings:onecrl-bundle-loaded" + ); + }); this.IntermediatePreloadsClient = new IntermediatePreloads(); ===================================== security/manager/ssl/tests/unit/test_onecrl_loads_bundle.js ===================================== @@ -0,0 +1,60 @@ +// -*- indent-tabs-mode: nil; js-indent-level: 2 -*- +// Any copyright is dedicated to the Public Domain. +// http://creativecommons.org/publicdomain/zero/1.0/ + +"use strict"; + +// Checks that OneCRL loads its remote settings bundle at startup by ensuring +// that each of the listed eMudhra intermediate certificates is revoked. + +do_get_profile(); + +const { TestUtils } = ChromeUtils.importESModule( + "resource://testing-common/TestUtils.sys.mjs" +); +const { RemoteSecuritySettings } = ChromeUtils.importESModule( + "resource://gre/modules/psm/RemoteSecuritySettings.sys.mjs" +); + +// DER encoding of the issuer name (CN=emSign Root TLS CA - G3), base64. +const ISSUER = + "MFYxCzAJBgNVBAYTAklOMSUwIwYDVQQKDBxlTXVkaHJhIFRlY2hub2xvZ2llcyBMaW1pdGVkMSAwHgYDVQQDDBdlbVNpZ24gUm9vdCBUTFMgQ0EgLSBHMw=="; + +// Serial numbers are the contents of the DER INTEGER, in hex. +const SERIALS = { + "EM DV TLS CA - G3B": "00a4aee678f23362b4c0e460b772f214", + "EM OV TLS CA - G3B": "6cada367fb1c100088120ea202d568", + "EM EV TLS CA - G3B": "00c7c41efe92d825f05eae841a4a82c1", + "EM DV TLS CA - G3C": "588b2e3c2bb8cda13e1a1209254321", + "EM OV TLS CA - G3C": "296926c227a3a1526392b19482a504", + "EM EV TLS CA - G3C": "008009928a64107601af05fa2f19f109", +}; + +function base64ToBytes(base64) { + return Array.from(atob(base64), c => c.charCodeAt(0)); +} + +function hexToBytes(hex) { + return hex.match(/../g).map(byte => parseInt(byte, 16)); +} + +add_task(async function test_emsign_intermediates_are_revoked() { + let loaded = TestUtils.topicObserved( + "remote-security-settings:onecrl-bundle-loaded" + ); + RemoteSecuritySettings.init(); + await loaded; + + let certStorage = Cc["@mozilla.org/security/certstorage;1"].getService( + Ci.nsICertStorage + ); + let issuer = base64ToBytes(ISSUER); + + for (let [name, serial] of Object.entries(SERIALS)) { + Assert.equal( + certStorage.getRevocationState(issuer, hexToBytes(serial), [], []), + Ci.nsICertStorage.STATE_ENFORCE, + `${name} should be revoked` + ); + } +}); ===================================== security/manager/ssl/tests/unit/xpcshell.toml ===================================== @@ -293,6 +293,8 @@ run-sequentially = ["true"] # hardcoded ports ["test_ocsp_url.js"] run-sequentially = ["true"] # hardcoded ports +["test_onecrl_loads_bundle.js"] + ["test_oskeystore.js"] skip-if = [ "os == 'mac' && os_version == '15.30' && arch == 'aarch64'", # Bug 1729538 ===================================== services/settings/dumps/security-state/onecrl.json ===================================== @@ -1,5 +1,575 @@ { "data": [ + { + "schema": 1793206881749, + "details": { + "bug": "https://bugzilla.mozilla.org/show_bug.cgi?id=2086683", + "who": "", + "why": "", + "name": "", + "created": "" + }, + "enabled": false, + "issuerName": "MIGIMQswCQYDVQQGEwJVUzETMBEGA1UECBMKTmV3IEplcnNleTEUMBIGA1UEBxMLSmVyc2V5IENpdHkxHjAcBgNVBAoTFVRoZSBVU0VSVFJVU1QgTmV0d29yazEuMCwGA1UEAxMlVVNFUlRydXN0IEVDQyBDZXJ0aWZpY2F0aW9uIEF1dGhvcml0eQ==", + "serialNumber": "RB/KBH9VgbnLpOJqZoTyow==", + "id": "7b77a8a9-4ff9-4190-a63d-480067cf8c4e", + "last_modified": 1793208925525 + }, + { + "schema": 1793206882065, + "details": { + "bug": "https://bugzilla.mozilla.org/show_bug.cgi?id=2086683", + "who": "", + "why": "", + "name": "", + "created": "" + }, + "enabled": false, + "issuerName": "MGkxCzAJBgNVBAYTAktSMSYwJAYDVQQKDB1OQVZFUiBCVVNJTkVTUyBQTEFURk9STSBDb3JwLjEyMDAGA1UEAwwpTkFWRVIgR2xvYmFsIFJvb3QgQ2VydGlmaWNhdGlvbiBBdXRob3JpdHk=", + "serialNumber": "FtGXvblx89aS5WAmhklF4wKyzbU=", + "id": "3fdd4f20-8e47-4578-997c-3016911d10a8", + "last_modified": 1793208925522 + }, + { + "schema": 1793206881998, + "details": { + "bug": "https://bugzilla.mozilla.org/show_bug.cgi?id=2086683", + "who": "", + "why": "", + "name": "", + "created": "" + }, + "enabled": false, + "issuerName": "MGkxCzAJBgNVBAYTAktSMSkwJwYDVQQKDCBOQVZFUiBDbG91ZCBUcnVzdCBTZXJ2aWNlcyBDb3JwLjEvMC0GA1UEAwwmTkFWRVIgQ2xvdWQgVHJ1c3QgU2VydmljZXMgRUNDIFJvb3QgRzE=", + "serialNumber": "DfW5xUaZ1oc/RsU7E9mA4Keg9GY=", + "id": "2d970c9e-ca3e-400f-a025-05a74762c688", + "last_modified": 1793208925519 + }, + { + "schema": 1793206881874, + "details": { + "bug": "https://bugzilla.mozilla.org/show_bug.cgi?id=2086683", + "who": "", + "why": "", + "name": "", + "created": "" + }, + "enabled": false, + "issuerName": "MEQxCzAJBgNVBAYTAkRFMQ8wDQYDVQQKEwZBbWF6b24xJDAiBgNVBAMTG0FtYXpvbiBFQ0RTQSAyNTYgUm9vdCBFVSBNMQ==", + "serialNumber": "B8DNf48cn1uAEWyCz2ilx7MfJQ==", + "id": "83ccc9c6-2121-4f62-b284-20d9514d1927", + "last_modified": 1793208925517 + }, + { + "schema": 1793206881808, + "details": { + "bug": "https://bugzilla.mozilla.org/show_bug.cgi?id=2086683", + "who": "", + "why": "", + "name": "", + "created": "" + }, + "enabled": false, + "issuerName": "MD0xCzAJBgNVBAYTAkNOMREwDwYDVQQKDAhVbmlUcnVzdDEbMBkGA1UEAwwSVUNBIEdsb2JhbCBHMiBSb290", + "serialNumber": "YCnHTexdqAT9NNVLh8z2QA==", + "id": "66ad7766-46e5-415b-8950-9d4143d68a47", + "last_modified": 1793208925514 + }, + { + "schema": 1793206881627, + "details": { + "bug": "https://bugzilla.mozilla.org/show_bug.cgi?id=2086683", + "who": "", + "why": "", + "name": "", + "created": "" + }, + "enabled": false, + "issuerName": "MGkxCzAJBgNVBAYTAktSMSYwJAYDVQQKDB1OQVZFUiBCVVNJTkVTUyBQTEFURk9STSBDb3JwLjEyMDAGA1UEAwwpTkFWRVIgR2xvYmFsIFJvb3QgQ2VydGlmaWNhdGlvbiBBdXRob3JpdHk=", + "serialNumber": "FJ5/xhegeJsX13b9RcYLaXVPUwA=", + "id": "a0372f6d-c436-429a-a41e-c16003c01ff0", + "last_modified": 1793208925511 + }, + { + "schema": 1793206881570, + "details": { + "bug": "https://bugzilla.mozilla.org/show_bug.cgi?id=2086683", + "who": "", + "why": "", + "name": "", + "created": "" + }, + "enabled": false, + "issuerName": "MGkxCzAJBgNVBAYTAktSMSkwJwYDVQQKDCBOQVZFUiBDbG91ZCBUcnVzdCBTZXJ2aWNlcyBDb3JwLjEvMC0GA1UEAwwmTkFWRVIgQ2xvdWQgVHJ1c3QgU2VydmljZXMgRUNDIFJvb3QgRzE=", + "serialNumber": "DjfBmWc92Uon9Pg7pj1ojD29WkA=", + "id": "fb78f960-2dd1-438c-a1e5-91f22ffb9aed", + "last_modified": 1793208925508 + }, + { + "schema": 1793206881509, + "details": { + "bug": "https://bugzilla.mozilla.org/show_bug.cgi?id=2086683", + "who": "", + "why": "", + "name": "", + "created": "" + }, + "enabled": false, + "issuerName": "MEQxCzAJBgNVBAYTAkRFMQ8wDQYDVQQKEwZBbWF6b24xJDAiBgNVBAMTG0FtYXpvbiBFQ0RTQSAzODQgUm9vdCBFVSBNMQ==", + "serialNumber": "B8DNf48ffwl2Bnue0WyZT7Yw+w==", + "id": "307e624d-6540-455a-80a5-a3a85c905e06", + "last_modified": 1793208925505 + }, + { + "schema": 1792622042163, + "details": { + "bug": "https://bugzilla.mozilla.org/show_bug.cgi?id=2086683", + "who": "", + "why": "", + "name": "", + "created": "" + }, + "enabled": false, + "issuerName": "MGkxCzAJBgNVBAYTAktSMSkwJwYDVQQKDCBOQVZFUiBDbG91ZCBUcnVzdCBTZXJ2aWNlcyBDb3JwLjEvMC0GA1UEAwwmTkFWRVIgQ2xvdWQgVHJ1c3QgU2VydmljZXMgUlNBIFJvb3QgRzE=", + "serialNumber": "C0Se6VVjeCepnsw2POzjSpEeSk0=", + "id": "beef1c99-a5f4-4bd6-8f9c-aa06e7d524df", + "last_modified": 1793208925502 + }, + { + "schema": 1793206881933, + "details": { + "bug": "https://bugzilla.mozilla.org/show_bug.cgi?id=2086683", + "who": "", + "why": "", + "name": "", + "created": "" + }, + "enabled": false, + "issuerName": "MGkxCzAJBgNVBAYTAktSMSkwJwYDVQQKDCBOQVZFUiBDbG91ZCBUcnVzdCBTZXJ2aWNlcyBDb3JwLjEvMC0GA1UEAwwmTkFWRVIgQ2xvdWQgVHJ1c3QgU2VydmljZXMgUlNBIFJvb3QgRzE=", + "serialNumber": "CuTxpDlw70JyQ/lqZvnIPouNYiQ=", + "id": "9b28e176-0b39-42b1-a4d8-ebb9f85d7558", + "last_modified": 1793208925499 + }, + { + "schema": 1793206881689, + "details": { + "bug": "https://bugzilla.mozilla.org/show_bug.cgi?id=2086683", + "who": "", + "why": "", + "name": "", + "created": "" + }, + "enabled": false, + "issuerName": "MEMxCzAJBgNVBAYTAkRFMQ8wDQYDVQQKEwZBbWF6b24xIzAhBgNVBAMTGkFtYXpvbiBSU0EgMjA0OCBSb290IEVVIE0x", + "serialNumber": "B8DNf48VlBcZv+FGxXhhwHG4iw==", + "id": "c2f70e6e-5a20-421f-8063-61f0854d171b", + "last_modified": 1793208925496 + }, + { + "schema": 1792621513680, + "details": { + "bug": "https://bugzilla.mozilla.org/show_bug.cgi?id=2083382", + "who": "", + "why": "", + "name": "", + "created": "" + }, + "enabled": true, + "subject": "MD8xCzAJBgNVBAYTAk5MMRMwEQYDVQQKDAplTXVkaHJhIEJWMRswGQYDVQQDDBJFTSBEViBUTFMgQ0EgLSBHM0I=", + "pubKeyHash": "6+1kdcLC2A/LGZ8/6N1uazzmC3frFdTkLD3sDG9bUQU=", + "id": "5ed2b441-a5bf-4f8e-89af-d01fa292a9b8", + "last_modified": 1792622041776 + }, + { + "schema": 1792621487501, + "details": { + "bug": "https://bugzilla.mozilla.org/show_bug.cgi?id=2083382", + "who": "", + "why": "", + "name": "", + "created": "" + }, + "enabled": true, + "subject": "MEAxCzAJBgNVBAYTAlVTMRQwEgYDVQQKDAtlTXVkaHJhIEluYzEbMBkGA1UEAwwSRU0gT1YgVExTIENBIC0gRzND", + "pubKeyHash": "Y+zv7kMdSVI7wRNsTnLQIGpHcIbK9gbNXPAlaGwQnPg=", + "id": "f67aff65-7a13-42ef-b818-74bb69053273", + "last_modified": 1792622041773 + }, + { + "schema": 1792621521702, + "details": { + "bug": "https://bugzilla.mozilla.org/show_bug.cgi?id=2083382", + "who": "", + "why": "", + "name": "", + "created": "" + }, + "enabled": true, + "subject": "MEAxCzAJBgNVBAYTAlVTMRQwEgYDVQQKDAtlTXVkaHJhIEluYzEbMBkGA1UEAwwSRU0gRFYgVExTIENBIC0gRzND", + "pubKeyHash": "Sd01PcK5GaN3BkURfQ5gwWEV59G8JZ7v5zSwDqQ0/l8=", + "id": "fb4be101-8280-4d4e-866e-fd0213bc3a02", + "last_modified": 1792622041769 + }, + { + "schema": 1792621504826, + "details": { + "bug": "https://bugzilla.mozilla.org/show_bug.cgi?id=2083382", + "who": "", + "why": "", + "name": "", + "created": "" + }, + "enabled": true, + "subject": "MEAxCzAJBgNVBAYTAlVTMRQwEgYDVQQKDAtlTXVkaHJhIEluYzEbMBkGA1UEAwwSRU0gRVYgVExTIENBIC0gRzND", + "pubKeyHash": "XAZgorFkkQ2P9YnBjhRVLp4Owjfc7+LLc21BpIEfgaA=", + "id": "adaa1e8d-57cb-4cfb-b647-305aa20f1b04", + "last_modified": 1792622041766 + }, + { + "schema": 1792621495984, + "details": { + "bug": "https://bugzilla.mozilla.org/show_bug.cgi?id=2083382", + "who": "", + "why": "", + "name": "", + "created": "" + }, + "enabled": true, + "subject": "MD8xCzAJBgNVBAYTAk5MMRMwEQYDVQQKDAplTXVkaHJhIEJWMRswGQYDVQQDDBJFTSBPViBUTFMgQ0EgLSBHM0I=", + "pubKeyHash": "Q7/6GdbvZRRR9oQeAZkx6W9vLq7/KKtLbxzkV23icXs=", + "id": "3f07ab6b-0d52-41a3-9a7b-de945502a3b8", + "last_modified": 1792622041762 + }, + { + "schema": 1792274405648, + "details": { + "bug": "https://bugzilla.mozilla.org/show_bug.cgi?id=2083382", + "who": "", + "why": "", + "name": "", + "created": "" + }, + "enabled": true, + "subject": "MD8xCzAJBgNVBAYTAk5MMRMwEQYDVQQKDAplTXVkaHJhIEJWMRswGQYDVQQDDBJFTSBFViBUTFMgQ0EgLSBHM0I=", + "pubKeyHash": "4CVMf83CZeLw+KQo2rt1q0rgObSHLWL17lfwh31aikE=", + "id": "5856b9af-5c14-46e7-b660-04d1cc3f2819", + "last_modified": 1792622041759 + }, + { + "schema": 1792270008816, + "details": { + "bug": "https://bugzilla.mozilla.org/show_bug.cgi?id=2083382", + "who": "", + "why": "", + "name": "", + "created": "" + }, + "enabled": true, + "issuerName": "MFYxCzAJBgNVBAYTAklOMSUwIwYDVQQKDBxlTXVkaHJhIFRlY2hub2xvZ2llcyBMaW1pdGVkMSAwHgYDVQQDDBdlbVNpZ24gUm9vdCBUTFMgQ0EgLSBHMw==", + "serialNumber": "WIsuPCu4zaE+GhIJJUMh", + "id": "dfebe454-5411-4f64-bd26-ff0346cad020", + "last_modified": 1792274405312 + }, + { + "schema": 1792269992087, + "details": { + "bug": "https://bugzilla.mozilla.org/show_bug.cgi?id=2083382", + "who": "", + "why": "", + "name": "", + "created": "" + }, + "enabled": true, + "issuerName": "MFYxCzAJBgNVBAYTAklOMSUwIwYDVQQKDBxlTXVkaHJhIFRlY2hub2xvZ2llcyBMaW1pdGVkMSAwHgYDVQQDDBdlbVNpZ24gUm9vdCBUTFMgQ0EgLSBHMw==", + "serialNumber": "AIAJkopkEHYBrwX6LxnxCQ==", + "id": "eefb82bf-3d32-4b7e-9dff-19fd2c21f755", + "last_modified": 1792274405309 + }, + { + "schema": 1792269983656, + "details": { + "bug": "https://bugzilla.mozilla.org/show_bug.cgi?id=2083382", + "who": "", + "why": "", + "name": "", + "created": "" + }, + "enabled": true, + "issuerName": "MFYxCzAJBgNVBAYTAklOMSUwIwYDVQQKDBxlTXVkaHJhIFRlY2hub2xvZ2llcyBMaW1pdGVkMSAwHgYDVQQDDBdlbVNpZ24gUm9vdCBUTFMgQ0EgLSBHMw==", + "serialNumber": "bK2jZ/scEACIEg6iAtVo", + "id": "a0ac0973-ae76-4860-8f9c-eff5fd57bc3b", + "last_modified": 1792274405304 + }, + { + "schema": 1792269975630, + "details": { + "bug": "https://bugzilla.mozilla.org/show_bug.cgi?id=2083382", + "who": "", + "why": "", + "name": "", + "created": "" + }, + "enabled": true, + "issuerName": "MFYxCzAJBgNVBAYTAklOMSUwIwYDVQQKDBxlTXVkaHJhIFRlY2hub2xvZ2llcyBMaW1pdGVkMSAwHgYDVQQDDBdlbVNpZ24gUm9vdCBUTFMgQ0EgLSBHMw==", + "serialNumber": "KWkmwiejoVJjkrGUgqUE", + "id": "c64301a0-888f-469d-86fc-a03bcd6bc407", + "last_modified": 1792274405300 + }, + { + "schema": 1792195211286, + "details": { + "bug": "https://bugzilla.mozilla.org/show_bug.cgi?id=2083382", + "who": "", + "why": "", + "name": "", + "created": "" + }, + "enabled": true, + "issuerName": "MFYxCzAJBgNVBAYTAklOMSUwIwYDVQQKDBxlTXVkaHJhIFRlY2hub2xvZ2llcyBMaW1pdGVkMSAwHgYDVQQDDBdlbVNpZ24gUm9vdCBUTFMgQ0EgLSBHMw==", + "serialNumber": "AMfEHv6S2CXwXq6EGkqCwQ==", + "id": "5fb952e1-b09f-4187-805c-b87a91dad9bb", + "last_modified": 1792274405297 + }, + { + "schema": 1792269999860, + "details": { + "bug": "https://bugzilla.mozilla.org/show_bug.cgi?id=2083382", + "who": "", + "why": "", + "name": "", + "created": "" + }, + "enabled": true, + "issuerName": "MFYxCzAJBgNVBAYTAklOMSUwIwYDVQQKDBxlTXVkaHJhIFRlY2hub2xvZ2llcyBMaW1pdGVkMSAwHgYDVQQDDBdlbVNpZ24gUm9vdCBUTFMgQ0EgLSBHMw==", + "serialNumber": "AKSu5njyM2K0wORgt3LyFA==", + "id": "bd595f9a-0da9-4980-bcc2-ef1e1bf70c7b", + "last_modified": 1792274405294 + }, + { + "schema": 1791318533037, + "details": { + "bug": "2078851", + "who": "", + "why": "", + "name": "", + "created": "" + }, + "enabled": true, + "issuerName": "MDsxCzAJBgNVBAYTAkVTMREwDwYDVQQKDAhGTk1ULVJDTTEZMBcGA1UECwwQQUMgUkFJWiBGTk1ULVJDTQ==", + "serialNumber": "LO0aXgKAW7xd34o67KqYWg==", + "id": "78e1876e-8832-49ee-9bd9-b36e3d53b059", + "last_modified": 1791573428759 + }, + { + "schema": 1791233394581, + "details": { + "bug": "2078851", + "who": "", + "why": "", + "name": "", + "created": "" + }, + "enabled": true, + "issuerName": "MDsxCzAJBgNVBAYTAkVTMREwDwYDVQQKDAhGTk1ULVJDTTEZMBcGA1UECwwQQUMgUkFJWiBGTk1ULVJDTQ==", + "serialNumber": "NIFgxR9e28td34nKtFczkg==", + "id": "2690f159-72d9-4a13-a7e4-ee2d2b5b7e5b", + "last_modified": 1791573428756 + }, + { + "schema": 1790874064811, + "details": { + "bug": "https://bugzilla.mozilla.org/show_bug.cgi?id=2077222", + "who": "", + "why": "", + "name": "", + "created": "" + }, + "enabled": false, + "issuerName": "MGExCzAJBgNVBAYTAkNOMTAwLgYDVQQKDCdDaGluYSBGaW5hbmNpYWwgQ2VydGlmaWNhdGlvbiBBdXRob3JpdHkxIDAeBgNVBAMMF0NGQ0EgR2xvYmFsIFJTQSBST09UIEcy", + "serialNumber": "ePkoIc73UVpF6g==", + "id": "68d677f4-b0d4-4632-aa81-a7409c34b8ee", + "last_modified": 1791233394201 + }, + { + "schema": 1790874064627, + "details": { + "bug": "https://bugzilla.mozilla.org/show_bug.cgi?id=2077222", + "who": "", + "why": "", + "name": "", + "created": "" + }, + "enabled": false, + "issuerName": "MFYxCzAJBgNVBAYTAkNOMTAwLgYDVQQKDCdDaGluYSBGaW5hbmNpYWwgQ2VydGlmaWNhdGlvbiBBdXRob3JpdHkxFTATBgNVBAMMDENGQ0EgRVYgUk9PVA==", + "serialNumber": "V1JfiijGiegkWPH+Fx5j7Q==", + "id": "d5bae32a-daac-403e-a2d0-ca2eb2231041", + "last_modified": 1791233394198 + }, + { + "schema": 1790874064530, + "details": { + "bug": "https://bugzilla.mozilla.org/show_bug.cgi?id=2077222", + "who": "", + "why": "", + "name": "", + "created": "" + }, + "enabled": false, + "issuerName": "MFoxCzAJBgNVBAYTAkpQMSYwJAYDVQQKEx1TRUNPTSBUcnVzdCBTeXN0ZW1zIENvLiwgTHRkLjEjMCEGA1UEAxMaU0VDT00gVExTIEVDQyBSb290IENBIDIwMjQ=", + "serialNumber": "T+az/YRcbK7igx8id+gKnw==", + "id": "cb98a089-ec7a-4e52-bc08-31eb43815728", + "last_modified": 1791233394194 + }, + { + "schema": 1790874064341, + "details": { + "bug": "https://bugzilla.mozilla.org/show_bug.cgi?id=2077222", + "who": "", + "why": "", + "name": "", + "created": "" + }, + "enabled": false, + "issuerName": "MF0xCzAJBgNVBAYTAkpQMSUwIwYDVQQKExxTRUNPTSBUcnVzdCBTeXN0ZW1zIENPLixMVEQuMScwJQYDVQQLEx5TZWN1cml0eSBDb21tdW5pY2F0aW9uIFJvb3RDQTI=", + "serialNumber": "IrmxL00F+e0T", + "id": "a1383d22-e6fd-4f35-8837-15ddb27ef93e", + "last_modified": 1791233394190 + }, + { + "schema": 1790874064147, + "details": { + "bug": "https://bugzilla.mozilla.org/show_bug.cgi?id=2077222", + "who": "", + "why": "", + "name": "", + "created": "" + }, + "enabled": false, + "issuerName": "MF8xCzAJBgNVBAYTAkdCMRgwFgYDVQQKEw9TZWN0aWdvIExpbWl0ZWQxNjA0BgNVBAMTLVNlY3RpZ28gUHVibGljIFNlcnZlciBBdXRoZW50aWNhdGlvbiBSb290IEU0Ng==", + "serialNumber": "AIEKq4YGXjOEvyglU57/P4g=", + "id": "637d67cd-89bf-4c25-90cd-ee7be62a591d", + "last_modified": 1791233394186 + }, + { + "schema": 1790874064043, + "details": { + "bug": "https://bugzilla.mozilla.org/show_bug.cgi?id=2077222", + "who": "", + "why": "", + "name": "", + "created": "" + }, + "enabled": false, + "issuerName": "MFoxCzAJBgNVBAYTAkpQMSYwJAYDVQQKEx1TRUNPTSBUcnVzdCBTeXN0ZW1zIENvLiwgTHRkLjEjMCEGA1UEAxMaU0VDT00gVExTIEVDQyBSb290IENBIDIwMjQ=", + "serialNumber": "T+az/DaNHzV/czSbV8b+SQ==", + "id": "489486f0-960f-4185-8afd-219198f6f558", + "last_modified": 1791233394183 + }, + { + "schema": 1790874063948, + "details": { + "bug": "https://bugzilla.mozilla.org/show_bug.cgi?id=2077222", + "who": "", + "why": "", + "name": "", + "created": "" + }, + "enabled": false, + "issuerName": "MDsxCzAJBgNVBAYTAkVTMREwDwYDVQQKDAhGTk1ULVJDTTEZMBcGA1UECwwQQUMgUkFJWiBGTk1ULVJDTQ==", + "serialNumber": "Pnz9VM3l4W1RzEmOGx7NSA==", + "id": "320d71d5-36a8-4410-9ad4-2124add204ff", + "last_modified": 1791233394178 + }, + { + "schema": 1790874063852, + "details": { + "bug": "https://bugzilla.mozilla.org/show_bug.cgi?id=2077222", + "who": "", + "why": "", + "name": "", + "created": "" + }, + "enabled": false, + "issuerName": "MFoxCzAJBgNVBAYTAkpQMSYwJAYDVQQKEx1TRUNPTSBUcnVzdCBTeXN0ZW1zIENvLiwgTHRkLjEjMCEGA1UEAxMaU0VDT00gVExTIFJTQSBSb290IENBIDIwMjQ=", + "serialNumber": "R6WRK1Xe7A7RwUFO96H4fQ==", + "id": "7891cb25-37db-4dcb-9218-c976cc847eb7", + "last_modified": 1791233394175 + }, + { + "schema": 1790874063748, + "details": { + "bug": "https://bugzilla.mozilla.org/show_bug.cgi?id=2077222", + "who": "", + "why": "", + "name": "", + "created": "" + }, + "enabled": false, + "issuerName": "MEYxCzAJBgNVBAYTAkJFMRkwFwYDVQQKExBHbG9iYWxTaWduIG52LXNhMRwwGgYDVQQDExNHbG9iYWxTaWduIFJvb3QgUjQ2", + "serialNumber": "AIRDWFKkZS7STOWUW1z2GLE=", + "id": "dd420d19-4bce-4fe6-911a-7025430247be", + "last_modified": 1791233394172 + }, + { + "schema": 1790874064721, + "details": { + "bug": "https://bugzilla.mozilla.org/show_bug.cgi?id=2077222", + "who": "", + "why": "", + "name": "", + "created": "" + }, + "enabled": false, + "issuerName": "MF8xCzAJBgNVBAYTAkdCMRgwFgYDVQQKEw9TZWN0aWdvIExpbWl0ZWQxNjA0BgNVBAMTLVNlY3RpZ28gUHVibGljIFNlcnZlciBBdXRoZW50aWNhdGlvbiBSb290IFI0Ng==", + "serialNumber": "VS0Duo3zjBUCApvCbQzk+Q==", + "id": "2823a801-82b1-43f1-858b-f740b601ddb4", + "last_modified": 1791233394168 + }, + { + "schema": 1790874064246, + "details": { + "bug": "https://bugzilla.mozilla.org/show_bug.cgi?id=2077222", + "who": "", + "why": "", + "name": "", + "created": "" + }, + "enabled": false, + "issuerName": "MFYxCzAJBgNVBAYTAkNOMTAwLgYDVQQKDCdDaGluYSBGaW5hbmNpYWwgQ2VydGlmaWNhdGlvbiBBdXRob3JpdHkxFTATBgNVBAMMDENGQ0EgRVYgUk9PVA==", + "serialNumber": "fXSiQkZ5TZ3Exw==", + "id": "0adcd89e-b78e-41c7-b10e-4d4f1ab97cc8", + "last_modified": 1791233394165 + }, + { + "schema": 1790874064436, + "details": { + "bug": "https://bugzilla.mozilla.org/show_bug.cgi?id=2077222", + "who": "", + "why": "", + "name": "", + "created": "" + }, + "enabled": false, + "issuerName": "MEgxCzAJBgNVBAYTAkRFMRUwEwYDVQQKEwxELVRydXN0IEdtYkgxIjAgBgNVBAMTGUQtVFJVU1QgRVYgUm9vdCBDQSAyIDIwMjM=", + "serialNumber": "Uu0msDCRjfC0h03kRtj2Ig==", + "id": "665bdb86-cf5b-451f-bb69-c0518c84b960", + "last_modified": 1791233394162 + }, + { + "schema": 1790553607419, + "details": { + "bug": "https://bugzilla.mozilla.org/show_bug.cgi?id=2077222", + "who": "", + "why": "", + "name": "", + "created": "" + }, + "enabled": false, + "issuerName": "MFoxCzAJBgNVBAYTAkpQMSYwJAYDVQQKEx1TRUNPTSBUcnVzdCBTeXN0ZW1zIENvLiwgTHRkLjEjMCEGA1UEAxMaU0VDT00gVExTIFJTQSBSb290IENBIDIwMjQ=", + "serialNumber": "R6WRKuXIqENrxkAQkNTByQ==", + "id": "94a3e6fb-153b-45e8-b50a-0f212970114d", + "last_modified": 1791233394158 + }, { "schema": 1781802209467, "details": { @@ -28199,5 +28769,5 @@ "last_modified": 1480349158647 } ], - "timestamp": 1781825227560 + "timestamp": 1793208925525 } ===================================== toolkit/components/tor-launcher/TorBootstrapRequest.sys.mjs ===================================== @@ -7,6 +7,8 @@ ChromeUtils.defineESModuleGetters(lazy, { "moz-src:///toolkit/components/tor-launcher/TorProviderBuilder.sys.mjs", TorProviderTopics: "moz-src:///toolkit/components/tor-launcher/TorProviderBuilder.sys.mjs", + TorBootstrapError: + "moz-src:///toolkit/components/tor-launcher/TorProviderBuilder.sys.mjs", }); const log = console.createInstance({ @@ -35,12 +37,18 @@ export class TorBootstrapRequest { #timeoutID = null; observe(subject, topic) { - const obj = subject?.wrappedJSObject; switch (topic) { case lazy.TorProviderTopics.BootstrapStatus: { - const progress = obj.PROGRESS; + let obj; + try { + obj = subject.QueryInterface(Ci.torITorBootstrapStatus); + } catch (e) { + log.warn("BootstrapStatus with an unexpected subject", e, subject); + break; + } + const progress = obj.progress; if (this.onbootstrapstatus) { - const status = obj.TAG; + const status = obj.tag; this.onbootstrapstatus(progress, status); } if (progress === 100) { @@ -55,8 +63,21 @@ export class TorBootstrapRequest { break; } case lazy.TorProviderTopics.BootstrapError: { + let obj; + try { + obj = subject.QueryInterface(Ci.torITorBootstrapStatus); + } catch (e) { + log.warn("TorBootstrapError with an unexpected subject", e, subject); + break; + } log.info("TorBootstrapRequest: observerd TorBootstrapError", obj); - this.#stop(obj); + this.#stop( + new lazy.TorBootstrapError({ + phase: obj.tag, + summary: obj.summary, + reason: obj.reason, + }) + ); break; } } ===================================== toolkit/components/tor-launcher/TorDomainIsolator.sys.mjs ===================================== @@ -288,8 +288,34 @@ class TorDomainIsolatorImpl { // TODO: What UX to use here? See tor-browser#41708 } } else if (topic === lazy.TorProviderTopics.CircuitCredentialsMatched) { - const { username, password, circuits } = subject.wrappedJSObject; - this.#updateCircuits(username, password, circuits); + let streamInfo; + try { + streamInfo = subject?.QueryInterface(Ci.torITorStreamInfo); + } catch (e) { + logger.warn( + "CircuitCredentialsMatched with invalid stream info", + e, + subject + ); + } + if (streamInfo) { + const circuits = streamInfo.circuits?.map(circuit => + circuit.map(node => { + return { + fingerprint: node.fingerprint, + ipAddrs: node.ipAddrs, + isBridge: node.isBridge, + bridgeType: node.bridgeType, + regionCode: node.regionCode, + }; + }) + ); + this.#updateCircuits( + streamInfo.socksUsername, + streamInfo.socksPassword, + circuits + ); + } } } ===================================== toolkit/components/tor-launcher/TorLauncherUtil.sys.mjs ===================================== @@ -644,17 +644,22 @@ export const TorLauncherUtil = { * Therefore, the messages should avoid containing any information that might * identify information about the user. * - * @param {string} type The log type (ERR, WARN, etc...) - * @param {string} msg The log message + * @param {string} severity The log severity (ERR, WARN, etc...) + * @param {string} message The log message */ - log(type, msg) { + log(severity, message) { const timestamp = new Date().toISOString().replace("T", " ").replace("Z", "") + " UTC"; Services.obs.notifyObservers( - { type, msg, timestamp }, + { + severity, + message, + timestamp, + QueryInterface: ChromeUtils.generateQI([Ci.torITorLogEntry]), + }, lazy.TorProviderTopics.TorLog ); - if (type === "WARN" || type === "ERR") { + if (severity === "WARN" || severity === "ERR") { // Notify so that Copy Log can be enabled. Services.obs.notifyObservers(null, lazy.TorProviderTopics.HasWarnOrErr); } ===================================== toolkit/components/tor-launcher/TorProvider.sys.mjs ===================================== @@ -7,10 +7,7 @@ import { clearTimeout, setTimeout } from "resource://gre/modules/Timer.sys.mjs"; import { TorLauncherUtil } from "moz-src:///toolkit/components/tor-launcher/TorLauncherUtil.sys.mjs"; import { TorParsers } from "moz-src:///toolkit/components/tor-launcher/TorParsers.sys.mjs"; import { TorProviderBase } from "moz-src:///toolkit/components/tor-launcher/TorProviderBase.sys.mjs"; -import { - TorBootstrapError, - TorProviderTopics, -} from "moz-src:///toolkit/components/tor-launcher/TorProviderBuilder.sys.mjs"; +import { TorProviderTopics } from "moz-src:///toolkit/components/tor-launcher/TorProviderBuilder.sys.mjs"; const lazy = {}; ChromeUtils.defineESModuleGetters(lazy, { @@ -57,12 +54,12 @@ const logger = console.createInstance({ * @typedef NodeData * @property {NodeFingerprint} fingerprint The node fingerprint * @property {string[]} ipAddrs The ip addresses associated with this node - * @property {string?} bridgeType The bridge type for this node, or "" if the - * node is a bridge but the type is unknown, or null if this is not a bridge - * node - * @property {string?} regionCode An upper case 2-letter ISO3166-1 code for the - * first ip address, or null if there is no region. This should also be a - * valid BCP47 Region subtag + * @property {boolean} isBridge Whether the node is a bridge. + * @property {string} bridgeType The bridge type for this node, or an empty + * string if the type is unknown or the node is not a bridge. + * @property {string} regionCode An upper case 2-letter ISO3166-1 code for the + * first ip address. This should also be a valid BCP47 Region subtag. + * It will be an empty string if the node is not a bridge. */ const Preferences = Object.freeze({ @@ -929,14 +926,16 @@ export class TorProvider extends TorProviderBase { const node = { fingerprint: id, ipAddrs: [], - bridgeType: null, - regionCode: null, + isBridge: false, + bridgeType: "", + regionCode: "", }; const bridge = (await this.#controller.getBridges())?.find( foundBridge => foundBridge.id?.toUpperCase() === id.toUpperCase() ); if (bridge) { + node.isBridge = true; node.bridgeType = bridge.transport ?? ""; // Attempt to get an IP address from bridge address string. const ip = bridge.addr.match(/^\[?([^\]]+)\]?:\d+$/)?.[1]; @@ -1006,7 +1005,10 @@ export class TorProvider extends TorProviderBase { */ #processBootstrapStatus(statusObj, isNotification) { // Notify observers - Services.obs.notifyObservers(statusObj, TorProviderTopics.BootstrapStatus); + Services.obs.notifyObservers( + new TorBootstrapStatus(statusObj), + TorProviderTopics.BootstrapStatus + ); if (statusObj.PROGRESS === 100) { return; @@ -1044,11 +1046,7 @@ export class TorProvider extends TorProviderBase { // anymore, since the first error eligible for notification will as a // matter of fact cancel the bootstrap. Services.obs.notifyObservers( - new TorBootstrapError({ - phase: statusObj.TAG, - reason: statusObj.REASON, - summary: statusObj.SUMMARY, - }), + new TorBootstrapStatus(statusObj), TorProviderTopics.BootstrapError ); } @@ -1147,14 +1145,24 @@ export class TorProvider extends TorProviderBase { * @param {StreamID} _streamId The ID of the stream that switched to the * sentconnect status. * @param {CircuitID} circuitId The ID of the circuit used by the stream - * @param {string} username The SOCKS username - * @param {string} password The SOCKS password + * @param {string} socksUsername The SOCKS username + * @param {string} socksPassword The SOCKS password */ - async onStreamSentConnect(_streamId, circuitId, username, password) { - if (!username || !password) { + async onStreamSentConnect( + _streamId, + circuitId, + socksUsername, + socksPassword + ) { + if (!socksUsername || !socksPassword) { return; } - logger.debug("Stream sentconnect event", username, password, circuitId); + logger.debug( + "Stream sentconnect event", + socksUsername, + socksPassword, + circuitId + ); if (!this.#circuits.has(circuitId)) { // tor-browser#42132: When using onion-grater (e.g., in Tails), we might // not receive the CIRC BUILT event, as it is impossible to know whether @@ -1194,13 +1202,33 @@ export class TorProvider extends TorProviderBase { ); Services.obs.notifyObservers( { - wrappedJSObject: { - username, - password, - circuits, - }, + socksUsername, + socksPassword, + circuits, + QueryInterface: ChromeUtils.generateQI([Ci.torITorStreamInfo]), }, TorProviderTopics.CircuitCredentialsMatched ); } } + +/** + * A JavaScript implementation of torITorBootstrapStatus. + */ +class TorBootstrapStatus { + /** + * Construct a typed status object from a status object sent by TorController. + * + * @param {object} statusObj The status object parsed by the controller + */ + constructor(statusObj) { + this.progress = statusObj.PROGRESS; + this.severity = statusObj.TYPE; + this.tag = statusObj.TAG; + this.summary = statusObj.SUMMARY; + this.reason = statusObj.REASON ?? ""; + this.shouldWarn = statusObj.RECOMMENDATION !== "ignore"; + } + + QueryInterface = ChromeUtils.generateQI([Ci.torITorBootstrapStatus]); +} ===================================== toolkit/components/tor-launcher/TorProviderBuilder.sys.mjs ===================================== @@ -81,9 +81,9 @@ export const TorProviders = Object.freeze({ /** * @typedef {object} LogEntry An object with a log message - * @property {string} timestamp The local date-time stamp at which we received the message - * @property {string} type The message level - * @property {string} msg The message + * @property {string} timestamp The UTC date-time stamp at which we received the message + * @property {string} severity The message severity + * @property {string} message The message */ /** @@ -182,7 +182,18 @@ export class TorProviderBuilder { 1000 ); this.#logObserver = subject => { - this.#addLogEntry(subject.wrappedJSObject); + let entry; + try { + entry = subject.QueryInterface(Ci.torITorLogEntry); + } catch (e) { + lazy.logger.warn("TorLog with invalid subject", e, subject); + return; + } + this.#addLogEntry({ + severity: entry.severity, + message: entry.message, + timestamp: entry.timestamp, + }); }; Services.obs.addObserver(this.#logObserver, TorProviderTopics.TorLog); ===================================== toolkit/components/tor-launcher/TorProviderMock.sys.mjs ===================================== @@ -3,19 +3,19 @@ import { TorProviderBase } from "moz-src:///toolkit/components/tor-launcher/TorP import { TorProviderTopics } from "moz-src:///toolkit/components/tor-launcher/TorProviderBuilder.sys.mjs"; const kBootstrapSteps = [ - { PROGRESS: 5, TAG: "starting", SUMMARY: "Starting" }, - { PROGRESS: 14, TAG: "handshake", SUMMARY: "Handshaking with a relay" }, + { progress: 5, tag: "starting", summary: "Starting" }, + { progress: 14, tag: "handshake", summary: "Handshaking with a relay" }, { - PROGRESS: 45, - TAG: "requesting_descriptors", - SUMMARY: "Asking for relay descriptors", + progress: 45, + tag: "requesting_descriptors", + summary: "Asking for relay descriptors", }, { - PROGRESS: 75, - TAG: "loading_descriptors", - SUMMARY: "Loading relay descriptors", + progress: 75, + tag: "loading_descriptors", + summary: "Loading relay descriptors", }, - { PROGRESS: 100, TAG: "done", SUMMARY: "Done" }, + { progress: 100, tag: "done", summary: "Done" }, ]; const kBootstrapStepDelayMs = 500; @@ -48,7 +48,15 @@ export class TorProviderMock extends TorProviderBase { const id = setTimeout( () => { Services.obs.notifyObservers( - { ...step, TYPE: "NOTICE" }, + { + ...step, + severity: "NOTICE", + reason: "", + shouldWarn: false, + QueryInterface: ChromeUtils.generateQI([ + Ci.torITorBootstrapStatus, + ]), + }, TorProviderTopics.BootstrapStatus ); }, ===================================== toolkit/components/tor-launcher/moz.build ===================================== @@ -1,3 +1,6 @@ +XPIDL_MODULE = "torlauncher" +XPIDL_SOURCES += ["tor-launcher.idl"] + MOZ_SRC_FILES += [ "TorBootstrapRequest.sys.mjs", "TorControlPort.sys.mjs", ===================================== toolkit/components/tor-launcher/tor-launcher.idl ===================================== @@ -0,0 +1,55 @@ +/* This Source Code Form is subject to the terms of the Mozilla Public + * License, v. 2.0. If a copy of the MPL was not distributed with this + * file, You can obtain one at http://mozilla.org/MPL/2.0/. */ + +#include "nsISupports.idl" + +/** + * The information about the bootstrap status. + * The values are extracted from C-tor's STATUS_CLIENT events, except for + * shouldWarn, which is derived from the RECOMMENDATION value. + */ +[scriptable, uuid(64b3197d-0b1e-46a1-aa18-8cea11bacf97)] +interface torITorBootstrapStatus : nsISupports { + readonly attribute octet progress; + readonly attribute ACString severity; + readonly attribute ACString tag; + readonly attribute ACString summary; + readonly attribute ACString reason; + readonly attribute boolean shouldWarn; +}; + +/** + * A tor log message. + * The severity might be also something not used by C-tor. + */ +[scriptable, uuid(f6d18f7d-c29c-4bf3-bf70-7d6bed3b427e)] +interface torITorLogEntry : nsISupports { + readonly attribute ACString severity; + readonly attribute ACString message; + readonly attribute ACString timestamp; +}; + +/** + * A tor node information. + * The bridgeType and regionCode should be used only when isBridge is true. + */ +[scriptable, uuid(7547d8e5-a93e-4a0c-9876-a7a2cc44e80f)] +interface torITorNodeInfo : nsISupports { + readonly attribute ACString fingerprint; + readonly attribute Array<ACString> ipAddrs; + readonly attribute boolean isBridge; + readonly attribute ACString bridgeType; + readonly attribute ACString regionCode; +}; + +/** + * The information to associate a SOCKS isolation credential (the combination + * of SOCKS username and password) to one or more circuits. + */ +[scriptable, uuid(a85269dc-1aca-4aec-87a6-88ff035f0eaa)] +interface torITorStreamInfo : nsISupports { + readonly attribute ACString socksUsername; + readonly attribute ACString socksPassword; + readonly attribute Array<Array<torITorNodeInfo> > circuits; +}; ===================================== toolkit/modules/TorAndroidIntegration.sys.mjs ===================================== @@ -130,13 +130,19 @@ class TorAndroidIntegrationImpl { {} ); break; - case lazy.TorProviderTopics.TorLog: - lazy.EventDispatcher.instance.sendRequest(EmittedEvents.torLogs, { - logType: subj.wrappedJSObject.type ?? "", - message: subj.wrappedJSObject.msg ?? "", - timestamp: subj.wrappedJSObject.timestamp ?? "", - }); + case lazy.TorProviderTopics.TorLog: { + try { + const entry = subj.QueryInterface(Ci.torITorLogEntry); + lazy.EventDispatcher.instance.sendRequest(EmittedEvents.torLogs, { + logType: entry.severity, + message: entry.message, + timestamp: entry.timestamp, + }); + } catch (e) { + logger.warn("Failed to dispatch a log event", e, subj); + } break; + } case lazy.TorSettingsTopics.Ready: lazy.EventDispatcher.instance.sendRequest(EmittedEvents.settingsReady, { settings: lazy.TorSettings.getSettings(), ===================================== tools/torbrowser/l10n/migrate.py ===================================== @@ -1,58 +1,252 @@ """ -Migration script for running migration recipes. +Migration script for running Tor Browser migration recipes. -Requires the mozilla fluent.migrate package (available with pip). +Based on the Firefox migration script, adapted for Tor Browser. See +https://firefox-source-docs.mozilla.org/l10n/migrations/ for more context and +information on how to write the migration recipes. -NOTE: This will likely send queries to the weblate API as part of the run. -Weblate may have a query limit of 100 requests per day for unauthorised users. +## Concepts + ++ String. Here, when we say "string", it refers to a piece of text that can be + translated. In practice, you can think of it as a being defined by a triple: + a file path relative to a locale's directory, an ID, and some content. For + example `("./tor-browser.ftl", "my-id", "my-id =\\n .label=Content")`. + When a string is translated the file path and ID stay the same, whilst the + translatable part of the content can vary. ++ Reference files. These are the en-US source Fluent files that contain the + *manual* string changes made by a developer. For most migrations, there is + only one reference file. ++ Migration recipe. A collection of instructions that take some string inputs + and transforms them into new strings. These are meant to mirror the manual + changes that were made in the reference files, in a way that can be applied + to any locale. ++ Translation repository. This refers to a local copy of the `translation` + repository (https://gitlab.torproject.org/tpo/translation) This repository + contains a different branch for different collections of strings. For + Tor Browser we usually use `tor-browser`, `base-browser` and + `fenix-torbrowserstringsxml`. Each branch will contain an `en-US` directory, + which contains the "source" files for translators to translate. These source + files are taken from the `tor-browser` repository. The branches also contain + a directory for each locale that is to be translated, in a file structure that + reflects the source `en-US` directory. This is where the migration script will + write changes to the translated files. + + +## Usage + +You will likely want to run this with `./mach python`, so that it uses the +`fluent.migrate` and `fluent.syntax` packages provided with the `tor-browser` +repository. + +For example, you might run this as -Based on the firefox migration script, adapted for tor browser. See -https://firefox-source-docs.mozilla.org/l10n/migrations/ for more context and -information on how to write the migration recipes. +``` + ./mach python migrate.py --translation-git /home/user/translation --locales 'es-ES sr' ./migrations/bug-xxxx-my-migration-recipe.py +``` + +Here `/home/user/translation` is a local translation repository with one of its +branches checked out (see below). This will run the migration recipe given in +`./migrations/bug-xxxx-my-migration-recipe.py` for the `es-ES` and `sr` +sub-directories within the `translation/` directory. + +### Changing the reference location + +You can optionally run the script with `--reference_directory` to read the +reference en-US files from somewhere other than the `translation` directory. +This can be useful when your manual changes have yet to reach the translation +repository, so you want to point the script to read the reference from the +`tor-browser` working directory instead. E.g. + +``` + --reference-directory /home/user/tor-browser/toolkit/locales/en-US/toolkit/global/ +``` + + +## Recipes + +Migration recipes are meant to mirror some manual change that was made to the +en-US source strings, in a way that can be reproduced in another locale. +### Example 1 -Usage with an example: +For this example, a developer wants to remove the string content from +`old-strings.properties`: -We have some en-US file "updated.ftl" in the tor-browser repository that we have -added new strings to. Some of these strings are copies or concatenations of some -already existing strings in a `.dtd`, `.properties` or `.ftl` file (including -strings within "updated.ftl"). +``` +my.old.string=Some string content +my.old.title=This is the tooltip +``` + +And add them to `tor-browser.ftl` instead, as: -So we also write a migration recipe in -`./l10n_migrations/bug-xxxx-my-migration-recipe.py` -which includes a `migrate(ctx)` method. This method takes the old strings and -create new Messages or Terms for "updated.ftl" using `ctx.add_transforms`. These -should match what was already done manually in "updated.ftl". +``` +my-new-string = Some string content + .title = This is the tooltip +``` + +For the en-US source strings, the developer simply deletes the two lines in the +`old-strings.properties` file (or the whole file, if these are the last two). +And they manually write in the content of `my-new-string` to `tor-browser.ftl`. +These manual changes will then eventually reach the `translation` repository to +be translated. + +But to save translators making the same manual changes, or re-translating, the +developer can write a migration recipe that mirror's their manual changes. In +this case, this migration recipe would contain: + +``` +ctx.add_transforms( + "tor-browser.ftl", # The path to write to. + "tor-browser.ftl", # The path to the reference file. + transforms_from( + \"\"\" +my-new-string = { COPY(path, "my.old.string") } + .title = { COPY(path, "my.old.title") } +\"\"\", + path="old-strings.properties", # The path to read from. + ), +) +``` + +Note, the first argument of `add_transforms` is the target path, which is +`tor-browser.ftl` in this case. This means that the migration script will try to +write the resulting `my-new-string` to the locale's `./tor-browser.ftl` file. +For example, to `translation/es-ES/tor-browser.ftl`. + +The second argument refers to the *reference* path. In almost all cases the +reference file will have the same name and path as the target path, but it has a +different role. Here the path is relative to the reference directory, which +contains the en-US source files. In this case, by default this means that +`translation/en-US/tor-browser.ftl` will act as the reference, but the exact +path could be changed with the `--reference-directory` argument. The existing +`my-new-string` will be read from this file to confirm that the migration +recipe produces the correct result. + +The file `old-strings.properties` is used as an input. This means that the +existing strings in the locale's `old-strings.properties` file will be read to +construct the new string `my-new-string`. For example, it will be read from +`translation/es-ES/old-strings.properties`. + +The `COPY` acts like a Fluent function, which asks the transformation to copy +the content from the translated string directly. + +### Example 2 + +For this example, a developer wants to swap a string in `tor-browser.ftl` from: + +``` +my-old-string2 = Some string content + .title = This is the tooltip +``` + +to + +``` +my-new-string2 = + .label = Some string content + .title = This is the tooltip +``` -Then we copy the en-US file to our local translation repository, say at -`/home/user/translation/en-US/updated.ftl`. The en-US file is needed as a -reference during migration to know where in the new file the new Messages or -Terms should be placed. Moreover, the migration can be verified against the -en-US file to ensure it is working as intended. +The developer removes `my-old-string2` from the file and adds `my-new-string2` +instead. This manual change will then eventually reach the `translation` +repository for `my-new-string2` to be translated. -Then we run the migration for individual locales: +The developer also writes a migration recipe to mirror this change: ``` - python migrate_l10n.py --translation-git /home/user/translation --locales 'es-ES sr' l10n_migrations.bug-xxxx-my-migration-recipe +ctx.add_transforms( + "tor-browser.ftl", + "tor-browser.ftl", + transforms_from( + \"\"\" +my-new-string2 = + .label = { COPY_PATTERN(path, "my-old-string2") } + .title = { COPY_PATTERN(path, "my-old-string2.title") } +\"\"\", + path="tor-browser.ftl", + ), +) ``` -This will first run a mock migration using the en-US localization files. This is -to help ensure that the migration will work as intended. I.e. if we ran the -migration recipe for en-US, using it as both the source of old strings and as a -reference, then each transformation using these old strings should produce a -Message or Term that is identical to the one in the reference file. +Note that in this case the locale's `tor-browser.ftl` file is being used as both +the input and the output. + +Note that for a Fluent-to-Fluent migration we use `COPY_PATTERN` rather than +`COPY`. The second argument provides an ID that either point's to the +Fluent Messages' value (roughly, the text content) by giving the Message's ID, +or one of its attributes by adding a corresponding suffix to the ID. + +### Other examples + +Migration recipes can support a lot more, including more advanced +transformations like removing or replacing HTML tags from content. There are +some existing recipes in the `./migrations` directory that can be used as a +reference. + + +## translation directory + +In most cases, you will just need to check out a `translation` branch that +contains the translated strings used by the migration recipe, as well as the +Fluent file that will be written to. Usually, this will be a branch based on +the origin/upstream `tor-browser` or `base-browser` branches. + +However, if the migration recipe requires source files that are in a different +branch to the Fluent file that is being written to, then you will need to +manually copy those files into the working directory before running the script. + + +## Script procedure -Then this will run `migrate(ctx)` for the es-ES and sr locales. Some of the -transformations will not be completed for a locale if that locale does not -contain a translated version of the old strings the transformation requires -(because there is no string to copy). +### Test en-US migration -If one of the old strings is from a `.dtd` or `.properties` file, this will -check whether they are marked as translated on weblate using its API. This -prevents copying "placeholder" en-US values into the locale's fluent files. +The script will first run a test migration using the en-US strings. It will run +the migration using the files from `translation/en-US/` as the input. In the +previous example 1, this will read `my.old.string` and `my.old.title` from +`translation/en-US/old-strings.properties`. And in example 2, this will read +`my-old-string2` from `translation/en-US/tor-browser.ftl`. -Any warnings or errors will be logged to stderr in the console. +If any of the input strings are missing, this indicates that there is likely a +mistake in the recipe or the translation directory and the script will be +aborted without running the migration for the other locales. + +Otherwise, it then compares the result of this migration with the strings found +in the reference files. In the previous example 1, this will read the entry for +`my-new-string` from `<reference-dir>/tor-browser.ftl`, where `<reference-dir>` +is `translation/en-US/` unless `--reference-directory` is given to use a +different directory. If the reference entry for `my-new-string` differs from the +entry that the recipe automatically produced, then this indicates that there is +a mistake in the recipe and the script will be aborted. + +### Locale migrations + +If the test migration passed, then the recipe will be re-run for each of the +specified locales. If the locale is missing some strings or files that a single +transformation requires, then that transformation will be skipped and the +corresponding entry won't be written to the Fluent file. Otherwise the new +entries will be appended to the existing Fluent files. + +#### Non-Fluent strings and Weblate queries + +Some non-Fluent formats, like DTD and properties, will fill untranslated strings +with the source en-US content. And we don't want these untranslated strings to +be used for migrations. There is no metadata within the files to determine this, +and simply being the same as the en-US string is not a reliable indicator. + +So for these strings the script will additionally make a remote query to the +Weblate API to fetch the metadata that indicates whether the string is +considered translated or not. If a transformation requires a string that is +marked as untranslated, it will be skipped. + +NOTE: The Weblate metadata is not always reliable, so you should still manually +review the changes. + +NOTE: When testing before, the Weblate API was limited to 100 requests per day +per IP address for unauthorised users. If this becomes an issue you may want to +run the migrations in limited batches of locales with new IP addresses. Or, you +can create a Weblate account and change this script to pass an authorisation +token with the request. """ import argparse @@ -63,142 +257,299 @@ import logging import os import sys import urllib.request +from collections.abc import Iterator +from typing import Any, NamedTuple, Protocol, cast from fluent.migrate.context import MigrationContext from fluent.migrate.errors import EmptyLocalizationError, UnreadableReferenceError from fluent.syntax import ast, parse, serialize -def in_red(string): +def in_red(string: str) -> str: """ Wrap a string so that it is shown as red in the console. + :param string: The string to wrap. + :returns: The wrapped string. """ return f"\x1b[1;91m{string}\x1b[0m" +def assert_keys( + check_dict: dict[Any, Any], expected_keys: tuple[str, ...], url: str +) -> None: + """ + Verify that the dictionary contains *at least* the expected keys, + otherwise this throws. + :param check_dict: The dictionary to check. + :param url: The url where the dictionary was fetched from. + """ + for key in expected_keys: + if key not in check_dict: + raise ValueError(f"Missing '{key}' in response for {url}") + + +class WeblateUnit: + """ + Represents a Weblate "unit": roughly corresponds to a translated string. + + See https://docs.weblate.org/en/latest/api.html#units. + """ + + def __init__(self, data: dict[Any, Any], url: str) -> None: + """ + :param data: The Weblate data response. + :param url: The URL that the data came from. + """ + assert_keys(data, ("context", "translated"), url) + string_id = data["context"] + translated = data["translated"] + if not isinstance(string_id, str): + raise ValueError(f"Unexpected 'context' type in response for {url}") + if not isinstance(translated, bool): + raise ValueError(f"Unexpected 'translated' type in response for {url}") + + self.string_id = string_id + self.translated = translated + + +class WeblateTranslation: + """ + Represents a Weblate "translation": roughly corresponds to the translation + of a file for a given locale/language. + + See https://docs.weblate.org/en/latest/api.html#translations. + """ + + def __init__(self, data: dict[Any, Any], url: str) -> None: + """ + :param data: The Weblate data response. + :param url: The URL that the data came from. + """ + assert_keys(data, ("filename", "units_list_url"), url) + locale_path = data["filename"] + units_url = data["units_list_url"] + if not isinstance(locale_path, str): + raise ValueError(f"Unexpected 'filename' type in response for {url}") + if not isinstance(units_url, str): + raise ValueError(f"Unexpected 'units_list_url' type in response for {url}") + + self.locale_path = locale_path + self.units_url = units_url + # The units, indexed by their string ID. Lazy-loaded and `None` whilst + # unfilled. + self.units: None | dict[str, WeblateUnit] = None + + +class WeblateComponent: + """ + Represents a Weblate "component": roughly corresponds to a source (en-US) + file that can be translated. + + See https://docs.weblate.org/en/latest/api.html#components. + And https://docs.weblate.org/en/latest/user/basic.html. + """ + + def __init__(self, data: dict[Any, Any], url: str) -> None: + """ + :param data: The Weblate data response. + :param url: The URL that the data came from. + """ + assert_keys(data, ("template", "translations_url", "branch"), url) + template_path = data["template"] + translations_url = data["translations_url"] + branch = data["branch"] + if not isinstance(template_path, str): + raise ValueError(f"Unexpected 'template' type in response for {url}") + if not isinstance(translations_url, str): + raise ValueError( + f"Unexpected 'translations_url' type in response for {url}" + ) + if not isinstance(branch, str): + raise ValueError(f"Unexpected 'branch' type in response for {url}") + + self.branch = branch + # The path to the source (en-US) file. + self.template_path = template_path + # The URL to fetch the component's translations from. + self.translations_url = translations_url + # The translations data, indexed by the locale file paths. Lazy-loaded + # and `None` whilst unfilled. + self.translations: None | dict[str, WeblateTranslation] = None + + class WeblateMetadata: """ - Manages and fetches string metadata from weblate. + Manages and fetches string metadata from Weblate. """ - def __init__(self): + # The translation branches that we expect to be used for Weblate. + _BROWSER_BRANCHES = ["tor-browser", "base-browser", "fenix-torbrowserstringsxml"] + _WEBLATE_API_URL = "https://hosted.weblate.org/api/" + _WEBLATE_API_TOR_COMPONENTS_URL = _WEBLATE_API_URL + "projects/tor/components/" + + def __init__(self) -> None: + """ + Create a new instance for fetching Weblate metadata. + """ self.logger = logging.getLogger("weblate-metadata") self.logger.setLevel(logging.INFO) - # The _components property will be lazy-loaded with weblate data. - # - # On weblate, for monolingual formats, the component will have a - # "template" file, which matches the en-US file path in the - # "translation" git repo. So we can find a component through a matching - # template. - # - # Each translation in that component will have a "filename" that will - # match the locale's file path in the "translation" git repo. So we can - # find a translation through a matching filename. - # - # Each translation can be queried for its units, including metadata. - # Each unit's "context" will match the string's id in the corresponding - # file. - # - # Expect the final structure to be: - # { - # template: { - # "translations-url": str, # Used for API translations query. - # "translations": { - # filename: { - # "units-url": str, # Used for API units query. - # "units": { - # context: { - # "translated": bool, - # }, - # }, - # }, - # }, - # }, - # } - self._components = None - - def _get_weblate_response(self, url): + # The _components property will be lazy-loaded with Weblate data. + self._components: dict[str, WeblateComponent] | None = None + + def _get_weblate_response(self, url: str) -> Any: + """ + Make a single query to Weblate. + :param url: The URL to fetch. + :returns: The JSON response. + """ + if not url.startswith(self._WEBLATE_API_URL): + raise ValueError(f"Unexpected URL request: {url}") + weblate_request = urllib.request.Request( url, # NOTE: can set the request header: # "Authorization": "Token WEBLATE_TOKEN" - # to get around rate-limiting + # to get around rate-limiting. + # NOTE: We empty the User-Agent to override the default + # value of "Python-urllib/{major}.{minor}". headers={"User-Agent": ""}, ) - with urllib.request.urlopen(weblate_request, timeout=20) as response: + with urllib.request.urlopen(weblate_request) as response: return json.load(response) - def _get_from_weblate(self, url): - ret = [] + def _get_from_weblate(self, url: str) -> list[dict[Any, Any]]: + """ + Fetch a list of results from Weblate. + :param url: The initial URL to fetch a response from. + :returns: The returned list of results. + """ + ret: list[dict[Any, Any]] = [] while url: response = self._get_weblate_response(url) + if not isinstance(response, dict): + raise ValueError(f"Unexpected response type for {url}") + assert_keys(response, ("results", "next"), url) + + res_val = response["results"] + if not isinstance(res_val, list): + raise ValueError(f"Unexpected 'results' type for {url}") + for res_item in res_val: + if not isinstance(res_item, dict): + raise ValueError(f"Unexpected 'results' item type for {url}") + ret.append(res_item) + # Continue to fetch the next page, if it is present. - # Expect the "next" url to be empty + # Expect the "next" url to be null # or the same url with "?page=2", etc. - url = response["next"] - ret.extend(response["results"]) + next_val = response["next"] + if next_val is not None and not isinstance(next_val, str): + raise ValueError(f"Unexpected 'next' type in response for {url}") + url = "" if next_val is None else next_val return ret - def _get_components(self): + def _get_components(self) -> dict[str, WeblateComponent]: + """ + Get all the Tor Browser Weblate components. + + We identify each Weblate component by its template path: the en-US + file path in the translation repository. + + NOTE: We assume that the template path uniquely identifies the + component, even though this is not guaranteed in general by Weblate. + + :returns: A dictionary of components, indexed by their en-US template + path. + """ + url = self._WEBLATE_API_TOR_COMPONENTS_URL if self._components is None: self._components = { - comp["template"]: { - "translations": None, - "translations-url": comp["translations_url"], - } - for comp in self._get_from_weblate( - "https://hosted.weblate.org/api/projects/tor/components/" + comp.template_path: comp + for comp in ( + WeblateComponent(data, url) for data in self._get_from_weblate(url) ) - if comp["template"] + if comp.template_path and comp.branch in self._BROWSER_BRANCHES } return self._components - def _get_translations(self, template): - component = self._get_components().get(template, None) + def _get_translations( + self, template_path: str + ) -> dict[str, WeblateTranslation] | None: + """ + Get the Weblate translations for a given Weblate component. + + We identify each translation by its file path: the locale-specific path + that the translated file is written to. + + :param template_path: The component's en-US template path. + :returns: A dictionary of translations, indexed by their locale file + paths. Or `None` if no components were found. + """ + component = self._get_components().get(template_path, None) if not component: - self.logger.warning(f"No component in weblate for {template}.") + self.logger.warning(f"No component in Weblate for {template_path}.") return None - if component["translations"] is None: - component["translations"] = { - trans["filename"]: { - "units": None, - "units-url": trans["units_list_url"], - } - for trans in self._get_from_weblate(component["translations-url"]) + if component.translations is None: + url = component.translations_url + component.translations = { + trans.locale_path: trans + for trans in ( + WeblateTranslation(data, url) + for data in self._get_from_weblate(url) + ) } - return component["translations"] + return component.translations + + def _get_units( + self, template_path: str, locale_path: str + ) -> dict[str, WeblateUnit] | None: + """ + Get the Weblate units for a given Weblate translation. + + We identify each unit by its string ID. - def _get_units(self, template, file): - translation_dict = self._get_translations(template) + :param template_path: The containing component's en-US template path. + :param locale_path: The translation's locale file path. + :returns: A dictionary of units, indexed by their string ID. Or `None` + if no translations could be found. + """ + translation_dict = self._get_translations(template_path) if translation_dict is None: return None - translation = translation_dict.get(file, None) + translation = translation_dict.get(locale_path, None) if translation is None: - self.logger.warning(f"No translation in weblate for {file}.") + self.logger.warning(f"No translation in Weblate for {locale_path}.") return None - if translation["units"] is None: - translation["units"] = { - unit["context"]: { - "translated": unit["translated"], - } - for unit in self._get_from_weblate(translation["units-url"]) + if translation.units is None: + url = translation.units_url + translation.units = { + unit.string_id: unit + for unit in ( + WeblateUnit(data, url) for data in self._get_from_weblate(url) + ) } - return translation["units"] + return translation.units - def is_translated(self, template_path, locale_path, string_id): + def is_translated(self, file_path: str, locale: str, string_id: str) -> bool: """ - Whether the given string is marked as translated on weblate. + Test whether the given string is marked as translated on Weblate. + + :param file_path: The file to check the translation of, relative to the + locale's directory. + :param locale: The locale to check for. + :param string_id: The ID of the string to check. """ + template_path = os.path.join("en-US", file_path) + locale_path = os.path.join(locale, file_path) unit_dict = self._get_units(template_path, locale_path) if unit_dict is None: return False unit = unit_dict.get(string_id, None) if unit is None: - self.logger.warning(f"No unit in weblate for {locale_path}:{string_id}.") + self.logger.warning(f"No unit in Weblate for {locale_path}:{string_id}.") return False - return unit["translated"] + return unit.translated class TorBrowserMigrationContext(MigrationContext): @@ -206,12 +557,18 @@ class TorBrowserMigrationContext(MigrationContext): Extension which adds some extra methods to use for tor-browser. """ - def __init__(self, locale, reference_dir, localization_dir): + def __init__(self, locale: str, reference_dir: str, localization_dir: str) -> None: super().__init__(locale, reference_dir, localization_dir) - def _fluent_keys(self, resource): + def _fluent_keys(self, resource: ast.Resource) -> Iterator[str]: + """ + Generate all the keys within a Fluent resource, where a key is an + identifier for a Term, a Message, a Term attribute, or a Message + attribute. + :yield: The Fluent keys. + """ # ast.Resource, want to extract all Message and Term identifiers, as - # well as their + # well as their attributes. for entry in resource.body: if not isinstance(entry, (ast.Term, ast.Message)): continue @@ -223,12 +580,13 @@ class TorBrowserMigrationContext(MigrationContext): for attr in entry.attributes: yield f"{key}.{attr.id.name}" - def tb_get_available_strings(self): + def tb_get_available_strings(self) -> set[tuple[str, str]]: """ - Return all the (path, string_id) pairs for all loaded localization + Get all the (path, string_id) pairs for all loaded localization resources. + :returns: The string pairs. """ - all_strings = set() + all_strings: set[tuple[str, str]] = set() # ctx.localization_resources is a dict containing all the loaded # localization_resources that have been added during add_transforms. # { localization_path: resource } @@ -237,14 +595,13 @@ class TorBrowserMigrationContext(MigrationContext): all_strings.update((path, key) for key in self._fluent_keys(resource)) else: # dictionary of { identifies: value } in legacy resource. - # For tor-browser, - # Only include references that differ from the en-US strings. all_strings.update((path, key) for key in resource.keys()) return all_strings - def tb_get_missing_resources(self): + def tb_get_missing_resources(self) -> set[str]: """ - Return the missing localization resources. + Get the missing localization resources. + :returns: The set of resource paths. """ return set( path @@ -253,25 +610,51 @@ class TorBrowserMigrationContext(MigrationContext): if path not in self.localization_resources ) - def tb_get_transform(self, target_path, transform_id): + def tb_get_transform( + self, target_path: str, transform_id: str + ) -> ast.Message | ast.Term | None: """ - Find the transformation node with the given id for the given path. + Find a transformation node. + :param target_path: The target path of the transformation. + :param transform_id: The ID of the transformation, AKA the string ID. + :returns: The matching transformation node, or `None` if there is no + match. """ for node in self.transforms[target_path]: if node.id.name == transform_id: return node return None - def tb_get_reference_entry(self, target_path, entry_id): + def tb_get_reference_entry( + self, target_path: str, transform_id: str + ) -> ast.Message | ast.Term | None: """ - Find the reference node that would be used for the given id and path. + Find a reference Message or Term for a given transformation. + :param target_path: The target path of the transformation. + :param transform_id: The ID of the transformation, AKA the string ID. + :returns: The matching reference entry for the transformation, or `None` + if there is no match. """ for entry in self.reference_resources[target_path].body: - if isinstance(entry, (ast.Term, ast.Message)) and entry.id.name == entry_id: + if ( + isinstance(entry, (ast.Term, ast.Message)) + and entry.id.name == transform_id + ): return entry.clone() return None +class RecipeModule(Protocol): + def migrate(self, context: MigrationContext) -> None: ... + + +class FluentSyntaxError(NamedTuple): + path: str + message: str + line: int + sample: str + + class TorBrowserMigrator: """ Performs a tor-browser migration. @@ -279,26 +662,38 @@ class TorBrowserMigrator: def __init__( self, - en_US_dir, - locale_dirs, - migrate_module, - weblate_metadata, - ): + en_US_ref_dir: str, + en_US_target_dir: str, + locale_dirs: dict[str, str], + recipe_module: RecipeModule, + weblate_metadata: WeblateMetadata, + ) -> None: + """ + :param en_US_ref_dir: The path to the en-US directory where reference + Fluent entries for the recipe can be found. + :param en_US_target_dir: The path to the en-US directory where existing + strings used by the recipe can be found. + :param locale_dirs: The collection of locales to run the migrations for. + :param recipe_module: The python module that contains the recipe + instructions. + :param weblate_metadata: The object to fetch Weblate metadata from. + """ self.logger = logging.getLogger("tor-browser-migrator") self.logger.setLevel(logging.INFO) - self.en_US_dir = en_US_dir + self.en_US_ref_dir = en_US_ref_dir + self.en_US_target_dir = en_US_target_dir self.locale_dirs = locale_dirs - self.migrate_module = migrate_module + self.recipe_module = recipe_module self.weblate_metadata = weblate_metadata - def run(self): + def run(self) -> None: """ Run the migration. """ if not self._check_en_US_resources(): sys.exit(1) - fluent_errors = [] + fluent_errors: list[FluentSyntaxError] = [] for locale, locale_dir in self.locale_dirs.items(): if locale == "en-US": print("", file=sys.stderr) @@ -316,12 +711,18 @@ class TorBrowserMigrator: "Migration does not need to be run again, but the following " "syntax errors should be fixed manually.\n" + "\n".join( - f"{in_red(full_path)}: line {line}: {message}: [[{sample}]]" - for full_path, message, line, sample in fluent_errors + f"{in_red(err.path)}: line {err.line}: {err.message}: [[{err.sample}]]" + for err in fluent_errors ) ) - def _run_locale(self, locale, locale_dir): + def _run_locale(self, locale: str, locale_dir: str) -> list[FluentSyntaxError]: + """ + Run a migration for a given locale. + :param locale: The name of the locale. + :param locale_dir: The path to the locale's directory. + :returns: A list of Fluent errors in the written files. + """ print("\n\n", file=sys.stderr) self.logger.info(f"Migrating '{in_red(locale)}' locale\n") @@ -358,7 +759,7 @@ class TorBrowserMigrator: available_strings = ctx.tb_get_available_strings() wrote_file = False - errors = [] + errors: list[FluentSyntaxError] = [] for target_path, reference in ctx.reference_resources.items(): translated_ids = [ @@ -422,13 +823,13 @@ class TorBrowserMigrator: break # Strings in legacy formats might have an entry in the file # that is just a copy of the en-US strings. - # For these we want to check the weblate metadata to ensure + # For these we want to check the Weblate metadata to ensure # it is a translated string. if not path.endswith( ".ftl" ) and not self.weblate_metadata.is_translated( - os.path.join("en-US", path), - os.path.join(locale, path), + path, + locale, string_id, ): can_transform = False @@ -436,7 +837,7 @@ class TorBrowserMigrator: f"Skipping transform {target_path}:{transform_id} " f"for '{locale}' locale because the string " f"{path}:{string_id} has not been translated on " - "weblate." + "Weblate." ) break if not can_transform: @@ -463,7 +864,9 @@ class TorBrowserMigrator: wrote_file = True # Collect any fluent parsing errors from the newly written file. errors.extend( - (full_path, message, line, sample) + FluentSyntaxError( + path=full_path, message=message, line=line, sample=sample + ) for message, line, sample in self._fluent_errors(full_content) ) @@ -471,27 +874,42 @@ class TorBrowserMigrator: self.logger.info(f"No files written for '{locale}' locale.") return errors - def _fluent_errors(self, fluent): + def _fluent_errors(self, fluent_content: str) -> Iterator[tuple[str, int, str]]: """ - Verify that the given fluent string can be parsed correctly. + Parse the given content as a Fluent file and generate all the syntax + errors. + :param fluent_content: The file content to parse. + :yield: Each encountered Fluent syntax error as a triple of the message, + the line number, and a sample. """ - resource = parse(fluent) + resource = parse(fluent_content) for entry in resource.body: if not isinstance(entry, ast.Junk): continue for annotation in entry.annotations: - line = fluent[0 : annotation.span.start].count("\n") + 1 - sample_start = max(annotation.span.start - 15, 0) - sample = "…" + fluent[sample_start : sample_start + 30] + "…" - yield annotation.message, line, sample - - def _get_migration_context(self, locale, locale_dir): - prev_missing_resources = set() + start = 0 + if annotation.span: + start = annotation.span.start + line = fluent_content[0:start].count("\n") + 1 + sample_start = max(start - 15, 0) + sample = "…" + fluent_content[sample_start : sample_start + 30] + "…" + yield annotation.message or "", line, sample + + def _get_migration_context( + self, locale: str, locale_dir: str + ) -> TorBrowserMigrationContext: + """ + Create a new migration context. + :param locale: The name of the locale. + :param locale_dir: The directory path for the locale. + :returns: The new context. + """ + prev_missing_resources: set[str] = set() while True: - ctx = TorBrowserMigrationContext(locale, self.en_US_dir, locale_dir) + ctx = TorBrowserMigrationContext(locale, self.en_US_ref_dir, locale_dir) try: - self.migrate_module.migrate(ctx) + self.recipe_module.migrate(ctx) except EmptyLocalizationError: # This case will be handled by missing_resources below. # NOTE: At the time of writing, add_transforms only throws if @@ -543,20 +961,28 @@ class TorBrowserMigrator: # before localization_resources was fully populated. prev_missing_resources = missing_resources - def _remove_if_empty(self, path): + def _remove_if_empty(self, path: str) -> None: + """ + Remove a file if it is empty. + :param path: The path to remove. + """ if os.stat(path).st_size: self.logger.warning(f"{path} is no longer empty. Not deleting.") return os.remove(path) - def _check_en_US_resources(self): - # We pass in the en-US directory as the localization directory, as well - # as the reference directory. - ctx = TorBrowserMigrationContext("en-US", self.en_US_dir, self.en_US_dir) + def _check_en_US_resources(self) -> bool: + """ + Verify that the migration recipe works as expected using en-US strings. + :returns: Whether the recipe worked as expected. + """ + ctx = TorBrowserMigrationContext( + "en-US", self.en_US_ref_dir, self.en_US_target_dir + ) have_error = False try: - self.migrate_module.migrate(ctx) + self.recipe_module.migrate(ctx) except EmptyLocalizationError: # Handle with localization_resources check. # NOTE: This throwing may have prevented further add_transforms from @@ -566,7 +992,9 @@ class TorBrowserMigrator: have_error = True except UnreadableReferenceError: # Reference filename is printed before this. - self.logger.error("Missing an en-US reference file.") + self.logger.error( + "Missing an en-US reference file. Is --reference-directory correct?" + ) return False # Check each transform would create the same entry in the target as the @@ -602,7 +1030,8 @@ class TorBrowserMigrator: reference_entry = ctx.tb_get_reference_entry(target_path, transform_id) if reference_entry is None: self.logger.error( - f"Missing en-US reference entry for transform {transform_name}" + f"Missing en-US reference entry for transform {transform_name}. " + "Is --reference-directory correct?" ) have_error = True continue @@ -625,9 +1054,50 @@ class TorBrowserMigrator: return not have_error +def main(recipe_path: str, translation_dir: str, locales: str, ref_dir: str) -> None: + """ + The main method. + :param recipe_path: The path to the recipe script. + :param translation_dir: The path to the translation repository directory. + :param locales: The locales to run the recipe for, separated by spaces. + :param ref_dir: The path to the reference directory. + """ + if not os.path.isfile(recipe_path): + print(f"{in_red(recipe_path)} is not a file.", file=sys.stderr) + sys.exit(1) + + recipe_module_spec = importlib.util.spec_from_file_location("_recipe", recipe_path) + recipe_module = cast( + RecipeModule, importlib.util.module_from_spec(recipe_module_spec) + ) + recipe_module_spec.loader.exec_module(recipe_module) + + def check_dir(path: str) -> str: + if not os.path.isdir(path): + print(f"{in_red(path)} is not a directory.", file=sys.stderr) + sys.exit(1) + return path + + translation_dir = check_dir(translation_dir) + en_US_target_dir = check_dir(os.path.join(translation_dir, "en-US")) + en_US_ref_dir = en_US_target_dir if ref_dir is None else check_dir(ref_dir) + + TorBrowserMigrator( + en_US_ref_dir, + en_US_target_dir, + { + locale: check_dir(os.path.join(translation_dir, locale)) + for locale in (l.strip() for l in locales.split(" ")) + if locale + }, + recipe_module, + WeblateMetadata(), + ).run() + + parser = argparse.ArgumentParser( description="Run a tor-browser localization migration. " - "See documentation within migrate_l10n.py." + "See documentation within migrate.py." ) parser.add_argument( "--translation-git", @@ -638,35 +1108,30 @@ parser.add_argument( parser.add_argument( "--locales", required=True, - metavar="<locale1> <locale2> ...", - help="Set of locales to restrict the migration to, separated by space.", + metavar="'<locale1> <locale2> ...'", + help="Set of locales to run the migration for, separated by space.", ) parser.add_argument( - "migration", - help="Migration to run, given as a python module. " - 'E.g. "l10n_migrations.my-migration-script".', + "--reference-directory", + metavar="<dir>", + help="Location of the en-US directory that contains the reference changes " + "that the recipe corresponds to. E.g. if the recipe constructs the string " + "`my-new-string` in `./tor-browser.ftl`, it should point to a directory " + "that contains the en-US `tor-browser.ftl` with `my-new-string` already " + "within it. Defaults to the `en-US/` sub-directory within the " + "translation-git directory.", +) +parser.add_argument( + "recipe", + help="The path to the migration recipe to run.", ) parsed_args = parser.parse_args() - -def check_dir(path): - if not os.path.isdir(path): - print(f"{in_red(path)} is not a directory.", file=sys.stderr) - sys.exit(1) - return path - - -translation_dir = check_dir(os.path.abspath(parsed_args.translation_git)) - -TorBrowserMigrator( - check_dir(os.path.join(translation_dir, "en-US")), - { - locale: check_dir(os.path.join(translation_dir, locale)) - for locale in (l.strip() for l in parsed_args.locales.split(" ")) - if locale - }, - importlib.import_module(parsed_args.migration), - WeblateMetadata(), -).run() +main( + parsed_args.recipe, + parsed_args.translation_git, + parsed_args.locales, + parsed_args.reference_directory, +) View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser/-/compare/6af75f… -- View it on GitLab: https://gitlab.torproject.org/tpo/applications/tor-browser/-/compare/6af75f… You're receiving this email because of your account on gitlab.torproject.org. Manage all notifications: https://gitlab.torproject.org/-/profile/notifications | Help: https://gitlab.torproject.org/help

1 0