February 2020 - tor-commits - lists.torproject.org

[torspec/master] Prop 312: Clarify IPv6 ORPort Binding
by teor＠torproject.org 11 Feb '20

11 Feb '20

commit 8af14dea81a2a39be2cb1f76a99a364827ec9d4e Author: teor <teor(a)torproject.org> Date: Tue Feb 11 14:11:09 2020 +1000 Prop 312: Clarify IPv6 ORPort Binding Part of 33049. --- proposals/312-relay-auto-ipv6-addr.txt | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/proposals/312-relay-auto-ipv6-addr.txt b/proposals/312-relay-auto-ipv6-addr.txt index 98c86c1..ba2ffcc 100644 --- a/proposals/312-relay-auto-ipv6-addr.txt +++ b/proposals/312-relay-auto-ipv6-addr.txt @@ -558,6 +558,12 @@ Ticket: #33073 IP address (in a single API call). Tor should support both styles of networking API. + In particular, if binding to all IPv6 addresses fails, relays should still + try to discover their public IPv6 address, and check the reachability of + that address. Some OSes may not support the IPV6_V6ONLY flag, but they may + instead bind to all addresses at runtime. (The tor install may also have + compile-time / runtime flag mismatches.) + If both reachability checks succeed, relays should publish their IPv4 and IPv6 ORPorts in their descriptor.

1 0

[torspec/master] Prop 312: Clarify OS hostname API behaviour
by teor＠torproject.org 11 Feb '20

11 Feb '20

commit e759c0ec4e6433e6d8cb010dda72b92b7477ae5e Author: teor <teor(a)torproject.org> Date: Tue Feb 11 13:38:17 2020 +1000 Prop 312: Clarify OS hostname API behaviour Part of 33049. --- proposals/312-relay-auto-ipv6-addr.txt | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/proposals/312-relay-auto-ipv6-addr.txt b/proposals/312-relay-auto-ipv6-addr.txt index 659e5a1..98c86c1 100644 --- a/proposals/312-relay-auto-ipv6-addr.txt +++ b/proposals/312-relay-auto-ipv6-addr.txt @@ -414,7 +414,8 @@ Ticket: #33073 and IPv6 addresses for: * the Address torrc option (see section 3.2.1), and * the local hostname. - However, OS APIs typically only return a single hostname. + However, OS APIs typically only return a single hostname. (Rather than a + separate hostname for IPv4 and IPv6.) For security reasons, directory authorities only use addresses that are explicitly configured in their torrc. Since hostname lookups may use DNS,

1 0

[tor-browser-build/master] Release preparations for 9.5a5
by sysrqb＠torproject.org 11 Feb '20

11 Feb '20

commit d3059ceb6288874b682076a5bf4aa1dff76511cd Author: Matthew Finkel <sysrqb(a)torproject.org> Date: Mon Feb 10 01:16:23 2020 +0000 Release preparations for 9.5a5 Versions bump and Changelog update --- projects/firefox-langpacks/config | 2 +- projects/firefox-locale-bundle/config | 2 +- projects/firefox/config | 2 +- .../tor-browser/Bundle-Data/Docs/ChangeLog.txt | 29 ++++++++++++++++++++++ projects/tor-browser/config | 4 +-- projects/tor-launcher/config | 2 +- projects/tor/config | 2 +- rbm.conf | 6 ++--- 8 files changed, 39 insertions(+), 10 deletions(-) diff --git a/projects/firefox-langpacks/config b/projects/firefox-langpacks/config index 34d6fad..17a707a 100644 --- a/projects/firefox-langpacks/config +++ b/projects/firefox-langpacks/config @@ -4,7 +4,7 @@ filename: '[% project %]-[% c("version") %]-[% c("var/osname") %]-[% c("var/buil var: ff_version: '[% pc("firefox", "var/firefox_version") %]' - ff_build: build1 + ff_build: build2 input_filename: 'dl-langpack-[% c("var/ff_arch") %]-[% c("version") %]' targets: diff --git a/projects/firefox-locale-bundle/config b/projects/firefox-locale-bundle/config index 2f4997f..2ea6db6 100644 --- a/projects/firefox-locale-bundle/config +++ b/projects/firefox-locale-bundle/config @@ -5,4 +5,4 @@ filename: '[% project %]-[% c("version") %]-[% c("var/build_id") %].tar.gz' var: use_container: 0 ff_version: '[% pc("firefox", "var/firefox_version") %]' - ff_build: build1 + ff_build: build2 diff --git a/projects/firefox/config b/projects/firefox/config index f780084..845f9c6 100644 --- a/projects/firefox/config +++ b/projects/firefox/config @@ -8,7 +8,7 @@ git_submodule: 1 gpg_keyring: torbutton.gpg var: - firefox_platform_version: 68.4.1 + firefox_platform_version: 68.5.0 firefox_version: '[% c("var/firefox_platform_version") %]esr' torbrowser_branch: 9.5 branding_directory: '[% IF c("var/android") %]mobile/android[% ELSE %]browser[% END %]/branding/alpha' diff --git a/projects/tor-browser/Bundle-Data/Docs/ChangeLog.txt b/projects/tor-browser/Bundle-Data/Docs/ChangeLog.txt index ad3cc9d..16acbbf 100644 --- a/projects/tor-browser/Bundle-Data/Docs/ChangeLog.txt +++ b/projects/tor-browser/Bundle-Data/Docs/ChangeLog.txt @@ -1,3 +1,32 @@ +Tor Browser 9.5a5 -- February 12 2020 + * All Platforms + * Update Firefox to 68.5.0esr + * Bump NoScript to 11.0.13 + * Translations update + * Bug 30237: Control port module improvements for v3 client authentication + * Bug 32891: Add new default bridges + * Bug 31395: Remove inline script in aboutTor.xhtml + * Bug 27268: Preferences clean-up in Torbutton code + * Bug 32470: Backport fix for bug 1590538 + * Bug 32414: Make Services.search.addEngine obey FPI + * Bug 32948: Make referer behavior consistent regardless of private browing mode status + * Bug 461204: Improve the random number generator for the boundaries in multipart/form-data + * Android + * Bug 30767: Custom obfs4 bridge does not work on Tor Browser for Android + * Windows + OS X + Linux + * Update Tor to 0.4.3.1-alpha + * Update Tor Launcher to 0.2.21.1 + * Translations update + * Bug 30237: Add v3 onion services client authentication prompt + * Bug 32870: Update version of pion-webrtc + * Bug 32767: Remove Disconnect search + * Bug 30237: Add v3 onion services client authentication prompt + * Build System + * Linux + * Bug 32976: Build and bundle geckodriver + * OS X + * Bug 33200: Fix permissions on bookmarks.html + Tor Browser 9.5a4 -- January 10 2020 * All Platforms * Update Firefox to 68.4.1esr diff --git a/projects/tor-browser/config b/projects/tor-browser/config index d3e8e61..7fd61a9 100644 --- a/projects/tor-browser/config +++ b/projects/tor-browser/config @@ -74,9 +74,9 @@ input_files: enable: '[% c("var/snowflake") %]' - filename: Bundle-Data enable: '[% ! c("var/android") %]' - - URL: https://addons.cdn.mozilla.net/user-media/addons/722/noscript_security_suit… + - URL: https://addons.cdn.mozilla.net/user-media/addons/722/noscript_security_suit… name: noscript - sha256sum: 1b0daa12e1b8fbc8e30e7bdb5689b12774aaa9f883cb3e9ad57943a883ce65b5 + sha256sum: ced718d9d9e719547db392ad85d24a7ad5c6b54cc0f93f9c57f2fcb44bb4de56 - filename: 'RelativeLink/start-tor-browser.desktop' enable: '[% c("var/linux") %]' - filename: 'RelativeLink/execdesktop' diff --git a/projects/tor-launcher/config b/projects/tor-launcher/config index 874ad15..fec0612 100644 --- a/projects/tor-launcher/config +++ b/projects/tor-launcher/config @@ -1,5 +1,5 @@ # vim: filetype=yaml sw=2 -version: 0.2.21 +version: 0.2.21.1 git_url: https://git.torproject.org/tor-launcher.git git_hash: '[% c("version") %]' gpg_keyring: torbutton.gpg diff --git a/projects/tor/config b/projects/tor/config index c1c41a6..9b5ba3d 100644 --- a/projects/tor/config +++ b/projects/tor/config @@ -1,6 +1,6 @@ # vim: filetype=yaml sw=2 filename: '[% project %]-[% c("version") %]-[% c("var/osname") %]-[% c("var/build_id") %]' -version: 0.4.3.1-alpha +version: 0.4.3.2-alpha git_hash: 'tor-[% c("version") %]' git_url: https://git.torproject.org/tor.git git_submodule: 1 diff --git a/rbm.conf b/rbm.conf index 77c6e4a..32b0575 100644 --- a/rbm.conf +++ b/rbm.conf @@ -24,10 +24,10 @@ buildconf: git_signtag_opt: '-s' var: - torbrowser_version: '9.5a4' - torbrowser_build: 'build2' + torbrowser_version: '9.5a5' + torbrowser_build: 'build1' torbrowser_incremental_from: - - 9.5a3 + - 9.5a4 project_name: tor-browser multi_lingual: 0 build_mar: 1

1 0

[tor-browser-build/master] Bug 30767: Support custom PT bridges on Android
by sysrqb＠torproject.org 11 Feb '20

11 Feb '20

commit 3ce57ded27baa63dfff074e9a10a62365dc3bc25 Author: Matthew Finkel <sysrqb(a)torproject.org> Date: Fri Feb 7 02:18:21 2020 +0000 Bug 30767: Support custom PT bridges on Android --- ...ustom-obfs4-bridge-does-not-work-on-Tor-B.patch | 146 +++++++++++++++++++++ projects/tor-onion-proxy-library/build | 1 + projects/tor-onion-proxy-library/config | 1 + 3 files changed, 148 insertions(+) diff --git a/projects/tor-onion-proxy-library/0001-Bug-30767-Custom-obfs4-bridge-does-not-work-on-Tor-B.patch b/projects/tor-onion-proxy-library/0001-Bug-30767-Custom-obfs4-bridge-does-not-work-on-Tor-B.patch new file mode 100644 index 0000000..2d1bc51 --- /dev/null +++ b/projects/tor-onion-proxy-library/0001-Bug-30767-Custom-obfs4-bridge-does-not-work-on-Tor-B.patch @@ -0,0 +1,146 @@ +From 0a482a749fd770827d3d6c71debb112a07e7fae3 Mon Sep 17 00:00:00 2001 +From: sisbell <shane.isbell(a)gmail.com> +Date: Thu, 14 Nov 2019 13:43:09 -0800 +Subject: [PATCH] Bug 30767: Custom obfs4 bridge does not work on Tor Browser + for Android + +TOPL fixes #134 +--- + .../thali/toronionproxy/TorConfigBuilder.java | 63 ++++++++----------- + 1 file changed, 26 insertions(+), 37 deletions(-) + +diff --git a/universal/src/main/java/com/msopentech/thali/toronionproxy/TorConfigBuilder.java b/universal/src/main/java/com/msopentech/thali/toronionproxy/TorConfigBuilder.java +index e324d9f..da9a6a2 100644 +--- a/universal/src/main/java/com/msopentech/thali/toronionproxy/TorConfigBuilder.java ++++ b/universal/src/main/java/com/msopentech/thali/toronionproxy/TorConfigBuilder.java +@@ -99,25 +99,32 @@ public final class TorConfigBuilder { + return this; + } + ++ @SettingsConfig ++ public TorConfigBuilder bridgesFromSettings() { ++ try { ++ addBridgesFromResources(); ++ } catch (IOException e) { ++ e.printStackTrace(); ++ } ++ return this; ++ } ++ + public TorConfigBuilder configurePluggableTransportsFromSettings(File pluggableTransportClient) throws IOException { +- List<String> supportedBridges = settings.getListOfSupportedBridges(); +- if (pluggableTransportClient == null || !settings.hasBridges() || supportedBridges.size() < 1) { ++ if (pluggableTransportClient == null) { + return this; + } + +- if (!pluggableTransportClient.exists() || !pluggableTransportClient.canExecute()) { ++ if (!pluggableTransportClient.exists()) { + throw new IOException("Bridge binary does not exist: " + pluggableTransportClient + .getCanonicalPath()); + } + +- if (supportedBridges.contains("obfs3") || supportedBridges.contains("obfs4")) { +- transportPluginObfs(pluggableTransportClient.getCanonicalPath()); +- } +- if (supportedBridges.contains("meek")) { +- transportPluginMeek(pluggableTransportClient.getCanonicalPath()); ++ if (!pluggableTransportClient.canExecute()) { ++ throw new IOException("Bridge binary is not executable: " + pluggableTransportClient ++ .getCanonicalPath()); + } +- String type = supportedBridges.contains("meek") ? "meek_lite" : "obfs4"; +- addBridgesFromResources(type, 2); ++ ++ transportPlugin(pluggableTransportClient.getCanonicalPath()); + return this; + } + +@@ -471,14 +478,8 @@ public final class TorConfigBuilder { + return transPort(settings.transPort()); + } + +- public TorConfigBuilder transportPluginMeek(String clientPath) { +- buffer.append("ClientTransportPlugin meek_lite exec ").append(clientPath).append('\n'); +- return this; +- } +- +- public TorConfigBuilder transportPluginObfs(String clientPath) { +- buffer.append("ClientTransportPlugin obfs3 exec ").append(clientPath).append('\n'); +- buffer.append("ClientTransportPlugin obfs4 exec ").append(clientPath).append('\n'); ++ public TorConfigBuilder transportPlugin(String clientPath) { ++ buffer.append("ClientTransportPlugin meek_lite,obfs3,obfs4 exec ").append(clientPath).append('\n'); + return this; + } + +@@ -489,7 +490,7 @@ public final class TorConfigBuilder { + + @SettingsConfig + public TorConfigBuilder useBridgesFromSettings() { +- return !settings.hasBridges() ? dontUseBridges() : this; ++ return settings.hasBridges() ? useBridges() : this; + } + + public TorConfigBuilder virtualAddressNetwork(String address) { +@@ -518,12 +519,12 @@ public final class TorConfigBuilder { + * </code> + * + */ +- TorConfigBuilder addBridgesFromResources(String type, int maxBridges) throws IOException { ++ TorConfigBuilder addBridgesFromResources() throws IOException { + if(settings.hasBridges()) { + InputStream bridgesStream = context.getInstaller().openBridgesStream(); + int formatType = bridgesStream.read(); +- if(formatType == 0) { +- addBridges(bridgesStream, type, maxBridges); ++ if (formatType == 0) { ++ addBridges(bridgesStream); + } else { + addCustomBridges(bridgesStream); + } +@@ -534,23 +535,14 @@ public final class TorConfigBuilder { + /** + * Add bridges from bridges.txt file. + */ +- private void addBridges(InputStream input, String bridgeType, int maxBridges) { +- if (input == null || isNullOrEmpty(bridgeType) || maxBridges < 1) { ++ private void addBridges(InputStream input) { ++ if (input == null) { + return; + } +- boolean hasAddedBridge = false; + List<Bridge> bridges = readBridgesFromStream(input); +- Collections.shuffle(bridges, new Random(System.nanoTime())); +- int bridgeCount = 0; + for (Bridge b : bridges) { +- if (b.type.equals(bridgeType)) { +- bridge(b.type, b.config); +- hasAddedBridge = true; +- if (++bridgeCount > maxBridges) +- break; +- } ++ bridge(b.type, b.config); + } +- if(hasAddedBridge) useBridges(); + } + + /** +@@ -560,15 +552,12 @@ public final class TorConfigBuilder { + if (input == null) { + return; + } +- boolean hasAddedBridge = false; + List<Bridge> bridges = readCustomBridgesFromStream(input); + for (Bridge b : bridges) { + if (b.type.equals("custom")) { + bridgeCustom(b.config); +- hasAddedBridge = true; + } + } +- if(hasAddedBridge) useBridges(); + } + + private static List<Bridge> readBridgesFromStream(InputStream input) { +-- +2.20.1 + diff --git a/projects/tor-onion-proxy-library/build b/projects/tor-onion-proxy-library/build index 0f2d976..9a8b31e 100644 --- a/projects/tor-onion-proxy-library/build +++ b/projects/tor-onion-proxy-library/build @@ -21,6 +21,7 @@ cd /var/tmp/build/[% project %]-[% c('version') %] # build will pull down the correct android tool versions patch -p1 < $rootdir/gradle.patch patch -p1 < $rootdir/canceldormant.patch +patch -p1 < $rootdir/0001-Bug-30767-Custom-obfs4-bridge-does-not-work-on-Tor-B.patch # Extract obfs4proxy from TorBrowser/Tor/PluggableTransports/obfs4proxy tar --strip-components=4 -xf $rootdir/[% c('input_files_by_name/obfs4') %] diff --git a/projects/tor-onion-proxy-library/config b/projects/tor-onion-proxy-library/config index ff3432c..d76949d 100644 --- a/projects/tor-onion-proxy-library/config +++ b/projects/tor-onion-proxy-library/config @@ -32,3 +32,4 @@ input_files: exec: '[% INCLUDE "fetch-gradle-dependencies" %]' - filename: gradle.patch - filename: canceldormant.patch + - filename: 0001-Bug-30767-Custom-obfs4-bridge-does-not-work-on-Tor-B.patch

1 0

[tor-browser-build/master] Bug 33200: Fix permissions on bookmarks.html
by sysrqb＠torproject.org 11 Feb '20

11 Feb '20

commit 241e805dbbefcb0860bf32a38b63fcd7edf71a68 Author: Nicolas Vigier <boklm(a)torproject.org> Date: Sun Feb 9 23:50:01 2020 +0100 Bug 33200: Fix permissions on bookmarks.html Fix a build reproducibility issue caused by different permissions on bookmarks.html. --- projects/tor-browser/build | 1 + 1 file changed, 1 insertion(+) diff --git a/projects/tor-browser/build b/projects/tor-browser/build index 2633ddc..b9732e8 100644 --- a/projects/tor-browser/build +++ b/projects/tor-browser/build @@ -170,6 +170,7 @@ cp defaults/preferences/000-tor-browser.js $rootdir mkdir -p chrome/en-US/locale/browser cp -p $rootdir/bookmarks.html chrome/en-US/locale/browser/ [% c("var/touch") %] chrome/en-US/locale/browser/bookmarks.html + chmod 600 chrome/en-US/locale/browser/bookmarks.html zip -Xm omni.ja chrome/en-US/locale/browser/bookmarks.html rm -rf chrome [% END %]

1 0

[torspec/master] Prop 311 & 312: Some changes affect IPv6 stats
by teor＠torproject.org 11 Feb '20

11 Feb '20

commit 01105c25434a7ead121d7f37adbb3a5afcd4f8a1 Author: teor <teor(a)torproject.org> Date: Mon Feb 10 12:45:49 2020 +1000 Prop 311 & 312: Some changes affect IPv6 stats When a change increases an IPv6 statistic, mention the impact in proposals 311 and 312. Related to proposal 313, which covers IPv6 statistics. Part of 33159. --- proposals/311-relay-ipv6-reachability.txt | 6 ++++++ proposals/312-relay-auto-ipv6-addr.txt | 5 +++++ 2 files changed, 11 insertions(+) diff --git a/proposals/311-relay-ipv6-reachability.txt b/proposals/311-relay-ipv6-reachability.txt index 85f089a..978dc9b 100644 --- a/proposals/311-relay-ipv6-reachability.txt +++ b/proposals/311-relay-ipv6-reachability.txt @@ -294,6 +294,12 @@ Ticket: #24404 The testing relay will confirm that test circuits can extend to both its IPv4 and IPv6 ORPorts. + Checking IPv6 ORPort reachability will create extra IPv6 connections on the + tor network. (See [Proposal 313: Relay IPv6 Statistics].) It won't directly + create much extra traffic, because reachability circuits don't send many + cells. But some client circuits may use the IPv6 connections created by + relay reachability self-tests. + 4.2.1. Selecting the Final Extending Relay IPv6 ORPort reachability checks require an IPv6 extend-capable relay as diff --git a/proposals/312-relay-auto-ipv6-addr.txt b/proposals/312-relay-auto-ipv6-addr.txt index 08fbcde..8914cb7 100644 --- a/proposals/312-relay-auto-ipv6-addr.txt +++ b/proposals/312-relay-auto-ipv6-addr.txt @@ -487,6 +487,11 @@ Ticket: #33073 IPv6 address, tor should use that address for reachability checks. If the reachability checks succeed, tor should use that address in its descriptor. + Doing relay directory fetches over IPv6 will create extra IPv6 connections + and IPv6 bandwidth on the tor network. (See + [Proposal 313: Relay IPv6 Statistics].) In addition, some client circuits + may use the IPv6 connections created by relay directory fetches. + 3.2.6. Disabling IPv6 Address Resolution Relays (and bridges) that have a reachable IPv6 address, but that address

1 0

[torspec/master] Prop 311: Add Optional IPv6 Bandwidth Self-Tests
by teor＠torproject.org 11 Feb '20

11 Feb '20

commit 58000de4faea96761d9e87dcd7cb1fcef63e961e Author: teor <teor(a)torproject.org> Date: Mon Feb 10 12:46:12 2020 +1000 Prop 311: Add Optional IPv6 Bandwidth Self-Tests Add an optional change that makes relays do bandwidth self-tests over IPv6. This change increases the connection and bandwidth IPv6 statistics. Related to proposal 313, which covers IPv6 statistics. Related to 33159. --- proposals/311-relay-ipv6-reachability.txt | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/proposals/311-relay-ipv6-reachability.txt b/proposals/311-relay-ipv6-reachability.txt index 978dc9b..e4c9414 100644 --- a/proposals/311-relay-ipv6-reachability.txt +++ b/proposals/311-relay-ipv6-reachability.txt @@ -573,6 +573,27 @@ Ticket: #24404 network makes it more likely that an existing connection can be used for the final hop of a relay IPv6 ORPort reachability check. +4.4.5. Relay Bandwidth Self-Tests Over IPv4 and IPv6 + + In this proposal, we only make relays (and bridges) use IPv6 for their + reachability self-tests. + + We propose this optional change, to improve the accuracy of relay (and + bridge) bandwidth self-tests: + + Relays (and bridges) perform bandwidth self-tests over IPv4 and IPv6. + + If we implement good abstractions for relay self-tests, then this change + will not need much extra code. + + If we implement IPv6 extends for all relay circuits (see section 4.4.4), + then this change will effectively be redundant. + + Doing relay bandwidth self-tests over IPv6 will create extra IPv6 + connections and IPv6 bandwidth on the tor network. (See + [Proposal 313: Relay IPv6 Statistics].) In addition, some client circuits + may use the IPv6 connections created by relay bandwidth self-tests. + 4.5. Alternate Reachability Designs We briefly mention some potential reachability designs, and the reasons that

1 0

[torspec/master] Prop 312: Clarify IPv6 Directory Failures
by teor＠torproject.org 11 Feb '20

11 Feb '20

commit aa53246772db5c0246ac71fb5c97acfac5854318 Author: teor <teor(a)torproject.org> Date: Mon Feb 10 12:49:43 2020 +1000 Prop 312: Clarify IPv6 Directory Failures Part of 33073. --- proposals/312-relay-auto-ipv6-addr.txt | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/proposals/312-relay-auto-ipv6-addr.txt b/proposals/312-relay-auto-ipv6-addr.txt index 8914cb7..c384d4c 100644 --- a/proposals/312-relay-auto-ipv6-addr.txt +++ b/proposals/312-relay-auto-ipv6-addr.txt @@ -465,7 +465,9 @@ Ticket: #33073 * explicitly configured with an IPv6 address, or * a publicly routable, reachable IPv6 address is discovered in an earlier step, - tor should start issuing IPv6 directory failure logs at warning level. + tor should start issuing IPv6 directory failure logs at warning level. Tor + may also record these directory failures as remote relay failures. (Rather + than ignoring them, as described in the previous paragraph.) (Alternately, tor could stop doing IPv6 directory requests entirely. But we prefer designs where all relays behave in a similar way, regardless of their

1 0

[torspec/master] Prop 311 & 312: Remove TODOs on Prop 313 link
by teor＠torproject.org 11 Feb '20

11 Feb '20

commit 638a9276d8cb622278efbe6ececf8985bef6de65 Author: teor <teor(a)torproject.org> Date: Mon Feb 10 12:50:25 2020 +1000 Prop 311 & 312: Remove TODOs on Prop 313 link Part of 33159. --- proposals/311-relay-ipv6-reachability.txt | 1 - proposals/312-relay-auto-ipv6-addr.txt | 2 +- 2 files changed, 1 insertion(+), 2 deletions(-) diff --git a/proposals/311-relay-ipv6-reachability.txt b/proposals/311-relay-ipv6-reachability.txt index e4c9414..9f536d3 100644 --- a/proposals/311-relay-ipv6-reachability.txt +++ b/proposals/311-relay-ipv6-reachability.txt @@ -770,7 +770,6 @@ References: [Proposal 313: Relay IPv6 Statistics]: https://gitweb.torproject.org/torspec.git/tree/proposals/313-relay-ipv6-sta… - (TODO) [Relay Search]: https://metrics.torproject.org/rs.html diff --git a/proposals/312-relay-auto-ipv6-addr.txt b/proposals/312-relay-auto-ipv6-addr.txt index c384d4c..5228c6c 100644 --- a/proposals/312-relay-auto-ipv6-addr.txt +++ b/proposals/312-relay-auto-ipv6-addr.txt @@ -1518,7 +1518,7 @@ References: https://gitweb.torproject.org/torspec.git/tree/proposals/311-relay-ipv6-rea… [Proposal 313: Relay IPv6 Statistics]: - https://gitweb.torproject.org/torspec.git/tree/proposals/313-relay-ipv6-sta… (TODO) + https://gitweb.torproject.org/torspec.git/tree/proposals/313-relay-ipv6-sta… [RFC 4941: Privacy Extensions for IPv6]: https://tools.ietf.org/html/rfc4941

1 0

[torspec/master] Prop 313: Relay IPv6 Statistics - Initial Draft
by teor＠torproject.org 11 Feb '20

11 Feb '20

commit 27de9d175b363f2ea744692d705319be06aed3f0 Author: teor <teor(a)torproject.org> Date: Mon Feb 10 16:11:14 2020 +1000 Prop 313: Relay IPv6 Statistics - Initial Draft Related to tickets: 33159 (proposal), 33051 and 33052 (implementation). --- proposals/313-relay-ipv6-stats.txt | 399 +++++++++++++++++++++++++++++++++++++ 1 file changed, 399 insertions(+) diff --git a/proposals/313-relay-ipv6-stats.txt b/proposals/313-relay-ipv6-stats.txt new file mode 100644 index 0000000..2452918 --- /dev/null +++ b/proposals/313-relay-ipv6-stats.txt @@ -0,0 +1,399 @@ +Filename: 313-relay-ipv6-stats.txt +Title: Relay IPv6 Statistics +Author: teor +Created: 10-February-2020 +Status: Draft +Ticket: #33159 + +0. Abstract + + We propose that Tor relays (and bridges) should log the number of relays in + the consensus that support IPv6 extends, and IPv6 client connections. + + We also propose that Tor relays (and bridges) should collect statistics on + IPv6 connections and consumed bandwidth. Like tor's existing connection + and consumed bandwidth statistics, these new IPv6 statistics will be + published in each relay's extra-info descriptor. + +1. Introduction + + Tor relays (and bridges) can accept IPv6 client connections via their + ORPort. But current versions of tor need to have an explicitly configured + IPv6 address (see [Proposal 312: Relay Auto IPv6 Address]), and they don't + perform IPv6 reachability self-checks (see + [Proposal 311: Relay IPv6 Reachability]). + + As we implement these new IPv6 features in tor, we want to monitor their + impact on the IPv6 connections and bandwidth in the tor network. + + Tor developers also need to know how many relays support these new IPv6 + features, so they can test tor's IPv6 reachability checks. (In particular, + see section 4.3.1 in [Proposal 311: Relay IPv6 Reachability]: Refusing to + Publish the Descriptor.) + +2. Scope + + This proposal modifies Tor's behaviour as follows: + + Relays, bridges, and directory authorities log the number of relays that + support IPv6 clients, and IPv6 relay reachability checks. They also log the + corresponding consensus weight fractions. + + As an optional change, tor clients may also log this information. + + Relays, bridges, and directory authorities collect statistics on: + * IPv6 connections, and + * IPv6 consumed bandwidth. + The design of these statistics will be based on tor's existing connection + and consumed bandwidth statistics. + + Tor's existing consumed bandwidth statistics truncate their totals to the + nearest kilobyte. The existing connection statistics do not perform any + binning. + + We do not proposed to add any extra noise or binning to these statistics. + Instead, we expect to leave these changes until we have a consistent + privacy-preserving statistics framwework for tor. As an example of this + kind of framework, see + [Proposal 288: Privacy-Preserving Stats with Privcount (Shamir version)]. + + We avoid: + * splitting connection statistics into clients and relays, and + * collecting circuit statistics. + These statistics are more sensitive, so we want to implement + privacy-preserving statistics, before we consider adding them. + + Throughout this proposal, "relays" includes directory authorities, except + where they are specifically excluded. "relays" does not include bridges, + except where they are specifically included. (The first mention of "relays" + in each section should specifically exclude or include these other roles.) + + Tor clients do not collect any statistics for public reporting. Therefore, + clients are out of scope in this proposal. (Except for some optional changes + to client logs, where they log the number of IPv6 relays in the consensus). + + When this proposal describes Tor's current behaviour, it covers all + supported Tor versions (0.3.5.7 to 0.4.2.5), as of January 2020, except + where another version is specifically mentioned. + +3. Logging IPv6 Relays in the Consensus + + We propose that relays (and bridges) log: + * the number of relays, and + * the consensus weight fraction of relays, + in the consensus that: + * have an IPv6 ORPort, + * support IPv6 reachability checks, and + * support IPv6 clients. + + In order to test these changes, and provide easy access to these + statistics, we propose implementing a script that: + * downloads a consensus, and + * calculates and reports these statistics. + + As well as the statistics listed above, this script should also report the + following relay statistic: + * support IPv6 reachability checks and IPv6 clients. + + The following consensus weight fractions should divide by the total + consensus weight: + * have an IPv6 ORPort (all relays have an IPv4 ORPort), and + * support IPv6 reachability checks (all relays support IPv4 reachability). + + The following consensus weight fractions should divide by the + "usable Guard" consensus weight: + * support IPv6 clients, and + * support IPv6 reachability checks and IPv6 clients. + + "Usable Guards" have the Guard flag, but do not have the Exit flag. If the + Guard also has the BadExit flag, the Exit flag should be ignored. + + We propose that these logs happen whenever tor: + * receives a consensus from a directory server, or + * loads a live, valid, cached consensus from disk. + + As an optional change, tor clients may also log this information. Some of + this information is not directly relevant to clients, but these logs may + help developers (and users). + +4. Collecting IPv6 Consumed Bandwidth Statistics + + We propose that relays (and bridges) collect IPv6 consumed bandwidth + statistics. + + To minimise development and testing effort, we propose re-using the existing + "bw_array" code in rephist.c. + + In particular, tor currently counts these bandwidth statistics: + * read, + * write, + * dir_read, and + * dir_write. + + We propose adding the following bandwidth statistics: + * ipv6_read, and + * ipv6_write. + (The IPv4 statistics can be calculated by subtracting the IPv6 statistics + from the existing total consumed bandwidth statistics.) + + We propose adding a new BandwidthStatistics torrc option and consensus + parameter, which activates reporting of all these statistics. Currently, + the existing statistics are controlled by ExtraInfoStatistics, but we + propose using the new BandwidthStatistics option for them as well. + + The default value of this option should be "auto", which checks the + consensus parameter. If there is no consensus parameter, the default should + be 1. (The existing bandwidth statistics are reported by default.) + + TODO: Should we collect IPv6 bandwidth statistics on bridges? + On bridges, should bandwidth statistics be on or off by default? + + If we do want to collect bridge statistics, and we think it's safe, + modify proposals 311 and 312 to allow bridge statistics. + +5. Collecting IPv6 Connection Statistics + + We propose that relays (and bridges) collect IPv6 connection statistics. + + To minimise development and testing effort, we propose re-using the existing + "bidi" code in rephist.c. (This code may require some refactoring, because + the "bidi" totals are globals, rather than a struct.) + + In particular, tor currently counts these connection statistics: + * below threshold, + * mostly read, + * mostly written, and + * both read and written. + + We propose adding IPv6 variants of all these statistics. (The IPv4 + statistics can be calculated by subtracting the IPv6 statistics from the + existing total connection statistics.) + + We propose using the existing ConnDirectionStatistics torrc option, and + adding a consensus parameter with the same name. This option will control + the new and existing connection statistics. + + The default value of this option should be "auto", which checks the + consensus parameter. If there is no consensus parameter, the default should + be 0. (The existing connection direction statistics are reported by + default.) + + TODO: Do enough relays report ConnDirectionStatistics, for accurate IPv6 + connection statistics? + * at least 25% of relays have IPv6 + * at the end of the project, we expect at least 33% of relays to have + deployed tor 0.4.4-stable + + If not, we should turn on ConnDirectionStatistics by default. (Or set the + consensus parameter for a few days, to collect these statistics.) + +6. Directory Protocol Specification Changes + + We propose adding IPv6 variants of the consumed bandwidth and connection + direction statistics to the tor directory protocol. + + We propose the following additions to the [Tor Directory Protocol] + specification, in section 2.1.2. Each addition should be inserted below the + existing consumed bandwidth and connection direction specifications. + + "ipv6-read-history" YYYY-MM-DD HH:MM:SS (NSEC s) NUM,NUM,NUM... NL + [At most once] + "ipv6-write-history" YYYY-MM-DD HH:MM:SS (NSEC s) NUM,NUM,NUM... NL + [At most once] + + Declare how much bandwidth the OR has used recently, on IPv6 + connections. See "read-history" and "write-history" for more details. + (The migration notes do not apply to IPv6.) + + "ipv6-conn-bi-direct" YYYY-MM-DD HH:MM:SS (NSEC s) BELOW,READ,WRITE,BOTH NL + [At most once] + + Number of IPv6 connections, that are used uni-directionally or + bi-directionally. See "conn-bi-direct" for more details. + + We also propose the following replacement, in the same section: + + "dirreq-read-history" YYYY-MM-DD HH:MM:SS (NSEC s) NUM,NUM,NUM... NL + [At most once] + "dirreq-write-history" YYYY-MM-DD HH:MM:SS (NSEC s) NUM,NUM,NUM... NL + [At most once] + + Declare how much bandwidth the OR has spent on answering directory + requests. See "read-history" and "write-history" for more details. + (The migration notes do not apply to dirreq.) + + This replacement is optional, but it may avoid the 3 *read-history + definitions getting out of sync. + +7. Optional Changes + + We propose some optional changes to help relay operators, tor developers, + and tor's network health. We also expect that these changes will drive IPv6 + relay adoption. + + Some of these changes may be more appropriate as future work, or along with + other proposed features. + +7.1. Log IPv6 Statistics in Tor's Heartbeat Logs + + We propose this optional change, so relay operators can see their own IPv6 + statistics: + + We propose that tor logs its IPv6 consumed bandwidth and connection + statistics in its regular "heartbeat" logs. + + These heartbeat statistics should be collected over the lifetime of the tor + process, rather than using the state file, like the statistics in sections + 4 and 5. + + Tor's existing heartbeat logs already show its consumed bandwidth and + connections (in the link protocol counts). + + We may also want to show IPv6 consumed bandwidth and connections as a + propotion of the total consumed bandwidth and connections. + + These statistics only show a relay's local bandwidth usage, so they can't + be used for reporting. + +7.2. Show IPv6 Statistics on Consensus Health + + The [Consensus Health] website displays a wide rage of tor statistics, + based on the most recent consensus. + + We propose this optional change, to: + * help relay operators diagnose issues with IPv6 on their relays, and + * to drive IPv6 adoption on tor relays. + + Consensus Health adds an IPv6 section, with the following relay statistics: + * have an IPv6 ORPort, + * support IPv6 reachability checks, + * support IPv6 clients, and + * support IPv6 reachability checks and IPv6 clients. + + The definitions of these statistics are in section 3. + + These changes can be tested using the script proposed in section 3. + +7.3. Add an IPv6 Reachability Pseudo-Flag on Relay Search + + The [Relay Search] website displays tor relay information, based on the + current consensus and relay descriptors. + + We propose this optional change, to: + * help relay operators diagnose issues with IPv6 on their relays, and + * drive IPv6 adoption on tor relays. + + Relay Search adds a pseudo-flag for relay IPv6 reachability support. + + This pseudo-flag should be given to relays that have: + * a reachable IPv6 ORPort (in the consensus), and + * support tor subprotocol version "Relay=3" (or later). + See [Proposal 311: Relay IPv6 Reachability] for details. + + TODO: Is this a useful change? + Are there better ways of driving IPv6 adoption? + +7.4. Add IPv6 Connections and Consumed Bandwidth Graphs to Tor Metrics + + The [Tor Metrics: Traffic] website displays connection and bandwidth + information for the tor network, based on relay extra-info descriptors. + + We propose these optional changes, to: + * help tor developers improve IPv6 support on the tor network, + * help diagnose issues with IPv6 on the tor network, and + * drive IPv6 adoption on tor relays. + + Tor Metrics adds the following information to the graphs on the Traffic + page: + + Consumed Bandwidth by IP version + * added to the existing [Tor Metrics: Advertised bandwidth by IP version] + page + * as a stacked graph, like + [Tor Metrics: Advertised and consumed bandwidth by relay flags] + + Fraction of connections used uni-/bidirectionally by IP version + * added to the existing + [Tor Metrics: Fraction of connections used uni-/bidirectionally] page + * as a stacked graph, like + [Tor Metrics: Advertised and consumed bandwidth by relay flags] + +8. Test Plan + + We provide a quick summary of our testing plans. + +8.1. Testing IPv6 Relay Consensus Count Logging + + We propose to test these changes using chutney networks. However, chutney + creates a limited number of relays, so we also need to test these changes + on the public tor network. + + Therefore, we propose to test these changes on the public network with a + small number of relays and bridges. + + We can use the script and the tor logs to cross-check these calculations. + The Tor Metrics team may also independently check these calculations. + + Once these changes are merged, they will be monitored by tor developers, as + more volunteer relay operators deploy the relevant tor versions. (And as the + number of IPv6 relays in the consensus increases.) + +8.2. Testing IPv6 Extra-Info Statistics + + We propose to test the connection and consumed bandwidth statistics using + chutney networks. However, chutney runs for a short amount of time, and + creates a limited amount of traffic, so we also need to test these changes + on the public tor network. + + In particular, we have struggled to test statistics using chutney, because + tor's hard-coded statistics period is 24 hours. (And most chutney networks + run for under 1 minute.) + + Therefore, we propose to test these changes on the public network with a + small number of relays and bridges. + + During 2020, the Tor Metrics team will analyse these statistics on the + public tor network, and provide IPv6 progress reports. We expect that we may + discover some bugs during the first analysis. + + Once these changes are merged, they will be monitored by tor developers, as + more volunteer relay operators deploy the relevant tor versions. (And as the + number of IPv6 relays in the consensus increases.) + +References: + +[Consensus Health]: + https://consensus-health.torproject.org/ + +[Proposal 288: Privacy-Preserving Stats with Privcount (Shamir version)]: + https://gitweb.torproject.org/torspec.git/tree/proposals/288-privcount-with… + +[Proposal 311: Relay IPv6 Reachability]: + https://gitweb.torproject.org/torspec.git/tree/proposals/311-relay-ipv6-rea… + +[Proposal 312: Relay Auto IPv6 Address]: + https://gitweb.torproject.org/torspec.git/tree/proposals/312-relay-auto-ipv… + +[Relay Search]: + https://metrics.torproject.org/rs.html + +[Tor Directory Protocol]: + (version 3) https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt + +[Tor Manual Page]: + https://2019.www.torproject.org/docs/tor-manual.html.en + +[Tor Metrics: Advertised and consumed bandwidth by relay flags]: + https://metrics.torproject.org/bandwidth-flags.html + +[Tor Metrics: Advertised bandwidth by IP version]: + https://metrics.torproject.org/advbw-ipv6.html + +[Tor Metrics: Fraction of connections used uni-/bidirectionally]: + https://metrics.torproject.org/connbidirect.html + +[Tor Metrics: Traffic]: + https://metrics.torproject.org/bandwidth-flags.html + +[Tor Specification]: + https://gitweb.torproject.org/torspec.git/tree/tor-spec.txt

1 0