- tbb-commits - lists.torproject.org

[torbutton/master] Bug 34196: add page info string for Onion Name
by sysrqb＠torproject.org 18 May '20

18 May '20

commit 62a78acdf5517c3cc5a08d9f419a5d3d3131cb44 Author: Alex Catarineu <acat(a)torproject.org> Date: Fri May 15 10:29:07 2020 +0200 Bug 34196: add page info string for Onion Name --- chrome/locale/en-US/torbutton.properties | 1 + 1 file changed, 1 insertion(+) diff --git a/chrome/locale/en-US/torbutton.properties b/chrome/locale/en-US/torbutton.properties index 507fa8d9..7b6c2d65 100644 --- a/chrome/locale/en-US/torbutton.properties +++ b/chrome/locale/en-US/torbutton.properties @@ -60,6 +60,7 @@ updateDownloadingPanelUILabel=Downloading %S update # .Onion Page Info prompt. Strings are kept here for ease of translation. pageInfo_OnionEncryptionWithBitsAndProtocol=Connection Encrypted (Onion Service, %1$S, %2$S bit keys, %3$S) pageInfo_OnionEncryption=Connection Encrypted (Onion Service) +pageInfo_OnionName=Onion Name: # Onion services strings. Strings are kept here for ease of translation. onionServices.learnMore=Learn more

1 0

[tor-browser-build/master] Bug 34247: Update and fix errors in README
by gk＠torproject.org 18 May '20

18 May '20

commit 2c48c7178644bc4ba219791ed56e4078ed9e9d18 Author: Georg Koppen <gk(a)torproject.org> Date: Mon May 18 07:22:16 2020 +0000 Bug 34247: Update and fix errors in README --- README | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/README b/README index 8e97e2f..a95f466 100644 --- a/README +++ b/README @@ -5,9 +5,7 @@ Installing build dependencies ----------------------------- To build Tor Browser, you need a Linux distribution that has support -for runc (such as Debian jessie, Ubuntu 16.04, Fedora 20, etc ...). -On Debian jessie, the runc package is available in backports. On Debian -stretch, the runc package is available in the main repository. +for runc (such as Debian Buster, Ubuntu 16.04, Fedora 30, etc ...). Your user account should have sudo access, which is required to be able to extract container file systems, start containers and copy files to and @@ -71,6 +69,7 @@ name to the makefile target: $ make nightly-linux-x86_64 $ make nightly-linux-i686 $ make nightly-windows-i686 + $ make nightly-windows-x86_64 $ make nightly-osx-x86_64 $ make nightly-android-armv7 $ make nightly-android-aarch64 @@ -81,13 +80,15 @@ When you want to quickly do a build to test a change, you can use the testbuild makefile target, and find the build in the testbuild directory. The build will be the same as regular alpha builds, except that in order to make the build faster, only the en-US locale will be built, and no -mar file will be created. +mar file will be created. If you want to base your testbuild on the latest +nightly code insted, rename rbm.local.conf.example to rbm.local.conf +and adapt the torbrowser-testbuild option accordingly. Updating git sources -------------------- -You can run "make fetch" to fetch the latest sources from git for all +You can run `make fetch` to fetch the latest sources from git for all components included in Tor Browser. You should run this if you want to make a nightly build with the latest commits, and you disabled automatic fetching of new commits for nightly builds in rbm.local.conf. @@ -122,14 +123,14 @@ option. You can look at the Makefile to find the rbm command for what you want to build, and add the --output-dir option. For example, if you want to build Tor Browser nightly for linux-x86_64: - ./rbm/rbm build release --output-dir=/var/builds/nightly/2017-01-23 \ + ./rbm/rbm build release --output-dir=/var/builds/nightly/2020-05-23 \ --target nightly --target torbrowser-linux-x86_64 The files will be put in the directory selected by --output-dir in a subdirectory named as the version number (or current date for nightly). To remove this version subdirectory, add the noversiondir target: - ./rbm/rbm build release --output-dir=/var/builds/nightly/2017-01-23 \ + ./rbm/rbm build release --output-dir=/var/builds/nightly/2020-05-23 \ --target nightly --target torbrowser-linux-x86_64 \ --target noversiondir @@ -166,14 +167,14 @@ in the rbm.local.conf file. Cleaning obsolete files and containers images --------------------------------------------- -You can run 'make clean' to clean old build files and containers that +You can run `make clean` to clean old build files and containers that are no longer used in current builds. Before doing that, you need to configure the branches and build targets you are using in the rbm.local.conf file. The cleaning script will check out all the configured branches to create a list of used build files, and delete the files from the 'out' directory that are not used. If you want to see the list of files and containers that would be removed without doing it, you can -use 'make clean-dry-run'. +use `make clean-dry-run`. Building without containers (Android builds only) @@ -181,12 +182,10 @@ Building without containers (Android builds only) By default the build is done inside containers. Adding the no_containers target will disable the use of containers. The following commands can -be used to build the alpha version for android-armv7 and android-x86: +be used to build the alpha version for e.g. android-armv7: ./rbm/rbm build release --target no_containers --target testbuild \ --target torbrowser-android-armv7 - ./rbm/rbm build release --target no_containers --target testbuild \ - --target torbrowser-android-x86 Note: the logs will still show the use and creation of a container image called "containers_disabled". This is due to the way we disable the use @@ -198,7 +197,7 @@ builds, and will require that you run Debian Stretch and install build dependencies for all the components that are built. This can be done with the following command: - # apt-get install build-essential python automake libtool zip unzip + # apt-get install build-essential python automake libtool zip unzip \ autoconf2.13 openjdk-8-jdk gettext-base autotools-dev \ automake autoconf libtool autopoint libssl-dev \ pkg-config zlib1g-dev libparallel-forkmanager-perl \ @@ -218,9 +217,10 @@ Hacking on the Tor Browser build The file README.HACKING tries to list the main things to know when making changes to the Tor Browser build. + Description of makefile rules ----------------------------- -You can find a description of the makefile rules in the README.MAKEFILE +You can find a description of the Makefile rules in the README.MAKEFILE file.

1 0

[tor-browser-build/master] Bug 33113: Bump NSIS version to 3.05
by sysrqb＠torproject.org 15 May '20

15 May '20

commit c6ef475b9388e935d77722a4d5fec860b372a6a5 Author: Georg Koppen <gk(a)torproject.org> Date: Sat May 2 09:36:54 2020 +0000 Bug 33113: Bump NSIS version to 3.05 --- projects/nsis/config | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/projects/nsis/config b/projects/nsis/config index 0e6f7ad..ed9823d 100644 --- a/projects/nsis/config +++ b/projects/nsis/config @@ -1,5 +1,5 @@ # vim: filetype=yaml sw=2 -version: 3.04 +version: 3.05 filename: 'nsis-[% c("version") %]-[% c("var/osname") %]-[% c("var/build_id") %].tar.gz' var: container: @@ -16,7 +16,7 @@ input_files: - project: container-image - filename: 'nsis-[% c("version") %].tar.bz2' URL: 'https://downloads.sourceforge.net/nsis/nsis-[% c("version") %]-src.tar.bz2' - sha256sum: 609536046c50f35cfd909dd7df2ab38f2e835d0da3c1048aa0d48c59c5a4f4f5 + sha256sum: b6e1b309ab907086c6797618ab2879cb95387ec144dab36656b0b5fb77e97ce9 - filename: no-insert-timestamp.patch - name: '[% c("var/compiler") %]' project: '[% c("var/compiler") %]'

1 0

[tor-browser-build/master] Merge remote-tracking branch 'gktpo/bug_33113_v2'
by sysrqb＠torproject.org 15 May '20

15 May '20

commit a4bf4b8a4fee918852c098edde7c89f784720d5b Merge: 8547db1 c6ef475 Author: Matthew Finkel <sysrqb(a)torproject.org> Date: Fri May 15 23:27:50 2020 +0000 Merge remote-tracking branch 'gktpo/bug_33113_v2' projects/nsis/config | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)

1 0

[tor-browser/tor-browser-68.8.0esr-9.5-1] (ESR68) Bug 1511941 - Don't expose PerformanceNavigationTiming in RFP mode
by sysrqb＠torproject.org 15 May '20

15 May '20

commit 6d7f6ea31a7629d3df6fd09fc579b5a60e599c35 Author: sanketh <sgmenda(a)uwaterloo.ca> Date: Sun May 10 16:50:41 2020 +0000 (ESR68) Bug 1511941 - Don't expose PerformanceNavigationTiming in RFP mode In RFP mode, we do not support PerformanceNavigationTiming, so don't expose it. In particular, window.PerformanceNavigationTiming should return undefined. Added a new method PerformanceNavigationTiming::Enabled which when used with the WebIDL Func attribute allows us to toggle whether window.PerformanceNavigationTiming is exposed. Created dom/tests/mochitest/general/test_toggling_performance_navigation_timing.html to test whether the toggling works. Updated browser/components/resistfingerprinting/test/browser/browser_performanceAPI.js to create a new window each time privacy.resistFingerprinting is flipped so this behavior does not leak into other tests. --- .../test/browser/browser_performanceAPI.js | 62 ++++++++-------------- dom/performance/PerformanceMainThread.cpp | 3 +- dom/performance/PerformanceNavigationTiming.cpp | 6 +++ dom/performance/PerformanceNavigationTiming.h | 6 +++ dom/tests/mochitest/general/mochitest.ini | 1 + ...est_toggling_performance_navigation_timing.html | 47 ++++++++++++++++ dom/webidl/PerformanceNavigationTiming.webidl | 2 + 7 files changed, 85 insertions(+), 42 deletions(-) diff --git a/browser/components/resistfingerprinting/test/browser/browser_performanceAPI.js b/browser/components/resistfingerprinting/test/browser/browser_performanceAPI.js index b4c6b4f9d461..eb51a98e7364 100644 --- a/browser/components/resistfingerprinting/test/browser/browser_performanceAPI.js +++ b/browser/components/resistfingerprinting/test/browser/browser_performanceAPI.js @@ -78,7 +78,6 @@ let isRounded = (x, expectedPrecision) => { }; let setupTest = async function( - tab, resistFingerprinting, reduceTimerPrecision, expectedPrecision, @@ -95,6 +94,13 @@ let setupTest = async function( ], ], }); + + let win = await BrowserTestUtils.openNewBrowserWindow(); + let tab = await BrowserTestUtils.openNewForegroundTab( + win.gBrowser, + TEST_PATH + "file_dummy.html" + ); + // No matter what we set the precision to, if we're in ResistFingerprinting mode // we use the larger of the precision pref and the constant 100ms if (resistFingerprinting) { @@ -110,15 +116,11 @@ let setupTest = async function( }, runTests ); + await BrowserTestUtils.closeWindow(win); }; // ================================================================================================ // ================================================================================================ add_task(async function runRPTests() { - let tab = await BrowserTestUtils.openNewForegroundTab( - gBrowser, - TEST_PATH + "file_dummy.html" - ); - let runTests = async function(data) { let timerlist = data.list; let expectedPrecision = data.precision; @@ -165,21 +167,14 @@ add_task(async function runRPTests() { ); }; - await setupTest(tab, true, true, 100, runTests); - await setupTest(tab, true, false, 13, runTests); - await setupTest(tab, true, false, 0.13, runTests); - - BrowserTestUtils.removeTab(tab); + await setupTest(true, true, 100, runTests); + await setupTest(true, false, 13, runTests); + await setupTest(true, false, 0.13, runTests); }); // ================================================================================================ // ================================================================================================ add_task(async function runRTPTests() { - let tab = await BrowserTestUtils.openNewForegroundTab( - gBrowser, - TEST_PATH + "file_dummy.html" - ); - let runTests = async function(data) { let timerlist = data.list; let expectedPrecision = data.precision; @@ -215,6 +210,7 @@ add_task(async function runRTPTests() { content.performance.getEntries().length, 4, "For reduceTimerPrecision, there should be 4 entries for performance.getEntries()" + // PerformanceNavigationTiming, PerformanceMark, PerformanceMark, PerformanceMeasure ); for (var i = 0; i < 4; i++) { let startTime = content.performance.getEntries()[i].startTime; @@ -253,11 +249,9 @@ add_task(async function runRTPTests() { content.performance.clearResourceTimings(); }; - await setupTest(tab, false, true, 100, runTests); - await setupTest(tab, false, true, 13, runTests); - await setupTest(tab, false, true, 0.13, runTests); - - BrowserTestUtils.removeTab(tab); + await setupTest(false, true, 100, runTests); + await setupTest(false, true, 13, runTests); + await setupTest(false, true, 0.13, runTests); }); // ================================================================================================ @@ -284,27 +278,13 @@ let runWorkerTest = async function(data) { }; add_task(async function runRPTestsForWorker() { - let tab = await BrowserTestUtils.openNewForegroundTab( - gBrowser, - TEST_PATH + "file_dummy.html" - ); - - await setupTest(tab, true, true, 100, runWorkerTest, "runRPTests"); - await setupTest(tab, true, false, 13, runWorkerTest, "runRPTests"); - await setupTest(tab, true, true, 0.13, runWorkerTest, "runRPTests"); - - BrowserTestUtils.removeTab(tab); + await setupTest(true, true, 100, runWorkerTest, "runRPTests"); + await setupTest(true, false, 13, runWorkerTest, "runRPTests"); + await setupTest(true, true, 0.13, runWorkerTest, "runRPTests"); }); add_task(async function runRTPTestsForWorker() { - let tab = await BrowserTestUtils.openNewForegroundTab( - gBrowser, - TEST_PATH + "file_dummy.html" - ); - - await setupTest(tab, false, true, 100, runWorkerTest, "runRTPTests"); - await setupTest(tab, false, true, 13, runWorkerTest, "runRTPTests"); - await setupTest(tab, false, true, 0.13, runWorkerTest, "runRTPTests"); - - BrowserTestUtils.removeTab(tab); + await setupTest(false, true, 100, runWorkerTest, "runRTPTests"); + await setupTest(false, true, 13, runWorkerTest, "runRTPTests"); + await setupTest(false, true, 0.13, runWorkerTest, "runRTPTests"); }); diff --git a/dom/performance/PerformanceMainThread.cpp b/dom/performance/PerformanceMainThread.cpp index 7c39b9350646..bfcbea4e1148 100644 --- a/dom/performance/PerformanceMainThread.cpp +++ b/dom/performance/PerformanceMainThread.cpp @@ -305,7 +305,8 @@ DOMHighResTimeStamp PerformanceMainThread::CreationTime() const { void PerformanceMainThread::CreateNavigationTimingEntry() { MOZ_ASSERT(!mDocEntry, "mDocEntry should be null."); - if (!StaticPrefs::dom_enable_performance_navigation_timing()) { + if (!StaticPrefs::dom_enable_performance_navigation_timing() || + StaticPrefs::privacy_resistFingerprinting()) { return; } diff --git a/dom/performance/PerformanceNavigationTiming.cpp b/dom/performance/PerformanceNavigationTiming.cpp index dbc6aa78c4d5..fb18b91aabe6 100644 --- a/dom/performance/PerformanceNavigationTiming.cpp +++ b/dom/performance/PerformanceNavigationTiming.cpp @@ -6,6 +6,7 @@ #include "mozilla/dom/PerformanceNavigationTiming.h" #include "mozilla/dom/PerformanceNavigationTimingBinding.h" +#include "mozilla/StaticPrefs.h" using namespace mozilla::dom; @@ -138,3 +139,8 @@ void PerformanceNavigationTiming::UpdatePropertiesFromHttpChannel( nsIHttpChannel* aHttpChannel, nsITimedChannel* aChannel) { mTimingData->SetPropertiesFromHttpChannel(aHttpChannel, aChannel); } + +bool PerformanceNavigationTiming::Enabled(JSContext* aCx, JSObject* aGlobal) { + return (StaticPrefs::dom_enable_performance_navigation_timing() && + !StaticPrefs::privacy_resistFingerprinting()); +} diff --git a/dom/performance/PerformanceNavigationTiming.h b/dom/performance/PerformanceNavigationTiming.h index 8ef79999a1bc..7bf76ee3d670 100644 --- a/dom/performance/PerformanceNavigationTiming.h +++ b/dom/performance/PerformanceNavigationTiming.h @@ -65,6 +65,12 @@ class PerformanceNavigationTiming final : public PerformanceResourceTiming { void UpdatePropertiesFromHttpChannel(nsIHttpChannel* aHttpChannel, nsITimedChannel* aChannel); + /* + * For use with the WebIDL Func attribute to determine whether + * window.PerformanceNavigationTiming is exposed. + */ + static bool Enabled(JSContext* aCx, JSObject* aGlobal); + private: ~PerformanceNavigationTiming() {} }; diff --git a/dom/tests/mochitest/general/mochitest.ini b/dom/tests/mochitest/general/mochitest.ini index cb233881397d..36f955500c09 100644 --- a/dom/tests/mochitest/general/mochitest.ini +++ b/dom/tests/mochitest/general/mochitest.ini @@ -141,6 +141,7 @@ skip-if = toolkit == 'android' # bug 1230232 - Mouse doesn't select in the same [test_storagePermissionsReject.html] [test_storagePermissionsRejectForeign.html] [test_stylesheetPI.html] +[test_toggling_performance_navigation_timing.html] [test_vibrator.html] [test_WebKitCSSMatrix.html] [test_windowedhistoryframes.html] diff --git a/dom/tests/mochitest/general/test_toggling_performance_navigation_timing.html b/dom/tests/mochitest/general/test_toggling_performance_navigation_timing.html new file mode 100644 index 000000000000..88f7154286cd --- /dev/null +++ b/dom/tests/mochitest/general/test_toggling_performance_navigation_timing.html @@ -0,0 +1,47 @@ +<!DOCTYPE HTML> +<html> +<head> + <meta charset="utf-8"> + <title>Test for Bug 1511941 - Don't expose PerformanceNavigationTiming when it is disabled</title> + <script src="/tests/SimpleTest/SimpleTest.js"></script> + <link rel="stylesheet" type="text/css" href="/tests/SimpleTest/test.css"/> +</head> +<body> + <div id="content"> </div> + <script type="application/javascript"> + async function testWhetherExposed(resistFingerprinting, enable_performance_navigation_timing) { + await SpecialPowers.pushPrefEnv({ + "set": [["privacy.resistFingerprinting", resistFingerprinting], + ["dom.enable_performance_navigation_timing", enable_performance_navigation_timing]], + }); + var iframe = document.createElement("iframe"); + document.body.append(iframe); + var p = iframe.contentWindow.PerformanceNavigationTiming; + if (resistFingerprinting) + is(p, undefined, "window.PerformanceNavigationTiming should not be exposed when" + + " dom.enable_performance_navigation_timing=" + enable_performance_navigation_timing + + " and privacy.resistFingerprinting="+ resistFingerprinting +"."); + if (!enable_performance_navigation_timing) + is(p, undefined, "window.PerformanceNavigationTiming should not be exposed when" + + " dom.enable_performance_navigation_timing=" + enable_performance_navigation_timing + + " and privacy.resistFingerprinting="+ resistFingerprinting +"."); + if (enable_performance_navigation_timing && !resistFingerprinting) { + isnot(p, undefined, "window.PerformanceNavigationTiming should be exposed when" + + " dom.enable_performance_navigation_timing=" + enable_performance_navigation_timing + + " and privacy.resistFingerprinting="+ resistFingerprinting +"."); + } + } + + async function start() { + await testWhetherExposed(true,true); + await testWhetherExposed(true,false); + await testWhetherExposed(false,true); + await testWhetherExposed(false,false); + SimpleTest.finish(); + } + + SimpleTest.waitForExplicitFinish(); + start(); + </script> +</body> +</html> diff --git a/dom/webidl/PerformanceNavigationTiming.webidl b/dom/webidl/PerformanceNavigationTiming.webidl index b26e1c18f688..cf029972304f 100644 --- a/dom/webidl/PerformanceNavigationTiming.webidl +++ b/dom/webidl/PerformanceNavigationTiming.webidl @@ -17,6 +17,8 @@ enum NavigationType { "prerender" }; +[Exposed=Window, + Func="mozilla::dom::PerformanceNavigationTiming::Enabled"] interface PerformanceNavigationTiming : PerformanceResourceTiming { readonly attribute DOMHighResTimeStamp unloadEventStart; readonly attribute DOMHighResTimeStamp unloadEventEnd;

1 0

[tor-browser/tor-browser-68.8.0esr-9.5-1] Bug 34196: Update site info URL with the onion name
by sysrqb＠torproject.org 14 May '20

14 May '20

commit 00f5a4c995a911423e10af5a2bbeceef1ae76be6 Author: Alex Catarineu <acat(a)torproject.org> Date: Wed May 13 11:13:24 2020 +0200 Bug 34196: Update site info URL with the onion name --- browser/base/content/browser-siteIdentity.js | 12 +++++++----- browser/base/content/browser.js | 3 ++- 2 files changed, 9 insertions(+), 6 deletions(-) diff --git a/browser/base/content/browser-siteIdentity.js b/browser/base/content/browser-siteIdentity.js index 32e939787fb7..d1cbf8708dfc 100644 --- a/browser/base/content/browser-siteIdentity.js +++ b/browser/base/content/browser-siteIdentity.js @@ -464,13 +464,13 @@ var gIdentityHandler = { * nsIURI for which the identity UI should be displayed, already * processed by nsIURIFixup.createExposableURI. */ - updateIdentity(state, uri) { + updateIdentity(state, uri, onionAliasURI) { let shouldHidePopup = this._uri && this._uri.spec != uri.spec; this._state = state; // Firstly, populate the state properties required to display the UI. See // the documentation of the individual properties for details. - this.setURI(uri); + this.setURI(uri, onionAliasURI); this._secInfo = gBrowser.securityUI.secInfo; // Then, update the user interface with the available data. @@ -549,17 +549,18 @@ var gIdentityHandler = { * Attempt to provide proper IDN treatment for host names */ getEffectiveHost() { + let uri = this._onionAliasURI || this._uri; if (!this._IDNService) { this._IDNService = Cc["@mozilla.org/network/idn-service;1"].getService( Ci.nsIIDNService ); } try { - return this._IDNService.convertToDisplayIDN(this._uri.host, {}); + return this._IDNService.convertToDisplayIDN(uri.host, {}); } catch (e) { // If something goes wrong (e.g. host is an IP address) just fail back // to the full domain. - return this._uri.host; + return uri.host; } }, @@ -984,8 +985,9 @@ var gIdentityHandler = { ContentBlocking.toggleReportBreakageButton(); }, - setURI(uri) { + setURI(uri, onionAliasURI) { this._uri = uri; + this._onionAliasURI = onionAliasURI; try { // Account for file: urls and catch when "" is the value diff --git a/browser/base/content/browser.js b/browser/base/content/browser.js index 67a1977fef6a..251b11dda03f 100644 --- a/browser/base/content/browser.js +++ b/browser/base/content/browser.js @@ -6083,6 +6083,7 @@ var XULBrowserWindow = { // Don't need to do anything if the data we use to update the UI hasn't // changed let uri = gBrowser.currentURI; + let onionAliasURI = gBrowser.selectedBrowser.currentOnionAliasURI; let spec = uri.spec; if (this._state == aState && this._lastLocation == spec) { // Switching to a tab of the same URL doesn't change most security @@ -6100,7 +6101,7 @@ var XULBrowserWindow = { try { uri = Services.uriFixup.createExposableURI(uri); } catch (e) {} - gIdentityHandler.updateIdentity(this._state, uri); + gIdentityHandler.updateIdentity(this._state, uri, onionAliasURI); }, // simulate all change notifications after switching tabs

1 0

[tor-browser/tor-browser-68.8.0esr-9.5-1] Merge remote-tracking branch 'acatgh/34196+1' into tor-browser-68.8.0esr-9.5-1
by sysrqb＠torproject.org 14 May '20

14 May '20

commit 2e16915ff1a6933090dfb47e6fc876ea056cf07c Merge: 4acec7610371 00f5a4c995a9 Author: Matthew Finkel <sysrqb(a)torproject.org> Date: Thu May 14 22:55:43 2020 +0000 Merge remote-tracking branch 'acatgh/34196+1' into tor-browser-68.8.0esr-9.5-1 browser/base/content/browser-siteIdentity.js | 12 +++++++----- browser/base/content/browser.js | 3 ++- 2 files changed, 9 insertions(+), 6 deletions(-)

1 0

Requesting feature for tor browser
by Keifer Bly 14 May '20

14 May '20

Hello, Is this the correct list to request features for tor browser? It would cool if there was an option to update tor browser automatically on start. Thanks very much. --Keifer

2 2

[tor-browser-build/master] Bug 34043: Update snowflake to persist sessions across proxies
by gk＠torproject.org 13 May '20

13 May '20

commit 8547db1c629adf1bbdabcb44e2245944920472e6 Author: David Fifield <david(a)bamsoftware.com> Date: Mon Feb 10 16:06:08 2020 -0700 Bug 34043: Update snowflake to persist sessions across proxies This brings in changes related to #33745 Merge a turbotunnel branch #33897 Remove buffering from WebRTCPeer #34042 Reduce DataChannelTimeout The new dependencies kcp-go and smux form the new inner session layer. There's a patch to kcp-go to remove dependencies required for features we don't use. --- projects/goxnet/config | 2 + projects/kcp-go/config | 24 + projects/kcp-go/remove-crypt-fec.patch | 1019 ++++++++++++++++++++++++++++++++ projects/smux/config | 16 + projects/snowflake/build | 3 + projects/snowflake/config | 6 +- 6 files changed, 1069 insertions(+), 1 deletion(-) diff --git a/projects/goxnet/config b/projects/goxnet/config index 5394875..7656795 100644 --- a/projects/goxnet/config +++ b/projects/goxnet/config @@ -11,6 +11,8 @@ var: use_container: 1 go_lib: golang.org/x/net go_lib_install: + - golang.org/x/net/ipv4 + - golang.org/x/net/ipv6 - golang.org/x/net/proxy - golang.org/x/net/dns/dnsmessage go_lib_deps: diff --git a/projects/kcp-go/config b/projects/kcp-go/config new file mode 100644 index 0000000..7551a5e --- /dev/null +++ b/projects/kcp-go/config @@ -0,0 +1,24 @@ +version: '[% c("abbrev") %]' +git_url: https://github.com/xtaci/kcp-go +git_hash: 831b5b267373f6fbd3548849a3925c4e70806de2 # v5.5.11 +filename: '[% project %]-[% c("version") %]-[% c("var/osname") %]-[% c("var/build_id") %].tar.gz' + +build: '[% c("projects/go/var/build_go_lib") %]' + +var: + container: + use_container: 1 + go_lib: github.com/xtaci/kcp-go + go_lib_deps: + - goerrors + - goxnet + +input_files: + - project: container-image + - name: go + project: go + - name: goerrors + project: goerrors + - name: goxnet + project: goxnet + - filename: remove-crypt-fec.patch diff --git a/projects/kcp-go/remove-crypt-fec.patch b/projects/kcp-go/remove-crypt-fec.patch new file mode 100644 index 0000000..5693ac5 --- /dev/null +++ b/projects/kcp-go/remove-crypt-fec.patch @@ -0,0 +1,1019 @@ +From 0b9d0759f979a5d828b747ea51771f307c53d221 Mon Sep 17 00:00:00 2001 +From: David Fifield <david(a)bamsoftware.com> +Date: Thu, 9 Apr 2020 11:27:44 -0600 +Subject: [PATCH] Remove crypt and FEC dependencies. + +--- + crypt.go | 618 ----------------------------------------------------- + fec.go | 337 ----------------------------- + removed.go | 29 +++ + 3 files changed, 29 insertions(+), 955 deletions(-) + delete mode 100644 crypt.go + delete mode 100644 fec.go + create mode 100644 removed.go + +diff --git a/crypt.go b/crypt.go +deleted file mode 100644 +index d882852..0000000 +--- a/crypt.go ++++ /dev/null +@@ -1,618 +0,0 @@ +-package kcp +- +-import ( +- "crypto/aes" +- "crypto/cipher" +- "crypto/des" +- "crypto/sha1" +- "unsafe" +- +- xor "github.com/templexxx/xorsimd" +- "github.com/tjfoc/gmsm/sm4" +- +- "golang.org/x/crypto/blowfish" +- "golang.org/x/crypto/cast5" +- "golang.org/x/crypto/pbkdf2" +- "golang.org/x/crypto/salsa20" +- "golang.org/x/crypto/tea" +- "golang.org/x/crypto/twofish" +- "golang.org/x/crypto/xtea" +-) +- +-var ( +- initialVector = []byte{167, 115, 79, 156, 18, 172, 27, 1, 164, 21, 242, 193, 252, 120, 230, 107} +- saltxor = `sH3CIVoF#rWLtJo6` +-) +- +-// BlockCrypt defines encryption/decryption methods for a given byte slice. +-// Notes on implementing: the data to be encrypted contains a builtin +-// nonce at the first 16 bytes +-type BlockCrypt interface { +- // Encrypt encrypts the whole block in src into dst. +- // Dst and src may point at the same memory. +- Encrypt(dst, src []byte) +- +- // Decrypt decrypts the whole block in src into dst. +- // Dst and src may point at the same memory. +- Decrypt(dst, src []byte) +-} +- +-type salsa20BlockCrypt struct { +- key [32]byte +-} +- +-// NewSalsa20BlockCrypt https://en.wikipedia.org/wiki/Salsa20 +-func NewSalsa20BlockCrypt(key []byte) (BlockCrypt, error) { +- c := new(salsa20BlockCrypt) +- copy(c.key[:], key) +- return c, nil +-} +- +-func (c *salsa20BlockCrypt) Encrypt(dst, src []byte) { +- salsa20.XORKeyStream(dst[8:], src[8:], src[:8], &c.key) +- copy(dst[:8], src[:8]) +-} +-func (c *salsa20BlockCrypt) Decrypt(dst, src []byte) { +- salsa20.XORKeyStream(dst[8:], src[8:], src[:8], &c.key) +- copy(dst[:8], src[:8]) +-} +- +-type sm4BlockCrypt struct { +- encbuf [sm4.BlockSize]byte // 64bit alignment enc/dec buffer +- decbuf [2 * sm4.BlockSize]byte +- block cipher.Block +-} +- +-// NewSM4BlockCrypt https://github.com/tjfoc/gmsm/tree/master/sm4 +-func NewSM4BlockCrypt(key []byte) (BlockCrypt, error) { +- c := new(sm4BlockCrypt) +- block, err := sm4.NewCipher(key) +- if err != nil { +- return nil, err +- } +- c.block = block +- return c, nil +-} +- +-func (c *sm4BlockCrypt) Encrypt(dst, src []byte) { encrypt(c.block, dst, src, c.encbuf[:]) } +-func (c *sm4BlockCrypt) Decrypt(dst, src []byte) { decrypt(c.block, dst, src, c.decbuf[:]) } +- +-type twofishBlockCrypt struct { +- encbuf [twofish.BlockSize]byte +- decbuf [2 * twofish.BlockSize]byte +- block cipher.Block +-} +- +-// NewTwofishBlockCrypt https://en.wikipedia.org/wiki/Twofish +-func NewTwofishBlockCrypt(key []byte) (BlockCrypt, error) { +- c := new(twofishBlockCrypt) +- block, err := twofish.NewCipher(key) +- if err != nil { +- return nil, err +- } +- c.block = block +- return c, nil +-} +- +-func (c *twofishBlockCrypt) Encrypt(dst, src []byte) { encrypt(c.block, dst, src, c.encbuf[:]) } +-func (c *twofishBlockCrypt) Decrypt(dst, src []byte) { decrypt(c.block, dst, src, c.decbuf[:]) } +- +-type tripleDESBlockCrypt struct { +- encbuf [des.BlockSize]byte +- decbuf [2 * des.BlockSize]byte +- block cipher.Block +-} +- +-// NewTripleDESBlockCrypt https://en.wikipedia.org/wiki/Triple_DES +-func NewTripleDESBlockCrypt(key []byte) (BlockCrypt, error) { +- c := new(tripleDESBlockCrypt) +- block, err := des.NewTripleDESCipher(key) +- if err != nil { +- return nil, err +- } +- c.block = block +- return c, nil +-} +- +-func (c *tripleDESBlockCrypt) Encrypt(dst, src []byte) { encrypt(c.block, dst, src, c.encbuf[:]) } +-func (c *tripleDESBlockCrypt) Decrypt(dst, src []byte) { decrypt(c.block, dst, src, c.decbuf[:]) } +- +-type cast5BlockCrypt struct { +- encbuf [cast5.BlockSize]byte +- decbuf [2 * cast5.BlockSize]byte +- block cipher.Block +-} +- +-// NewCast5BlockCrypt https://en.wikipedia.org/wiki/CAST-128 +-func NewCast5BlockCrypt(key []byte) (BlockCrypt, error) { +- c := new(cast5BlockCrypt) +- block, err := cast5.NewCipher(key) +- if err != nil { +- return nil, err +- } +- c.block = block +- return c, nil +-} +- +-func (c *cast5BlockCrypt) Encrypt(dst, src []byte) { encrypt(c.block, dst, src, c.encbuf[:]) } +-func (c *cast5BlockCrypt) Decrypt(dst, src []byte) { decrypt(c.block, dst, src, c.decbuf[:]) } +- +-type blowfishBlockCrypt struct { +- encbuf [blowfish.BlockSize]byte +- decbuf [2 * blowfish.BlockSize]byte +- block cipher.Block +-} +- +-// NewBlowfishBlockCrypt https://en.wikipedia.org/wiki/Blowfish_(cipher) +-func NewBlowfishBlockCrypt(key []byte) (BlockCrypt, error) { +- c := new(blowfishBlockCrypt) +- block, err := blowfish.NewCipher(key) +- if err != nil { +- return nil, err +- } +- c.block = block +- return c, nil +-} +- +-func (c *blowfishBlockCrypt) Encrypt(dst, src []byte) { encrypt(c.block, dst, src, c.encbuf[:]) } +-func (c *blowfishBlockCrypt) Decrypt(dst, src []byte) { decrypt(c.block, dst, src, c.decbuf[:]) } +- +-type aesBlockCrypt struct { +- encbuf [aes.BlockSize]byte +- decbuf [2 * aes.BlockSize]byte +- block cipher.Block +-} +- +-// NewAESBlockCrypt https://en.wikipedia.org/wiki/Advanced_Encryption_Standard +-func NewAESBlockCrypt(key []byte) (BlockCrypt, error) { +- c := new(aesBlockCrypt) +- block, err := aes.NewCipher(key) +- if err != nil { +- return nil, err +- } +- c.block = block +- return c, nil +-} +- +-func (c *aesBlockCrypt) Encrypt(dst, src []byte) { encrypt(c.block, dst, src, c.encbuf[:]) } +-func (c *aesBlockCrypt) Decrypt(dst, src []byte) { decrypt(c.block, dst, src, c.decbuf[:]) } +- +-type teaBlockCrypt struct { +- encbuf [tea.BlockSize]byte +- decbuf [2 * tea.BlockSize]byte +- block cipher.Block +-} +- +-// NewTEABlockCrypt https://en.wikipedia.org/wiki/Tiny_Encryption_Algorithm +-func NewTEABlockCrypt(key []byte) (BlockCrypt, error) { +- c := new(teaBlockCrypt) +- block, err := tea.NewCipherWithRounds(key, 16) +- if err != nil { +- return nil, err +- } +- c.block = block +- return c, nil +-} +- +-func (c *teaBlockCrypt) Encrypt(dst, src []byte) { encrypt(c.block, dst, src, c.encbuf[:]) } +-func (c *teaBlockCrypt) Decrypt(dst, src []byte) { decrypt(c.block, dst, src, c.decbuf[:]) } +- +-type xteaBlockCrypt struct { +- encbuf [xtea.BlockSize]byte +- decbuf [2 * xtea.BlockSize]byte +- block cipher.Block +-} +- +-// NewXTEABlockCrypt https://en.wikipedia.org/wiki/XTEA +-func NewXTEABlockCrypt(key []byte) (BlockCrypt, error) { +- c := new(xteaBlockCrypt) +- block, err := xtea.NewCipher(key) +- if err != nil { +- return nil, err +- } +- c.block = block +- return c, nil +-} +- +-func (c *xteaBlockCrypt) Encrypt(dst, src []byte) { encrypt(c.block, dst, src, c.encbuf[:]) } +-func (c *xteaBlockCrypt) Decrypt(dst, src []byte) { decrypt(c.block, dst, src, c.decbuf[:]) } +- +-type simpleXORBlockCrypt struct { +- xortbl []byte +-} +- +-// NewSimpleXORBlockCrypt simple xor with key expanding +-func NewSimpleXORBlockCrypt(key []byte) (BlockCrypt, error) { +- c := new(simpleXORBlockCrypt) +- c.xortbl = pbkdf2.Key(key, []byte(saltxor), 32, mtuLimit, sha1.New) +- return c, nil +-} +- +-func (c *simpleXORBlockCrypt) Encrypt(dst, src []byte) { xor.Bytes(dst, src, c.xortbl) } +-func (c *simpleXORBlockCrypt) Decrypt(dst, src []byte) { xor.Bytes(dst, src, c.xortbl) } +- +-type noneBlockCrypt struct{} +- +-// NewNoneBlockCrypt does nothing but copying +-func NewNoneBlockCrypt(key []byte) (BlockCrypt, error) { +- return new(noneBlockCrypt), nil +-} +- +-func (c *noneBlockCrypt) Encrypt(dst, src []byte) { copy(dst, src) } +-func (c *noneBlockCrypt) Decrypt(dst, src []byte) { copy(dst, src) } +- +-// packet encryption with local CFB mode +-func encrypt(block cipher.Block, dst, src, buf []byte) { +- switch block.BlockSize() { +- case 8: +- encrypt8(block, dst, src, buf) +- case 16: +- encrypt16(block, dst, src, buf) +- default: +- panic("unsupported cipher block size") +- } +-} +- +-// optimized encryption for the ciphers which works in 8-bytes +-func encrypt8(block cipher.Block, dst, src, buf []byte) { +- tbl := buf[:8] +- block.Encrypt(tbl, initialVector) +- n := len(src) / 8 +- base := 0 +- repeat := n / 8 +- left := n % 8 +- ptr_tbl := (*uint64)(unsafe.Pointer(&tbl[0])) +- +- for i := 0; i < repeat; i++ { +- s := src[base:][0:64] +- d := dst[base:][0:64] +- // 1 +- *(*uint64)(unsafe.Pointer(&d[0])) = *(*uint64)(unsafe.Pointer(&s[0])) ^ *ptr_tbl +- block.Encrypt(tbl, d[0:8]) +- // 2 +- *(*uint64)(unsafe.Pointer(&d[8])) = *(*uint64)(unsafe.Pointer(&s[8])) ^ *ptr_tbl +- block.Encrypt(tbl, d[8:16]) +- // 3 +- *(*uint64)(unsafe.Pointer(&d[16])) = *(*uint64)(unsafe.Pointer(&s[16])) ^ *ptr_tbl +- block.Encrypt(tbl, d[16:24]) +- // 4 +- *(*uint64)(unsafe.Pointer(&d[24])) = *(*uint64)(unsafe.Pointer(&s[24])) ^ *ptr_tbl +- block.Encrypt(tbl, d[24:32]) +- // 5 +- *(*uint64)(unsafe.Pointer(&d[32])) = *(*uint64)(unsafe.Pointer(&s[32])) ^ *ptr_tbl +- block.Encrypt(tbl, d[32:40]) +- // 6 +- *(*uint64)(unsafe.Pointer(&d[40])) = *(*uint64)(unsafe.Pointer(&s[40])) ^ *ptr_tbl +- block.Encrypt(tbl, d[40:48]) +- // 7 +- *(*uint64)(unsafe.Pointer(&d[48])) = *(*uint64)(unsafe.Pointer(&s[48])) ^ *ptr_tbl +- block.Encrypt(tbl, d[48:56]) +- // 8 +- *(*uint64)(unsafe.Pointer(&d[56])) = *(*uint64)(unsafe.Pointer(&s[56])) ^ *ptr_tbl +- block.Encrypt(tbl, d[56:64]) +- base += 64 +- } +- +- switch left { +- case 7: +- *(*uint64)(unsafe.Pointer(&dst[base])) = *(*uint64)(unsafe.Pointer(&src[base])) ^ *ptr_tbl +- block.Encrypt(tbl, dst[base:]) +- base += 8 +- fallthrough +- case 6: +- *(*uint64)(unsafe.Pointer(&dst[base])) = *(*uint64)(unsafe.Pointer(&src[base])) ^ *ptr_tbl +- block.Encrypt(tbl, dst[base:]) +- base += 8 +- fallthrough +- case 5: +- *(*uint64)(unsafe.Pointer(&dst[base])) = *(*uint64)(unsafe.Pointer(&src[base])) ^ *ptr_tbl +- block.Encrypt(tbl, dst[base:]) +- base += 8 +- fallthrough +- case 4: +- *(*uint64)(unsafe.Pointer(&dst[base])) = *(*uint64)(unsafe.Pointer(&src[base])) ^ *ptr_tbl +- block.Encrypt(tbl, dst[base:]) +- base += 8 +- fallthrough +- case 3: +- *(*uint64)(unsafe.Pointer(&dst[base])) = *(*uint64)(unsafe.Pointer(&src[base])) ^ *ptr_tbl +- block.Encrypt(tbl, dst[base:]) +- base += 8 +- fallthrough +- case 2: +- *(*uint64)(unsafe.Pointer(&dst[base])) = *(*uint64)(unsafe.Pointer(&src[base])) ^ *ptr_tbl +- block.Encrypt(tbl, dst[base:]) +- base += 8 +- fallthrough +- case 1: +- *(*uint64)(unsafe.Pointer(&dst[base])) = *(*uint64)(unsafe.Pointer(&src[base])) ^ *ptr_tbl +- block.Encrypt(tbl, dst[base:]) +- base += 8 +- fallthrough +- case 0: +- xorBytes(dst[base:], src[base:], tbl) +- } +-} +- +-// optimized encryption for the ciphers which works in 16-bytes +-func encrypt16(block cipher.Block, dst, src, buf []byte) { +- tbl := buf[:16] +- block.Encrypt(tbl, initialVector) +- n := len(src) / 16 +- base := 0 +- repeat := n / 8 +- left := n % 8 +- for i := 0; i < repeat; i++ { +- s := src[base:][0:128] +- d := dst[base:][0:128] +- // 1 +- xor.Bytes16Align(d[0:16], s[0:16], tbl) +- block.Encrypt(tbl, d[0:16]) +- // 2 +- xor.Bytes16Align(d[16:32], s[16:32], tbl) +- block.Encrypt(tbl, d[16:32]) +- // 3 +- xor.Bytes16Align(d[32:48], s[32:48], tbl) +- block.Encrypt(tbl, d[32:48]) +- // 4 +- xor.Bytes16Align(d[48:64], s[48:64], tbl) +- block.Encrypt(tbl, d[48:64]) +- // 5 +- xor.Bytes16Align(d[64:80], s[64:80], tbl) +- block.Encrypt(tbl, d[64:80]) +- // 6 +- xor.Bytes16Align(d[80:96], s[80:96], tbl) +- block.Encrypt(tbl, d[80:96]) +- // 7 +- xor.Bytes16Align(d[96:112], s[96:112], tbl) +- block.Encrypt(tbl, d[96:112]) +- // 8 +- xor.Bytes16Align(d[112:128], s[112:128], tbl) +- block.Encrypt(tbl, d[112:128]) +- base += 128 +- } +- +- switch left { +- case 7: +- xor.Bytes16Align(dst[base:], src[base:], tbl) +- block.Encrypt(tbl, dst[base:]) +- base += 16 +- fallthrough +- case 6: +- xor.Bytes16Align(dst[base:], src[base:], tbl) +- block.Encrypt(tbl, dst[base:]) +- base += 16 +- fallthrough +- case 5: +- xor.Bytes16Align(dst[base:], src[base:], tbl) +- block.Encrypt(tbl, dst[base:]) +- base += 16 +- fallthrough +- case 4: +- xor.Bytes16Align(dst[base:], src[base:], tbl) +- block.Encrypt(tbl, dst[base:]) +- base += 16 +- fallthrough +- case 3: +- xor.Bytes16Align(dst[base:], src[base:], tbl) +- block.Encrypt(tbl, dst[base:]) +- base += 16 +- fallthrough +- case 2: +- xor.Bytes16Align(dst[base:], src[base:], tbl) +- block.Encrypt(tbl, dst[base:]) +- base += 16 +- fallthrough +- case 1: +- xor.Bytes16Align(dst[base:], src[base:], tbl) +- block.Encrypt(tbl, dst[base:]) +- base += 16 +- fallthrough +- case 0: +- xorBytes(dst[base:], src[base:], tbl) +- } +-} +- +-// decryption +-func decrypt(block cipher.Block, dst, src, buf []byte) { +- switch block.BlockSize() { +- case 8: +- decrypt8(block, dst, src, buf) +- case 16: +- decrypt16(block, dst, src, buf) +- default: +- panic("unsupported cipher block size") +- } +-} +- +-// decrypt 8 bytes block, all byte slices are supposed to be 64bit aligned +-func decrypt8(block cipher.Block, dst, src, buf []byte) { +- tbl := buf[0:8] +- next := buf[8:16] +- block.Encrypt(tbl, initialVector) +- n := len(src) / 8 +- base := 0 +- repeat := n / 8 +- left := n % 8 +- ptr_tbl := (*uint64)(unsafe.Pointer(&tbl[0])) +- ptr_next := (*uint64)(unsafe.Pointer(&next[0])) +- +- for i := 0; i < repeat; i++ { +- s := src[base:][0:64] +- d := dst[base:][0:64] +- // 1 +- block.Encrypt(next, s[0:8]) +- *(*uint64)(unsafe.Pointer(&d[0])) = *(*uint64)(unsafe.Pointer(&s[0])) ^ *ptr_tbl +- // 2 +- block.Encrypt(tbl, s[8:16]) +- *(*uint64)(unsafe.Pointer(&d[8])) = *(*uint64)(unsafe.Pointer(&s[8])) ^ *ptr_next +- // 3 +- block.Encrypt(next, s[16:24]) +- *(*uint64)(unsafe.Pointer(&d[16])) = *(*uint64)(unsafe.Pointer(&s[16])) ^ *ptr_tbl +- // 4 +- block.Encrypt(tbl, s[24:32]) +- *(*uint64)(unsafe.Pointer(&d[24])) = *(*uint64)(unsafe.Pointer(&s[24])) ^ *ptr_next +- // 5 +- block.Encrypt(next, s[32:40]) +- *(*uint64)(unsafe.Pointer(&d[32])) = *(*uint64)(unsafe.Pointer(&s[32])) ^ *ptr_tbl +- // 6 +- block.Encrypt(tbl, s[40:48]) +- *(*uint64)(unsafe.Pointer(&d[40])) = *(*uint64)(unsafe.Pointer(&s[40])) ^ *ptr_next +- // 7 +- block.Encrypt(next, s[48:56]) +- *(*uint64)(unsafe.Pointer(&d[48])) = *(*uint64)(unsafe.Pointer(&s[48])) ^ *ptr_tbl +- // 8 +- block.Encrypt(tbl, s[56:64]) +- *(*uint64)(unsafe.Pointer(&d[56])) = *(*uint64)(unsafe.Pointer(&s[56])) ^ *ptr_next +- base += 64 +- } +- +- switch left { +- case 7: +- block.Encrypt(next, src[base:]) +- *(*uint64)(unsafe.Pointer(&dst[base])) = *(*uint64)(unsafe.Pointer(&src[base])) ^ *(*uint64)(unsafe.Pointer(&tbl[0])) +- tbl, next = next, tbl +- base += 8 +- fallthrough +- case 6: +- block.Encrypt(next, src[base:]) +- *(*uint64)(unsafe.Pointer(&dst[base])) = *(*uint64)(unsafe.Pointer(&src[base])) ^ *(*uint64)(unsafe.Pointer(&tbl[0])) +- tbl, next = next, tbl +- base += 8 +- fallthrough +- case 5: +- block.Encrypt(next, src[base:]) +- *(*uint64)(unsafe.Pointer(&dst[base])) = *(*uint64)(unsafe.Pointer(&src[base])) ^ *(*uint64)(unsafe.Pointer(&tbl[0])) +- tbl, next = next, tbl +- base += 8 +- fallthrough +- case 4: +- block.Encrypt(next, src[base:]) +- *(*uint64)(unsafe.Pointer(&dst[base])) = *(*uint64)(unsafe.Pointer(&src[base])) ^ *(*uint64)(unsafe.Pointer(&tbl[0])) +- tbl, next = next, tbl +- base += 8 +- fallthrough +- case 3: +- block.Encrypt(next, src[base:]) +- *(*uint64)(unsafe.Pointer(&dst[base])) = *(*uint64)(unsafe.Pointer(&src[base])) ^ *(*uint64)(unsafe.Pointer(&tbl[0])) +- tbl, next = next, tbl +- base += 8 +- fallthrough +- case 2: +- block.Encrypt(next, src[base:]) +- *(*uint64)(unsafe.Pointer(&dst[base])) = *(*uint64)(unsafe.Pointer(&src[base])) ^ *(*uint64)(unsafe.Pointer(&tbl[0])) +- tbl, next = next, tbl +- base += 8 +- fallthrough +- case 1: +- block.Encrypt(next, src[base:]) +- *(*uint64)(unsafe.Pointer(&dst[base])) = *(*uint64)(unsafe.Pointer(&src[base])) ^ *(*uint64)(unsafe.Pointer(&tbl[0])) +- tbl, next = next, tbl +- base += 8 +- fallthrough +- case 0: +- xorBytes(dst[base:], src[base:], tbl) +- } +-} +- +-func decrypt16(block cipher.Block, dst, src, buf []byte) { +- tbl := buf[0:16] +- next := buf[16:32] +- block.Encrypt(tbl, initialVector) +- n := len(src) / 16 +- base := 0 +- repeat := n / 8 +- left := n % 8 +- for i := 0; i < repeat; i++ { +- s := src[base:][0:128] +- d := dst[base:][0:128] +- // 1 +- block.Encrypt(next, s[0:16]) +- xor.Bytes16Align(d[0:16], s[0:16], tbl) +- // 2 +- block.Encrypt(tbl, s[16:32]) +- xor.Bytes16Align(d[16:32], s[16:32], next) +- // 3 +- block.Encrypt(next, s[32:48]) +- xor.Bytes16Align(d[32:48], s[32:48], tbl) +- // 4 +- block.Encrypt(tbl, s[48:64]) +- xor.Bytes16Align(d[48:64], s[48:64], next) +- // 5 +- block.Encrypt(next, s[64:80]) +- xor.Bytes16Align(d[64:80], s[64:80], tbl) +- // 6 +- block.Encrypt(tbl, s[80:96]) +- xor.Bytes16Align(d[80:96], s[80:96], next) +- // 7 +- block.Encrypt(next, s[96:112]) +- xor.Bytes16Align(d[96:112], s[96:112], tbl) +- // 8 +- block.Encrypt(tbl, s[112:128]) +- xor.Bytes16Align(d[112:128], s[112:128], next) +- base += 128 +- } +- +- switch left { +- case 7: +- block.Encrypt(next, src[base:]) +- xor.Bytes16Align(dst[base:], src[base:], tbl) +- tbl, next = next, tbl +- base += 16 +- fallthrough +- case 6: +- block.Encrypt(next, src[base:]) +- xor.Bytes16Align(dst[base:], src[base:], tbl) +- tbl, next = next, tbl +- base += 16 +- fallthrough +- case 5: +- block.Encrypt(next, src[base:]) +- xor.Bytes16Align(dst[base:], src[base:], tbl) +- tbl, next = next, tbl +- base += 16 +- fallthrough +- case 4: +- block.Encrypt(next, src[base:]) +- xor.Bytes16Align(dst[base:], src[base:], tbl) +- tbl, next = next, tbl +- base += 16 +- fallthrough +- case 3: +- block.Encrypt(next, src[base:]) +- xor.Bytes16Align(dst[base:], src[base:], tbl) +- tbl, next = next, tbl +- base += 16 +- fallthrough +- case 2: +- block.Encrypt(next, src[base:]) +- xor.Bytes16Align(dst[base:], src[base:], tbl) +- tbl, next = next, tbl +- base += 16 +- fallthrough +- case 1: +- block.Encrypt(next, src[base:]) +- xor.Bytes16Align(dst[base:], src[base:], tbl) +- tbl, next = next, tbl +- base += 16 +- fallthrough +- case 0: +- xorBytes(dst[base:], src[base:], tbl) +- } +-} +- +-// per bytes xors +-func xorBytes(dst, a, b []byte) int { +- n := len(a) +- if len(b) < n { +- n = len(b) +- } +- if n == 0 { +- return 0 +- } +- +- for i := 0; i < n; i++ { +- dst[i] = a[i] ^ b[i] +- } +- return n +-} +diff --git a/fec.go b/fec.go +deleted file mode 100644 +index 97cd40b..0000000 +--- a/fec.go ++++ /dev/null +@@ -1,337 +0,0 @@ +-package kcp +- +-import ( +- "encoding/binary" +- "sync/atomic" +- +- "github.com/klauspost/reedsolomon" +-) +- +-const ( +- fecHeaderSize = 6 +- fecHeaderSizePlus2 = fecHeaderSize + 2 // plus 2B data size +- typeData = 0xf1 +- typeParity = 0xf2 +- fecExpire = 60000 +-) +- +-// fecPacket is a decoded FEC packet +-type fecPacket []byte +- +-func (bts fecPacket) seqid() uint32 { return binary.LittleEndian.Uint32(bts) } +-func (bts fecPacket) flag() uint16 { return binary.LittleEndian.Uint16(bts[4:]) } +-func (bts fecPacket) data() []byte { return bts[6:] } +- +-// fecElement has auxcilliary time field +-type fecElement struct { +- fecPacket +- ts uint32 +-} +- +-// fecDecoder for decoding incoming packets +-type fecDecoder struct { +- rxlimit int // queue size limit +- dataShards int +- parityShards int +- shardSize int +- rx []fecElement // ordered receive queue +- +- // caches +- decodeCache [][]byte +- flagCache []bool +- +- // zeros +- zeros []byte +- +- // RS decoder +- codec reedsolomon.Encoder +-} +- +-func newFECDecoder(rxlimit, dataShards, parityShards int) *fecDecoder { +- if dataShards <= 0 || parityShards <= 0 { +- return nil +- } +- if rxlimit < dataShards+parityShards { +- return nil +- } +- +- dec := new(fecDecoder) +- dec.rxlimit = rxlimit +- dec.dataShards = dataShards +- dec.parityShards = parityShards +- dec.shardSize = dataShards + parityShards +- codec, err := reedsolomon.New(dataShards, parityShards) +- if err != nil { +- return nil +- } +- dec.codec = codec +- dec.decodeCache = make([][]byte, dec.shardSize) +- dec.flagCache = make([]bool, dec.shardSize) +- dec.zeros = make([]byte, mtuLimit) +- return dec +-} +- +-// decode a fec packet +-func (dec *fecDecoder) decode(in fecPacket) (recovered [][]byte) { +- // insertion +- n := len(dec.rx) - 1 +- insertIdx := 0 +- for i := n; i >= 0; i-- { +- if in.seqid() == dec.rx[i].seqid() { // de-duplicate +- return nil +- } else if _itimediff(in.seqid(), dec.rx[i].seqid()) > 0 { // insertion +- insertIdx = i + 1 +- break +- } +- } +- +- // make a copy +- pkt := fecPacket(xmitBuf.Get().([]byte)[:len(in)]) +- copy(pkt, in) +- elem := fecElement{pkt, currentMs()} +- +- // insert into ordered rx queue +- if insertIdx == n+1 { +- dec.rx = append(dec.rx, elem) +- } else { +- dec.rx = append(dec.rx, fecElement{}) +- copy(dec.rx[insertIdx+1:], dec.rx[insertIdx:]) // shift right +- dec.rx[insertIdx] = elem +- } +- +- // shard range for current packet +- shardBegin := pkt.seqid() - pkt.seqid()%uint32(dec.shardSize) +- shardEnd := shardBegin + uint32(dec.shardSize) - 1 +- +- // max search range in ordered queue for current shard +- searchBegin := insertIdx - int(pkt.seqid()%uint32(dec.shardSize)) +- if searchBegin < 0 { +- searchBegin = 0 +- } +- searchEnd := searchBegin + dec.shardSize - 1 +- if searchEnd >= len(dec.rx) { +- searchEnd = len(dec.rx) - 1 +- } +- +- // re-construct datashards +- if searchEnd-searchBegin+1 >= dec.dataShards { +- var numshard, numDataShard, first, maxlen int +- +- // zero caches +- shards := dec.decodeCache +- shardsflag := dec.flagCache +- for k := range dec.decodeCache { +- shards[k] = nil +- shardsflag[k] = false +- } +- +- // shard assembly +- for i := searchBegin; i <= searchEnd; i++ { +- seqid := dec.rx[i].seqid() +- if _itimediff(seqid, shardEnd) > 0 { +- break +- } else if _itimediff(seqid, shardBegin) >= 0 { +- shards[seqid%uint32(dec.shardSize)] = dec.rx[i].data() +- shardsflag[seqid%uint32(dec.shardSize)] = true +- numshard++ +- if dec.rx[i].flag() == typeData { +- numDataShard++ +- } +- if numshard == 1 { +- first = i +- } +- if len(dec.rx[i].data()) > maxlen { +- maxlen = len(dec.rx[i].data()) +- } +- } +- } +- +- if numDataShard == dec.dataShards { +- // case 1: no loss on data shards +- dec.rx = dec.freeRange(first, numshard, dec.rx) +- } else if numshard >= dec.dataShards { +- // case 2: loss on data shards, but it's recoverable from parity shards +- for k := range shards { +- if shards[k] != nil { +- dlen := len(shards[k]) +- shards[k] = shards[k][:maxlen] +- copy(shards[k][dlen:], dec.zeros) +- } else if k < dec.dataShards { +- shards[k] = xmitBuf.Get().([]byte)[:0] +- } +- } +- if err := dec.codec.ReconstructData(shards); err == nil { +- for k := range shards[:dec.dataShards] { +- if !shardsflag[k] { +- // recovered data should be recycled +- recovered = append(recovered, shards[k]) +- } +- } +- } +- dec.rx = dec.freeRange(first, numshard, dec.rx) +- } +- } +- +- // keep rxlimit +- if len(dec.rx) > dec.rxlimit { +- if dec.rx[0].flag() == typeData { // track the unrecoverable data +- atomic.AddUint64(&DefaultSnmp.FECShortShards, 1) +- } +- dec.rx = dec.freeRange(0, 1, dec.rx) +- } +- +- // timeout policy +- current := currentMs() +- numExpired := 0 +- for k := range dec.rx { +- if _itimediff(current, dec.rx[k].ts) > fecExpire { +- numExpired++ +- continue +- } +- break +- } +- if numExpired > 0 { +- dec.rx = dec.freeRange(0, numExpired, dec.rx) +- } +- return +-} +- +-// free a range of fecPacket +-func (dec *fecDecoder) freeRange(first, n int, q []fecElement) []fecElement { +- for i := first; i < first+n; i++ { // recycle buffer +- xmitBuf.Put([]byte(q[i].fecPacket)) +- } +- +- if first == 0 && n < cap(q)/2 { +- return q[n:] +- } +- copy(q[first:], q[first+n:]) +- return q[:len(q)-n] +-} +- +-// release all segments back to xmitBuf +-func (dec *fecDecoder) release() { +- if n := len(dec.rx); n > 0 { +- dec.rx = dec.freeRange(0, n, dec.rx) +- } +-} +- +-type ( +- // fecEncoder for encoding outgoing packets +- fecEncoder struct { +- dataShards int +- parityShards int +- shardSize int +- paws uint32 // Protect Against Wrapped Sequence numbers +- next uint32 // next seqid +- +- shardCount int // count the number of datashards collected +- maxSize int // track maximum data length in datashard +- +- headerOffset int // FEC header offset +- payloadOffset int // FEC payload offset +- +- // caches +- shardCache [][]byte +- encodeCache [][]byte +- +- // zeros +- zeros []byte +- +- // RS encoder +- codec reedsolomon.Encoder +- } +-) +- +-func newFECEncoder(dataShards, parityShards, offset int) *fecEncoder { +- if dataShards <= 0 || parityShards <= 0 { +- return nil +- } +- enc := new(fecEncoder) +- enc.dataShards = dataShards +- enc.parityShards = parityShards +- enc.shardSize = dataShards + parityShards +- enc.paws = 0xffffffff / uint32(enc.shardSize) * uint32(enc.shardSize) +- enc.headerOffset = offset +- enc.payloadOffset = enc.headerOffset + fecHeaderSize +- +- codec, err := reedsolomon.New(dataShards, parityShards) +- if err != nil { +- return nil +- } +- enc.codec = codec +- +- // caches +- enc.encodeCache = make([][]byte, enc.shardSize) +- enc.shardCache = make([][]byte, enc.shardSize) +- for k := range enc.shardCache { +- enc.shardCache[k] = make([]byte, mtuLimit) +- } +- enc.zeros = make([]byte, mtuLimit) +- return enc +-} +- +-// encodes the packet, outputs parity shards if we have collected quorum datashards +-// notice: the contents of 'ps' will be re-written in successive calling +-func (enc *fecEncoder) encode(b []byte) (ps [][]byte) { +- // The header format: +- // | FEC SEQID(4B) | FEC TYPE(2B) | SIZE (2B) | PAYLOAD(SIZE-2) | +- // |<-headerOffset |<-payloadOffset +- enc.markData(b[enc.headerOffset:]) +- binary.LittleEndian.PutUint16(b[enc.payloadOffset:], uint16(len(b[enc.payloadOffset:]))) +- +- // copy data from payloadOffset to fec shard cache +- sz := len(b) +- enc.shardCache[enc.shardCount] = enc.shardCache[enc.shardCount][:sz] +- copy(enc.shardCache[enc.shardCount][enc.payloadOffset:], b[enc.payloadOffset:]) +- enc.shardCount++ +- +- // track max datashard length +- if sz > enc.maxSize { +- enc.maxSize = sz +- } +- +- // Generation of Reed-Solomon Erasure Code +- if enc.shardCount == enc.dataShards { +- // fill '0' into the tail of each datashard +- for i := 0; i < enc.dataShards; i++ { +- shard := enc.shardCache[i] +- slen := len(shard) +- copy(shard[slen:enc.maxSize], enc.zeros) +- } +- +- // construct equal-sized slice with stripped header +- cache := enc.encodeCache +- for k := range cache { +- cache[k] = enc.shardCache[k][enc.payloadOffset:enc.maxSize] +- } +- +- // encoding +- if err := enc.codec.Encode(cache); err == nil { +- ps = enc.shardCache[enc.dataShards:] +- for k := range ps { +- enc.markParity(ps[k][enc.headerOffset:]) +- ps[k] = ps[k][:enc.maxSize] +- } +- } +- +- // counters resetting +- enc.shardCount = 0 +- enc.maxSize = 0 +- } +- +- return +-} +- +-func (enc *fecEncoder) markData(data []byte) { +- binary.LittleEndian.PutUint32(data, enc.next) +- binary.LittleEndian.PutUint16(data[4:], typeData) +- enc.next++ +-} +- +-func (enc *fecEncoder) markParity(data []byte) { +- binary.LittleEndian.PutUint32(data, enc.next) +- binary.LittleEndian.PutUint16(data[4:], typeParity) +- // sequence wrap will only happen at parity shard +- enc.next = (enc.next + 1) % enc.paws +-} +diff --git a/removed.go b/removed.go +new file mode 100644 +index 0000000..5ecf446 +--- /dev/null ++++ b/removed.go +@@ -0,0 +1,29 @@ ++package kcp ++ ++// Dummy implementations for types from crypt.go and fec.go, removed to reduce ++// dependencies. ++ ++const ( ++ fecHeaderSize = 6 ++ fecHeaderSizePlus2 = fecHeaderSize + 2 ++ typeData = 0xf1 ++ typeParity = 0xf2 ++) ++ ++type ( ++ BlockCrypt interface { ++ Encrypt(_, _ []byte) ++ Decrypt(_, _ []byte) ++ } ++ fecDecoder struct{} ++ fecEncoder struct{} ++ fecPacket []byte ++) ++ ++func newFECDecoder(rxlimit, dataShards, parityShards int) *fecDecoder { return nil } ++func newFECEncoder(dataShards, parityShards, offset int) *fecEncoder { return nil } ++ ++func (_ *fecDecoder) decode(in fecPacket) [][]byte { panic("disabled") } ++func (_ *fecDecoder) release() { panic("disabled") } ++func (_ *fecEncoder) encode(b []byte) [][]byte { panic("disabled") } ++func (_ fecPacket) flag() uint16 { panic("disabled") } +-- +2.20.1 + diff --git a/projects/smux/config b/projects/smux/config new file mode 100644 index 0000000..54d2a11 --- /dev/null +++ b/projects/smux/config @@ -0,0 +1,16 @@ +version: '[% c("abbrev") %]' +git_url: https://github.com/xtaci/smux +git_hash: c6969d8a76874342611f4b544d05b1590f1a76b1 # v1.5.12 +filename: '[% project %]-[% c("version") %]-[% c("var/osname") %]-[% c("var/build_id") %].tar.gz' + +build: '[% c("projects/go/var/build_go_lib") %]' + +var: + container: + use_container: 1 + go_lib: github.com/xtaci/smux + +input_files: + - project: container-image + - name: go + project: go diff --git a/projects/snowflake/build b/projects/snowflake/build index 1aede45..2724dd9 100644 --- a/projects/snowflake/build +++ b/projects/snowflake/build @@ -8,6 +8,8 @@ mkdir -p $PTDIR $DOCSDIR tar -C /var/tmp/dist -xf [% c('input_files_by_name/uniuri') %] tar -C /var/tmp/dist -xf [% c('input_files_by_name/goptlib') %] tar -C /var/tmp/dist -xf [% c('input_files_by_name/pion-webrtc') %] +tar -C /var/tmp/dist -xf [% c('input_files_by_name/kcp-go') %] +tar -C /var/tmp/dist -xf [% c('input_files_by_name/smux') %] mkdir -p /var/tmp/build tar -C /var/tmp/build -xf [% project %]-[% c('version') %].tar.gz @@ -18,6 +20,7 @@ ln -sf "$PWD" "$GOPATH/src/git.torproject.org/pluggable-transports/snowflake.git # Fix gopath location of versioned dependencies ln -sf "$GOPATH/src/github.com/pion/webrtc" "$GOPATH/src/github.com/pion/webrtc/v2" ln -sf "$GOPATH/src/github.com/pion/sdp" "$GOPATH/src/github.com/pion/sdp/v2" +ln -sf "$GOPATH/src/github.com/xtaci/kcp-go" "$GOPATH/src/github.com/xtaci/kcp-go/v5" cd client go build -ldflags '-s' diff --git a/projects/snowflake/config b/projects/snowflake/config index b81f1f5..efd0707 100644 --- a/projects/snowflake/config +++ b/projects/snowflake/config @@ -1,7 +1,7 @@ # vim: filetype=yaml sw=2 version: '[% c("abbrev") %]' git_url: https://git.torproject.org/pluggable-transports/snowflake.git -git_hash: ea01bf41c3011590938b079ed96c7b35cb40588b +git_hash: 7043a055f9fb0680281ecffd7d458a43f2ce65b5 filename: '[% project %]-[% c("version") %]-[% c("var/osname") %]-[% c("var/build_id") %].tar.gz' var: @@ -18,3 +18,7 @@ input_files: project: goptlib - name: pion-webrtc project: pion-webrtc + - name: kcp-go + project: kcp-go + - name: smux + project: smux

1 0

[tor-browser-build/master] Typo fix, spotted by a cypherpunk, thanks!
by gk＠torproject.org 08 May '20

08 May '20

commit fab8c5736378dfba4b8524e72f3796ffcf732a6d Author: Georg Koppen <gk(a)torproject.org> Date: Fri May 8 07:34:10 2020 +0000 Typo fix, spotted by a cypherpunk, thanks! --- projects/tor-browser/Bundle-Data/Docs/ChangeLog.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/projects/tor-browser/Bundle-Data/Docs/ChangeLog.txt b/projects/tor-browser/Bundle-Data/Docs/ChangeLog.txt index eaf2df8..a4b0f8a 100644 --- a/projects/tor-browser/Bundle-Data/Docs/ChangeLog.txt +++ b/projects/tor-browser/Bundle-Data/Docs/ChangeLog.txt @@ -13,7 +13,7 @@ Tor Browser 9.5a12 -- May 7 2020 * Windows + OS X + Linux * Update Tor Launcher to 0.2.21.6 * Translations update - * Bug 33573: Update pion-webrtc version to 2.2.3 + * Bug 33576: Update pion-webrtc version to 2.2.3 * Bug 32418: Allow updates to be disabled via an enterprise policy. * Bug 34032: Use Securedrop's Official https-everywhere ruleset * Windows

1 0