October 2020 - tbb-commits - lists.torproject.org

[tor-browser-spec/master] Remove superfluous whitespace in audit script
by gk＠torproject.org 23 Oct '20

23 Oct '20

commit f936ca02d854b2578fc712aabd08e0f42188da48 Author: Georg Koppen <gk(a)torproject.org> Date: Fri Oct 23 08:09:24 2020 +0000 Remove superfluous whitespace in audit script --- audits/java_audit.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/audits/java_audit.sh b/audits/java_audit.sh index 57524eb..e085b57 100644 --- a/audits/java_audit.sh +++ b/audits/java_audit.sh @@ -78,7 +78,7 @@ GREP_LINE="$(join_by \| ${KEYWORDS[@]})" export GREP_COLOR="05;37;41" … [View More]

1 0

[tor-browser-spec/master] Bug 40005: Add Fenix82 net audit
by gk＠torproject.org 23 Oct '20

23 Oct '20

commit d3790ada30eb10772b4a7e0cd810e191fc3d44e8 Author: Matthew Finkel <sysrqb(a)torproject.org> Date: Mon Oct 12 21:15:23 2020 +0000 Bug 40005: Add Fenix82 net audit Add java_audit.sh, authored by Mike (see https://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/40132#n…) --- audits/FF82_NETWORK_AUDIT | 125 ++++++++++++++++++++++++++++++++++++++++++++++ audits/java_audit.sh | 85 +++++++++++++++++++++++++++++++ 2 files changed, 210 insertions(+) … [View More]diff --git a/audits/FF82_NETWORK_AUDIT b/audits/FF82_NETWORK_AUDIT new file mode 100644 index 0000000..705a544 --- /dev/null +++ b/audits/FF82_NETWORK_AUDIT @@ -0,0 +1,125 @@ +`git diff cb11d5556759bd5bf174fbac719f51b2f02e2f0b 763b45bd9edb0073a2c6058dd3edc9254ec901e9` +and then go over all the changes containing the +above mentioned potentially dangerous calls and features. Grep the diff for +the following strings and examine surrounding usage. + +=============== Native DNS Portion ============= + +PR_GetHostByName +PR_GetIPNodeByName +PR_GetAddrInfoByName +PR_StringToNetAddr + +MDNS +TRR (DNS Trusted Recursive Resolver) + - Adds |doh-rollout.clearModeOnShutdown| pref for resetting |doh-rollout.mode| when the browser shuts down + +Direct Paths to DNS resolution +nsDNSService::Resolve +nsDNSService::AsyncResolve +nsHostResolver::ResolveHost + +# FF82: Nothing of interest + +============ Misc Socket Portion ============== + +SOCK_ +SOCKET_ +_SOCKET +UDPSocket +TCPSocket + PR_NewTCPSocket + AsyncTCPSocket + +Misc PR_Socket + +# FF82: Nothing of interest + +=========== Misc XPCOM Portion ================ + +Misc XPCOM (including commands for pre-diff review approach) + *SocketProvider + grep -R udp-socket . + grep -R tcp-socket . + grep for tcpsocket + grep -R "NS_" | grep SOCKET | grep "_C" + grep -R "@mozilla.org/network/" . | grep socket | grep -v udp-socket + + - New usage of ResolveNative in nsHttpConnectionMgr::nsHalfOpenSocket::SetupStreams + (netwerk/protocol/http/nsHttpConnectionMgr.cpp) but resolution is blocked by + DNSForbiddenByActiveProxy + - New usage of @mozilla.org/network/dns-service;1 in toolkit/content/aboutNetworking.js + but it only allows clearing the DNS cache + +============ Rust Portion ================ + +Rust + - XXX: What do we grep for here? Or do we rely on Ritter's compile-time tool? + - Check for new sendmsg and recvmsg usage + +# FF82: Zero new instances of sendmsg/recvmsg/connect + +============ Android Portion ============= + +Android Java calls + - URLConnection + - XXX: getInputStream? other methods? + - HttpURLConnection + - UrlConnectionDownloader + - ch.boye.httpclientandroidlib.impl.client.* (look for execute() calls) + - grep -n openConnection\( mobile/android/thirdparty/ + - java.net.URL -- has SEVERAL proxy bypass URL fetching methods :/ + - java.net + - javax.net + - ch.boye.httpclientandroidlib.conn.* (esp ssl) + - ch.boye.httpclientandroidlib.impl.conn.* (esp ssl) + - Sudden appearance of thirdparty libs: + - OkHttp + - Retrofit + - Glide + - com.amitshekhar.android + - IntentHelper + - openUriExternal (can come from GeckoAppShell too) + - getHandlersForMimeType + - getHandlersForURL + - getHandlersForIntent + - android.content.Intent - too common; instead find launch methods: + - startActivity + - startActivities + - sendBroadcast + - sendOrderedBroadcast + - startService + - bindService + - android.app.PendingIntent + - android.app.DownloadManager + - ActivityHandlerHelper.startIntentAndCatch + +# FF82: Nothing of interest (using `java_audit.sh`) + +============ Application Services Portion ============= + +Start: 160239424a37088ec84e15fb1bae82aed2cbee8f +End: 8e63363359c3d20385ed55f5308d19e321816898 # v63.0.0 + +Zero new usage found of known proxy-bypass APIs + +============ Android Components Portion ============= + +Start: c84cf8e7736ee77c22c75ca9f0397b202e489991 +End: 0a93a5ecd39e5a7f80e453a0d1a863057465aca0 # v60.0.3 + +Zero new usage found of known proxy-bypass APIs (using `java_audit.sh`) + +============ Fenix Portion ============= + +Start: b54949e58f9fda3698ada3e64b9f4337177d84f0 +End: 998b62866dee35929ca0d81641df101c83ac1224 # v82.0.0-beta.4 + +Zero new usage found of known proxy-bypass APIs (using `java_audit.sh`) + +============ Regression/Prior Vuln Review ========= + +Review proxy bypass bugs; check for new vectors to look for: + - https://trac.torproject.org/projects/tor/query?keywords=~tbb-proxy + - Look for new features like these. Especially external app launch vectors + diff --git a/audits/java_audit.sh b/audits/java_audit.sh new file mode 100644 index 0000000..57524eb --- /dev/null +++ b/audits/java_audit.sh @@ -0,0 +1,85 @@ +#!/bin/bash -e + +if [ $# -ne 3 ]; then + echo "usage: <path/to/repo> <old commit> <new commit>" + exit 1 +fi + +REPO_DIR=$1 + +OLD=$2 +NEW=$3 + +SCOPE="java" # string: this is the java audit + +declare -a KEYWORDS + +#KEYWORDS+=('\+\+\+\ ') + +# URL access +KEYWORDS+=(URLConnection) +KEYWORDS+=(UrlConnectionDownloader) + +# Proxy settings +KEYWORDS+=(ProxySelector) + +# Android and java networking and 3rd party libs +KEYWORDS+=("openConnection\(") +KEYWORDS+=("java.net") +KEYWORDS+=("javax.net") +KEYWORDS+=(android.net) +KEYWORDS+=(android.webkit) + +# Third Party http libs +KEYWORDS+=(ch.boye.httpclientandroidlib.impl.client) +KEYWORDS+=(okhttp) + +# Intents +KEYWORDS+=(IntentHelper) +KEYWORDS+=(openUriExternal) +KEYWORDS+=(getHandlersForMimeType) +KEYWORDS+=(getHandlersForURL) +KEYWORDS+=(getHandlersForIntent) +# KEYOWRDS+=(android.content.Intent) # Common +KEYWORDS+=(startActivity) +KEYWORDS+=(startActivities) +KEYWORDS+=(startBroadcast) +KEYWORDS+=(sendBroadcast) +KEYWORDS+=(sendOrderedBroadcast) +KEYWORDS+=(startService) +KEYWORDS+=(bindService) +KEYWORDS+=(android.app.PendingIntent) +KEYWORDS+=(ActivityHandlerHelper.startIntentAndCatch) +KEYWORDS+=(AppLinksInterceptor) +KEYWORDS+=(AppLinksUseCases) + +cd $REPO_DIR +#function join_by { local d=$1; shift; local f=$1; shift; printf %s "$f" "${@/#/ $d}"; } +#GREP_LINE="$(join_by \-G ${KEYWORDS[@]})" + +base=`git merge-base ${OLD} ${NEW}` + +#if [ ! -f "release-${OLD}-${NEW}.diff" ]; +if [ ! -f "release-${base}-${NEW}.diff" ]; +then + #echo "Diffing release-${OLD}-${NEW}.diff" + echo "Diffing release-${base}-${NEW}.diff" + #git diff --color=always --color-moved origin/$OLD origin/$NEW -U20 > release-${OLD}-${NEW}.diff + git diff --color=always --color-moved $base $NEW -U20 > release-${base}-${NEW}.diff + #git diff --color=always --color-moved -G${GREP_LINE} $OLD $NEW -U20 > release-${OLD}-${NEW}-G.diff +fi + +echo "Done with diff" + +function join_by { local d=$1; shift; local f=$1; shift; printf %s "$f" "${@/#/$d}"; } + +GREP_LINE="$(join_by \| ${KEYWORDS[@]})" +#GREP_LINE="\+\+\+ |$(join_by \| ${KEYWORDS[@]})" + +export GREP_COLOR="05;37;41" + +# XXX: Arg this sometimes misses file context +egrep -A40 -B40 --color=always "${GREP_LINE}" release-${base}-${NEW}.diff > keywords-${base}-${NEW}-$SCOPE.diff + +echo "Diff generated. View it with:" +echo " less -R $REPO_DIR/keywords-$base-$NEW-$SCOPE.diff" [View Less]

1 0

[tor-browser-build/master] Bug 40125: Fold in fenix-*rust and fenix-cbindgen projects
by sysrqb＠torproject.org 22 Oct '20

22 Oct '20

commit 1e88b1aeda67811065a87cfa6441fa6224abcef6 Author: Georg Koppen <gk(a)torproject.org> Date: Sun Oct 18 07:16:00 2020 +0000 Bug 40125: Fold in fenix-*rust and fenix-cbindgen projects --- projects/application-services/build | 2 +- projects/application-services/config | 8 +- projects/cbindgen/config | 17 +- projects/fenix-as-rust/build | 65 ------ projects/fenix-as-rust/config … [View More] | 103 --------- projects/fenix-as-rust/unwind.patch | 162 ------------- projects/fenix-cbindgen/build | 32 --- projects/fenix-cbindgen/config | 16 -- projects/fenix-rust/43909.patch | 252 --------------------- projects/fenix-rust/build | 65 ------ projects/fenix-rust/config | 103 --------- projects/fenix-rust/unwind.patch | 162 ------------- projects/geckoview/build | 2 +- projects/geckoview/config | 4 +- .../43909.patch => rust/43909_145.patch} | 0 projects/rust/build | 6 +- projects/rust/config | 40 ++-- 17 files changed, 41 insertions(+), 998 deletions(-) diff --git a/projects/application-services/build b/projects/application-services/build index 55ec519..9fb0f50 100644 --- a/projects/application-services/build +++ b/projects/application-services/build @@ -17,7 +17,7 @@ cd /var/tmp/dist tar -xf $rootdir/[% c('input_files_by_name/sqlcipher-' _ arch) %] mv sqlcipher sqlcipher-[% arch %] [% END -%] -export PATH=/var/tmp/dist/fenix-as-rust/bin:$PATH +export PATH=/var/tmp/dist/rust/bin:$PATH cd $rootdir # application-services uses a newer NDK, 21, than everything else... diff --git a/projects/application-services/config b/projects/application-services/config index d877002..68dc794 100644 --- a/projects/application-services/config +++ b/projects/application-services/config @@ -24,19 +24,19 @@ input_files: - project: '[% c("var/compiler") %]' name: '[% c("var/compiler") %]' # We need the torbrowser-* targets for Rust, see: #40094. - - project: fenix-as-rust + - project: rust name: rust-armv7 target_prepend: - torbrowser-android-armv7 - - project: fenix-as-rust + - project: rust name: rust-aarch64 target_prepend: - torbrowser-android-aarch64 - - project: fenix-as-rust + - project: rust name: rust-x86 target_prepend: - torbrowser-android-x86 - - project: fenix-as-rust + - project: rust name: rust-x86_64 target_prepend: - torbrowser-android-x86_64 diff --git a/projects/cbindgen/config b/projects/cbindgen/config index 3dd94f6..1258b4f 100644 --- a/projects/cbindgen/config +++ b/projects/cbindgen/config @@ -1,16 +1,27 @@ # vim: filetype=yaml sw=2 -version: 0.14.1 +version: '[% c("var/cbindgen_version") %]' git_url: https://github.com/eqrion/cbindgen -git_hash: 0761b9bbe48d01ded1bbec45bbeea5544b3b1002 # v0.14.1 +git_hash: '[% c("var/cbindgen_hash") %]' filename: '[% project %]-[% c("version") %]-[% c("var/osname") %]-[% c("var/build_id") %].tar.gz' var: + cbindgen_version: 0.14.1 + cbindgen_hash: 0761b9bbe48d01ded1bbec45bbeea5544b3b1002 + cbindgen_vendor_hash: 1c500bc1b8de4eb400f90a528c4b5c6b2c7357e401a95c5354469a18fd365dbc container: use_container: 1 + +targets: + android: + var: + cbindgen_version: 0.14.3 + cbindgen_hash: cc2876f709808a52d132a5f84e53ca1964eb92db + cbindgen_vendor_hash: cc47db55a1c937e7bcd1cbd8664964656a4922bc35175d95c4015d452f2dcb2b + input_files: - project: container-image - name: rust project: rust # `cargo vendor vendor` in the `cbindgen` directory has vendored the sources. - URL: https://people.torproject.org/~gk/mirrors/sources/cbindgen-vendor-[% c('version') %].tar.bz2 - sha256sum: 1c500bc1b8de4eb400f90a528c4b5c6b2c7357e401a95c5354469a18fd365dbc + sha256sum: '[% c("var/cbindgen_vendor_hash") %]' diff --git a/projects/fenix-as-rust/build b/projects/fenix-as-rust/build deleted file mode 100644 index 5d3d5c1..0000000 --- a/projects/fenix-as-rust/build +++ /dev/null @@ -1,65 +0,0 @@ -#!/bin/bash -[% c("var/set_default_env") -%] -distdir=/var/tmp/dist/[% project %] -mkdir -p $distdir -tar -C /var/tmp/dist -xf [% c('input_files_by_name/cmake') %] -export PATH="/var/tmp/dist/cmake/bin:$PATH" -tar -C /var/tmp/dist -xf [% c('input_files_by_name/prev_rust') %] -cd /var/tmp/dist/rust-[% c('var/prev_version') %]-x86_64-unknown-linux-gnu -./install.sh --prefix=$distdir-rust-old -export PATH="$distdir-rust-old/bin:$PATH" - -[% pc(c('var/compiler'), 'var/setup', { compiler_tarfile => c('input_files_by_name/' _ c('var/compiler')) }) %] - -[% IF c("var/osx") %] - # We need to clear `CC` and `LDFLAGS` as they are used for the host platform - # (i.e. Linux). - unset CC - unset LDFLAGS - - mkdir $distdir/helper - # We need to adapt our CFLAGS and make sure our flags are passed down to all - # dependencies. Using `CFLAGS_x86_apple-darwin` did not do the trick, so resort - # to a wrapper script. - cat > $distdir/helper/x86_64-apple-darwin-clang << 'EOF' -#!/bin/sh -BASEDIR=/var/tmp/dist/macosx-toolchain -$BASEDIR/cctools/bin/x86_64-apple-darwin-clang -target x86_64-apple-darwin -B $BASEDIR/cctools/bin -isysroot $BASEDIR/MacOSX10.11.sdk/ -Wl,-syslibroot,$BASEDIR/MacOSX10.11.sdk/ -Wl,-dead_strip -Wl,-pie "$@" -EOF - - chmod +x $distdir/helper/x86_64-apple-darwin-clang - export PATH=$distdir/helper:$PATH -[% END %] - -cd $rootdir -mkdir /var/tmp/build -tar -C /var/tmp/build -xf [% c('input_files_by_name/rust') %] -cd /var/tmp/build/rustc-[% c('version') %]-src - -# LLVM has reproducibility issues when optimizing bitcode, which we need to -# patch. See: #32053 for more details. -cd src/llvm-project -patch -p1 < $rootdir/43909.patch -cd ../../ - -[% IF c("var/windows-i686") %] - # Cross-compiling for Windows 32bit is currently not possible without any - # patches. The reason for that is libstd expecting DWARF unwinding while most - # toolchains on Linux, targeting Windows 32bit, use SjLj unwinding. - # See: https://github.com/rust-lang/rust/issues/12859 for discussion about - # that and https://github.com/rust-lang/rust/pull/49633 for a newer attempt to - # fix this problem. We apply the patch from neersighted. - patch -p1 < $rootdir/unwind.patch -[% END %] - -mkdir build -cd build -../configure --prefix=$distdir [% c("var/configure_opt") %] - -make -j[% c("buildconf/num_procs") %] -make install -cd /var/tmp/dist -[% c('tar', { - tar_src => [ project ], - tar_args => '-czf ' _ dest_dir _ '/' _ c('filename'), - }) %] diff --git a/projects/fenix-as-rust/config b/projects/fenix-as-rust/config deleted file mode 100644 index 4a4ae3c..0000000 --- a/projects/fenix-as-rust/config +++ /dev/null @@ -1,103 +0,0 @@ -# vim: filetype=yaml sw=2 -filename: '[% project %]-[% c("version") %]-[% c("var/build_id") %].tar.gz' -version: '[% c("input_file_var/rust_version") %]' - -# Those values can be changed from the input_files section of other -# projects. See bug 32436. -input_file_var: - rust_version: 1.45.2 - prev_version: 1.44.1 - -var: - prev_version: '[% c("input_file_var/prev_version") %]' - container: - use_container: 1 - -targets: - android: - var: - arch_deps: - - libssl-dev - - pkg-config - - zlib1g-dev - configure_opt: --enable-local-rust --enable-vendor --enable-extended --release-channel=stable --sysconfdir=etc --target=[% c("var/cross_prefix") %] --set=target.[% c("var/cross_prefix") %].cc=[% c("var/CC") %] --set=target.[% c("var/cross_prefix") %].ar=[% c("var/cross_prefix") %]-ar - - android-armv7: - var: - configure_opt: --enable-local-rust --enable-vendor --enable-extended --release-channel=stable --sysconfdir=etc --target=[% IF pc(c("origin_project"), "var/application_services") %]armv7-linux-androideabi[% ELSE %]thumbv7neon-linux-androideabi[% END %] --set=target.[% IF pc(c("origin_project"), "var/application_services") %]armv7-linux-androideabi[% ELSE %]thumbv7neon-linux-androideabi[% END %].cc=[% c("var/CC") %] --set=target.[% IF pc(c("origin_project"), "var/application_services") %]armv7-linux-androideabi[% ELSE %]thumbv7neon-linux-androideabi[% END %].ar=[% c("var/cross_prefix") %]-ar - - linux: - var: - deps: - - libc6-dev-i386 - - lib32stdc++6 - - build-essential - - python - - automake - - libssl-dev - - pkg-config - - hardening-wrapper - # We use - # `--enable-local-rust` to avoid downloading the required compiler during - # build time - # - # `--enable-vendor` to avoid downloading crates during build time and just - # use the ones which are shipped with the source - # - # `--enable-extended` to build not only rustc but cargo as well - # - # `--enable-llvm-static-stdccp` to take a libstdc++ on Jessie into account - # which is too old and if used gives undefined reference errors - # - # `--release-channel=stable` to just include stable features in the - # compiler - # - # `--sysconfdir=etc` to avoid install failures as |make install| wants to - # write to /etc otherwise - # - # the `target` triple to explicitly specify the architecture and platform - # for the compiler/std lib. Ideally, it should not be needed unless one is - # cross-compiling, but compiling `alloc_jemalloc` fails without that in a - # 32bit container. "--host=x86_64-unknown-linux-gnu" is used in its - # configure script in this case. - # `--set=` to explicitly specify the C compiler. We need to compile the - # bundled LLVM and it wants to use `cc`. However, we don't have that in - # our compiled GCC resulting in weird errors due to C and C++ compiler - # version mismatch. We avoid that with this configure option. - configure_opt: --enable-local-rust --enable-vendor --enable-extended --enable-llvm-static-stdcpp --release-channel=stable --sysconfdir=etc --target=x86_64-unknown-linux-gnu,i686-unknown-linux-gnu --set=target.x86_64-unknown-linux-gnu.cc=gcc --set=target.i686-unknown-linux-gnu.cc=gcc - - osx-x86_64: - var: - arch_deps: - - libssl-dev - - pkg-config - - zlib1g-dev - configure_opt: --enable-local-rust --enable-vendor --enable-extended --release-channel=stable --sysconfdir=etc --target=x86_64-apple-darwin --set=target.x86_64-apple-darwin.cc=x86_64-apple-darwin-clang - - windows: - var: - arch_deps: - - libssl-dev - - pkg-config - - zlib1g-dev - configure_opt: --enable-local-rust --enable-vendor --enable-extended --release-channel=stable --sysconfdir=etc --target=[% c("arch") %]-pc-windows-gnu - -input_files: - - project: container-image - - project: cmake - name: cmake - - project: '[% c("var/compiler") %]' - name: '[% c("var/compiler") %]' - - URL: 'https://static.rust-lang.org/dist/rustc-[% c("version") %]-src.tar.gz' - name: rust - sig_ext: asc - file_gpg_id: 1 - gpg_keyring: rust.gpg - - URL: 'https://static.rust-lang.org/dist/rust-[% c("var/prev_version") %]-x86_64-unknown-linux-gnu.tar.xz' - name: prev_rust - sig_ext: asc - file_gpg_id: 1 - gpg_keyring: rust.gpg - - filename: unwind.patch - enable: '[% c("var/windows-i686") %]' - - filename: 43909.patch diff --git a/projects/fenix-as-rust/unwind.patch b/projects/fenix-as-rust/unwind.patch deleted file mode 100644 index 7b22dcb..0000000 --- a/projects/fenix-as-rust/unwind.patch +++ /dev/null @@ -1,162 +0,0 @@ -From b3bea7008ece7a5bdf9b5a5dcc95e82febad1854 Mon Sep 17 00:00:00 2001 -From: Bjorn Neergaard <bjorn(a)neersighted.com> -Date: Sat, 9 Feb 2019 19:39:23 +0000 -Subject: [PATCH] Fix cross-compiling i686-pc-windows-gnu from Linux - -This is still very rough and serves as a proof-of-concept for fixing -Linux -> 32-bit MinGW cross compilation workflow. Currently, clang and -GCC's MinGW targets both only support DW2 (DWARF) or SJLJ (Set Jump Long -Jump) unwinding on 32-bit Windows. - -The default for GCC (and the way it is shipped on every major distro) is -to use SJLJ on Windows, as DWARF cannot traverse non-DWARF frames. This -would work fine, except for the fact that libgcc (our C runtime on the -MinGW platform) exports symbols under a different name when configured -to use SJLJ-style unwinding, and uses a preprocessor macro internally to -alias them. - -Because of this, we have to detect this scenario and link to the correct -symbols ourselves. Linking has been tested with a full bootstrap on both -x86_64-unknown-linux-gnu and i686-pc-windows-gnu, as well as -cross-compilation of some of my own projects. - -Obviously, the detection is a bit unrefined. Right now we -unconditionally use SJLJ when compiling Linux -> MinGW. I'd like to add -feature detection using compiler build flags or autotools-style -compilation and object analysis. Input on the best way to proceed here -is welcome. - -Also, currently there is copy-pasted/duplicated code in libunwind. -Ideally, this could be reduced, but this would likely require a -rethinking of how iOS is special-cased above, to avoid further -duplication. Input on how to best structure this file is requested. - -diff --git a/src/bootstrap/compile.rs b/src/bootstrap/compile.rs -index 249a183189..df08d6eb0c 100644 ---- a/src/bootstrap/compile.rs -+++ b/src/bootstrap/compile.rs -@@ -162,7 +162,12 @@ pub fn std_cargo(builder: &Builder<'_>, - .arg("--features") - .arg("compiler-builtins-mem"); - } else { -- let features = builder.std_features(); -+ let mut features = builder.std_features(); -+ -+ // FIXME: Temporary detection of SJLJ MinGW compilers. -+ if builder.config.build.contains("linux") && target == "i686-pc-windows-gnu" { -+ features.push_str(" sjlj_eh"); -+ } - - if compiler.stage != 0 && builder.config.sanitizers { - // This variable is used by the sanitizer runtime crates, e.g. -diff --git a/src/libstd/Cargo.toml b/src/libstd/Cargo.toml -index 7d60a17042..d876d0b89a 100644 ---- a/src/libstd/Cargo.toml -+++ b/src/libstd/Cargo.toml -@@ -71,3 +71,4 @@ wasm-bindgen-threads = [] - # https://github.com/rust-lang-nursery/stdsimd/blob/master/crates/std_detect/… - std_detect_file_io = [] - std_detect_dlsym_getauxval = [] -+sjlj_eh = ["unwind/sjlj_eh"] -diff --git a/src/libunwind/Cargo.toml b/src/libunwind/Cargo.toml -index 2378b0a315..0b5979ed62 100644 ---- a/src/libunwind/Cargo.toml -+++ b/src/libunwind/Cargo.toml -@@ -16,3 +16,6 @@ doc = false - core = { path = "../libcore" } - libc = { version = "0.2.43", features = ['rustc-dep-of-std'], default-features = false } - compiler_builtins = "0.1.0" -+ -+[features] -+sjlj_eh = [] -diff --git a/src/libunwind/libunwind.rs b/src/libunwind/libunwind.rs -index 339b554ed6..ec2f93ed60 100644 ---- a/src/libunwind/libunwind.rs -+++ b/src/libunwind/libunwind.rs -@@ -1,10 +1,5 @@ - #![allow(nonstandard_style)] - --macro_rules! cfg_if { -- ( $( if #[cfg( $meta:meta )] { $($it1:item)* } else { $($it2:item)* } )* ) => -- ( $( $( #[cfg($meta)] $it1)* $( #[cfg(not($meta))] $it2)* )* ) --} -- - use libc::{c_int, c_void, uintptr_t}; - - #[repr(C)] -@@ -73,8 +68,8 @@ pub enum _Unwind_Context {} - pub type _Unwind_Exception_Cleanup_Fn = extern "C" fn(unwind_code: _Unwind_Reason_Code, - exception: *mut _Unwind_Exception); - extern "C" { -- #[unwind(allowed)] -- pub fn _Unwind_Resume(exception: *mut _Unwind_Exception) -> !; -+ #[cfg_attr(stage0, unwind)] -+ #[cfg_attr(not(stage0), unwind(allowed))] - pub fn _Unwind_DeleteException(exception: *mut _Unwind_Exception); - pub fn _Unwind_GetLanguageSpecificData(ctx: *mut _Unwind_Context) -> *mut c_void; - pub fn _Unwind_GetRegionStart(ctx: *mut _Unwind_Context) -> _Unwind_Ptr; -@@ -206,26 +201,52 @@ if #[cfg(all(any(target_os = "ios", target_os = "netbsd", not(target_arch = "arm - pc - } - } -+} // cfg_if! - --if #[cfg(not(all(target_os = "ios", target_arch = "arm")))] { -- // Not 32-bit iOS -+cfg_if! { -+if #[cfg(all(target_os = "ios", target_arch = "arm"))] { -+ // 32-bit iOS uses SjLj and does not provide _Unwind_Backtrace() - extern "C" { -- #[unwind(allowed)] -- pub fn _Unwind_RaiseException(exception: *mut _Unwind_Exception) -> _Unwind_Reason_Code; -+ #[cfg_attr(stage0, unwind)] -+ #[cfg_attr(not(stage0), unwind(allowed))] -+ pub fn _Unwind_Resume(exception: *mut _Unwind_Exception) -> !; -+ pub fn _Unwind_SjLj_RaiseException(e: *mut _Unwind_Exception) -> _Unwind_Reason_Code; -+ } -+ -+ #[inline] -+ pub unsafe fn _Unwind_RaiseException(exc: *mut _Unwind_Exception) -> _Unwind_Reason_Code { -+ _Unwind_SjLj_RaiseException(exc) -+ } -+ -+} else if #[cfg(feature = "sjlj_eh")] { -+ extern "C" { -+ #[cfg_attr(stage0, unwind)] -+ #[cfg_attr(not(stage0), unwind(allowed))] -+ pub fn _Unwind_SjLj_Resume(e: *mut _Unwind_Exception) -> !; -+ pub fn _Unwind_SjLj_RaiseException(e: *mut _Unwind_Exception) -> _Unwind_Reason_Code; - pub fn _Unwind_Backtrace(trace: _Unwind_Trace_Fn, - trace_argument: *mut c_void) - -> _Unwind_Reason_Code; - } --} else { -- // 32-bit iOS uses SjLj and does not provide _Unwind_Backtrace() -- extern "C" { -- #[unwind(allowed)] -- pub fn _Unwind_SjLj_RaiseException(e: *mut _Unwind_Exception) -> _Unwind_Reason_Code; -+ -+ #[inline] -+ pub unsafe fn _Unwind_Resume(exc: *mut _Unwind_Exception) -> ! { -+ _Unwind_SjLj_Resume(exc) - } - - #[inline] - pub unsafe fn _Unwind_RaiseException(exc: *mut _Unwind_Exception) -> _Unwind_Reason_Code { - _Unwind_SjLj_RaiseException(exc) - } -+} else { -+ extern "C" { -+ #[cfg_attr(stage0, unwind)] -+ #[cfg_attr(not(stage0), unwind(allowed))] -+ pub fn _Unwind_Resume(exception: *mut _Unwind_Exception) -> !; -+ pub fn _Unwind_RaiseException(exception: *mut _Unwind_Exception) -> _Unwind_Reason_Code; -+ pub fn _Unwind_Backtrace(trace: _Unwind_Trace_Fn, -+ trace_argument: *mut c_void) -+ -> _Unwind_Reason_Code; -+ } - } - } // cfg_if! --- -2.23.0.rc0 - diff --git a/projects/fenix-cbindgen/build b/projects/fenix-cbindgen/build deleted file mode 100644 index c63a783..0000000 --- a/projects/fenix-cbindgen/build +++ /dev/null @@ -1,32 +0,0 @@ -#!/bin/bash -[% c("var/set_default_env") -%] -distdir=/var/tmp/dist -builddir=/var/tmp/build/[% project %] -mkdir -p $distdir/[% project %] -tar -C /var/tmp/dist -xf $rootdir/[% c('input_files_by_name/rust') %] -export PATH="/var/tmp/dist/fenix-rust/bin:$PATH" -mkdir -p /var/tmp/build -tar -C /var/tmp/build -xf [% project %]-[% c('version') %].tar.gz - -# Now prepare the offline build -# Move the directory for hardcoding the path in .cargo/config -mv /var/tmp/build/[% project %]-[% c('version') %] $builddir -tar -C $builddir -xjf cbindgen-vendor-[% c('version') %].tar.bz2 -cd $builddir -mkdir .cargo -cat > .cargo/config << 'EOF' -[source.crates-io] -replace-with = "vendored-sources" - -[source.vendored-sources] -directory = "/var/tmp/build/fenix-cbindgen/vendor" -EOF - -cargo build --release --frozen --target x86_64-unknown-linux-gnu -mv target/x86_64-unknown-linux-gnu/release/cbindgen $distdir/[% project %] - -cd $distdir -[% c('tar', { - tar_src => [ project ], - tar_args => '-czf ' _ dest_dir _ '/' _ c('filename'), - }) %] diff --git a/projects/fenix-cbindgen/config b/projects/fenix-cbindgen/config deleted file mode 100644 index c081451..0000000 --- a/projects/fenix-cbindgen/config +++ /dev/null @@ -1,16 +0,0 @@ -# vim: filetype=yaml sw=2 -version: 0.14.3 -git_url: https://github.com/eqrion/cbindgen -git_hash: cc2876f709808a52d132a5f84e53ca1964eb92db # v0.14.3 -filename: '[% project %]-[% c("version") %]-[% c("var/osname") %]-[% c("var/build_id") %].tar.gz' - -var: - container: - use_container: 1 -input_files: - - project: container-image - - name: rust - project: fenix-rust - # `cargo vendor vendor` in the `cbindgen` directory has vendored the sources. - - URL: https://people.torproject.org/~gk/mirrors/sources/cbindgen-vendor-[% c('version') %].tar.bz2 - sha256sum: cc47db55a1c937e7bcd1cbd8664964656a4922bc35175d95c4015d452f2dcb2b diff --git a/projects/fenix-rust/43909.patch b/projects/fenix-rust/43909.patch deleted file mode 100644 index 78d2a75..0000000 --- a/projects/fenix-rust/43909.patch +++ /dev/null @@ -1,252 +0,0 @@ -From c95310f2d4fd3c88241c3b5d6dbf6251d34a3256 Mon Sep 17 00:00:00 2001 -From: Nikita Popov <nikita.ppv(a)gmail.com> -Date: Sat, 16 Nov 2019 16:22:18 +0100 -Subject: [PATCH] Restructure caching - -Variant on D70103. The caching is switched to always use a BB to -cache entry map, which then contains per-value caches. A separate -set contains value handles with a deletion callback. This allows us -to properly invalidate overdefined values. - -A possible alternative would be to always cache by value first and -have per-BB maps/sets in the each cache entry. In that case we could -use a ValueMap and would avoid the separate value handle set. I went -with the BB indexing at the top level to make it easier to integrate -D69914, but possibly that's not the right choice. - -Differential Revision: https://reviews.llvm.org/D70376 - -diff --git a/llvm/lib/Analysis/LazyValueInfo.cpp b/llvm/lib/Analysis/LazyValueInfo.cpp -index 110c085d3f3..aa6862cb588 100644 ---- a/llvm/lib/Analysis/LazyValueInfo.cpp -+++ b/llvm/lib/Analysis/LazyValueInfo.cpp -@@ -133,12 +133,9 @@ namespace { - /// A callback value handle updates the cache when values are erased. - class LazyValueInfoCache; - struct LVIValueHandle final : public CallbackVH { -- // Needs to access getValPtr(), which is protected. -- friend struct DenseMapInfo<LVIValueHandle>; -- - LazyValueInfoCache *Parent; - -- LVIValueHandle(Value *V, LazyValueInfoCache *P) -+ LVIValueHandle(Value *V, LazyValueInfoCache *P = nullptr) - : CallbackVH(V), Parent(P) { } - - void deleted() override; -@@ -152,89 +149,63 @@ namespace { - /// This is the cache kept by LazyValueInfo which - /// maintains information about queries across the clients' queries. - class LazyValueInfoCache { -- /// This is all of the cached block information for exactly one Value*. -- /// The entries are sorted by the BasicBlock* of the -- /// entries, allowing us to do a lookup with a binary search. -- /// Over-defined lattice values are recorded in OverDefinedCache to reduce -- /// memory overhead. -- struct ValueCacheEntryTy { -- ValueCacheEntryTy(Value *V, LazyValueInfoCache *P) : Handle(V, P) {} -- LVIValueHandle Handle; -- SmallDenseMap<PoisoningVH<BasicBlock>, ValueLatticeElement, 4> BlockVals; -+ /// This is all of the cached information for one basic block. It contains -+ /// the per-value lattice elements, as well as a separate set for -+ /// overdefined values to reduce memory usage. -+ struct BlockCacheEntryTy { -+ SmallDenseMap<AssertingVH<Value>, ValueLatticeElement, 4> LatticeElements; -+ SmallDenseSet<AssertingVH<Value>, 4> OverDefined; - }; - -- /// This tracks, on a per-block basis, the set of values that are -- /// over-defined at the end of that block. -- typedef DenseMap<PoisoningVH<BasicBlock>, SmallPtrSet<Value *, 4>> -- OverDefinedCacheTy; -- /// Keep track of all blocks that we have ever seen, so we -- /// don't spend time removing unused blocks from our caches. -- DenseSet<PoisoningVH<BasicBlock> > SeenBlocks; -- -- /// This is all of the cached information for all values, -- /// mapped from Value* to key information. -- DenseMap<Value *, std::unique_ptr<ValueCacheEntryTy>> ValueCache; -- OverDefinedCacheTy OverDefinedCache; -- -+ /// Cached information per basic block. -+ DenseMap<PoisoningVH<BasicBlock>, BlockCacheEntryTy> BlockCache; -+ /// Set of value handles used to erase values from the cache on deletion. -+ DenseSet<LVIValueHandle, DenseMapInfo<Value *>> ValueHandles; - - public: - void insertResult(Value *Val, BasicBlock *BB, - const ValueLatticeElement &Result) { -- SeenBlocks.insert(BB); -- -+ auto &CacheEntry = BlockCache.try_emplace(BB).first->second; - // Insert over-defined values into their own cache to reduce memory - // overhead. - if (Result.isOverdefined()) -- OverDefinedCache[BB].insert(Val); -- else { -- auto It = ValueCache.find_as(Val); -- if (It == ValueCache.end()) { -- ValueCache[Val] = make_unique<ValueCacheEntryTy>(Val, this); -- It = ValueCache.find_as(Val); -- assert(It != ValueCache.end() && "Val was just added to the map!"); -- } -- It->second->BlockVals[BB] = Result; -- } -- } -- -- bool isOverdefined(Value *V, BasicBlock *BB) const { -- auto ODI = OverDefinedCache.find(BB); -- -- if (ODI == OverDefinedCache.end()) -- return false; -+ CacheEntry.OverDefined.insert(Val); -+ else -+ CacheEntry.LatticeElements.insert({ Val, Result }); - -- return ODI->second.count(V); -+ auto HandleIt = ValueHandles.find_as(Val); -+ if (HandleIt == ValueHandles.end()) -+ ValueHandles.insert({ Val, this }); - } - - bool hasCachedValueInfo(Value *V, BasicBlock *BB) const { -- if (isOverdefined(V, BB)) -- return true; -- -- auto I = ValueCache.find_as(V); -- if (I == ValueCache.end()) -+ auto It = BlockCache.find(BB); -+ if (It == BlockCache.end()) - return false; - -- return I->second->BlockVals.count(BB); -+ return It->second.OverDefined.count(V) || -+ It->second.LatticeElements.count(V); - } - - ValueLatticeElement getCachedValueInfo(Value *V, BasicBlock *BB) const { -- if (isOverdefined(V, BB)) -+ auto It = BlockCache.find(BB); -+ if (It == BlockCache.end()) -+ return ValueLatticeElement(); -+ -+ if (It->second.OverDefined.count(V)) - return ValueLatticeElement::getOverdefined(); - -- auto I = ValueCache.find_as(V); -- if (I == ValueCache.end()) -+ auto LatticeIt = It->second.LatticeElements.find(V); -+ if (LatticeIt == It->second.LatticeElements.end()) - return ValueLatticeElement(); -- auto BBI = I->second->BlockVals.find(BB); -- if (BBI == I->second->BlockVals.end()) -- return ValueLatticeElement(); -- return BBI->second; -+ -+ return LatticeIt->second; - } - - /// clear - Empty the cache. - void clear() { -- SeenBlocks.clear(); -- ValueCache.clear(); -- OverDefinedCache.clear(); -+ BlockCache.clear(); -+ ValueHandles.clear(); - } - - /// Inform the cache that a given value has been deleted. -@@ -248,23 +219,18 @@ namespace { - /// OldSucc might have (unless also overdefined in NewSucc). This just - /// flushes elements from the cache and does not add any. - void threadEdgeImpl(BasicBlock *OldSucc,BasicBlock *NewSucc); -- -- friend struct LVIValueHandle; - }; - } - - void LazyValueInfoCache::eraseValue(Value *V) { -- for (auto I = OverDefinedCache.begin(), E = OverDefinedCache.end(); I != E;) { -- // Copy and increment the iterator immediately so we can erase behind -- // ourselves. -- auto Iter = I++; -- SmallPtrSetImpl<Value *> &ValueSet = Iter->second; -- ValueSet.erase(V); -- if (ValueSet.empty()) -- OverDefinedCache.erase(Iter); -+ for (auto &Pair : BlockCache) { -+ Pair.second.LatticeElements.erase(V); -+ Pair.second.OverDefined.erase(V); - } - -- ValueCache.erase(V); -+ auto HandleIt = ValueHandles.find_as(V); -+ if (HandleIt != ValueHandles.end()) -+ ValueHandles.erase(HandleIt); - } - - void LVIValueHandle::deleted() { -@@ -274,18 +240,7 @@ void LVIValueHandle::deleted() { - } - - void LazyValueInfoCache::eraseBlock(BasicBlock *BB) { -- // Shortcut if we have never seen this block. -- DenseSet<PoisoningVH<BasicBlock> >::iterator I = SeenBlocks.find(BB); -- if (I == SeenBlocks.end()) -- return; -- SeenBlocks.erase(I); -- -- auto ODI = OverDefinedCache.find(BB); -- if (ODI != OverDefinedCache.end()) -- OverDefinedCache.erase(ODI); -- -- for (auto &I : ValueCache) -- I.second->BlockVals.erase(BB); -+ BlockCache.erase(BB); - } - - void LazyValueInfoCache::threadEdgeImpl(BasicBlock *OldSucc, -@@ -303,10 +258,11 @@ void LazyValueInfoCache::threadEdgeImpl(BasicBlock *OldSucc, - std::vector<BasicBlock*> worklist; - worklist.push_back(OldSucc); - -- auto I = OverDefinedCache.find(OldSucc); -- if (I == OverDefinedCache.end()) -+ auto I = BlockCache.find(OldSucc); -+ if (I == BlockCache.end() || I->second.OverDefined.empty()) - return; // Nothing to process here. -- SmallVector<Value *, 4> ValsToClear(I->second.begin(), I->second.end()); -+ SmallVector<Value *, 4> ValsToClear(I->second.OverDefined.begin(), -+ I->second.OverDefined.end()); - - // Use a worklist to perform a depth-first search of OldSucc's successors. - // NOTE: We do not need a visited list since any blocks we have already -@@ -320,10 +276,10 @@ void LazyValueInfoCache::threadEdgeImpl(BasicBlock *OldSucc, - if (ToUpdate == NewSucc) continue; - - // If a value was marked overdefined in OldSucc, and is here too... -- auto OI = OverDefinedCache.find(ToUpdate); -- if (OI == OverDefinedCache.end()) -+ auto OI = BlockCache.find(ToUpdate); -+ if (OI == BlockCache.end() || OI->second.OverDefined.empty()) - continue; -- SmallPtrSetImpl<Value *> &ValueSet = OI->second; -+ auto &ValueSet = OI->second.OverDefined; - - bool changed = false; - for (Value *V : ValsToClear) { -@@ -333,11 +289,6 @@ void LazyValueInfoCache::threadEdgeImpl(BasicBlock *OldSucc, - // If we removed anything, then we potentially need to update - // blocks successors too. - changed = true; -- -- if (ValueSet.empty()) { -- OverDefinedCache.erase(OI); -- break; -- } - } - - if (!changed) continue; --- -2.24.0 - diff --git a/projects/fenix-rust/build b/projects/fenix-rust/build deleted file mode 100644 index 5d3d5c1..0000000 --- a/projects/fenix-rust/build +++ /dev/null @@ -1,65 +0,0 @@ -#!/bin/bash -[% c("var/set_default_env") -%] -distdir=/var/tmp/dist/[% project %] -mkdir -p $distdir -tar -C /var/tmp/dist -xf [% c('input_files_by_name/cmake') %] -export PATH="/var/tmp/dist/cmake/bin:$PATH" -tar -C /var/tmp/dist -xf [% c('input_files_by_name/prev_rust') %] -cd /var/tmp/dist/rust-[% c('var/prev_version') %]-x86_64-unknown-linux-gnu -./install.sh --prefix=$distdir-rust-old -export PATH="$distdir-rust-old/bin:$PATH" - -[% pc(c('var/compiler'), 'var/setup', { compiler_tarfile => c('input_files_by_name/' _ c('var/compiler')) }) %] - -[% IF c("var/osx") %] - # We need to clear `CC` and `LDFLAGS` as they are used for the host platform - # (i.e. Linux). - unset CC - unset LDFLAGS - - mkdir $distdir/helper - # We need to adapt our CFLAGS and make sure our flags are passed down to all - # dependencies. Using `CFLAGS_x86_apple-darwin` did not do the trick, so resort - # to a wrapper script. - cat > $distdir/helper/x86_64-apple-darwin-clang << 'EOF' -#!/bin/sh -BASEDIR=/var/tmp/dist/macosx-toolchain -$BASEDIR/cctools/bin/x86_64-apple-darwin-clang -target x86_64-apple-darwin -B $BASEDIR/cctools/bin -isysroot $BASEDIR/MacOSX10.11.sdk/ -Wl,-syslibroot,$BASEDIR/MacOSX10.11.sdk/ -Wl,-dead_strip -Wl,-pie "$@" -EOF - - chmod +x $distdir/helper/x86_64-apple-darwin-clang - export PATH=$distdir/helper:$PATH -[% END %] - -cd $rootdir -mkdir /var/tmp/build -tar -C /var/tmp/build -xf [% c('input_files_by_name/rust') %] -cd /var/tmp/build/rustc-[% c('version') %]-src - -# LLVM has reproducibility issues when optimizing bitcode, which we need to -# patch. See: #32053 for more details. -cd src/llvm-project -patch -p1 < $rootdir/43909.patch -cd ../../ - -[% IF c("var/windows-i686") %] - # Cross-compiling for Windows 32bit is currently not possible without any - # patches. The reason for that is libstd expecting DWARF unwinding while most - # toolchains on Linux, targeting Windows 32bit, use SjLj unwinding. - # See: https://github.com/rust-lang/rust/issues/12859 for discussion about - # that and https://github.com/rust-lang/rust/pull/49633 for a newer attempt to - # fix this problem. We apply the patch from neersighted. - patch -p1 < $rootdir/unwind.patch -[% END %] - -mkdir build -cd build -../configure --prefix=$distdir [% c("var/configure_opt") %] - -make -j[% c("buildconf/num_procs") %] -make install -cd /var/tmp/dist -[% c('tar', { - tar_src => [ project ], - tar_args => '-czf ' _ dest_dir _ '/' _ c('filename'), - }) %] diff --git a/projects/fenix-rust/config b/projects/fenix-rust/config deleted file mode 100644 index 0f08463..0000000 --- a/projects/fenix-rust/config +++ /dev/null @@ -1,103 +0,0 @@ -# vim: filetype=yaml sw=2 -filename: '[% project %]-[% c("version") %]-[% c("var/build_id") %].tar.gz' -version: '[% c("input_file_var/rust_version") %]' - -# Those values can be changed from the input_files section of other -# projects. See bug 32436. -input_file_var: - rust_version: 1.43.0 - prev_version: 1.42.0 - -var: - prev_version: '[% c("input_file_var/prev_version") %]' - container: - use_container: 1 - -targets: - android: - var: - arch_deps: - - libssl-dev - - pkg-config - - zlib1g-dev - configure_opt: --enable-local-rust --enable-vendor --enable-extended --release-channel=stable --sysconfdir=etc --target=[% c("var/cross_prefix") %] --set=target.[% c("var/cross_prefix") %].cc=[% c("var/CC") %] --set=target.[% c("var/cross_prefix") %].ar=[% c("var/cross_prefix") %]-ar - - android-armv7: - var: - configure_opt: --enable-local-rust --enable-vendor --enable-extended --release-channel=stable --sysconfdir=etc --target=[% IF pc(c("origin_project"), "var/application_services") %]armv7-linux-androideabi[% ELSE %]thumbv7neon-linux-androideabi[% END %] --set=target.[% IF pc(c("origin_project"), "var/application_services") %]armv7-linux-androideabi[% ELSE %]thumbv7neon-linux-androideabi[% END %].cc=[% c("var/CC") %] --set=target.[% IF pc(c("origin_project"), "var/application_services") %]armv7-linux-androideabi[% ELSE %]thumbv7neon-linux-androideabi[% END %].ar=[% c("var/cross_prefix") %]-ar - - linux: - var: - deps: - - libc6-dev-i386 - - lib32stdc++6 - - build-essential - - python - - automake - - libssl-dev - - pkg-config - - hardening-wrapper - # We use - # `--enable-local-rust` to avoid downloading the required compiler during - # build time - # - # `--enable-vendor` to avoid downloading crates during build time and just - # use the ones which are shipped with the source - # - # `--enable-extended` to build not only rustc but cargo as well - # - # `--enable-llvm-static-stdccp` to take a libstdc++ on Jessie into account - # which is too old and if used gives undefined reference errors - # - # `--release-channel=stable` to just include stable features in the - # compiler - # - # `--sysconfdir=etc` to avoid install failures as |make install| wants to - # write to /etc otherwise - # - # the `target` triple to explicitly specify the architecture and platform - # for the compiler/std lib. Ideally, it should not be needed unless one is - # cross-compiling, but compiling `alloc_jemalloc` fails without that in a - # 32bit container. "--host=x86_64-unknown-linux-gnu" is used in its - # configure script in this case. - # `--set=` to explicitly specify the C compiler. We need to compile the - # bundled LLVM and it wants to use `cc`. However, we don't have that in - # our compiled GCC resulting in weird errors due to C and C++ compiler - # version mismatch. We avoid that with this configure option. - configure_opt: --enable-local-rust --enable-vendor --enable-extended --enable-llvm-static-stdcpp --release-channel=stable --sysconfdir=etc --target=x86_64-unknown-linux-gnu,i686-unknown-linux-gnu --set=target.x86_64-unknown-linux-gnu.cc=gcc --set=target.i686-unknown-linux-gnu.cc=gcc - - osx-x86_64: - var: - arch_deps: - - libssl-dev - - pkg-config - - zlib1g-dev - configure_opt: --enable-local-rust --enable-vendor --enable-extended --release-channel=stable --sysconfdir=etc --target=x86_64-apple-darwin --set=target.x86_64-apple-darwin.cc=x86_64-apple-darwin-clang - - windows: - var: - arch_deps: - - libssl-dev - - pkg-config - - zlib1g-dev - configure_opt: --enable-local-rust --enable-vendor --enable-extended --release-channel=stable --sysconfdir=etc --target=[% c("arch") %]-pc-windows-gnu - -input_files: - - project: container-image - - project: cmake - name: cmake - - project: '[% c("var/compiler") %]' - name: '[% c("var/compiler") %]' - - URL: 'https://static.rust-lang.org/dist/rustc-[% c("version") %]-src.tar.gz' - name: rust - sig_ext: asc - file_gpg_id: 1 - gpg_keyring: rust.gpg - - URL: 'https://static.rust-lang.org/dist/rust-[% c("var/prev_version") %]-x86_64-unknown-linux-gnu.tar.xz' - name: prev_rust - sig_ext: asc - file_gpg_id: 1 - gpg_keyring: rust.gpg - - filename: unwind.patch - enable: '[% c("var/windows-i686") %]' - - filename: 43909.patch diff --git a/projects/fenix-rust/unwind.patch b/projects/fenix-rust/unwind.patch deleted file mode 100644 index 7b22dcb..0000000 --- a/projects/fenix-rust/unwind.patch +++ /dev/null @@ -1,162 +0,0 @@ -From b3bea7008ece7a5bdf9b5a5dcc95e82febad1854 Mon Sep 17 00:00:00 2001 -From: Bjorn Neergaard <bjorn(a)neersighted.com> -Date: Sat, 9 Feb 2019 19:39:23 +0000 -Subject: [PATCH] Fix cross-compiling i686-pc-windows-gnu from Linux - -This is still very rough and serves as a proof-of-concept for fixing -Linux -> 32-bit MinGW cross compilation workflow. Currently, clang and -GCC's MinGW targets both only support DW2 (DWARF) or SJLJ (Set Jump Long -Jump) unwinding on 32-bit Windows. - -The default for GCC (and the way it is shipped on every major distro) is -to use SJLJ on Windows, as DWARF cannot traverse non-DWARF frames. This -would work fine, except for the fact that libgcc (our C runtime on the -MinGW platform) exports symbols under a different name when configured -to use SJLJ-style unwinding, and uses a preprocessor macro internally to -alias them. - -Because of this, we have to detect this scenario and link to the correct -symbols ourselves. Linking has been tested with a full bootstrap on both -x86_64-unknown-linux-gnu and i686-pc-windows-gnu, as well as -cross-compilation of some of my own projects. - -Obviously, the detection is a bit unrefined. Right now we -unconditionally use SJLJ when compiling Linux -> MinGW. I'd like to add -feature detection using compiler build flags or autotools-style -compilation and object analysis. Input on the best way to proceed here -is welcome. - -Also, currently there is copy-pasted/duplicated code in libunwind. -Ideally, this could be reduced, but this would likely require a -rethinking of how iOS is special-cased above, to avoid further -duplication. Input on how to best structure this file is requested. - -diff --git a/src/bootstrap/compile.rs b/src/bootstrap/compile.rs -index 249a183189..df08d6eb0c 100644 ---- a/src/bootstrap/compile.rs -+++ b/src/bootstrap/compile.rs -@@ -162,7 +162,12 @@ pub fn std_cargo(builder: &Builder<'_>, - .arg("--features") - .arg("compiler-builtins-mem"); - } else { -- let features = builder.std_features(); -+ let mut features = builder.std_features(); -+ -+ // FIXME: Temporary detection of SJLJ MinGW compilers. -+ if builder.config.build.contains("linux") && target == "i686-pc-windows-gnu" { -+ features.push_str(" sjlj_eh"); -+ } - - if compiler.stage != 0 && builder.config.sanitizers { - // This variable is used by the sanitizer runtime crates, e.g. -diff --git a/src/libstd/Cargo.toml b/src/libstd/Cargo.toml -index 7d60a17042..d876d0b89a 100644 ---- a/src/libstd/Cargo.toml -+++ b/src/libstd/Cargo.toml -@@ -71,3 +71,4 @@ wasm-bindgen-threads = [] - # https://github.com/rust-lang-nursery/stdsimd/blob/master/crates/std_detect/… - std_detect_file_io = [] - std_detect_dlsym_getauxval = [] -+sjlj_eh = ["unwind/sjlj_eh"] -diff --git a/src/libunwind/Cargo.toml b/src/libunwind/Cargo.toml -index 2378b0a315..0b5979ed62 100644 ---- a/src/libunwind/Cargo.toml -+++ b/src/libunwind/Cargo.toml -@@ -16,3 +16,6 @@ doc = false - core = { path = "../libcore" } - libc = { version = "0.2.43", features = ['rustc-dep-of-std'], default-features = false } - compiler_builtins = "0.1.0" -+ -+[features] -+sjlj_eh = [] -diff --git a/src/libunwind/libunwind.rs b/src/libunwind/libunwind.rs -index 339b554ed6..ec2f93ed60 100644 ---- a/src/libunwind/libunwind.rs -+++ b/src/libunwind/libunwind.rs -@@ -1,10 +1,5 @@ - #![allow(nonstandard_style)] - --macro_rules! cfg_if { -- ( $( if #[cfg( $meta:meta )] { $($it1:item)* } else { $($it2:item)* } )* ) => -- ( $( $( #[cfg($meta)] $it1)* $( #[cfg(not($meta))] $it2)* )* ) --} -- - use libc::{c_int, c_void, uintptr_t}; - - #[repr(C)] -@@ -73,8 +68,8 @@ pub enum _Unwind_Context {} - pub type _Unwind_Exception_Cleanup_Fn = extern "C" fn(unwind_code: _Unwind_Reason_Code, - exception: *mut _Unwind_Exception); - extern "C" { -- #[unwind(allowed)] -- pub fn _Unwind_Resume(exception: *mut _Unwind_Exception) -> !; -+ #[cfg_attr(stage0, unwind)] -+ #[cfg_attr(not(stage0), unwind(allowed))] - pub fn _Unwind_DeleteException(exception: *mut _Unwind_Exception); - pub fn _Unwind_GetLanguageSpecificData(ctx: *mut _Unwind_Context) -> *mut c_void; - pub fn _Unwind_GetRegionStart(ctx: *mut _Unwind_Context) -> _Unwind_Ptr; -@@ -206,26 +201,52 @@ if #[cfg(all(any(target_os = "ios", target_os = "netbsd", not(target_arch = "arm - pc - } - } -+} // cfg_if! - --if #[cfg(not(all(target_os = "ios", target_arch = "arm")))] { -- // Not 32-bit iOS -+cfg_if! { -+if #[cfg(all(target_os = "ios", target_arch = "arm"))] { -+ // 32-bit iOS uses SjLj and does not provide _Unwind_Backtrace() - extern "C" { -- #[unwind(allowed)] -- pub fn _Unwind_RaiseException(exception: *mut _Unwind_Exception) -> _Unwind_Reason_Code; -+ #[cfg_attr(stage0, unwind)] -+ #[cfg_attr(not(stage0), unwind(allowed))] -+ pub fn _Unwind_Resume(exception: *mut _Unwind_Exception) -> !; -+ pub fn _Unwind_SjLj_RaiseException(e: *mut _Unwind_Exception) -> _Unwind_Reason_Code; -+ } -+ -+ #[inline] -+ pub unsafe fn _Unwind_RaiseException(exc: *mut _Unwind_Exception) -> _Unwind_Reason_Code { -+ _Unwind_SjLj_RaiseException(exc) -+ } -+ -+} else if #[cfg(feature = "sjlj_eh")] { -+ extern "C" { -+ #[cfg_attr(stage0, unwind)] -+ #[cfg_attr(not(stage0), unwind(allowed))] -+ pub fn _Unwind_SjLj_Resume(e: *mut _Unwind_Exception) -> !; -+ pub fn _Unwind_SjLj_RaiseException(e: *mut _Unwind_Exception) -> _Unwind_Reason_Code; - pub fn _Unwind_Backtrace(trace: _Unwind_Trace_Fn, - trace_argument: *mut c_void) - -> _Unwind_Reason_Code; - } --} else { -- // 32-bit iOS uses SjLj and does not provide _Unwind_Backtrace() -- extern "C" { -- #[unwind(allowed)] -- pub fn _Unwind_SjLj_RaiseException(e: *mut _Unwind_Exception) -> _Unwind_Reason_Code; -+ -+ #[inline] -+ pub unsafe fn _Unwind_Resume(exc: *mut _Unwind_Exception) -> ! { -+ _Unwind_SjLj_Resume(exc) - } - - #[inline] - pub unsafe fn _Unwind_RaiseException(exc: *mut _Unwind_Exception) -> _Unwind_Reason_Code { - _Unwind_SjLj_RaiseException(exc) - } -+} else { -+ extern "C" { -+ #[cfg_attr(stage0, unwind)] -+ #[cfg_attr(not(stage0), unwind(allowed))] -+ pub fn _Unwind_Resume(exception: *mut _Unwind_Exception) -> !; -+ pub fn _Unwind_RaiseException(exception: *mut _Unwind_Exception) -> _Unwind_Reason_Code; -+ pub fn _Unwind_Backtrace(trace: _Unwind_Trace_Fn, -+ trace_argument: *mut c_void) -+ -> _Unwind_Reason_Code; -+ } - } - } // cfg_if! --- -2.23.0.rc0 - diff --git a/projects/geckoview/build b/projects/geckoview/build index b07bd2b..8296856 100644 --- a/projects/geckoview/build +++ b/projects/geckoview/build @@ -20,7 +20,7 @@ tar -C /var/tmp/dist -xf [% c('input_files_by_name/node') %] tar -C /var/tmp/dist -xf [% c('input_files_by_name/clang') %] export LLVM_CONFIG="/var/tmp/dist/clang/bin/llvm-config" tar -C /var/tmp/dist -xf [% c('input_files_by_name/binutils') %] -export PATH="/var/tmp/dist/fenix-rust/bin:/var/tmp/dist/fenix-cbindgen:/var/tmp/dist/nasm/bin:/var/tmp/dist/python/bin:/var/tmp/dist/node/bin:/var/tmp/dist/fenix-clang/bin:/var/tmp/dist/binutils/bin:$PATH" +export PATH="/var/tmp/dist/rust/bin:/var/tmp/dist/cbindgen:/var/tmp/dist/nasm/bin:/var/tmp/dist/python/bin:/var/tmp/dist/node/bin:/var/tmp/dist/fenix-clang/bin:/var/tmp/dist/binutils/bin:$PATH" tar -C /var/tmp/build -xf [% project %]-[% c('version') %].tar.gz diff --git a/projects/geckoview/config b/projects/geckoview/config index 92d4a1b..c9eb23e 100644 --- a/projects/geckoview/config +++ b/projects/geckoview/config @@ -144,9 +144,9 @@ input_files: name: mozconfig - project: binutils name: binutils - - project: fenix-rust + - project: rust name: rust - - project: fenix-cbindgen + - project: cbindgen name: cbindgen - project: node name: node diff --git a/projects/fenix-as-rust/43909.patch b/projects/rust/43909_145.patch similarity index 100% rename from projects/fenix-as-rust/43909.patch rename to projects/rust/43909_145.patch diff --git a/projects/rust/build b/projects/rust/build index 5d3d5c1..8accf56 100644 --- a/projects/rust/build +++ b/projects/rust/build @@ -4,8 +4,8 @@ distdir=/var/tmp/dist/[% project %] mkdir -p $distdir tar -C /var/tmp/dist -xf [% c('input_files_by_name/cmake') %] export PATH="/var/tmp/dist/cmake/bin:$PATH" -tar -C /var/tmp/dist -xf [% c('input_files_by_name/prev_rust') %] -cd /var/tmp/dist/rust-[% c('var/prev_version') %]-x86_64-unknown-linux-gnu +tar -C /var/tmp/dist -xf [% c('input_files_by_name/previous_rust') %] +cd /var/tmp/dist/rust-[% c('var/previous_version') %]-x86_64-unknown-linux-gnu ./install.sh --prefix=$distdir-rust-old export PATH="$distdir-rust-old/bin:$PATH" @@ -39,7 +39,7 @@ cd /var/tmp/build/rustc-[% c('version') %]-src # LLVM has reproducibility issues when optimizing bitcode, which we need to # patch. See: #32053 for more details. cd src/llvm-project -patch -p1 < $rootdir/43909.patch +patch -p1 < $rootdir/[% c('input_files_by_name/43909') %] cd ../../ [% IF c("var/windows-i686") %] diff --git a/projects/rust/config b/projects/rust/config index af94f05..d4529c8 100644 --- a/projects/rust/config +++ b/projects/rust/config @@ -1,40 +1,27 @@ # vim: filetype=yaml sw=2 filename: '[% project %]-[% c("version") %]-[% c("var/build_id") %].tar.gz' -version: '[% c("input_file_var/rust_version") %]' - -# Those values can be changed from the input_files section of other -# projects. See bug 32436. -input_file_var: - rust_version: 1.43.0 - prev_version: 1.42.0 +version: '[% c("var/current_version") %]' var: - prev_version: '[% c("input_file_var/prev_version") %]' + current_version: 1.43.0 + previous_version: 1.42.0 container: use_container: 1 targets: android: var: + current_version: '[% IF pc(c("origin_project"), "var/application_services") %]1.45.2[% ELSE %]1.43.0[% END %]' + previous_version: '[% IF pc(c("origin_project"), "var/application_services") %]1.44.1[% ELSE %]1.42.0[% END %]' arch_deps: - libssl-dev - pkg-config - zlib1g-dev - android-armv7: - var: - configure_opt: --enable-local-rust --enable-vendor --enable-extended --release-channel=stable --sysconfdir=etc --target=thumbv7neon-linux-androideabi --set=target.thumbv7neon-linux-androideabi.cc=$ANDROID_NDK_HOME/arm/bin/arm-linux-androideabi-gcc - - android-x86: - var: - configure_opt: --enable-local-rust --enable-vendor --enable-extended --release-channel=stable --sysconfdir=etc --target=i686-linux-android --set=target.i686-linux-android.cc=$ANDROID_NDK_HOME/x86/bin/i686-linux-android-gcc - - android-x86_64: - var: - configure_opt: --enable-local-rust --enable-vendor --enable-extended --release-channel=stable --sysconfdir=etc --target=x86_64-linux-android --set=target.x86_64-linux-android.cc=$ANDROID_NDK_HOME/x86_64/bin/x86_64-linux-android-gcc + configure_opt: --enable-local-rust --enable-vendor --enable-extended --release-channel=stable --sysconfdir=etc --target=[% c("var/cross_prefix") %] --set=target.[% c("var/cross_prefix") %].cc=[% c("var/CC") %] --set=target.[% c("var/cross_prefix") %].ar=[% c("var/cross_prefix") %]-ar - android-aarch64: + android-armv7: var: - configure_opt: --enable-local-rust --enable-vendor --enable-extended --release-channel=stable --sysconfdir=etc --target=aarch64-linux-android --set=target.aarch64-linux-android.cc=$ANDROID_NDK_HOME/arm64/bin/aarch64-linux-android-gcc + configure_opt: --enable-local-rust --enable-vendor --enable-extended --release-channel=stable --sysconfdir=etc --target=[% IF pc(c("origin_project"), "var/application_services") %]armv7-linux-androideabi[% ELSE %]thumbv7neon-linux-androideabi[% END %] --set=target.[% IF pc(c("origin_project"), "var/application_services") %]armv7-linux-androideabi[% ELSE %]thumbv7neon-linux-androideabi[% END %].cc=[% c("var/CC") %] --set=target.[% IF pc(c("origin_project"), "var/application_services") %]armv7-linux-androideabi[% ELSE %]thumbv7neon-linux-androideabi[% END %].ar=[% c("var/cross_prefix") %]-ar linux: var: @@ -98,16 +85,21 @@ input_files: name: cmake - project: '[% c("var/compiler") %]' name: '[% c("var/compiler") %]' - - URL: 'https://static.rust-lang.org/dist/rustc-[% c("version") %]-src.tar.gz' + - URL: 'https://static.rust-lang.org/dist/rustc-[% c("var/current_version") %]-src.tar.gz' name: rust sig_ext: asc file_gpg_id: 1 gpg_keyring: rust.gpg - - URL: 'https://static.rust-lang.org/dist/rust-[% c("var/prev_version") %]-x86_64-unknown-linux-gnu.tar.xz' - name: prev_rust + - URL: 'https://static.rust-lang.org/dist/rust-[% c("var/previous_version") %]-x86_64-unknown-linux-gnu.tar.xz' + name: previous_rust sig_ext: asc file_gpg_id: 1 gpg_keyring: rust.gpg - filename: unwind.patch enable: '[% c("var/windows-i686") %]' - filename: 43909.patch + name: 43909 + enable: '[% ! pc(c("origin_project"), "var/application_services") %]' + - filename: 43909_145.patch + name: 43909 + enable: '[% pc(c("origin_project"), "var/application_services") %]' [View Less]

1 0

[tor-browser-build/master] Bug 40125: Merge clang and fenix-clang projects
by sysrqb＠torproject.org 22 Oct '20

22 Oct '20

commit bfd59379b6b0934cb4a0771e1a668e447a540971 Author: Georg Koppen <gk(a)torproject.org> Date: Sun Oct 18 07:17:14 2020 +0000 Bug 40125: Merge clang and fenix-clang projects --- projects/clang/build | 31 +++++++++++++++++++++---------- projects/clang/config | 14 ++++++++++++-- projects/fenix-clang/build | 34 ---------------------------------- projects/fenix-clang/config | 19 ------------------- projects/geckoview/build | 2 +- projects/geckoview/config | … [View More]2 +- projects/llvm-project/build | 4 ++-- 7 files changed, 37 insertions(+), 69 deletions(-) diff --git a/projects/clang/build b/projects/clang/build index 10700ce..c4c16d9 100644 --- a/projects/clang/build +++ b/projects/clang/build @@ -4,12 +4,14 @@ distdir=/var/tmp/dist/[% project %] mkdir -p /var/tmp/dist tar -C /var/tmp/dist -xf [% c('input_files_by_name/cmake') %] export PATH="/var/tmp/dist/cmake/bin:$PATH" -[% IF c("var/linux") %] - # We need a link to our GCC, otherwise the system cc gets used which points to - # /usr/bin/gcc. - [% pc('gcc', 'var/setup', { compiler_tarfile => c('input_files_by_name/gcc'), - hardened_gcc => 0 }) %] - ln -s gcc /var/tmp/dist/gcc/bin/cc +[% IF c("var/linux") || c("var/android") %] + [% IF c("var/linux") %] + # We need a link to our GCC, otherwise the system cc gets used which points + # to /usr/bin/gcc. + [% pc('gcc', 'var/setup', { compiler_tarfile => c('input_files_by_name/gcc'), + hardened_gcc => 0 }) %] + ln -s gcc /var/tmp/dist/gcc/bin/cc + [% END %] tar -C /var/tmp/dist -xf [% c('input_files_by_name/binutils') %] export PATH="/var/tmp/dist/binutils/bin:$PATH" [% END -%] @@ -20,10 +22,19 @@ cd clang-source export LLVM_HOME=$(pwd) mkdir build cd build -cmake .. -G "Unix Makefiles" -DCMAKE_INSTALL_PREFIX=$distdir \ - -DCMAKE_BUILD_TYPE:STRING=Release \ - [% IF c("var/rlbox") -%]-DLLVM_EXPERIMENTAL_TARGETS_TO_BUILD=WebAssembly \[% END -%] - $LLVM_HOME +# XXX: remove duplicated parts while fixing tor-browser-build#40048 +[% IF ! c("var/android") %] + cmake .. -G "Unix Makefiles" -DCMAKE_INSTALL_PREFIX=$distdir \ + -DCMAKE_BUILD_TYPE:STRING=Release \ + [% IF c("var/rlbox") -%]-DLLVM_EXPERIMENTAL_TARGETS_TO_BUILD=WebAssembly \[% END -%] + $LLVM_HOME +[% ELSE %] + cmake ../llvm -G "Unix Makefiles" -DCMAKE_INSTALL_PREFIX=$distdir \ + -DCMAKE_BUILD_TYPE=Release \ + -DLLVM_TARGETS_TO_BUILD="X86;ARM;AArch64" \ + [% IF c("var/rlbox") -%]-DLLVM_EXPERIMENTAL_TARGETS_TO_BUILD=WebAssembly \[% END -%] + -DLLVM_ENABLE_PROJECTS="clang;clang-tools-extra;compiler-rt;libcxx;libcxxabi;lld" +[% END -%] make -j[% c("buildconf/num_procs") %] make install cd /var/tmp/dist diff --git a/projects/clang/config b/projects/clang/config index 46d308a..6038eab 100644 --- a/projects/clang/config +++ b/projects/clang/config @@ -1,11 +1,17 @@ # vim: filetype=yaml sw=2 -version: 9.0.1 +version: '[% c("var/llvm_version") %]' filename: '[% project %]-[% c("version") %]-[% c("var/build_id") %].tar.gz' var: + llvm_version: '[% pc("clang-source", "version") %]' container: use_container: 1 +targets: + android: + var: + llvm_version: '[% pc("llvm-project", "version") %]' + input_files: - project: container-image - name: '[% c("var/compiler") %]' @@ -13,8 +19,12 @@ input_files: enable: '[% c("var/linux") %]' - name: binutils project: binutils - enable: '[% c("var/linux") %]' + enable: '[% c("var/linux") %] || [% c("var/android") %]' - project: cmake name: cmake - project: clang-source name: clang-source + enable: '[% ! c("var/android") %]' + - project: llvm-project + name: clang-source + enable: '[% c("var/android") %]' diff --git a/projects/fenix-clang/build b/projects/fenix-clang/build deleted file mode 100644 index c3c15f1..0000000 --- a/projects/fenix-clang/build +++ /dev/null @@ -1,34 +0,0 @@ -#!/bin/bash -[% c("var/set_default_env") -%] -distdir=/var/tmp/dist/[% project %] -mkdir -p /var/tmp/dist -tar -C /var/tmp/dist -xf [% c('input_files_by_name/cmake') %] -export PATH="/var/tmp/dist/cmake/bin:$PATH" -[% IF c("var/linux") %] - # We need a link to our GCC, otherwise the system cc gets used which points to - # /usr/bin/gcc. - [% pc('gcc', 'var/setup', { compiler_tarfile => c('input_files_by_name/gcc'), - hardened_gcc => 0 }) %] - ln -s gcc /var/tmp/dist/gcc/bin/cc -[% END -%] -tar -C /var/tmp/dist -xf [% c('input_files_by_name/binutils') %] -export PATH="/var/tmp/dist/binutils/bin:$PATH" - -mkdir -p /var/tmp/build -cd /var/tmp/build -tar -xf $rootdir/[% c('input_files_by_name/llvm-project') %] -cd llvm-project -mkdir build -cd build -cmake ../llvm -G "Unix Makefiles" -DCMAKE_INSTALL_PREFIX=$distdir \ - -DCMAKE_BUILD_TYPE=Release \ - -DLLVM_TARGETS_TO_BUILD="X86;ARM;AArch64" \ - [% IF c("var/rlbox") -%]-DLLVM_EXPERIMENTAL_TARGETS_TO_BUILD=WebAssembly \[% END -%] - -DLLVM_ENABLE_PROJECTS="clang;clang-tools-extra;compiler-rt;libcxx;libcxxabi;lld" -make -j[% c("buildconf/num_procs") %] -make install -cd /var/tmp/dist -[% c('tar', { - tar_src => [ project ], - tar_args => '-czf ' _ dest_dir _ '/' _ c('filename'), - }) %] diff --git a/projects/fenix-clang/config b/projects/fenix-clang/config deleted file mode 100644 index 4bc55f3..0000000 --- a/projects/fenix-clang/config +++ /dev/null @@ -1,19 +0,0 @@ -# vim: filetype=yaml sw=2 -version: '[% pc("llvm-project", "version") %]' -filename: '[% project %]-[% c("version") %]-[% c("var/build_id") %].tar.gz' - -var: - container: - use_container: 1 - -input_files: - - project: container-image - - name: '[% c("var/compiler") %]' - project: '[% c("var/compiler") %]' - enable: '[% c("var/linux") %]' - - name: binutils - project: binutils - - project: cmake - name: cmake - - project: llvm-project - name: llvm-project diff --git a/projects/geckoview/build b/projects/geckoview/build index 8296856..1c6582c 100644 --- a/projects/geckoview/build +++ b/projects/geckoview/build @@ -20,7 +20,7 @@ tar -C /var/tmp/dist -xf [% c('input_files_by_name/node') %] tar -C /var/tmp/dist -xf [% c('input_files_by_name/clang') %] export LLVM_CONFIG="/var/tmp/dist/clang/bin/llvm-config" tar -C /var/tmp/dist -xf [% c('input_files_by_name/binutils') %] -export PATH="/var/tmp/dist/rust/bin:/var/tmp/dist/cbindgen:/var/tmp/dist/nasm/bin:/var/tmp/dist/python/bin:/var/tmp/dist/node/bin:/var/tmp/dist/fenix-clang/bin:/var/tmp/dist/binutils/bin:$PATH" +export PATH="/var/tmp/dist/rust/bin:/var/tmp/dist/cbindgen:/var/tmp/dist/nasm/bin:/var/tmp/dist/python/bin:/var/tmp/dist/node/bin:/var/tmp/dist/clang/bin:/var/tmp/dist/binutils/bin:$PATH" tar -C /var/tmp/build -xf [% project %]-[% c('version') %].tar.gz diff --git a/projects/geckoview/config b/projects/geckoview/config index c9eb23e..36d1212 100644 --- a/projects/geckoview/config +++ b/projects/geckoview/config @@ -154,7 +154,7 @@ input_files: name: nasm - project: python name: python - - project: fenix-clang + - project: clang name: clang - filename: 'gradle-dependencies-[% c("var/gradle_dependencies_version") %]' name: gradle-dependencies diff --git a/projects/llvm-project/build b/projects/llvm-project/build index ac0f21b..e35adbf 100644 --- a/projects/llvm-project/build +++ b/projects/llvm-project/build @@ -1,8 +1,8 @@ #!/bin/bash [% c("var/set_default_env") -%] tar xf [% project %]-[% c("version") %].tar.gz -mv [% project %]-[% c("version") %] [% project %] +mv [% project %]-[% c("version") %] clang-source [% c('tar', { - tar_src => [ project ], + tar_src => [ 'clang-source' ], tar_args => '-czf ' _ dest_dir _ '/' _ c('filename'), }) %] [View Less]

1 0

[Git][tpo/applications/fenix][tor-browser-82.0.0b4-10.0-1] 2 commits: Bug 40095: Hide "Sign in to sync" in bookmarks
by Matthew Finkel 22 Oct '20

22 Oct '20

Matthew Finkel pushed to branch tor-browser-82.0.0b4-10.0-1 at The Tor Project / Applications / fenix Commits: d0ee6fe3 by Alex Catarineu at 2020-10-22T13:25:22+02:00 Bug 40095: Hide "Sign in to sync" in bookmarks - - - - - 8c969c0c by Matthew Finkel at 2020-10-22T22:18:33+00:00 Merge remote-tracking branch 'acatgl/40095' into tor-browser-82.0.0b4-10.0-1 - - - - - 1 changed file: - app/src/main/java/org/mozilla/fenix/library/bookmarks/BookmarkFragment.kt … [View More]

1 0

[Git][tpo/applications/android-components] Pushed new branch android-components-63.0.0-10.0-1
by Georg Koppen 22 Oct '20

22 Oct '20

Georg Koppen pushed new branch android-components-63.0.0-10.0-1 at The Tor Project / Applications / android-components -- View it on GitLab: https://gitlab.torproject.org/tpo/applications/android-components/-/tree/an… You're receiving this email because of your account on gitlab.torproject.org.

1 0

[Git][tpo/applications/android-components][android-components-60.0.3-10.0-1] 2 commits: Bug 40022: Migrate Tor security level from Fennec
by Matthew Finkel 21 Oct '20

21 Oct '20

Matthew Finkel pushed to branch android-components-60.0.3-10.0-1 at The Tor Project / Applications / android-components Commits: 9eb30f13 by Alex Catarineu at 2020-10-21T22:15:39+02:00 Bug 40022: Migrate Tor security level from Fennec - - - - - 6b75c81e by Matthew Finkel at 2020-10-21T22:09:37+00:00 Merge remote-tracking branch 'acatgl/40022' into android-components-60.0.3-10.0-1 - - - - - 2 changed files: - components/support/migration/src/main/java/mozilla/components/… [View More]support/migration/FennecMigrator.kt - components/support/migration/src/main/java/mozilla/components/support/migration/GeckoMigration.kt Changes: ===================================== components/support/migration/src/main/java/mozilla/components/support/migration/FennecMigrator.kt ===================================== @@ -975,7 +975,7 @@ class FennecMigrator private constructor( return try { logger.debug("Migrating gecko files...") - val result = GeckoMigration.migrate(profile.path, migrationVersion) + val result = GeckoMigration.migrate(profile.path, migrationVersion, context) logger.debug("Migrated gecko files.") if (result is Result.Failure<GeckoMigrationResult>) { ===================================== components/support/migration/src/main/java/mozilla/components/support/migration/GeckoMigration.kt ===================================== @@ -4,6 +4,7 @@ package mozilla.components.support.migration +import android.content.Context import android.util.AtomicFile import androidx.annotation.VisibleForTesting import mozilla.components.support.ktx.util.writeString @@ -95,7 +96,8 @@ internal object GeckoMigration { */ fun migrate( profilePath: String, - migrationVersion: Int + migrationVersion: Int, + context: Context ): Result<GeckoMigrationResult> { // GeckoView will happily pick up the profile from Fennec and reuse all data in it. So this // migration is mostly focused on removing all prefs that we do not want to reuse. @@ -118,6 +120,23 @@ internal object GeckoMigration { }.toList() } + val secLevel = prefsjs.useLines { + // The prefs are auto-generated, so we can expect the offset to be always the same + it.find { line -> line.startsWith("$USER_PREF(\"extensions.torbutton.security_slider\"") } + ?.substring(50, 51)?.toIntOrNull() + } + if (secLevel != null && secLevel > 0 && secLevel < 4) { + val prefVal = when (secLevel) { + 1 -> "pref_key_tor_security_level_safest_option" + else -> "pref_key_tor_security_level_safer_option" + } + val fenixAppPrefs = context.getSharedPreferences(FennecSettingsMigration.FENIX_SHARED_PREFS_NAME, Context.MODE_PRIVATE) + fenixAppPrefs.edit() + .putBoolean("pref_key_tor_security_level_standard_option", false) + .putBoolean(prefVal, true) + .apply() + } + return if (transformed.isEmpty()) { removePrefsFile(prefsjs) } else { View it on GitLab: https://gitlab.torproject.org/tpo/applications/android-components/-/compare… -- View it on GitLab: https://gitlab.torproject.org/tpo/applications/android-components/-/compare… You're receiving this email because of your account on gitlab.torproject.org. [View Less]

1 0

[Git][tpo/applications/fenix][tor-browser-82.0.0b4-10.0-1] 2 commits: Bug 40094: Do not use MasterPasswordTipProvider in HomeFragment
by Matthew Finkel 21 Oct '20

21 Oct '20

Matthew Finkel pushed to branch tor-browser-82.0.0b4-10.0-1 at The Tor Project / Applications / fenix Commits: 7edfd5b5 by Alex Catarineu at 2020-10-21T12:51:36+02:00 Bug 40094: Do not use MasterPasswordTipProvider in HomeFragment - - - - - 03b642ac by Matthew Finkel at 2020-10-21T21:46:59+00:00 Merge remote-tracking branch 'acatgl/40094' into tor-browser-82.0.0b4-10.0-1 - - - - - 1 changed file: - app/src/main/java/org/mozilla/fenix/home/HomeFragment.kt Changes: ======… [View More]=============================== app/src/main/java/org/mozilla/fenix/home/HomeFragment.kt ===================================== @@ -87,9 +87,7 @@ import org.mozilla.fenix.components.PrivateShortcutCreateManager import org.mozilla.fenix.components.StoreProvider import org.mozilla.fenix.components.TabCollectionStorage import org.mozilla.fenix.components.metrics.Event -import org.mozilla.fenix.components.tips.FenixTipManager import org.mozilla.fenix.components.tips.Tip -import org.mozilla.fenix.components.tips.providers.MasterPasswordTipProvider import org.mozilla.fenix.components.toolbar.TabCounterMenu import org.mozilla.fenix.components.toolbar.ToolbarPosition import org.mozilla.fenix.ext.components @@ -218,17 +216,6 @@ class HomeFragment : Fragment() { expandedCollections = emptySet(), mode = currentMode.getCurrentMode(), topSites = components.core.topSitesStorage.cachedTopSites, - tip = StrictMode.allowThreadDiskReads().resetPoliciesAfter { - FenixTipManager( - listOf( - MasterPasswordTipProvider( - requireContext(), - ::navToSavedLogins, - ::dismissTip - ) - ) - ).getTip() - }, showCollectionPlaceholder = components.settings.showCollectionsPlaceholderOnHome ) ) @@ -685,17 +672,6 @@ class HomeFragment : Fragment() { collections = components.core.tabCollectionStorage.cachedTabCollections, mode = currentMode.getCurrentMode(), topSites = components.core.topSitesStorage.cachedTopSites, - tip = StrictMode.allowThreadDiskReads().resetPoliciesAfter { - FenixTipManager( - listOf( - MasterPasswordTipProvider( - requireContext(), - ::navToSavedLogins, - ::dismissTip - ) - ) - ).getTip() - }, showCollectionPlaceholder = components.settings.showCollectionsPlaceholderOnHome ) ) View it on GitLab: https://gitlab.torproject.org/tpo/applications/fenix/-/compare/67007f534ee6… -- View it on GitLab: https://gitlab.torproject.org/tpo/applications/fenix/-/compare/67007f534ee6… You're receiving this email because of your account on gitlab.torproject.org. [View Less]

1 0

[Git][tpo/applications/android-components][android-components-60.0.3-10.0-1] 2 commits: Bug 40021: Force telemetry=false in Fennec settings migration
by Matthew Finkel 21 Oct '20

21 Oct '20

Matthew Finkel pushed to branch android-components-60.0.3-10.0-1 at The Tor Project / Applications / android-components Commits: 4693b3c0 by Alex Catarineu at 2020-10-21T16:17:57+02:00 Bug 40021: Force telemetry=false in Fennec settings migration - - - - - cda23d34 by Matthew Finkel at 2020-10-21T20:13:36+00:00 Merge remote-tracking branch 'acatgl/40021' into android-components-60.0.3-10.0-1 - - - - - 2 changed files: - components/support/migration/src/main/java/mozilla/… [View More]components/support/migration/FennecMigrator.kt - components/support/migration/src/main/java/mozilla/components/support/migration/FennecSettingsMigrator.kt Changes: ===================================== components/support/migration/src/main/java/mozilla/components/support/migration/FennecMigrator.kt ===================================== @@ -92,7 +92,7 @@ sealed class Migration(val currentVersion: Int) { /** * Migrates all Fennec settings backed by SharedPreferences. */ - object Settings : Migration(currentVersion = 2) + object Settings : Migration(currentVersion = 3) /** * Migrates / Disables all currently unsupported Add-ons. ===================================== components/support/migration/src/main/java/mozilla/components/support/migration/FennecSettingsMigrator.kt ===================================== @@ -44,16 +44,11 @@ internal object FennecSettingsMigration { return migrateTelemetryOptInStatus(fennecAppPrefs, fenixAppPrefs) } + @Suppress("UNUSED_PARAMETER") private fun migrateTelemetryOptInStatus( fennecPrefs: SharedPreferences, fenixPrefs: SharedPreferences ): Result<SettingsMigrationResult> { - // Sanity check: make sure we actually have an FHR value set. - if (!fennecPrefs.contains(FENNEC_PREFS_FHR_KEY)) { - logger.warn("Missing FHR pref value") - return Result.Failure(SettingsMigrationException(SettingsMigrationResult.Failure.MissingFHRPrefValue)) - } - // Fennec has two telemetry settings: // - Firefox Health Report (FHR) - defaults to 'on', // - Telemetry - defaults to 'off'. @@ -66,7 +61,7 @@ internal object FennecSettingsMigration { // If FHR is disabled by the user, we'll disable telemetry in Fenix. Otherwise, it will be enabled. // Read Fennec prefs. - val fennecFHRState = fennecPrefs.getBoolean(FENNEC_PREFS_FHR_KEY, false) + val fennecFHRState = false logger.info("Fennec FHR state is: $fennecFHRState") // Update Fenix prefs. View it on GitLab: https://gitlab.torproject.org/tpo/applications/android-components/-/compare… -- View it on GitLab: https://gitlab.torproject.org/tpo/applications/android-components/-/compare… You're receiving this email because of your account on gitlab.torproject.org. [View Less]

1 0

[tor-browser-build/maint-9.5] Release prep for 9.5.5
by sysrqb＠torproject.org 21 Oct '20

21 Oct '20

commit c684b01184c8bb0623411ce3e442daa45769273b Author: Matthew Finkel <sysrqb(a)torproject.org> Date: Wed Oct 21 13:52:42 2020 +0000 Release prep for 9.5.5 Changelog update and versions bump --- projects/firefox/config | 2 +- projects/tor-browser/Bundle-Data/Docs/ChangeLog.txt | 4 ++++ rbm.conf | 2 +- 3 files changed, 6 insertions(+), 2 deletions(-) diff --git a/projects/firefox/config b/projects/… [View More]

1 0