August 2018 - tor-commits - lists.torproject.org

[tor/release-0.3.3] Merge remote-tracking branch 'teor/travis-osx-033' into maint-0.3.3
by nickm＠torproject.org 22 Aug '18

22 Aug '18

commit f0633bc49164a10ca163aa94139222682af376f7 Merge: 245025a3d 6463cdea5 Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Aug 21 20:03:56 2018 -0400 Merge remote-tracking branch 'teor/travis-osx-033' into maint-0.3.3 .travis.yml | 262 +++++++++++++++++++++++++------------------ changes/bug26497 | 3 + changes/bug26497-backport | 3 + changes/bug26497-cd | 4 + changes/bug27088 | 5 + changes/bug27090 | 3 + changes/bug27093 | 3 + changes/ticket24629 | 3 + changes/ticket24629-backport | 3 + changes/ticket26560 | 3 + changes/ticket26952-cargo | 3 + changes/ticket26952-ccache | 3 + changes/ticket27087 | 3 + src/rust/tor_rust/include.am | 2 - src/test/test_rust.sh | 6 +- 15 files changed, 191 insertions(+), 118 deletions(-)

1 0

[tor/release-0.3.4] Add more optional packages to Travis
by nickm＠torproject.org 22 Aug '18

22 Aug '18

commit 340da669be373174550e038d1d20ff2bff49eb6b Author: Taylor Yu <catalyst(a)torproject.org> Date: Thu Jun 28 14:43:31 2018 -0500 Add more optional packages to Travis Apparently we weren't building with either libcap or libseccomp on Travis. Install libcap-dev and libseccomp-dev in .travis.yml. Closes ticket 26560. --- .travis.yml | 3 ++- changes/ticket26560 | 3 +++ 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index 6a3e1bfc0..e3735f7d5 100644 --- a/.travis.yml +++ b/.travis.yml @@ -42,11 +42,12 @@ addons: packages: ## Required dependencies - libevent-dev - - libseccomp2 - zlib1g-dev ## Optional dependencies + - libcap-dev - liblzma-dev - libscrypt-dev + - libseccomp-dev ## zstd doesn't exist in Ubuntu Trusty #- libzstd diff --git a/changes/ticket26560 b/changes/ticket26560 new file mode 100644 index 000000000..5b4fb1bfe --- /dev/null +++ b/changes/ticket26560 @@ -0,0 +1,3 @@ + o Minor features (continuous integration): + - Install libcap-dev and libseccomp2-dev so these optional + dependencies get tested on Travis CI. Closes ticket 26560.

1 0

[tor/release-0.3.4] Travis: enable macOS builds
by nickm＠torproject.org 22 Aug '18

22 Aug '18

commit 515d190b2ca9249871d04067c3bdbd312669997e Author: teor <teor(a)torproject.org> Date: Thu Aug 9 18:03:18 2018 +1000 Travis: enable macOS builds Also: * explain why we don't install zlib Part of 24629. --- .travis.yml | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/.travis.yml b/.travis.yml index dfdf20f31..d75c74ebb 100644 --- a/.travis.yml +++ b/.travis.yml @@ -6,6 +6,7 @@ compiler: os: - linux + - osx ## The build matrix in the following stanza expands into builds for each ## OS and compiler. @@ -81,7 +82,26 @@ addons: - libscrypt-dev - libseccomp-dev +## (OSX only) Use the default OSX image +## See https://docs.travis-ci.com/user/reference/osx#os-x-version +## Default is Xcode 9.4 on macOS 10.13 as of August 2018 +#osx_image: xcode9.4 + +before_install: + ## If we're on OSX, homebrew usually needs to updated first + - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then brew update; fi + ## We might be upgrading some useless packages, but that's better than missing an upgrade + - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then brew upgrade; fi + install: + ## If we're on OSX use brew to install required dependencies (for Linux, see the "apt:" section above) + - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then brew install libevent; fi + - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then brew install openssl; fi + - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then brew install pkg-config; fi + ## macOS comes with zlib by default, so the homebrew install is keg-only + # - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then brew install zlib; fi + ## If we're on OSX also install the optional dependencies + - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then brew install libscrypt; fi ## Install conditional features ## Install coveralls - if [[ "$COVERAGE_OPTIONS" != "" ]]; then pip install --user cpp-coveralls; fi

1 0

[tor/release-0.3.4] Travis: run an asciidoc build
by nickm＠torproject.org 22 Aug '18

22 Aug '18

commit 23b242104ba5cefb5fd16effda33a611e521050d Author: teor <teor(a)torproject.org> Date: Thu Aug 9 18:29:29 2018 +1000 Travis: run an asciidoc build Implements 27087. --- .travis.yml | 15 +++++++++++++-- changes/ticket27087 | 3 +++ 2 files changed, 16 insertions(+), 2 deletions(-) diff --git a/.travis.yml b/.travis.yml index 424f0d070..d822780cd 100644 --- a/.travis.yml +++ b/.travis.yml @@ -20,6 +20,8 @@ env: ## We turn on hardening by default ## Also known as --enable-fragile-hardening in 0.3.0.3-alpha and later - HARDENING_OPTIONS="--enable-expensive-hardening" + ## We turn off asciidoc by default, because it's slow + - ASCIIDOC_OPTIONS="--disable-asciidoc" matrix: ## We want to use each build option at least once ## @@ -34,8 +36,9 @@ matrix: - env: COVERAGE_OPTIONS="--enable-coverage" HARDENING_OPTIONS="" ## We only want to check these build option combinations once ## (they shouldn't vary by compiler or OS) - - env: DISTCHECK="yes" - env: HARDENING_OPTIONS="" + ## We check asciidoc with distcheck, to make sure we remove doc products + - env: ASCIIDOC_OPTIONS="" DISTCHECK="yes" ## Uncomment to allow the build to report success (with non-required ## sub-builds continuing to run) if all required sub-builds have @@ -84,6 +87,12 @@ addons: - libcap-dev - libscrypt-dev - libseccomp-dev + ## Conditional dependencies + ## Always installed, so we don't need sudo + - asciidoc + - docbook-xsl + - docbook-xml + - xmlto ## (OSX only) Use the default OSX image ## See https://docs.travis-ci.com/user/reference/osx#os-x-version @@ -113,10 +122,12 @@ install: ## Install conditional features ## Install coveralls - if [[ "$COVERAGE_OPTIONS" != "" ]]; then pip install --user cpp-coveralls; fi + ## If we're on OSX, and using asciidoc, install asciidoc + - if [[ "$ASCIIDOC_OPTIONS" == "" ]] && [[ "$TRAVIS_OS_NAME" == "osx" ]]; then brew install asciidoc; fi script: - ./autogen.sh - - CONFIGURE_FLAGS="$COVERAGE_OPTIONS $HARDENING_OPTIONS $OPENSSL_OPTIONS --disable-asciidoc --enable-fatal-warnings --disable-silent-rules" + - CONFIGURE_FLAGS="$ASCIIDOC_OPTIONS $COVERAGE_OPTIONS $HARDENING_OPTIONS $OPENSSL_OPTIONS --enable-fatal-warnings --disable-silent-rules" - echo $CONFIGURE_FLAGS - ./configure $CONFIGURE_FLAGS ## We run `make check` because that's what https://jenkins.torproject.org does. diff --git a/changes/ticket27087 b/changes/ticket27087 new file mode 100644 index 000000000..b8af70aaa --- /dev/null +++ b/changes/ticket27087 @@ -0,0 +1,3 @@ + o Minor features (continuous integration): + - Run asciidoc during Travis CI. + Implements ticket 27087.

1 0

[tor/release-0.3.4] Travis: Use ccache
by nickm＠torproject.org 22 Aug '18

22 Aug '18

commit 286a6bc3b8fd2c369583d0d48c09769fed78e39f Author: teor <teor(a)torproject.org> Date: Thu Aug 9 15:40:27 2018 +1000 Travis: Use ccache Part of ticket 26952. --- .travis.yml | 6 ++++++ changes/ticket26952-ccache | 3 +++ 2 files changed, 9 insertions(+) diff --git a/.travis.yml b/.travis.yml index c2db6b78b..424f0d070 100644 --- a/.travis.yml +++ b/.travis.yml @@ -1,5 +1,8 @@ language: c +cache: + ccache: true + compiler: - gcc - clang @@ -94,6 +97,9 @@ before_install: - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then brew upgrade; fi install: + ## If we're on OSX use brew to install ccache (ccache is automatically installed on Linux) + - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then brew install ccache; fi + - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then export PATH="/usr/local/opt/ccache/libexec:$PATH"; fi ## If we're on OSX use brew to install required dependencies (for Linux, see the "apt:" section above) - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then brew install libevent; fi - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then brew install openssl; fi diff --git a/changes/ticket26952-ccache b/changes/ticket26952-ccache new file mode 100644 index 000000000..edc115e9d --- /dev/null +++ b/changes/ticket26952-ccache @@ -0,0 +1,3 @@ + o Minor features (continuous integration): + - Use ccache in our Travis CI configuration. + Closes ticket 26952.

1 0

[tor/release-0.3.4] Travis: make macOS builds work for Tor 0.2.9
by nickm＠torproject.org 22 Aug '18

22 Aug '18

commit 74b3a340df0df1eb803b4b17039725fc42d5a86a Author: teor <teor(a)torproject.org> Date: Thu Aug 9 17:44:11 2018 +1000 Travis: make macOS builds work for Tor 0.2.9 Tor 0.2.9 needs extra help to find OpenSSL on macOS. Part of 24629. --- .travis.yml | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index 6169c808a..c2db6b78b 100644 --- a/.travis.yml +++ b/.travis.yml @@ -102,13 +102,15 @@ install: # - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then brew install zlib; fi ## If we're on OSX also install the optional dependencies - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then brew install libscrypt; fi + ## If we're on OSX, OpenSSL is keg-only, so tor 0.2.9 and later need to be configured --with-openssl-dir= to build + - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then OPENSSL_OPTIONS=--with-openssl-dir=`brew --prefix openssl`; fi ## Install conditional features ## Install coveralls - if [[ "$COVERAGE_OPTIONS" != "" ]]; then pip install --user cpp-coveralls; fi script: - ./autogen.sh - - CONFIGURE_FLAGS="$COVERAGE_OPTIONS $HARDENING_OPTIONS --disable-asciidoc --enable-fatal-warnings --disable-silent-rules" + - CONFIGURE_FLAGS="$COVERAGE_OPTIONS $HARDENING_OPTIONS $OPENSSL_OPTIONS --disable-asciidoc --enable-fatal-warnings --disable-silent-rules" - echo $CONFIGURE_FLAGS - ./configure $CONFIGURE_FLAGS ## We run `make check` because that's what https://jenkins.torproject.org does.

1 0

[tor/release-0.3.4] Travis: create configure flags once, then echo the flags
by nickm＠torproject.org 22 Aug '18

22 Aug '18

commit 7cf7b52fcab81c55a4c493bad68eddc0d27a0d63 Author: teor <teor(a)torproject.org> Date: Thu Aug 9 17:06:13 2018 +1000 Travis: create configure flags once, then echo the flags Creating the configure flags once avoids inconsistent flags between configure and distcheck configure. Echoing the flags helps developers work out what configure is doing. (Backported to 0.2.9 and later as a precaution.) Fixes 27088 on 0.3.4.1-alpha, adds logging in previous releases. --- .travis.yml | 6 ++++-- changes/bug27088 | 5 +++++ 2 files changed, 9 insertions(+), 2 deletions(-) diff --git a/.travis.yml b/.travis.yml index d75c74ebb..6169c808a 100644 --- a/.travis.yml +++ b/.travis.yml @@ -108,10 +108,12 @@ install: script: - ./autogen.sh - - ./configure $COVERAGE_OPTIONS $HARDENING_OPTIONS --disable-asciidoc --enable-fatal-warnings --disable-silent-rules + - CONFIGURE_FLAGS="$COVERAGE_OPTIONS $HARDENING_OPTIONS --disable-asciidoc --enable-fatal-warnings --disable-silent-rules" + - echo $CONFIGURE_FLAGS + - ./configure $CONFIGURE_FLAGS ## We run `make check` because that's what https://jenkins.torproject.org does. - if [[ "$DISTCHECK" == "" ]]; then make check; fi - - if [[ "$DISTCHECK" != "" ]]; then make distcheck DISTCHECK_CONFIGURE_FLAGS="$HARDENING_OPTIONS $COVERAGE_OPTIONS --disable-asciidoc --enable-fatal-warnings --disable-silent-rules"; fi + - if [[ "$DISTCHECK" != "" ]]; then make distcheck DISTCHECK_CONFIGURE_FLAGS="$CONFIGURE_FLAGS"; fi after_failure: ## configure will leave a log file with more details of config failures. diff --git a/changes/bug27088 b/changes/bug27088 new file mode 100644 index 000000000..d4d3b292c --- /dev/null +++ b/changes/bug27088 @@ -0,0 +1,5 @@ + o Minor bugfixes (continuous integration): + - Pass the module flags to distcheck configure, and + log the flags before running configure. (Backported + to 0.2.9 and later as a precaution.) + Fixes bug 27088; bugfix on 0.3.4.1-alpha.

1 0

[tor/release-0.3.4] Travis: add rust cargo online mode
by nickm＠torproject.org 22 Aug '18

22 Aug '18

commit 14ceee0fd1cc82da9fc6cd51d69173546bc4d7fd Author: teor <teor(a)torproject.org> Date: Thu Aug 9 22:32:20 2018 +1000 Travis: add rust cargo online mode Forward-ports parts of the 0.3.2 travis config on top of the 0.2.9 merge. Also: * build rust on clang and gcc, Linux and macOS * build combinations of non-default options on gcc Linux * exclude broken builds * log the rustup version Part of 24629. --- .travis.yml | 22 ++++++++++++++++++++-- 1 file changed, 20 insertions(+), 2 deletions(-) diff --git a/.travis.yml b/.travis.yml index e61c3ba8b..43392b3f4 100644 --- a/.travis.yml +++ b/.travis.yml @@ -28,6 +28,11 @@ env: ## We don't list default variable values, because we set the defaults ## in global (or the default is unset) - + ## We turn off hardening for Rust builds, because they are incompatible, + ## and it's going to take a while for them to be fixed. See: + ## https:/trac.torproject.org/projects/tor/ticket/25386 + ## https:/trac.torproject.org/projects/tor/ticket/26398 + - RUST_OPTIONS="--enable-rust --enable-cargo-online-mode" HARDENING_OPTIONS="" matrix: ## include creates builds with gcc, linux, sudo: false @@ -36,7 +41,8 @@ matrix: - env: COVERAGE_OPTIONS="--enable-coverage" HARDENING_OPTIONS="" ## We only want to check these build option combinations once ## (they shouldn't vary by compiler or OS) - - env: HARDENING_OPTIONS="" + ## We run rust and coverage with hardening off, which seems like enough + # - env: HARDENING_OPTIONS="" ## We check asciidoc with distcheck, to make sure we remove doc products - env: ASCIIDOC_OPTIONS="" DISTCHECK="yes" @@ -129,14 +135,26 @@ install: - if [[ "$COVERAGE_OPTIONS" != "" ]]; then pip install --user cpp-coveralls; fi ## If we're on OSX, and using asciidoc, install asciidoc - if [[ "$ASCIIDOC_OPTIONS" == "" ]] && [[ "$TRAVIS_OS_NAME" == "osx" ]]; then brew install asciidoc; fi + ## If we're using Rust, download rustup + - if [[ "$RUST_OPTIONS" != "" ]]; then curl -Ssf -o rustup.sh https://sh.rustup.rs; fi + ## Install the stable channels of rustc and cargo and setup our toolchain environment + - if [[ "$RUST_OPTIONS" != "" ]]; then sh rustup.sh -y --default-toolchain stable; fi + - if [[ "$RUST_OPTIONS" != "" ]]; then source $HOME/.cargo/env; fi ## ## Finally, list installed package versions - if [[ "$TRAVIS_OS_NAME" == "linux" ]]; then dpkg-query --show; fi - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then brew list --versions; fi + ## Get some info about rustup, rustc and cargo + - if [[ "$RUST_OPTIONS" != "" ]]; then which rustup; fi + - if [[ "$RUST_OPTIONS" != "" ]]; then which rustc; fi + - if [[ "$RUST_OPTIONS" != "" ]]; then which cargo; fi + - if [[ "$RUST_OPTIONS" != "" ]]; then rustup --version; fi + - if [[ "$RUST_OPTIONS" != "" ]]; then rustc --version; fi + - if [[ "$RUST_OPTIONS" != "" ]]; then cargo --version; fi script: - ./autogen.sh - - CONFIGURE_FLAGS="$ASCIIDOC_OPTIONS $COVERAGE_OPTIONS $HARDENING_OPTIONS $OPENSSL_OPTIONS --enable-fatal-warnings --disable-silent-rules" + - CONFIGURE_FLAGS="$ASCIIDOC_OPTIONS $COVERAGE_OPTIONS $HARDENING_OPTIONS $OPENSSL_OPTIONS $RUST_OPTIONS --enable-fatal-warnings --disable-silent-rules" - echo $CONFIGURE_FLAGS - ./configure $CONFIGURE_FLAGS ## We run `make check` because that's what https://jenkins.torproject.org does.

1 0

[tor/release-0.3.4] Rust: backport src/test/test_rust.sh from master
by nickm＠torproject.org 22 Aug '18

22 Aug '18

commit 2a35b085eeee977376b5c5e2dd084f9f505d8f8a Author: teor <teor(a)torproject.org> Date: Fri Aug 10 11:09:18 2018 +1000 Rust: backport src/test/test_rust.sh from master Preparation for 26497. --- src/test/test_rust.sh | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-) diff --git a/src/test/test_rust.sh b/src/test/test_rust.sh index d559f94ce..4ecca7834 100755 --- a/src/test/test_rust.sh +++ b/src/test/test_rust.sh @@ -1,13 +1,19 @@ #!/bin/sh -# Test all the Rust crates we're using +# Test all Rust crates -crates=tor_util +set -e -exitcode=0 +export LSAN_OPTIONS=suppressions=${abs_top_srcdir}/src/test/rust_supp.txt -for crate in $crates; do - cd "${abs_top_srcdir:-.}/src/rust/${crate}" - CARGO_TARGET_DIR="${abs_top_builddir}/src/rust/target" CARGO_HOME="${abs_top_builddir}/src/rust" "${CARGO:-cargo}" test ${CARGO_ONLINE-"--frozen"} || exitcode=1 +for cargo_toml_dir in "${abs_top_srcdir:-../../..}"/src/rust/*; do + if [ -e "${cargo_toml_dir}/Cargo.toml" ]; then + cd "${cargo_toml_dir}" && \ + CARGO_TARGET_DIR="${abs_top_builddir:-../../..}/src/rust/target" \ + CARGO_HOME="${abs_top_builddir:-../../..}/src/rust/.cargo" \ + "${CARGO:-cargo}" test ${CARGO_ONLINE-"--frozen"} \ + ${EXTRA_CARGO_OPTIONS} \ + --manifest-path "${cargo_toml_dir}/Cargo.toml" || exitcode=1 + fi done exit $exitcode

1 0

[tor/release-0.3.4] Stop setting $CARGO_HOME
by nickm＠torproject.org 22 Aug '18

22 Aug '18

commit ce19477ffcdf64255f050308182f3d682e6f6de1 Author: teor <teor(a)torproject.org> Date: Fri Aug 10 11:00:46 2018 +1000 Stop setting $CARGO_HOME cargo will use the user's $CARGO_HOME, or $HOME/.cargo by default. Fixes bug 26497; bugfix on 0.3.1.5-alpha. --- changes/bug26497 | 3 +++ src/rust/tor_util/include.am | 1 - src/test/test_rust.sh | 1 - 3 files changed, 3 insertions(+), 2 deletions(-) diff --git a/changes/bug26497 b/changes/bug26497 new file mode 100644 index 000000000..d0c05ff3e --- /dev/null +++ b/changes/bug26497 @@ -0,0 +1,3 @@ + o Minor bugfixes (rust): + - Stop setting $CARGO_HOME. cargo will use the user's $CARGO_HOME, or + $HOME/.cargo by default. Fixes bug 26497; bugfix on 0.3.1.5-alpha. diff --git a/src/rust/tor_util/include.am b/src/rust/tor_util/include.am index ec3898577..c11060289 100644 --- a/src/rust/tor_util/include.am +++ b/src/rust/tor_util/include.am @@ -7,7 +7,6 @@ EXTRA_DIST +=\ src/rust/target/release/@TOR_RUST_UTIL_STATIC_NAME@: FORCE ( cd "$(abs_top_srcdir)/src/rust/tor_util" ; \ CARGO_TARGET_DIR="$(abs_top_builddir)/src/rust/target" \ - CARGO_HOME="$(abs_top_builddir)/src/rust" \ $(CARGO) build --release --quiet $(CARGO_ONLINE) ) FORCE: diff --git a/src/test/test_rust.sh b/src/test/test_rust.sh index 4ecca7834..9ad7a698a 100755 --- a/src/test/test_rust.sh +++ b/src/test/test_rust.sh @@ -9,7 +9,6 @@ for cargo_toml_dir in "${abs_top_srcdir:-../../..}"/src/rust/*; do if [ -e "${cargo_toml_dir}/Cargo.toml" ]; then cd "${cargo_toml_dir}" && \ CARGO_TARGET_DIR="${abs_top_builddir:-../../..}/src/rust/target" \ - CARGO_HOME="${abs_top_builddir:-../../..}/src/rust/.cargo" \ "${CARGO:-cargo}" test ${CARGO_ONLINE-"--frozen"} \ ${EXTRA_CARGO_OPTIONS} \ --manifest-path "${cargo_toml_dir}/Cargo.toml" || exitcode=1

1 0