August 2018 - tor-commits - lists.torproject.org

[tor/maint-0.3.3] Merge remote-tracking branch 'teor/travis-osx-033' into maint-0.3.3
by nickm＠torproject.org 22 Aug '18

22 Aug '18

commit f0633bc49164a10ca163aa94139222682af376f7 Merge: 245025a3d 6463cdea5 Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Aug 21 20:03:56 2018 -0400 Merge remote-tracking branch 'teor/travis-osx-033' into maint-0.3.3 .travis.yml | 262 +++++++++++++++++++++++++------------------ changes/bug26497 | 3 + changes/bug26497-backport | 3 + changes/bug26497-cd | 4 + changes/bug27088 | 5 + changes/bug27090 | 3 + changes/bug27093 | 3 + changes/ticket24629 | 3 + changes/ticket24629-backport | 3 + changes/ticket26560 | 3 + changes/ticket26952-cargo | 3 + changes/ticket26952-ccache | 3 + changes/ticket27087 | 3 + src/rust/tor_rust/include.am | 2 - src/test/test_rust.sh | 6 +- 15 files changed, 191 insertions(+), 118 deletions(-)

1 0

[tor/maint-0.3.3] Merge remote-tracking branch 'teor/travis-osx-029' into maint-0.2.9
by nickm＠torproject.org 22 Aug '18

22 Aug '18

commit 05d62a3b73ed092ec1643ead56d775bb5d06f648 Merge: 85a879234 d514e9866 Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Aug 21 20:03:19 2018 -0400 Merge remote-tracking branch 'teor/travis-osx-029' into maint-0.2.9 .travis.yml | 213 ++++++++++++++++++++++++++------------------- changes/bug27088 | 5 ++ changes/ticket24629 | 3 + changes/ticket26560 | 3 + changes/ticket26952-ccache | 3 + changes/ticket27087 | 3 + 6 files changed, 140 insertions(+), 90 deletions(-)

1 0

[tor/maint-0.3.3] Merge branch 'travis-osx-029' into travis-osx-032
by nickm＠torproject.org 22 Aug '18

22 Aug '18

commit e2eec40c4469c10dfb3006dfb07240d7e1144aa7 Merge: 69adc68ac d514e9866 Author: teor <teor(a)torproject.org> Date: Fri Aug 10 19:05:49 2018 +1000 Merge branch 'travis-osx-029' into travis-osx-032 Merge asciidoc from 029 with rust from 032. .travis.yml | 1 + 1 file changed, 1 insertion(+) diff --cc .travis.yml index f92a3d8d4,8b8621007..1b43c721a --- a/.travis.yml +++ b/.travis.yml @@@ -147,13 -125,7 +147,14 @@@ install ## If we're on OSX, and using asciidoc, install asciidoc - if [[ "$ASCIIDOC_OPTIONS" == "" ]] && [[ "$TRAVIS_OS_NAME" == "osx" ]]; then brew install asciidoc; fi - if [[ "$ASCIIDOC_OPTIONS" == "" ]] && [[ "$TRAVIS_OS_NAME" == "osx" ]]; then brew install xmlto; fi + - if [[ "$ASCIIDOC_OPTIONS" == "" ]] && [[ "$TRAVIS_OS_NAME" == "osx" ]]; then export XML_CATALOG_FILES="/usr/local/etc/xml/catalog"; fi + ## If we're using Rust, download rustup + - if [[ "$RUST_OPTIONS" != "" ]]; then curl -Ssf -o rustup.sh https://sh.rustup.rs; fi + ## Install the stable channels of rustc and cargo and setup our toolchain environment + - if [[ "$RUST_OPTIONS" != "" ]]; then sh rustup.sh -y --default-toolchain stable; fi + - if [[ "$RUST_OPTIONS" != "" ]]; then source $HOME/.cargo/env; fi + ## If we're testing rust builds in offline-mode, then set up our vendored dependencies + - if [[ "$RUST_DEPENDENCIES" == "true" ]]; then export RUST_DEPENDENCIES=$PWD/src/ext/rust/crates; fi ## ## Finally, list installed package versions - if [[ "$TRAVIS_OS_NAME" == "linux" ]]; then dpkg-query --show; fi

1 0

[tor/maint-0.3.4] Travis: create configure flags once, then echo the flags
by nickm＠torproject.org 22 Aug '18

22 Aug '18

commit 7cf7b52fcab81c55a4c493bad68eddc0d27a0d63 Author: teor <teor(a)torproject.org> Date: Thu Aug 9 17:06:13 2018 +1000 Travis: create configure flags once, then echo the flags Creating the configure flags once avoids inconsistent flags between configure and distcheck configure. Echoing the flags helps developers work out what configure is doing. (Backported to 0.2.9 and later as a precaution.) Fixes 27088 on 0.3.4.1-alpha, adds logging in previous releases. --- .travis.yml | 6 ++++-- changes/bug27088 | 5 +++++ 2 files changed, 9 insertions(+), 2 deletions(-) diff --git a/.travis.yml b/.travis.yml index d75c74ebb..6169c808a 100644 --- a/.travis.yml +++ b/.travis.yml @@ -108,10 +108,12 @@ install: script: - ./autogen.sh - - ./configure $COVERAGE_OPTIONS $HARDENING_OPTIONS --disable-asciidoc --enable-fatal-warnings --disable-silent-rules + - CONFIGURE_FLAGS="$COVERAGE_OPTIONS $HARDENING_OPTIONS --disable-asciidoc --enable-fatal-warnings --disable-silent-rules" + - echo $CONFIGURE_FLAGS + - ./configure $CONFIGURE_FLAGS ## We run `make check` because that's what https://jenkins.torproject.org does. - if [[ "$DISTCHECK" == "" ]]; then make check; fi - - if [[ "$DISTCHECK" != "" ]]; then make distcheck DISTCHECK_CONFIGURE_FLAGS="$HARDENING_OPTIONS $COVERAGE_OPTIONS --disable-asciidoc --enable-fatal-warnings --disable-silent-rules"; fi + - if [[ "$DISTCHECK" != "" ]]; then make distcheck DISTCHECK_CONFIGURE_FLAGS="$CONFIGURE_FLAGS"; fi after_failure: ## configure will leave a log file with more details of config failures. diff --git a/changes/bug27088 b/changes/bug27088 new file mode 100644 index 000000000..d4d3b292c --- /dev/null +++ b/changes/bug27088 @@ -0,0 +1,5 @@ + o Minor bugfixes (continuous integration): + - Pass the module flags to distcheck configure, and + log the flags before running configure. (Backported + to 0.2.9 and later as a precaution.) + Fixes bug 27088; bugfix on 0.3.4.1-alpha.

1 0

[tor/maint-0.3.4] Travis: Rewrite .travis.yml
by nickm＠torproject.org 22 Aug '18

22 Aug '18

commit c53f17fb1a3b787567cee1e87f03a887a5cee0bd Author: teor <teor(a)torproject.org> Date: Thu Aug 9 16:42:05 2018 +1000 Travis: Rewrite .travis.yml Build on all compilers: * default options + hardening Build on gcc: * coverage (+ no hardening) * distcheck * no hardening Add some extra logging: * tail config.log on failure (config.log is too long for travis to render) Put the config in a more logical order * Sort config items in chronological order * Put related items together Part of 24629. --- .travis.yml | 175 +++++++++++++++++++++++++++--------------------------------- 1 file changed, 80 insertions(+), 95 deletions(-) diff --git a/.travis.yml b/.travis.yml index e3735f7d5..dfdf20f31 100644 --- a/.travis.yml +++ b/.travis.yml @@ -1,68 +1,38 @@ language: c -## Comment out the compiler list for now to allow an explicit build -## matrix. -# compiler: -# - gcc -# - clang - -notifications: - irc: - channels: - - "irc.oftc.net#tor-ci" - template: - - "%{repository} %{branch} %{commit} - %{author}: %{commit_subject}" - - "Build #%{build_number} %{result}. Details: %{build_url}" - on_success: change - on_failure: change - email: - on_success: never - on_failure: change +compiler: + - gcc + - clang os: - linux - ## Uncomment the following line to also run the entire build matrix on OSX. - ## This will make your CI builds take roughly ten times longer to finish. - # - osx - -## Use the Ubuntu Trusty images. -dist: trusty - -## We don't need sudo. (The "apt:" stanza after this allows us to not need sudo; -## otherwise, we would need it for getting dependencies.) -## -## We override this in the explicit build matrix to work around a -## Travis CI environment regression -## https://github.com/travis-ci/travis-ci/issues/9033 -sudo: false -## (Linux only) Download our dependencies -addons: - apt: - packages: - ## Required dependencies - - libevent-dev - - zlib1g-dev - ## Optional dependencies - - libcap-dev - - liblzma-dev - - libscrypt-dev - - libseccomp-dev - ## zstd doesn't exist in Ubuntu Trusty - #- libzstd - -## The build matrix in the following two stanzas expands into four builds (per OS): -## -## * with GCC, with Rust -## * with GCC, without Rust -## * with Clang, with Rust -## * with Clang, without Rust +## The build matrix in the following stanza expands into builds for each +## OS and compiler. env: global: ## The Travis CI environment allows us two cores, so let's use both. - MAKEFLAGS="-j 2" + ## We turn on hardening by default + ## Also known as --enable-fragile-hardening in 0.3.0.3-alpha and later + - HARDENING_OPTIONS="--enable-expensive-hardening" + matrix: + ## We want to use each build option at least once + ## + ## We don't list default variable values, because we set the defaults + ## in global (or the default is unset) + - matrix: + ## include creates builds with gcc, linux, sudo: false + include: + ## We include a single coverage build with the best options for coverage + - env: COVERAGE_OPTIONS="--enable-coverage" HARDENING_OPTIONS="" + ## We only want to check these build option combinations once + ## (they shouldn't vary by compiler or OS) + - env: DISTCHECK="yes" + - env: HARDENING_OPTIONS="" + ## Uncomment to allow the build to report success (with non-required ## sub-builds continuing to run) if all required sub-builds have ## succeeded. This is somewhat buggy currently: it can cause @@ -71,60 +41,62 @@ matrix: ## https://github.com/travis-ci/travis-ci/issues/1696 # fast_finish: true - ## Uncomment the appropriate lines below to allow the build to - ## report success even if some less-critical sub-builds fail and it - ## seems likely to take a while for someone to fix it. Currently - ## Travis CI doesn't distinguish "all builds succeeded" from "some - ## non-required sub-builds failed" except on the individual build's - ## page, which makes it somewhat annoying to detect from the - ## branches and build history pages. See - ## https://github.com/travis-ci/travis-ci/issues/8716 - allow_failures: - # - env: RUST_OPTIONS="--enable-rust" TOR_RUST_DEPENDENCIES=true - # - env: RUST_OPTIONS="--enable-rust --enable-cargo-online-mode - # - compiler: clang - - ## Create explicit matrix entries to work around a Travis CI - ## environment issue. Missing keys inherit from the first list - ## entry under that key outside the "include" clause. - include: - - compiler: gcc - - compiler: gcc - env: COVERAGE_OPTIONS="--enable-coverage" - - compiler: gcc - env: DISTCHECK="yes" - ## The "sudo: required" forces non-containerized builds, working - ## around a Travis CI environment issue: clang LeakAnalyzer fails - ## because it requires ptrace and the containerized environment no - ## longer allows ptrace. + ## Careful! We use global envs, which makes it hard to exclude or + ## allow failures by env: + ## https://docs.travis-ci.com/user/customizing-the-build#matching-jobs-with-al… + exclude: + ## Clang doesn't work in containerized builds, see below. - compiler: clang + sudo: false + ## We also exclude non-containerized gcc, because they're slow and redundant. + - compiler: gcc sudo: required -before_install: - ## If we're on OSX, homebrew usually needs to updated first - - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then brew update ; fi - ## Download rustup - - curl -Ssf -o rustup.sh https://sh.rustup.rs - - if [[ "$COVERAGE_OPTIONS" != "" ]]; then pip install --user cpp-coveralls; fi +## We don't need sudo. (The "apt:" stanza after this allows us to not need +## sudo; otherwise, we would need it for getting dependencies.) +## +## But we use "sudo: required" to force non-containerized builds, working +## around a Travis CI environment issue: clang LeakAnalyzer fails +## because it requires ptrace and the containerized environment no +## longer allows ptrace. +## https://github.com/travis-ci/travis-ci/issues/9033 +## +## In the matrix above, we exclude redundant combinations. +sudo: + - false + - required + +## (Linux only) Use the latest Linux image (Ubuntu Trusty) +dist: trusty + +## (Linux only) Download our dependencies +addons: + apt: + packages: + ## Required dependencies + - libevent-dev + - zlib1g-dev + ## Optional dependencies + - libcap-dev + - libscrypt-dev + - libseccomp-dev install: - ## If we're on OSX use brew to install required dependencies (for Linux, see the "apt:" section above) - - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then { brew outdated openssl || brew upgrade openssl; }; fi - - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then { brew outdated libevent || brew upgrade libevent; }; fi - - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then { brew outdated pkg-config || brew upgrade pkg-config; }; fi - ## If we're on OSX also install the optional dependencies - - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then { brew outdated xz || brew upgrade xz; }; fi - - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then { brew outdated libscrypt || brew upgrade libscrypt; }; fi - - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then { brew outdated zstd || brew upgrade zstd; }; fi + ## Install conditional features + ## Install coveralls + - if [[ "$COVERAGE_OPTIONS" != "" ]]; then pip install --user cpp-coveralls; fi script: - ./autogen.sh - - ./configure $RUST_OPTIONS $COVERAGE_OPTIONS --disable-asciidoc --enable-fatal-warnings --disable-silent-rules --enable-fragile-hardening + - ./configure $COVERAGE_OPTIONS $HARDENING_OPTIONS --disable-asciidoc --enable-fatal-warnings --disable-silent-rules ## We run `make check` because that's what https://jenkins.torproject.org does. - if [[ "$DISTCHECK" == "" ]]; then make check; fi - - if [[ "$DISTCHECK" != "" ]]; then make distcheck DISTCHECK_CONFIGURE_FLAGS="$RUST_OPTIONS $COVERAGE_OPTIONS --disable-asciidoc --enable-fatal-warnings --disable-silent-rules --enable-fragile-hardening"; fi + - if [[ "$DISTCHECK" != "" ]]; then make distcheck DISTCHECK_CONFIGURE_FLAGS="$HARDENING_OPTIONS $COVERAGE_OPTIONS --disable-asciidoc --enable-fatal-warnings --disable-silent-rules"; fi after_failure: + ## configure will leave a log file with more details of config failures. + ## But the log is too long for travis' rendered view, so tail it. + - tail -1000 config.log ## `make check` will leave a log file with more details of test failures. - if [[ "$DISTCHECK" == "" ]]; then cat test-suite.log; fi ## `make distcheck` puts it somewhere different. @@ -133,3 +105,16 @@ after_failure: after_success: ## If this build was one that produced coverage, upload it. - if [[ "$COVERAGE_OPTIONS" != "" ]]; then coveralls -b . --exclude src/test --exclude src/trunnel --gcov-options '\-p'; fi + +notifications: + irc: + channels: + - "irc.oftc.net#tor-ci" + template: + - "%{repository} %{branch} %{commit} - %{author}: %{commit_subject}" + - "Build #%{build_number} %{result}. Details: %{build_url}" + on_success: change + on_failure: change + email: + on_success: never + on_failure: change

1 0

[tor/maint-0.3.4] Travis: Use ccache
by nickm＠torproject.org 22 Aug '18

22 Aug '18

commit 286a6bc3b8fd2c369583d0d48c09769fed78e39f Author: teor <teor(a)torproject.org> Date: Thu Aug 9 15:40:27 2018 +1000 Travis: Use ccache Part of ticket 26952. --- .travis.yml | 6 ++++++ changes/ticket26952-ccache | 3 +++ 2 files changed, 9 insertions(+) diff --git a/.travis.yml b/.travis.yml index c2db6b78b..424f0d070 100644 --- a/.travis.yml +++ b/.travis.yml @@ -1,5 +1,8 @@ language: c +cache: + ccache: true + compiler: - gcc - clang @@ -94,6 +97,9 @@ before_install: - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then brew upgrade; fi install: + ## If we're on OSX use brew to install ccache (ccache is automatically installed on Linux) + - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then brew install ccache; fi + - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then export PATH="/usr/local/opt/ccache/libexec:$PATH"; fi ## If we're on OSX use brew to install required dependencies (for Linux, see the "apt:" section above) - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then brew install libevent; fi - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then brew install openssl; fi diff --git a/changes/ticket26952-ccache b/changes/ticket26952-ccache new file mode 100644 index 000000000..edc115e9d --- /dev/null +++ b/changes/ticket26952-ccache @@ -0,0 +1,3 @@ + o Minor features (continuous integration): + - Use ccache in our Travis CI configuration. + Closes ticket 26952.

1 0

[tor/maint-0.3.3] Merge branch 'maint-0.2.9' into maint-0.3.2
by nickm＠torproject.org 22 Aug '18

22 Aug '18

commit a5bbe3f1d0d8162e70af01f6b3956412dcc75946 Merge: a29e9a901 05d62a3b7 Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Aug 21 20:03:46 2018 -0400 Merge branch 'maint-0.2.9' into maint-0.3.2

1 0

[tor/maint-0.3.4] Travis: run an asciidoc build
by nickm＠torproject.org 22 Aug '18

22 Aug '18

commit 23b242104ba5cefb5fd16effda33a611e521050d Author: teor <teor(a)torproject.org> Date: Thu Aug 9 18:29:29 2018 +1000 Travis: run an asciidoc build Implements 27087. --- .travis.yml | 15 +++++++++++++-- changes/ticket27087 | 3 +++ 2 files changed, 16 insertions(+), 2 deletions(-) diff --git a/.travis.yml b/.travis.yml index 424f0d070..d822780cd 100644 --- a/.travis.yml +++ b/.travis.yml @@ -20,6 +20,8 @@ env: ## We turn on hardening by default ## Also known as --enable-fragile-hardening in 0.3.0.3-alpha and later - HARDENING_OPTIONS="--enable-expensive-hardening" + ## We turn off asciidoc by default, because it's slow + - ASCIIDOC_OPTIONS="--disable-asciidoc" matrix: ## We want to use each build option at least once ## @@ -34,8 +36,9 @@ matrix: - env: COVERAGE_OPTIONS="--enable-coverage" HARDENING_OPTIONS="" ## We only want to check these build option combinations once ## (they shouldn't vary by compiler or OS) - - env: DISTCHECK="yes" - env: HARDENING_OPTIONS="" + ## We check asciidoc with distcheck, to make sure we remove doc products + - env: ASCIIDOC_OPTIONS="" DISTCHECK="yes" ## Uncomment to allow the build to report success (with non-required ## sub-builds continuing to run) if all required sub-builds have @@ -84,6 +87,12 @@ addons: - libcap-dev - libscrypt-dev - libseccomp-dev + ## Conditional dependencies + ## Always installed, so we don't need sudo + - asciidoc + - docbook-xsl + - docbook-xml + - xmlto ## (OSX only) Use the default OSX image ## See https://docs.travis-ci.com/user/reference/osx#os-x-version @@ -113,10 +122,12 @@ install: ## Install conditional features ## Install coveralls - if [[ "$COVERAGE_OPTIONS" != "" ]]; then pip install --user cpp-coveralls; fi + ## If we're on OSX, and using asciidoc, install asciidoc + - if [[ "$ASCIIDOC_OPTIONS" == "" ]] && [[ "$TRAVIS_OS_NAME" == "osx" ]]; then brew install asciidoc; fi script: - ./autogen.sh - - CONFIGURE_FLAGS="$COVERAGE_OPTIONS $HARDENING_OPTIONS $OPENSSL_OPTIONS --disable-asciidoc --enable-fatal-warnings --disable-silent-rules" + - CONFIGURE_FLAGS="$ASCIIDOC_OPTIONS $COVERAGE_OPTIONS $HARDENING_OPTIONS $OPENSSL_OPTIONS --enable-fatal-warnings --disable-silent-rules" - echo $CONFIGURE_FLAGS - ./configure $CONFIGURE_FLAGS ## We run `make check` because that's what https://jenkins.torproject.org does. diff --git a/changes/ticket27087 b/changes/ticket27087 new file mode 100644 index 000000000..b8af70aaa --- /dev/null +++ b/changes/ticket27087 @@ -0,0 +1,3 @@ + o Minor features (continuous integration): + - Run asciidoc during Travis CI. + Implements ticket 27087.

1 0

[tor/maint-0.3.3] Merge branch 'maint-0.3.2' into maint-0.3.3
by nickm＠torproject.org 22 Aug '18

22 Aug '18

commit 3cfceeb45d51f8f3c4960cabaad1849dcecb14a3 Merge: f0633bc49 a5bbe3f1d Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Aug 21 20:04:17 2018 -0400 Merge branch 'maint-0.3.2' into maint-0.3.3

1 0

[tor/maint-0.3.4] Add more optional packages to Travis
by nickm＠torproject.org 22 Aug '18

22 Aug '18

commit 340da669be373174550e038d1d20ff2bff49eb6b Author: Taylor Yu <catalyst(a)torproject.org> Date: Thu Jun 28 14:43:31 2018 -0500 Add more optional packages to Travis Apparently we weren't building with either libcap or libseccomp on Travis. Install libcap-dev and libseccomp-dev in .travis.yml. Closes ticket 26560. --- .travis.yml | 3 ++- changes/ticket26560 | 3 +++ 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index 6a3e1bfc0..e3735f7d5 100644 --- a/.travis.yml +++ b/.travis.yml @@ -42,11 +42,12 @@ addons: packages: ## Required dependencies - libevent-dev - - libseccomp2 - zlib1g-dev ## Optional dependencies + - libcap-dev - liblzma-dev - libscrypt-dev + - libseccomp-dev ## zstd doesn't exist in Ubuntu Trusty #- libzstd diff --git a/changes/ticket26560 b/changes/ticket26560 new file mode 100644 index 000000000..5b4fb1bfe --- /dev/null +++ b/changes/ticket26560 @@ -0,0 +1,3 @@ + o Minor features (continuous integration): + - Install libcap-dev and libseccomp2-dev so these optional + dependencies get tested on Travis CI. Closes ticket 26560.

1 0