August 2018 - tor-commits - lists.torproject.org

[tor/release-0.3.2] Merge branch 'travis-osx-029' into travis-osx-032
by nickm＠torproject.org 22 Aug '18

22 Aug '18

commit e2eec40c4469c10dfb3006dfb07240d7e1144aa7 Merge: 69adc68ac d514e9866 Author: teor <teor(a)torproject.org> Date: Fri Aug 10 19:05:49 2018 +1000 Merge branch 'travis-osx-029' into travis-osx-032 Merge asciidoc from 029 with rust from 032. .travis.yml | 1 + 1 file changed, 1 insertion(+) diff --cc .travis.yml index f92a3d8d4,8b8621007..1b43c721a --- a/.travis.yml +++ b/.travis.yml @@@ -147,13 -125,7 +147,14 @@@ install ## If we're on OSX, and using asciidoc, install asciidoc - if [[ "$ASCIIDOC_OPTIONS" == "" ]] && [[ "$TRAVIS_OS_NAME" == "osx" ]]; then brew install asciidoc; fi - if [[ "$ASCIIDOC_OPTIONS" == "" ]] && [[ "$TRAVIS_OS_NAME" == "osx" ]]; then brew install xmlto; fi + - if [[ "$ASCIIDOC_OPTIONS" == "" ]] && [[ "$TRAVIS_OS_NAME" == "osx" ]]; then export XML_CATALOG_FILES="/usr/local/etc/xml/catalog"; fi + ## If we're using Rust, download rustup + - if [[ "$RUST_OPTIONS" != "" ]]; then curl -Ssf -o rustup.sh https://sh.rustup.rs; fi + ## Install the stable channels of rustc and cargo and setup our toolchain environment + - if [[ "$RUST_OPTIONS" != "" ]]; then sh rustup.sh -y --default-toolchain stable; fi + - if [[ "$RUST_OPTIONS" != "" ]]; then source $HOME/.cargo/env; fi + ## If we're testing rust builds in offline-mode, then set up our vendored dependencies + - if [[ "$RUST_DEPENDENCIES" == "true" ]]; then export RUST_DEPENDENCIES=$PWD/src/ext/rust/crates; fi ## ## Finally, list installed package versions - if [[ "$TRAVIS_OS_NAME" == "linux" ]]; then dpkg-query --show; fi

1 0

[tor/release-0.3.3] Add more optional packages to Travis
by nickm＠torproject.org 22 Aug '18

22 Aug '18

commit 340da669be373174550e038d1d20ff2bff49eb6b Author: Taylor Yu <catalyst(a)torproject.org> Date: Thu Jun 28 14:43:31 2018 -0500 Add more optional packages to Travis Apparently we weren't building with either libcap or libseccomp on Travis. Install libcap-dev and libseccomp-dev in .travis.yml. Closes ticket 26560. --- .travis.yml | 3 ++- changes/ticket26560 | 3 +++ 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index 6a3e1bfc0..e3735f7d5 100644 --- a/.travis.yml +++ b/.travis.yml @@ -42,11 +42,12 @@ addons: packages: ## Required dependencies - libevent-dev - - libseccomp2 - zlib1g-dev ## Optional dependencies + - libcap-dev - liblzma-dev - libscrypt-dev + - libseccomp-dev ## zstd doesn't exist in Ubuntu Trusty #- libzstd diff --git a/changes/ticket26560 b/changes/ticket26560 new file mode 100644 index 000000000..5b4fb1bfe --- /dev/null +++ b/changes/ticket26560 @@ -0,0 +1,3 @@ + o Minor features (continuous integration): + - Install libcap-dev and libseccomp2-dev so these optional + dependencies get tested on Travis CI. Closes ticket 26560.

1 0

[tor/release-0.3.3] Travis: Rewrite .travis.yml
by nickm＠torproject.org 22 Aug '18

22 Aug '18

commit c53f17fb1a3b787567cee1e87f03a887a5cee0bd Author: teor <teor(a)torproject.org> Date: Thu Aug 9 16:42:05 2018 +1000 Travis: Rewrite .travis.yml Build on all compilers: * default options + hardening Build on gcc: * coverage (+ no hardening) * distcheck * no hardening Add some extra logging: * tail config.log on failure (config.log is too long for travis to render) Put the config in a more logical order * Sort config items in chronological order * Put related items together Part of 24629. --- .travis.yml | 175 +++++++++++++++++++++++++++--------------------------------- 1 file changed, 80 insertions(+), 95 deletions(-) diff --git a/.travis.yml b/.travis.yml index e3735f7d5..dfdf20f31 100644 --- a/.travis.yml +++ b/.travis.yml @@ -1,68 +1,38 @@ language: c -## Comment out the compiler list for now to allow an explicit build -## matrix. -# compiler: -# - gcc -# - clang - -notifications: - irc: - channels: - - "irc.oftc.net#tor-ci" - template: - - "%{repository} %{branch} %{commit} - %{author}: %{commit_subject}" - - "Build #%{build_number} %{result}. Details: %{build_url}" - on_success: change - on_failure: change - email: - on_success: never - on_failure: change +compiler: + - gcc + - clang os: - linux - ## Uncomment the following line to also run the entire build matrix on OSX. - ## This will make your CI builds take roughly ten times longer to finish. - # - osx - -## Use the Ubuntu Trusty images. -dist: trusty - -## We don't need sudo. (The "apt:" stanza after this allows us to not need sudo; -## otherwise, we would need it for getting dependencies.) -## -## We override this in the explicit build matrix to work around a -## Travis CI environment regression -## https://github.com/travis-ci/travis-ci/issues/9033 -sudo: false -## (Linux only) Download our dependencies -addons: - apt: - packages: - ## Required dependencies - - libevent-dev - - zlib1g-dev - ## Optional dependencies - - libcap-dev - - liblzma-dev - - libscrypt-dev - - libseccomp-dev - ## zstd doesn't exist in Ubuntu Trusty - #- libzstd - -## The build matrix in the following two stanzas expands into four builds (per OS): -## -## * with GCC, with Rust -## * with GCC, without Rust -## * with Clang, with Rust -## * with Clang, without Rust +## The build matrix in the following stanza expands into builds for each +## OS and compiler. env: global: ## The Travis CI environment allows us two cores, so let's use both. - MAKEFLAGS="-j 2" + ## We turn on hardening by default + ## Also known as --enable-fragile-hardening in 0.3.0.3-alpha and later + - HARDENING_OPTIONS="--enable-expensive-hardening" + matrix: + ## We want to use each build option at least once + ## + ## We don't list default variable values, because we set the defaults + ## in global (or the default is unset) + - matrix: + ## include creates builds with gcc, linux, sudo: false + include: + ## We include a single coverage build with the best options for coverage + - env: COVERAGE_OPTIONS="--enable-coverage" HARDENING_OPTIONS="" + ## We only want to check these build option combinations once + ## (they shouldn't vary by compiler or OS) + - env: DISTCHECK="yes" + - env: HARDENING_OPTIONS="" + ## Uncomment to allow the build to report success (with non-required ## sub-builds continuing to run) if all required sub-builds have ## succeeded. This is somewhat buggy currently: it can cause @@ -71,60 +41,62 @@ matrix: ## https://github.com/travis-ci/travis-ci/issues/1696 # fast_finish: true - ## Uncomment the appropriate lines below to allow the build to - ## report success even if some less-critical sub-builds fail and it - ## seems likely to take a while for someone to fix it. Currently - ## Travis CI doesn't distinguish "all builds succeeded" from "some - ## non-required sub-builds failed" except on the individual build's - ## page, which makes it somewhat annoying to detect from the - ## branches and build history pages. See - ## https://github.com/travis-ci/travis-ci/issues/8716 - allow_failures: - # - env: RUST_OPTIONS="--enable-rust" TOR_RUST_DEPENDENCIES=true - # - env: RUST_OPTIONS="--enable-rust --enable-cargo-online-mode - # - compiler: clang - - ## Create explicit matrix entries to work around a Travis CI - ## environment issue. Missing keys inherit from the first list - ## entry under that key outside the "include" clause. - include: - - compiler: gcc - - compiler: gcc - env: COVERAGE_OPTIONS="--enable-coverage" - - compiler: gcc - env: DISTCHECK="yes" - ## The "sudo: required" forces non-containerized builds, working - ## around a Travis CI environment issue: clang LeakAnalyzer fails - ## because it requires ptrace and the containerized environment no - ## longer allows ptrace. + ## Careful! We use global envs, which makes it hard to exclude or + ## allow failures by env: + ## https://docs.travis-ci.com/user/customizing-the-build#matching-jobs-with-al… + exclude: + ## Clang doesn't work in containerized builds, see below. - compiler: clang + sudo: false + ## We also exclude non-containerized gcc, because they're slow and redundant. + - compiler: gcc sudo: required -before_install: - ## If we're on OSX, homebrew usually needs to updated first - - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then brew update ; fi - ## Download rustup - - curl -Ssf -o rustup.sh https://sh.rustup.rs - - if [[ "$COVERAGE_OPTIONS" != "" ]]; then pip install --user cpp-coveralls; fi +## We don't need sudo. (The "apt:" stanza after this allows us to not need +## sudo; otherwise, we would need it for getting dependencies.) +## +## But we use "sudo: required" to force non-containerized builds, working +## around a Travis CI environment issue: clang LeakAnalyzer fails +## because it requires ptrace and the containerized environment no +## longer allows ptrace. +## https://github.com/travis-ci/travis-ci/issues/9033 +## +## In the matrix above, we exclude redundant combinations. +sudo: + - false + - required + +## (Linux only) Use the latest Linux image (Ubuntu Trusty) +dist: trusty + +## (Linux only) Download our dependencies +addons: + apt: + packages: + ## Required dependencies + - libevent-dev + - zlib1g-dev + ## Optional dependencies + - libcap-dev + - libscrypt-dev + - libseccomp-dev install: - ## If we're on OSX use brew to install required dependencies (for Linux, see the "apt:" section above) - - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then { brew outdated openssl || brew upgrade openssl; }; fi - - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then { brew outdated libevent || brew upgrade libevent; }; fi - - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then { brew outdated pkg-config || brew upgrade pkg-config; }; fi - ## If we're on OSX also install the optional dependencies - - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then { brew outdated xz || brew upgrade xz; }; fi - - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then { brew outdated libscrypt || brew upgrade libscrypt; }; fi - - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then { brew outdated zstd || brew upgrade zstd; }; fi + ## Install conditional features + ## Install coveralls + - if [[ "$COVERAGE_OPTIONS" != "" ]]; then pip install --user cpp-coveralls; fi script: - ./autogen.sh - - ./configure $RUST_OPTIONS $COVERAGE_OPTIONS --disable-asciidoc --enable-fatal-warnings --disable-silent-rules --enable-fragile-hardening + - ./configure $COVERAGE_OPTIONS $HARDENING_OPTIONS --disable-asciidoc --enable-fatal-warnings --disable-silent-rules ## We run `make check` because that's what https://jenkins.torproject.org does. - if [[ "$DISTCHECK" == "" ]]; then make check; fi - - if [[ "$DISTCHECK" != "" ]]; then make distcheck DISTCHECK_CONFIGURE_FLAGS="$RUST_OPTIONS $COVERAGE_OPTIONS --disable-asciidoc --enable-fatal-warnings --disable-silent-rules --enable-fragile-hardening"; fi + - if [[ "$DISTCHECK" != "" ]]; then make distcheck DISTCHECK_CONFIGURE_FLAGS="$HARDENING_OPTIONS $COVERAGE_OPTIONS --disable-asciidoc --enable-fatal-warnings --disable-silent-rules"; fi after_failure: + ## configure will leave a log file with more details of config failures. + ## But the log is too long for travis' rendered view, so tail it. + - tail -1000 config.log ## `make check` will leave a log file with more details of test failures. - if [[ "$DISTCHECK" == "" ]]; then cat test-suite.log; fi ## `make distcheck` puts it somewhere different. @@ -133,3 +105,16 @@ after_failure: after_success: ## If this build was one that produced coverage, upload it. - if [[ "$COVERAGE_OPTIONS" != "" ]]; then coveralls -b . --exclude src/test --exclude src/trunnel --gcov-options '\-p'; fi + +notifications: + irc: + channels: + - "irc.oftc.net#tor-ci" + template: + - "%{repository} %{branch} %{commit} - %{author}: %{commit_subject}" + - "Build #%{build_number} %{result}. Details: %{build_url}" + on_success: change + on_failure: change + email: + on_success: never + on_failure: change

1 0

[tor/release-0.3.2] Merge branch 'maint-0.3.2' into release-0.3.2
by nickm＠torproject.org 22 Aug '18

22 Aug '18

commit 54e2a3315a94b86454b687600bb02ada96b3699c Merge: 22414c050 a5bbe3f1d Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Aug 21 20:04:17 2018 -0400 Merge branch 'maint-0.3.2' into release-0.3.2 .travis.yml | 253 ++++++++++++++++++++++++++----------------- changes/bug26497 | 3 + changes/bug26497-backport | 3 + changes/bug27088 | 5 + changes/bug27090 | 3 + changes/bug27093 | 3 + changes/ticket24629 | 3 + changes/ticket26560 | 3 + changes/ticket26952-cargo | 3 + changes/ticket26952-ccache | 3 + changes/ticket27087 | 3 + src/rust/tor_util/include.am | 1 - src/test/test_rust.sh | 17 ++- 13 files changed, 194 insertions(+), 109 deletions(-)

1 0

[tor/release-0.3.2] Merge remote-tracking branch 'teor/travis-osx-032' into maint-0.3.2
by nickm＠torproject.org 22 Aug '18

22 Aug '18

commit a29e9a901d196243791f9210be1fed1c177521df Merge: d52f40600 e2eec40c4 Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Aug 21 20:03:41 2018 -0400 Merge remote-tracking branch 'teor/travis-osx-032' into maint-0.3.2 .travis.yml | 253 ++++++++++++++++++++++++++----------------- changes/bug26497 | 3 + changes/bug26497-backport | 3 + changes/bug27088 | 5 + changes/bug27090 | 3 + changes/bug27093 | 3 + changes/ticket24629 | 3 + changes/ticket26560 | 3 + changes/ticket26952-cargo | 3 + changes/ticket26952-ccache | 3 + changes/ticket27087 | 3 + src/rust/tor_util/include.am | 1 - src/test/test_rust.sh | 17 ++- 13 files changed, 194 insertions(+), 109 deletions(-)

1 0

[tor/release-0.3.2] Merge branch 'maint-0.2.9' into maint-0.3.2
by nickm＠torproject.org 22 Aug '18

22 Aug '18

commit a5bbe3f1d0d8162e70af01f6b3956412dcc75946 Merge: a29e9a901 05d62a3b7 Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Aug 21 20:03:46 2018 -0400 Merge branch 'maint-0.2.9' into maint-0.3.2

1 0

[tor/release-0.3.3] Travis: make macOS builds work for Tor 0.2.9
by nickm＠torproject.org 22 Aug '18

22 Aug '18

commit 74b3a340df0df1eb803b4b17039725fc42d5a86a Author: teor <teor(a)torproject.org> Date: Thu Aug 9 17:44:11 2018 +1000 Travis: make macOS builds work for Tor 0.2.9 Tor 0.2.9 needs extra help to find OpenSSL on macOS. Part of 24629. --- .travis.yml | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index 6169c808a..c2db6b78b 100644 --- a/.travis.yml +++ b/.travis.yml @@ -102,13 +102,15 @@ install: # - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then brew install zlib; fi ## If we're on OSX also install the optional dependencies - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then brew install libscrypt; fi + ## If we're on OSX, OpenSSL is keg-only, so tor 0.2.9 and later need to be configured --with-openssl-dir= to build + - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then OPENSSL_OPTIONS=--with-openssl-dir=`brew --prefix openssl`; fi ## Install conditional features ## Install coveralls - if [[ "$COVERAGE_OPTIONS" != "" ]]; then pip install --user cpp-coveralls; fi script: - ./autogen.sh - - CONFIGURE_FLAGS="$COVERAGE_OPTIONS $HARDENING_OPTIONS --disable-asciidoc --enable-fatal-warnings --disable-silent-rules" + - CONFIGURE_FLAGS="$COVERAGE_OPTIONS $HARDENING_OPTIONS $OPENSSL_OPTIONS --disable-asciidoc --enable-fatal-warnings --disable-silent-rules" - echo $CONFIGURE_FLAGS - ./configure $CONFIGURE_FLAGS ## We run `make check` because that's what https://jenkins.torproject.org does.

1 0

[tor/release-0.3.3] Travis: run an asciidoc build
by nickm＠torproject.org 22 Aug '18

22 Aug '18

commit 23b242104ba5cefb5fd16effda33a611e521050d Author: teor <teor(a)torproject.org> Date: Thu Aug 9 18:29:29 2018 +1000 Travis: run an asciidoc build Implements 27087. --- .travis.yml | 15 +++++++++++++-- changes/ticket27087 | 3 +++ 2 files changed, 16 insertions(+), 2 deletions(-) diff --git a/.travis.yml b/.travis.yml index 424f0d070..d822780cd 100644 --- a/.travis.yml +++ b/.travis.yml @@ -20,6 +20,8 @@ env: ## We turn on hardening by default ## Also known as --enable-fragile-hardening in 0.3.0.3-alpha and later - HARDENING_OPTIONS="--enable-expensive-hardening" + ## We turn off asciidoc by default, because it's slow + - ASCIIDOC_OPTIONS="--disable-asciidoc" matrix: ## We want to use each build option at least once ## @@ -34,8 +36,9 @@ matrix: - env: COVERAGE_OPTIONS="--enable-coverage" HARDENING_OPTIONS="" ## We only want to check these build option combinations once ## (they shouldn't vary by compiler or OS) - - env: DISTCHECK="yes" - env: HARDENING_OPTIONS="" + ## We check asciidoc with distcheck, to make sure we remove doc products + - env: ASCIIDOC_OPTIONS="" DISTCHECK="yes" ## Uncomment to allow the build to report success (with non-required ## sub-builds continuing to run) if all required sub-builds have @@ -84,6 +87,12 @@ addons: - libcap-dev - libscrypt-dev - libseccomp-dev + ## Conditional dependencies + ## Always installed, so we don't need sudo + - asciidoc + - docbook-xsl + - docbook-xml + - xmlto ## (OSX only) Use the default OSX image ## See https://docs.travis-ci.com/user/reference/osx#os-x-version @@ -113,10 +122,12 @@ install: ## Install conditional features ## Install coveralls - if [[ "$COVERAGE_OPTIONS" != "" ]]; then pip install --user cpp-coveralls; fi + ## If we're on OSX, and using asciidoc, install asciidoc + - if [[ "$ASCIIDOC_OPTIONS" == "" ]] && [[ "$TRAVIS_OS_NAME" == "osx" ]]; then brew install asciidoc; fi script: - ./autogen.sh - - CONFIGURE_FLAGS="$COVERAGE_OPTIONS $HARDENING_OPTIONS $OPENSSL_OPTIONS --disable-asciidoc --enable-fatal-warnings --disable-silent-rules" + - CONFIGURE_FLAGS="$ASCIIDOC_OPTIONS $COVERAGE_OPTIONS $HARDENING_OPTIONS $OPENSSL_OPTIONS --enable-fatal-warnings --disable-silent-rules" - echo $CONFIGURE_FLAGS - ./configure $CONFIGURE_FLAGS ## We run `make check` because that's what https://jenkins.torproject.org does. diff --git a/changes/ticket27087 b/changes/ticket27087 new file mode 100644 index 000000000..b8af70aaa --- /dev/null +++ b/changes/ticket27087 @@ -0,0 +1,3 @@ + o Minor features (continuous integration): + - Run asciidoc during Travis CI. + Implements ticket 27087.

1 0

[tor/release-0.3.3] Travis: Use ccache
by nickm＠torproject.org 22 Aug '18

22 Aug '18

commit 286a6bc3b8fd2c369583d0d48c09769fed78e39f Author: teor <teor(a)torproject.org> Date: Thu Aug 9 15:40:27 2018 +1000 Travis: Use ccache Part of ticket 26952. --- .travis.yml | 6 ++++++ changes/ticket26952-ccache | 3 +++ 2 files changed, 9 insertions(+) diff --git a/.travis.yml b/.travis.yml index c2db6b78b..424f0d070 100644 --- a/.travis.yml +++ b/.travis.yml @@ -1,5 +1,8 @@ language: c +cache: + ccache: true + compiler: - gcc - clang @@ -94,6 +97,9 @@ before_install: - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then brew upgrade; fi install: + ## If we're on OSX use brew to install ccache (ccache is automatically installed on Linux) + - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then brew install ccache; fi + - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then export PATH="/usr/local/opt/ccache/libexec:$PATH"; fi ## If we're on OSX use brew to install required dependencies (for Linux, see the "apt:" section above) - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then brew install libevent; fi - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then brew install openssl; fi diff --git a/changes/ticket26952-ccache b/changes/ticket26952-ccache new file mode 100644 index 000000000..edc115e9d --- /dev/null +++ b/changes/ticket26952-ccache @@ -0,0 +1,3 @@ + o Minor features (continuous integration): + - Use ccache in our Travis CI configuration. + Closes ticket 26952.

1 0

[tor/release-0.3.3] Travis: enable macOS builds
by nickm＠torproject.org 22 Aug '18

22 Aug '18

commit 515d190b2ca9249871d04067c3bdbd312669997e Author: teor <teor(a)torproject.org> Date: Thu Aug 9 18:03:18 2018 +1000 Travis: enable macOS builds Also: * explain why we don't install zlib Part of 24629. --- .travis.yml | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/.travis.yml b/.travis.yml index dfdf20f31..d75c74ebb 100644 --- a/.travis.yml +++ b/.travis.yml @@ -6,6 +6,7 @@ compiler: os: - linux + - osx ## The build matrix in the following stanza expands into builds for each ## OS and compiler. @@ -81,7 +82,26 @@ addons: - libscrypt-dev - libseccomp-dev +## (OSX only) Use the default OSX image +## See https://docs.travis-ci.com/user/reference/osx#os-x-version +## Default is Xcode 9.4 on macOS 10.13 as of August 2018 +#osx_image: xcode9.4 + +before_install: + ## If we're on OSX, homebrew usually needs to updated first + - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then brew update; fi + ## We might be upgrading some useless packages, but that's better than missing an upgrade + - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then brew upgrade; fi + install: + ## If we're on OSX use brew to install required dependencies (for Linux, see the "apt:" section above) + - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then brew install libevent; fi + - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then brew install openssl; fi + - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then brew install pkg-config; fi + ## macOS comes with zlib by default, so the homebrew install is keg-only + # - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then brew install zlib; fi + ## If we're on OSX also install the optional dependencies + - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then brew install libscrypt; fi ## Install conditional features ## Install coveralls - if [[ "$COVERAGE_OPTIONS" != "" ]]; then pip install --user cpp-coveralls; fi

1 0