July 2019 - tor-commits - lists.torproject.org

[tor/maint-0.4.1] Travis: control "make check" with its own flag
by dgoulet＠torproject.org 23 Jul '19

23 Jul '19

commit b0826346a64e58b81e00ec6fcb0d2b75b302c0d3 Author: Nick Mathewson <nickm(a)torproject.org> Date: Wed Feb 27 07:51:52 2019 -0500 Travis: control "make check" with its own flag Previously we had "make check" launched whenever DISTCHECK was false. Now we'd like to turn it off in a few other circumstances, like running chutney. Maybe stem too? --- .travis.yml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/.travis.yml b/.travis.yml index 456b5abc5..a6727923d 100644 --- a/.travis.yml +++ b/.travis.yml @@ -39,7 +39,7 @@ matrix: ## We run coverage with hardening off, which seems like enough # - env: HARDENING_OPTIONS="" ## We check asciidoc with distcheck, to make sure we remove doc products - - env: DISTCHECK="yes" ASCIIDOC_OPTIONS="" + - env: DISTCHECK="yes" ASCIIDOC_OPTIONS="" SKIP_MAKE_CHECK="yes" ## Uncomment to allow the build to report success (with non-required ## sub-builds continuing to run) if all required sub-builds have @@ -153,7 +153,7 @@ script: - echo "Configure flags are $CONFIGURE_FLAGS" - ./configure $CONFIGURE_FLAGS ## We run `make check` because that's what https://jenkins.torproject.org does. - - if [[ "$DISTCHECK" == "" ]]; then make check; fi + - if [[ "$SKIP_MAKE_CHECK" == "" ]]; then make check; fi - if [[ "$DISTCHECK" != "" ]]; then make distcheck DISTCHECK_CONFIGURE_FLAGS="$CONFIGURE_FLAGS"; fi ## If this build was one that produced coverage, upload it. - if [[ "$COVERAGE_OPTIONS" != "" ]]; then coveralls -b . --exclude src/test --exclude src/trunnel --gcov-options '\-p' || echo "Coverage failed"; fi @@ -163,7 +163,7 @@ after_failure: ## But the log is too long for travis' rendered view, so tail it. - tail -1000 config.log || echo "tail failed" ## `make check` will leave a log file with more details of test failures. - - if [[ "$DISTCHECK" == "" ]]; then cat test-suite.log || echo "cat failed"; fi + - if [[ "$SKIP_MAKE_CHECK" == "" ]]; then cat test-suite.log || echo "cat failed"; fi ## `make distcheck` puts it somewhere different. - if [[ "$DISTCHECK" != "" ]]; then make show-distdir-testlog || echo "make failed"; fi

1 0

[tor/maint-0.4.1] Travis: remove a redundant os: linux line
by dgoulet＠torproject.org 23 Jul '19

23 Jul '19

commit 75f9ae9029b4244658042ce7a4243202bfb4c527 Author: teor <teor(a)torproject.org> Date: Mon Jul 1 14:11:25 2019 +1000 Travis: remove a redundant os: linux line Linux is the first item in the os matrix, so we don't need to specify it under matrix: include: . Part of 29280. --- .travis.yml | 1 - 1 file changed, 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index 562036e51..190f43bd0 100644 --- a/.travis.yml +++ b/.travis.yml @@ -43,7 +43,6 @@ matrix: # We also try running a hardened clang build with chutney on Linux. - env: CHUTNEY="yes" SKIP_MAKE_CHECK="yes" CHUTNEY_ALLOW_FAILURES="2" compiler: clang - os: linux ## Uncomment to allow the build to report success (with non-required ## sub-builds continuing to run) if all required sub-builds have

1 0

[tor/maint-0.4.1] Travis: show the chutney commit and python version
by dgoulet＠torproject.org 23 Jul '19

23 Jul '19

commit ac7ce179834d8d97342fc17bc385730eccc109a8 Author: teor <teor(a)torproject.org> Date: Mon Jul 1 14:10:07 2019 +1000 Travis: show the chutney commit and python version Part of 29280. --- .travis.yml | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/.travis.yml b/.travis.yml index cb1351a18..562036e51 100644 --- a/.travis.yml +++ b/.travis.yml @@ -151,6 +151,10 @@ install: ## Finally, list installed package versions - if [[ "$TRAVIS_OS_NAME" == "linux" ]]; then dpkg-query --show; fi - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then brew list --versions; fi + ## Get python version + - python --version + ## If we're running chutney, show the chutney commit + - if [[ "$CHUTNEY" != "" ]]; then pushd "$CHUTNEY_PATH"; git log -1 ; popd ; fi script: - ./autogen.sh

1 0

[tor/maint-0.4.1] Add a chutney mode to travis.yml.
by dgoulet＠torproject.org 23 Jul '19

23 Jul '19

commit 63325c05964ca5906aad4b51a53a5dc471fc7347 Author: Nick Mathewson <nickm(a)torproject.org> Date: Wed Feb 27 07:59:39 2019 -0500 Add a chutney mode to travis.yml. --- .travis.yml | 8 +++++++- changes/chutney_ci | 3 +++ 2 files changed, 10 insertions(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index a6727923d..a8ad1f122 100644 --- a/.travis.yml +++ b/.travis.yml @@ -40,6 +40,10 @@ matrix: # - env: HARDENING_OPTIONS="" ## We check asciidoc with distcheck, to make sure we remove doc products - env: DISTCHECK="yes" ASCIIDOC_OPTIONS="" SKIP_MAKE_CHECK="yes" + # We also try running a hardened clang build with chutney on Linux. + - env: CHUTNEY="yes" SKIP_MAKE_CHECK="yes" CHUTNEY_ALLOW_FAILURES="2" + compiler: clang + os: linux ## Uncomment to allow the build to report success (with non-required ## sub-builds continuing to run) if all required sub-builds have @@ -142,7 +146,8 @@ install: - if [[ "$COVERAGE_OPTIONS" != "" ]]; then pip install --user cpp-coveralls; fi ## If we're on OSX, and using asciidoc, configure asciidoc - if [[ "$ASCIIDOC_OPTIONS" == "" ]] && [[ "$TRAVIS_OS_NAME" == "osx" ]]; then export XML_CATALOG_FILES="/usr/local/etc/xml/catalog"; fi - ## + ## If we're running chutney, install it. + - if [[ "$CHUTNEY" != "" ]]; then git clone --depth 1 https://github.com/torproject/chutney.git ; export CHUTNEY_PATH="$(pwd)/chutney"; fi ## Finally, list installed package versions - if [[ "$TRAVIS_OS_NAME" == "linux" ]]; then dpkg-query --show; fi - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then brew list --versions; fi @@ -155,6 +160,7 @@ script: ## We run `make check` because that's what https://jenkins.torproject.org does. - if [[ "$SKIP_MAKE_CHECK" == "" ]]; then make check; fi - if [[ "$DISTCHECK" != "" ]]; then make distcheck DISTCHECK_CONFIGURE_FLAGS="$CONFIGURE_FLAGS"; fi + - if [[ "$CHUTNEY" != "" ]]; then make test-network-all; fi ## If this build was one that produced coverage, upload it. - if [[ "$COVERAGE_OPTIONS" != "" ]]; then coveralls -b . --exclude src/test --exclude src/trunnel --gcov-options '\-p' || echo "Coverage failed"; fi diff --git a/changes/chutney_ci b/changes/chutney_ci new file mode 100644 index 000000000..b17d58732 --- /dev/null +++ b/changes/chutney_ci @@ -0,0 +1,3 @@ + o Minor features (continuous integration): + - Our Travis configuration now uses Chutney to run some network + integration tests automatically. Closes ticket 29280.

1 0

[tor/maint-0.4.1] Travis: show chutney logs on chutney job failure
by dgoulet＠torproject.org 23 Jul '19

23 Jul '19

commit b26e3e4c565f4f764ee13a8afd5ea569547bc402 Author: teor <teor(a)torproject.org> Date: Mon Jul 1 14:05:26 2019 +1000 Travis: show chutney logs on chutney job failure Part of 29280. --- .travis.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/.travis.yml b/.travis.yml index a8ad1f122..cb1351a18 100644 --- a/.travis.yml +++ b/.travis.yml @@ -172,6 +172,7 @@ after_failure: - if [[ "$SKIP_MAKE_CHECK" == "" ]]; then cat test-suite.log || echo "cat failed"; fi ## `make distcheck` puts it somewhere different. - if [[ "$DISTCHECK" != "" ]]; then make show-distdir-testlog || echo "make failed"; fi + - if [[ "$CHUTNEY" != "" ]]; then ls test_network_log || echo "ls failed"; cat test_network_log/* || echo "cat failed"; fi before_cache: ## Delete all gcov files.

1 0

[tor/maint-0.4.1] Merge branch 'chutney_ci_v4_029' into chutney_ci_v4_035
by dgoulet＠torproject.org 23 Jul '19

23 Jul '19

commit 84f2c0affbace6227882a9fd440da74515aa7c12 Merge: b0fa1f4fb 75f9ae902 Author: teor <teor(a)torproject.org> Date: Mon Jul 1 14:31:17 2019 +1000 Merge branch 'chutney_ci_v4_029' into chutney_ci_v4_035 .travis.yml | 19 +++++++++++++++---- changes/chutney_ci | 3 +++ 2 files changed, 18 insertions(+), 4 deletions(-) diff --cc .travis.yml index 861b37f08,190f43bd0..8493589f7 --- a/.travis.yml +++ b/.travis.yml @@@ -45,19 -36,13 +45,22 @@@ matrix - env: COVERAGE_OPTIONS="--enable-coverage" HARDENING_OPTIONS="" ## We only want to check these build option combinations once ## (they shouldn't vary by compiler or OS) - ## We run coverage with hardening off, which seems like enough + ## We run rust and coverage with hardening off, which seems like enough # - env: HARDENING_OPTIONS="" ## We check asciidoc with distcheck, to make sure we remove doc products - - env: DISTCHECK="yes" ASCIIDOC_OPTIONS="" + - env: DISTCHECK="yes" ASCIIDOC_OPTIONS="" SKIP_MAKE_CHECK="yes" + # We also try running a hardened clang build with chutney on Linux. + - env: CHUTNEY="yes" SKIP_MAKE_CHECK="yes" CHUTNEY_ALLOW_FAILURES="2" + compiler: clang + # We clone our stem repo and run `make test-stem` + - env: TEST_STEM="yes" + ## Check rust online with distcheck, to make sure we remove rust products + - env: DISTCHECK="yes" RUST_OPTIONS="--enable-rust --enable-cargo-online-mode" + ## Check disable module dirauth with and without rust + - env: MODULES_OPTIONS="--disable-module-dirauth" RUST_OPTIONS="--enable-rust" TOR_RUST_DEPENDENCIES=true + - env: MODULES_OPTIONS="--disable-module-dirauth" + ## Check NSS + - env: NSS_OPTIONS="--enable-nss" ## Uncomment to allow the build to report success (with non-required ## sub-builds continuing to run) if all required sub-builds have @@@ -166,39 -145,25 +169,46 @@@ install - if [[ "$COVERAGE_OPTIONS" != "" ]]; then pip install --user cpp-coveralls; fi ## If we're on OSX, and using asciidoc, configure asciidoc - if [[ "$ASCIIDOC_OPTIONS" == "" ]] && [[ "$TRAVIS_OS_NAME" == "osx" ]]; then export XML_CATALOG_FILES="/usr/local/etc/xml/catalog"; fi + ## If we're using Rust, download rustup + - if [[ "$RUST_OPTIONS" != "" ]]; then curl -Ssf -o rustup.sh https://sh.rustup.rs; fi + ## Install the nightly channels of rustc and cargo and setup our toolchain environment + - if [[ "$RUST_OPTIONS" != "" ]]; then sh rustup.sh -y --default-toolchain nightly; fi + - if [[ "$RUST_OPTIONS" != "" ]]; then source $HOME/.cargo/env; fi + ## If we're testing rust builds in offline-mode, then set up our vendored dependencies + - if [[ "$TOR_RUST_DEPENDENCIES" == "true" ]]; then export TOR_RUST_DEPENDENCIES=$PWD/src/ext/rust/crates; fi + ## If we're running chutney, install it. + - if [[ "$CHUTNEY" != "" ]]; then git clone --depth 1 https://github.com/torproject/chutney.git ; export CHUTNEY_PATH="$(pwd)/chutney"; fi + - if [[ "$TEST_STEM" != "" ]]; then git clone --depth 1 https://github.com/torproject/stem.git ; export STEM_SOURCE_DIR=`pwd`/stem; fi + ## ## Finally, list installed package versions - if [[ "$TRAVIS_OS_NAME" == "linux" ]]; then dpkg-query --show; fi - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then brew list --versions; fi + ## Get some info about rustup, rustc and cargo + - if [[ "$RUST_OPTIONS" != "" ]]; then which rustup; fi + - if [[ "$RUST_OPTIONS" != "" ]]; then which rustc; fi + - if [[ "$RUST_OPTIONS" != "" ]]; then which cargo; fi + - if [[ "$RUST_OPTIONS" != "" ]]; then rustup --version; fi + - if [[ "$RUST_OPTIONS" != "" ]]; then rustc --version; fi + - if [[ "$RUST_OPTIONS" != "" ]]; then cargo --version; fi + ## Get python version + - python --version + ## If we're running chutney, show the chutney commit + - if [[ "$CHUTNEY" != "" ]]; then pushd "$CHUTNEY_PATH"; git log -1 ; popd ; fi + - if [[ "$TEST_STEM" != "" ]]; then pushd stem; python -c "from stem import stem; print(stem.__version__);"; git log -1; popd; fi script: + # Skip test_rebind on macOS + - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then export TOR_SKIP_TEST_REBIND=true; fi - ./autogen.sh - - CONFIGURE_FLAGS="$ASCIIDOC_OPTIONS $COVERAGE_OPTIONS $HARDENING_OPTIONS $OPENSSL_OPTIONS --enable-fatal-warnings --disable-silent-rules" + - CONFIGURE_FLAGS="$ASCIIDOC_OPTIONS $COVERAGE_OPTIONS $HARDENING_OPTIONS $MODULES_OPTIONS $NSS_OPTIONS $OPENSSL_OPTIONS $RUST_OPTIONS --enable-fatal-warnings --disable-silent-rules" - echo "Configure flags are $CONFIGURE_FLAGS" - ./configure $CONFIGURE_FLAGS ## We run `make check` because that's what https://jenkins.torproject.org does. - - if [[ "$DISTCHECK" == "" && "$TEST_STEM" == "" ]]; then make check; fi + - if [[ "$SKIP_MAKE_CHECK" == "" ]]; then make check; fi + - if [[ "$DISTCHECK" != "" ]]; then make distcheck DISTCHECK_CONFIGURE_FLAGS="$CONFIGURE_FLAGS"; fi + - if [[ "$CHUTNEY" != "" ]]; then make test-network-all; fi + ## Diagnostic for bug 29437: kill stem if it hangs for 15 minutes + - if [[ "$TEST_STEM" != "" ]]; then make src/app/tor; timelimit -p -t 540 -s USR1 -T 30 -S ABRT python3 "$STEM_SOURCE_DIR"/run_tests.py --tor src/app/tor --integ --log notice --target RUN_ALL; fi - - if [[ "$DISTCHECK" != "" && "$TEST_STEM" == "" ]]; then make distcheck DISTCHECK_CONFIGURE_FLAGS="$CONFIGURE_FLAGS"; fi ## If this build was one that produced coverage, upload it. - if [[ "$COVERAGE_OPTIONS" != "" ]]; then coveralls -b . --exclude src/test --exclude src/trunnel --gcov-options '\-p' || echo "Coverage failed"; fi @@@ -207,11 -172,10 +217,12 @@@ after_failure ## But the log is too long for travis' rendered view, so tail it. - tail -1000 config.log || echo "tail failed" ## `make check` will leave a log file with more details of test failures. - - if [[ "$DISTCHECK" == "" ]]; then cat test-suite.log || echo "cat failed"; fi + - if [[ "$SKIP_MAKE_CHECK" == "" ]]; then cat test-suite.log || echo "cat failed"; fi ## `make distcheck` puts it somewhere different. - if [[ "$DISTCHECK" != "" ]]; then make show-distdir-testlog || echo "make failed"; fi + - if [[ "$DISTCHECK" != "" ]]; then make show-distdir-core || echo "make failed"; fi + - if [[ "$CHUTNEY" != "" ]]; then ls test_network_log || echo "ls failed"; cat test_network_log/* || echo "cat failed"; fi + - if [[ "$TEST_STEM" != "" ]]; then cat "$STEM_SOURCE_DIR"/test/data/tor_log || echo "cat failed"; fi before_cache: ## Delete all gcov files.

1 0

[tor/maint-0.4.1] Travis: SKIP_MAKE_CHECK="yes" when running stem
by dgoulet＠torproject.org 23 Jul '19

23 Jul '19

commit 6bc8dedbed31abc40ca3017ac4f26a3373eac40d Author: teor <teor(a)torproject.org> Date: Mon Jul 1 14:33:54 2019 +1000 Travis: SKIP_MAKE_CHECK="yes" when running stem And add some comments about stem Part of 29280. --- .travis.yml | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/.travis.yml b/.travis.yml index 8493589f7..66d39f69c 100644 --- a/.travis.yml +++ b/.travis.yml @@ -53,7 +53,7 @@ matrix: - env: CHUTNEY="yes" SKIP_MAKE_CHECK="yes" CHUTNEY_ALLOW_FAILURES="2" compiler: clang # We clone our stem repo and run `make test-stem` - - env: TEST_STEM="yes" + - env: TEST_STEM="yes" SKIP_MAKE_CHECK="yes" ## Check rust online with distcheck, to make sure we remove rust products - env: DISTCHECK="yes" RUST_OPTIONS="--enable-rust --enable-cargo-online-mode" ## Check disable module dirauth with and without rust @@ -75,7 +75,7 @@ matrix: ## https://docs.travis-ci.com/user/customizing-the-build#matching-jobs-with-al… allow_failures: ## test-stem sometimes hangs on Travis - - env: TEST_STEM="yes" + - env: TEST_STEM="yes" SKIP_MAKE_CHECK="yes" exclude: ## gcc on OSX is less useful, because the default compiler is clang. @@ -178,6 +178,7 @@ install: - if [[ "$TOR_RUST_DEPENDENCIES" == "true" ]]; then export TOR_RUST_DEPENDENCIES=$PWD/src/ext/rust/crates; fi ## If we're running chutney, install it. - if [[ "$CHUTNEY" != "" ]]; then git clone --depth 1 https://github.com/torproject/chutney.git ; export CHUTNEY_PATH="$(pwd)/chutney"; fi + ## If we're running stem, install it. - if [[ "$TEST_STEM" != "" ]]; then git clone --depth 1 https://github.com/torproject/stem.git ; export STEM_SOURCE_DIR=`pwd`/stem; fi ## ## Finally, list installed package versions @@ -194,6 +195,7 @@ install: - python --version ## If we're running chutney, show the chutney commit - if [[ "$CHUTNEY" != "" ]]; then pushd "$CHUTNEY_PATH"; git log -1 ; popd ; fi + ## If we're running stem, show the stem version and commit - if [[ "$TEST_STEM" != "" ]]; then pushd stem; python -c "from stem import stem; print(stem.__version__);"; git log -1; popd; fi script:

1 0

[community/master] Merge branch 'relays' into 'master'
by hiro＠torproject.org 23 Jul '19

23 Jul '19

commit 6b6168afce194e4ff7f003e042671f9fca588890 Merge: b9ecab7 ab90ee0 Author: gus <gus(a)riseup.net> Date: Mon Jul 8 13:24:36 2019 -0700 Merge branch 'relays' into 'master' Relay-operations: Include Good ISPs, Swag page and fix broken links See merge request gus/communitytpo!2 .../community-resources/contents.lr | 12 +- .../community-resources/good-bad-isps/contents.lr | 324 +++++++++++++++++++-- .../community-resources/swags/contents.lr | 25 ++ .../tor-abuse-templates/contents.lr | 4 +- .../tor-exit-guidelines/contents.lr | 34 +-- .../tor-relay-universities/contents.lr | 5 +- .../technical-setup/centosrhel/contents.lr | 58 ---- .../relay-operations/technical-setup/contents.lr | 10 +- .../technical-setup/debianubuntu/contents.lr | 44 --- .../technical-setup/fedora/contents.lr | 37 --- .../technical-setup/freebsd/contents.lr | 75 ----- .../contents.lr | 52 ++-- 12 files changed, 389 insertions(+), 291 deletions(-)

1 0

[community/master] Merge branch 'relays' into 'master'
by hiro＠torproject.org 23 Jul '19

23 Jul '19

commit b9ecab71dbffba433c46f1c264235592be7b61df Merge: 95ba41c 9f08fa9 Author: gus <gus(a)riseup.net> Date: Fri Jul 5 07:50:48 2019 -0700 Merge branch 'relays' into 'master' Relay Operations: Technical Considerations, Technical Setup, Community Resources See merge request gus/communitytpo!1 .../community-resources/contents.lr | 58 ++- .../community-resources/good-bad-isps/contents.lr | 45 +++ .../tor-abuse-templates/contents.lr | 302 ++++++++++++++ .../tor-exit-guidelines/contents.lr | 117 ++++++ .../tor-relay-universities/contents.lr | 49 +++ content/relay-operations/contents.lr | 2 +- .../relays-requirements/contents.lr | 6 +- .../technical-considerations/contents.lr | 95 +++++ .../centos-rhel-opensuse/contents.lr | 101 +++++ .../bridge-deployment-guide/contents.lr | 23 ++ .../debian-ubuntu/contents.lr | 80 ++++ .../bridge-deployment-guide/freebsd/contents.lr | 98 +++++ .../bridge-deployment-guide/openbsd/contents.lr | 74 ++++ .../post-install/contents.lr | 22 + .../technical-setup/centosrhel/contents.lr | 45 +++ .../relay-operations/technical-setup/contents.lr | 444 +-------------------- .../technical-setup/exit-relay/contents.lr | 8 +- .../technical-setup/fedora/contents.lr | 26 ++ .../technical-setup/freebsd/contents.lr | 66 ++- .../middleguard-relay/centosrhel/contents.lr | 56 +++ .../technical-setup/middleguard-relay/contents.lr | 15 + .../middleguard-relay/debianubuntu/contents.lr | 46 +++ .../middleguard-relay/fedora/contents.lr | 37 ++ .../middleguard-relay/freebsd/contents.lr | 73 ++++ .../contents.lr | 175 ++++++++ .../relay-operations/types-of-relays/contents.lr | 5 +- 26 files changed, 1618 insertions(+), 450 deletions(-)

1 0

[community/master] Merge remote-tracking branch 'gus/master'
by hiro＠torproject.org 23 Jul '19

23 Jul '19

commit d48c8c055e13b3edc3f08273191db34aeb4e48e6 Merge: 3d74e69 6b6168a Author: hiro <hiro(a)torproject.org> Date: Tue Jul 23 14:04:09 2019 +0200 Merge remote-tracking branch 'gus/master'

1 0