September 2021 - tor-commits - lists.torproject.org

[tor/release-0.4.5] Only check for bindable ports if we are unsure if it will fail.
by dgoulet＠torproject.org 13 Sep '21

13 Sep '21

commit fcef8e3f75cf39ae7eb6195713d4855b4fc4d08c Author: Alexander Færøy <ahf(a)torproject.org> Date: Thu Feb 4 23:11:11 2021 +0000 Only check for bindable ports if we are unsure if it will fail. We currently assume that the only way for Tor to listen on ports in the privileged port range (1 to 1023), on Linux, is if we are granted the NET_BIND_SERVICE capability. Today on Linux, it's possible to specify the beginning of the unprivileged port range using a sysctl configuration option. Docker (and thus the CI service Tor uses) recently changed this sysctl value to 0, which causes our tests to fail as they assume that we should NOT be able to bind to a privileged port *without* the NET_BIND_SERVICE capability. In this patch, we read the value of the sysctl value via the /proc/sys/ filesystem iff it's present, otherwise we assume the default unprivileged port range begins at port 1024. See: tor#40275 --- src/test/test_switch_id.c | 60 ++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 57 insertions(+), 3 deletions(-) diff --git a/src/test/test_switch_id.c b/src/test/test_switch_id.c index baddf8d66e..91277e374f 100644 --- a/src/test/test_switch_id.c +++ b/src/test/test_switch_id.c @@ -31,7 +31,47 @@ static const struct { { NULL, 0 } }; +/* Returns the first port that we think we can bind to without special + * permissions. Usually this function returns 1024. */ +static uint16_t +unprivileged_port_range_start(void) +{ + uint16_t result = 1024; + +#if defined(__linux__) + char *content = NULL; + + content = read_file_to_str( + "/proc/sys/net/ipv4/ip_unprivileged_port_start", + 0, + NULL); + + if (content != NULL) { + int ok = 1; + uint16_t tmp_result; + + tmp_result = (uint16_t)tor_parse_long(content, 10, 0, 65535, &ok, NULL); + + if (ok) { + result = tmp_result; + } else { + fprintf(stderr, + "Unable to convert ip_unprivileged_port_start to integer: %s\n", + content); + } + } + + tor_free(content); +#endif /* defined(__linux__) */ + + return result; +} + #if !defined(_WIN32) + +#define PORT_TEST_RANGE_START 600 +#define PORT_TEST_RANGE_END 1024 + /* 0 on no, 1 on yes, -1 on failure. */ static int check_can_bind_low_ports(void) @@ -41,7 +81,7 @@ check_can_bind_low_ports(void) memset(&sin, 0, sizeof(sin)); sin.sin_family = AF_INET; - for (port = 600; port < 1024; ++port) { + for (port = PORT_TEST_RANGE_START; port < PORT_TEST_RANGE_END; ++port) { sin.sin_port = htons(port); tor_socket_t fd = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP); if (! SOCKET_OK(fd)) { @@ -149,10 +189,24 @@ main(int argc, char **argv) /* Succeed if we can do a setuid with capability retention, and doing so * does not make us lose the ability to bind low ports */ { - int keepcaps = (test_id == TEST_SETUID_KEEPCAPS); + const int keepcaps = (test_id == TEST_SETUID_KEEPCAPS); okay = switch_id(username, keepcaps ? SWITCH_ID_KEEP_BINDLOW : 0) == 0; + if (okay) { - okay = check_can_bind_low_ports() == keepcaps; + /* Only run this check if there are ports we may not be able to bind + * to. */ + const uint16_t min_port = unprivileged_port_range_start(); + + if (min_port >= PORT_TEST_RANGE_START && + min_port < PORT_TEST_RANGE_END) { + okay = check_can_bind_low_ports() == keepcaps; + } else { + fprintf(stderr, + "Skipping check for whether we can bind to any " + "privileged ports as the user system seems to " + "allow us to bind to ports even without any " + "capabilities set.\n"); + } } break; }

1 0

[tor/release-0.4.5] Force amd64 for CI builds.
by dgoulet＠torproject.org 13 Sep '21

13 Sep '21

commit 84d6f977e72aa71c19c65c9a8fa3ff922854510b Author: Alexander Færøy <ahf(a)torproject.org> Date: Mon Sep 13 18:05:58 2021 +0200 Force amd64 for CI builds. --- .gitlab-ci.yml | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 1e9d03caf5..915f242f26 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -37,6 +37,12 @@ variables: paths: - artifacts/ + +# This template is used for x86-64 builds. +.x86-64-template: &x86-64-template + tags: + - amd64 + # This template should be usable on any system that's based on apt. .apt-template: &apt-template | export LC_ALL=C.UTF-8 @@ -57,6 +63,7 @@ variables: # This template sets us up for Debian system in particular. .debian-template: &debian-template <<: *artifacts-template + <<: *x86-64-template variables: DEBIAN_FRONTEND: "noninteractive" # TODO: Using "cache" in this way speeds up our downloads. It would be

1 0

[tor/release-0.4.6] Merge branch 'maint-0.4.5' into maint-0.4.6
by dgoulet＠torproject.org 13 Sep '21

13 Sep '21

commit f728e09ebe611d6858e721eaa37637025bfbf259 Merge: 2be49cdbe5 0dbb2f53b9 Author: David Goulet <dgoulet(a)torproject.org> Date: Mon Sep 13 13:34:48 2021 -0400 Merge branch 'maint-0.4.5' into maint-0.4.6 .gitlab-ci.yml | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)

1 0

[tor/release-0.4.6] Merge branch 'maint-0.3.5' into maint-0.4.5
by dgoulet＠torproject.org 13 Sep '21

13 Sep '21

commit 0dbb2f53b933cb089ed18d72717bbf10938ffa76 Merge: da9ff3936d fcef8e3f75 Author: David Goulet <dgoulet(a)torproject.org> Date: Mon Sep 13 13:34:44 2021 -0400 Merge branch 'maint-0.3.5' into maint-0.4.5 .gitlab-ci.yml | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)

1 0

[tor/release-0.4.6] Only check for bindable ports if we are unsure if it will fail.
by dgoulet＠torproject.org 13 Sep '21

13 Sep '21

commit fcef8e3f75cf39ae7eb6195713d4855b4fc4d08c Author: Alexander Færøy <ahf(a)torproject.org> Date: Thu Feb 4 23:11:11 2021 +0000 Only check for bindable ports if we are unsure if it will fail. We currently assume that the only way for Tor to listen on ports in the privileged port range (1 to 1023), on Linux, is if we are granted the NET_BIND_SERVICE capability. Today on Linux, it's possible to specify the beginning of the unprivileged port range using a sysctl configuration option. Docker (and thus the CI service Tor uses) recently changed this sysctl value to 0, which causes our tests to fail as they assume that we should NOT be able to bind to a privileged port *without* the NET_BIND_SERVICE capability. In this patch, we read the value of the sysctl value via the /proc/sys/ filesystem iff it's present, otherwise we assume the default unprivileged port range begins at port 1024. See: tor#40275 --- src/test/test_switch_id.c | 60 ++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 57 insertions(+), 3 deletions(-) diff --git a/src/test/test_switch_id.c b/src/test/test_switch_id.c index baddf8d66e..91277e374f 100644 --- a/src/test/test_switch_id.c +++ b/src/test/test_switch_id.c @@ -31,7 +31,47 @@ static const struct { { NULL, 0 } }; +/* Returns the first port that we think we can bind to without special + * permissions. Usually this function returns 1024. */ +static uint16_t +unprivileged_port_range_start(void) +{ + uint16_t result = 1024; + +#if defined(__linux__) + char *content = NULL; + + content = read_file_to_str( + "/proc/sys/net/ipv4/ip_unprivileged_port_start", + 0, + NULL); + + if (content != NULL) { + int ok = 1; + uint16_t tmp_result; + + tmp_result = (uint16_t)tor_parse_long(content, 10, 0, 65535, &ok, NULL); + + if (ok) { + result = tmp_result; + } else { + fprintf(stderr, + "Unable to convert ip_unprivileged_port_start to integer: %s\n", + content); + } + } + + tor_free(content); +#endif /* defined(__linux__) */ + + return result; +} + #if !defined(_WIN32) + +#define PORT_TEST_RANGE_START 600 +#define PORT_TEST_RANGE_END 1024 + /* 0 on no, 1 on yes, -1 on failure. */ static int check_can_bind_low_ports(void) @@ -41,7 +81,7 @@ check_can_bind_low_ports(void) memset(&sin, 0, sizeof(sin)); sin.sin_family = AF_INET; - for (port = 600; port < 1024; ++port) { + for (port = PORT_TEST_RANGE_START; port < PORT_TEST_RANGE_END; ++port) { sin.sin_port = htons(port); tor_socket_t fd = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP); if (! SOCKET_OK(fd)) { @@ -149,10 +189,24 @@ main(int argc, char **argv) /* Succeed if we can do a setuid with capability retention, and doing so * does not make us lose the ability to bind low ports */ { - int keepcaps = (test_id == TEST_SETUID_KEEPCAPS); + const int keepcaps = (test_id == TEST_SETUID_KEEPCAPS); okay = switch_id(username, keepcaps ? SWITCH_ID_KEEP_BINDLOW : 0) == 0; + if (okay) { - okay = check_can_bind_low_ports() == keepcaps; + /* Only run this check if there are ports we may not be able to bind + * to. */ + const uint16_t min_port = unprivileged_port_range_start(); + + if (min_port >= PORT_TEST_RANGE_START && + min_port < PORT_TEST_RANGE_END) { + okay = check_can_bind_low_ports() == keepcaps; + } else { + fprintf(stderr, + "Skipping check for whether we can bind to any " + "privileged ports as the user system seems to " + "allow us to bind to ports even without any " + "capabilities set.\n"); + } } break; }

1 0

[tor/release-0.4.6] Merge branch 'maint-0.4.6' into release-0.4.6
by dgoulet＠torproject.org 13 Sep '21

13 Sep '21

commit fb7c8325973d704ca84546a04c24244e50d46b6f Merge: 7456f6d777 f728e09ebe Author: David Goulet <dgoulet(a)torproject.org> Date: Mon Sep 13 13:34:48 2021 -0400 Merge branch 'maint-0.4.6' into release-0.4.6 .gitlab-ci.yml | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)

1 0

[community/master] cache is per job. also: separate test for different situations
by emmapeel＠torproject.org 13 Sep '21

13 Sep '21

commit eb777acae11247d8695fba8308eade237f77108d Author: emma peel <emma.peel(a)riseup.net> Date: Mon Sep 13 18:53:36 2021 +0200 cache is per job. also: separate test for different situations --- .gitlab-ci.yml | 96 +++++++++++++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 81 insertions(+), 15 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index bd05699..e753821 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -1,11 +1,5 @@ image: debian:buster-slim -cache: - key: $CI_PROJECT_PATH_SLUG.${CI_COMMIT_REF_SLUG} - paths: - - packages - - lego - - apt-cache # This template should be usable on any system that's based on apt. # taken from tor gitlabci @@ -27,26 +21,98 @@ cache: apt-get upgrade -qy variables: - GIT_SUBMODULE_STRATEGY: recursive + GIT_SUBMODULE_STRATEGY: recursive + PIP_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pip" + TRANSLATION_BRANCH: "communitytpo-contentspot" + + +stages: + - build + - test_l10n pages: + cache: + key: $CI_PROJECT_PATH_SLUG.${CI_COMMIT_REF_SLUG} + paths: + - packages + - lego + - apt-cache + - venv + - .cache/pip + - .cache/lektor/builds/ + stage: build script: - *apt-template - - DEBIAN_FRONTEND=noninteractive apt-get install gettext python3-babel python3-pip git python3-inifile python3-dev python3-setuptools python3-openssl python3-cryptography i18nspector -y + - DEBIAN_FRONTEND=noninteractive apt-get install gettext python3-babel python3-pip git python3-inifile python3-dev python3-setuptools python3-openssl python3-cryptography i18nspector apt-utils ca-certificates -y + - pip3 install virtualenv + - virtualenv venv + - source venv/bin/activate - pip3 install lektor - echo 'checking out translations' - rm -rf i18n - - git clone --branch communitytpo-contentspot https://git.torproject.org/translation.git i18n - - echo 'reinstalling plugins to have last version' + - git clone --branch $TRANSLATION_BRANCH https://git.torproject.org/translation.git i18n + - echo 'reinstall lektor plugins' + - lektor project-info --output-path - lektor plugins reinstall - - echo 'building lektor 3 times to get translations in place' + - echo 'building lektor 3 more times to get translations in place' - lektor build --output-path public && lektor build --output-path public && lektor build --output-path public - - echo 'lets see if there are any updates in the strings for translation' + artifacts: + paths: + - public + - i18n + rules: + - when: always + +check_new_strings: + cache: + key: $CI_PROJECT_PATH_SLUG.${CI_COMMIT_REF_SLUG} + paths: + - packages + - lego + - apt-cache + - venv + - .cache/pip + + stage: test_l10n + needs: [pages] + allow_failure: true + rules: + - if: '$CI_PIPELINE_SOURCE == "merge_request_event"' + - if: $CI_COMMIT_REF_NAME == $CI_DEFAULT_BRANCH + - if: '$CI_COMMIT_BRANCH == "translations"' + when: never + - changes: + - content/**/*.lr + - templates/**/* + script: + - *apt-template + - DEBIAN_FRONTEND=noninteractive apt-get install gettext git python3-dev python3-setuptools i18nspector python3-polib python3-requests ca-certificates apt-utils -y - git clone https://gitlab.torproject.org/tpo/community/l10n.git - - pip3 install polib - - l10n/bin/check_po_status.py communitytpo-contentspot - - l10n/bin/check_markdown_links.py i18n/ + - echo 'lets see if there are any updates in the strings for translation' + - l10n/bin/check_po_status.py $TRANSLATION_BRANCH artifacts: paths: - public - i18n + - l10n + +check_l10n: + allow_failure: true + cache: + key: $CI_PROJECT_PATH_SLUG.${CI_COMMIT_REF_SLUG} + paths: + - packages + - lego + - apt-cache + - venv + - i18n + - .cache/pip + stage: test_l10n + needs: [pages] + only: + - translations + script: + - DEBIAN_FRONTEND=noninteractive apt-get install gettext i18nspector python3-polib ca-certificates -y + - echo 'lets see if there are any broken links on the translations' + - l10n/bin/check_markdown_links.py i18n/ +

1 0

[torspec/main] Add error 451 for ONION_CLIENT_AUTH_ADD
by nickm＠torproject.org 13 Sep '21

13 Sep '21

commit 04eb311e87c61dba797459eb81329f1ed8817cf5 Author: Neel Chauhan <neel(a)neelc.org> Date: Sun Sep 12 11:17:08 2021 -0700 Add error 451 for ONION_CLIENT_AUTH_ADD --- control-spec.txt | 1 + 1 file changed, 1 insertion(+) diff --git a/control-spec.txt b/control-spec.txt index 616387c..25977b9 100644 --- a/control-spec.txt +++ b/control-spec.txt @@ -1935,6 +1935,7 @@ 251 - Client auth credentials for this onion service already existed and replaced. 252 - Added client auth credentials and successfully decrypted a cached descriptor. + 451 - We reached authorized client capacity 512 - Syntax error in "HSAddress", or "PrivateKeyBlob" or "Nickname" 551 - Client with with this "Nickname" already exists 552 - Unrecognized KeyType

1 0

[torspec/main] Merge remote-tracking branch 'tor-gitlab/mr/44'
by nickm＠torproject.org 13 Sep '21

13 Sep '21

commit 498023bd188be99435391a2b192519f9ce3a572c Merge: 363ea98 04eb311 Author: Nick Mathewson <nickm(a)torproject.org> Date: Mon Sep 13 09:42:47 2021 -0400 Merge remote-tracking branch 'tor-gitlab/mr/44' control-spec.txt | 1 + 1 file changed, 1 insertion(+)

1 0

[tor/main] Clear asciidoc-helper.sh syntax
by dgoulet＠torproject.org 13 Sep '21

13 Sep '21

commit eac7bdce91f1ab341d801cc2f317dd408ed85cb4 Author: skaluzka <skaluzka(a)protonmail.com> Date: Wed Sep 8 22:25:47 2021 +0200 Clear asciidoc-helper.sh syntax Remove all unnecessary ";" characters at the end of several lines. Align all indentations to 4 spaces. Update console messages related to XML_CATALOG_FILES and .bashrc file. Signed-off-by: skaluzka <skaluzka(a)protonmail.com> --- doc/asciidoc-helper.sh | 47 +++++++++++++++++++++++------------------------ 1 file changed, 23 insertions(+), 24 deletions(-) diff --git a/doc/asciidoc-helper.sh b/doc/asciidoc-helper.sh index 765850a125..edc9b5b0ea 100755 --- a/doc/asciidoc-helper.sh +++ b/doc/asciidoc-helper.sh @@ -9,7 +9,7 @@ set -e if [ $# != 3 ]; then - exit 1; + exit 1 fi output=$3 @@ -19,50 +19,49 @@ if [ "$1" = "html" ]; then base=${output%%.html.in} if [ "$2" != none ]; then - TZ=UTC "$2" -d manpage -o "$output" "$input"; + TZ=UTC "$2" -d manpage -o "$output" "$input" else - echo "=================================="; - echo; - echo "You need asciidoc installed to be able to build the manpage."; - echo "To build without manpages, use the --disable-asciidoc argument"; - echo "when calling configure."; - echo; - echo "=================================="; - exit 1; + echo "==================================" + echo + echo "You need asciidoc installed to be able to build the manpage." + echo "To build without manpages, use the --disable-asciidoc argument" + echo "when calling configure." + echo + echo "==================================" + exit 1 fi elif [ "$1" = "man" ]; then input=${output%%.1.in}.1.txt base=${output%%.1.in} if test "$2" = none; then - echo "=================================="; - echo; - echo "You need asciidoc installed to be able to build the manpage."; - echo "To build without manpages, use the --disable-asciidoc argument"; - echo "when calling configure."; - echo; - echo "=================================="; - exit 1; + echo "==================================" + echo + echo "You need asciidoc installed to be able to build the manpage." + echo "To build without manpages, use the --disable-asciidoc argument" + echo "when calling configure." + echo + echo "==================================" + exit 1 fi if "$2" -f manpage "$input"; then - mv "$base.1" "$output"; + mv "$base.1" "$output" else - cat<<EOF + cat<<EOF ================================== You need a working asciidoc installed to be able to build the manpage. a2x is installed, but for some reason it isn't working. Sometimes this happens because required docbook support files are missing. Please install docbook-xsl, docbook-xml, and xmlto (Debian) or -similar. If you use homebrew on Mac OS X, install the docbook formula +similar. If you use homebrew on Mac OS X, install the docbook formula and add "export XML_CATALOG_FILES=/usr/local/etc/xml/catalog" to your -.bashrc +.bashrc file. Alternatively, to build without manpages, use the --disable-asciidoc argument when calling configure. ================================== EOF - exit 1; + exit 1 fi fi -

1 0