August 2020 - tor-commits - lists.torproject.org

[tor/release-0.4.4] Changelog draft for 0.4.4.4-rc
by nickm＠torproject.org 12 Aug '20

12 Aug '20

commit 9b3e721d74fa6c40d1dabb678054de277d0e3f9a Author: Nick Mathewson <nickm(a)torproject.org> Date: Wed Aug 12 14:52:23 2020 -0400 Changelog draft for 0.4.4.4-rc --- ChangeLog | 56 +++++++++++++++++++++++++++++++++++++++++++++++++++++ changes/bug31036 | 3 --- changes/bug34086 | 3 --- changes/bug40072 | 4 ---- changes/bug40076 | 5 ----- changes/bug40083 | 5 ----- changes/bug40095 | 4 ---- changes/ticket33747 | 7 ------- changes/ticket40081 | 6 ------ changes/ticket6198 | 3 --- 10 files changed, 56 insertions(+), 40 deletions(-) diff --git a/ChangeLog b/ChangeLog index 65891fefa9..7ef8c543c3 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,59 @@ +Changes in version 0.4.4.4-rc - 2020-08-12 + Tor 0.4.4.4-rc is the first release candidate in its series. It fixes + several bugs in previous versions, including some that caused annoying + behavior for relay and bridge operators. + + o Minor features (security): + - Channels using obsolete versions of the Tor link protocol are no + longer allowed to circumvent address-canonicity checks. (This is + only a minor issue, since such channels have no way to set ed25519 + keys, and therefore should always be rejected for circuits that + specify ed25519 identities.) Closes ticket 40081. + + o Minor features (defense in depth): + - Wipe more data from connection address fields before returning + them to the memory heap. Closes ticket 6198. + + o Minor bugfixes (correctness, buffers): + - Fix a correctness bug that could cause an assertion failure if we + ever tried using the buf_move_all() function with an empty input + buffer. As far as we know, no released versions of Tor do this. + Fixes bug 40076; bugfix on 0.3.3.1-alpha. + + o Minor bugfixes (linux seccomp2 sandbox): + - Fix startup crash with seccomp sandbox enabled when tor tries to + open the data directory. Patch from Daniel Pinto. Fixes bug 40072; + bugfix on 0.4.4.3-alpha-dev. + + o Minor bugfixes (onion service v3): + - Remove a BUG() warning that could trigger in certain unlikely + edge-cases. Fixes bug 34086; bugfix on 0.3.2.1-alpha. + + o Minor bugfixes (rate limiting, bridges, pluggable transports): + - On a bridge, treat all connections from an ExtORPort as remote by + default for the purposes of rate-limiting. Previously, bridges + would treat the connection as local unless they explicitly + received a "USERADDR" command. ExtORPort connections still count + as local if there is a USERADDR command with an explicit local + address. Fixes bug 33747; bugfix on 0.2.5.1-alpha. + + o Minor bugfixes (relay, self-testing): + - When starting up as a relay, if we haven't been able to verify + that we're reachable, only launch reachability tests at most once + a minute. Previously, we had been launching tests up to once a + second, which was needlessly noisy. Fixes bug 40083; bugfix + on 0.2.8.1-alpha. + + o Minor bugfixes (testing): + - When running the subsystem order check, use the python binary + configured with the PYTHON environment variable. Fixes bug 40095; + bugfix on 0.4.4.1-alpha. + + o Minor bugfixes (windows): + - Fix a bug that prevented Tor from starting if its log file grew + above 2GB. Fixes bug 31036; bugfix on 0.2.1.8-alpha. + + Changes in version 0.4.4.3-alpha - 2020-07-27 Tor 0.4.4.3-alpha fixes several annoyances in previous versions, including one affecting NSS users, and several affecting the Linux diff --git a/changes/bug31036 b/changes/bug31036 deleted file mode 100644 index d9921dba43..0000000000 --- a/changes/bug31036 +++ /dev/null @@ -1,3 +0,0 @@ - o Minor bugfixes (windows): - - Fix a bug that prevented Tor from starting if its log file - grew above 2GB. Fixes bug 31036; bugfix on 0.2.1.8-alpha. diff --git a/changes/bug34086 b/changes/bug34086 deleted file mode 100644 index 245992f8f4..0000000000 --- a/changes/bug34086 +++ /dev/null @@ -1,3 +0,0 @@ - o Minor bugfixes (onion service v3): - - Remove a BUG() warning that could trigger in certain unlikely edge-cases. - Fixes bug 34086; bugfix on 0.3.2.1-alpha. diff --git a/changes/bug40072 b/changes/bug40072 deleted file mode 100644 index 2b82f3f18b..0000000000 --- a/changes/bug40072 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes (linux seccomp2 sandbox): - - Fix startup crash with seccomp sandbox enabled when tor tries to - open the data directory. Patch from Daniel Pinto. Fixes bug 40072; - bugfix on 0.4.4.3-alpha-dev. diff --git a/changes/bug40076 b/changes/bug40076 deleted file mode 100644 index 9ef5969ae8..0000000000 --- a/changes/bug40076 +++ /dev/null @@ -1,5 +0,0 @@ - o Minor bugfixes (correctness, buffers): - - Fix a correctness bug that could cause an assertion failure if we ever - tried using the buf_move_all() function with an empty input. - As far as we know, no released versions of Tor do this. - Fixes bug 40076; bugfix on 0.3.3.1-alpha. diff --git a/changes/bug40083 b/changes/bug40083 deleted file mode 100644 index db26017664..0000000000 --- a/changes/bug40083 +++ /dev/null @@ -1,5 +0,0 @@ - o Minor bugfixes (relay, self-testing): - - When starting up as a relay, if we haven't been able to verify that - we're reachable, only launch reachability tests at most once a minute. - Previously, we had been launching tests up to once a second, which - was needlessly noisy. Fixes bug 40083; bugfix on 0.2.8.1-alpha. diff --git a/changes/bug40095 b/changes/bug40095 deleted file mode 100644 index 5c4b3a2b7e..0000000000 --- a/changes/bug40095 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes (testing): - - When running the subsystem order check, use the python binary - configured with the PYTHON environment variable. Fixes bug 40095; - bugfix on 0.4.4.1-alpha. diff --git a/changes/ticket33747 b/changes/ticket33747 deleted file mode 100644 index 57c72e9d0a..0000000000 --- a/changes/ticket33747 +++ /dev/null @@ -1,7 +0,0 @@ - o Minor bugfixes (rate limiting, bridges, pluggable transports): - - On a bridge, treat all connections from an ExtORPort as remote - by default for the purposes of rate-limiting. Previously, - bridges would treat the connection as local unless they explicitly - received a "USERADDR" command. ExtORPort connections still - count as local if there is a USERADDR command with an explicit local - address. Fixes bug 33747; bugfix on 0.2.5.1-alpha. diff --git a/changes/ticket40081 b/changes/ticket40081 deleted file mode 100644 index 683ae33518..0000000000 --- a/changes/ticket40081 +++ /dev/null @@ -1,6 +0,0 @@ - o Minor features (security): - - Channels using obsolete versions of the Tor link protocol are no - longer allowed to circumvent address-canonicity checks. - (This is only a minor issue, since such channels have no way to - set ed25519 keys, and therefore should always be rejected.) - Closes ticket 40081. diff --git a/changes/ticket6198 b/changes/ticket6198 deleted file mode 100644 index 7f3fdf2fa7..0000000000 --- a/changes/ticket6198 +++ /dev/null @@ -1,3 +0,0 @@ - o Minor features (defense in depth): - - Wipe more data from connection address fields before returning them to - the memory heap. Closes ticket 6198.

1 0

[tor/release-0.4.4] small edits to 0444-rc changelog
by nickm＠torproject.org 12 Aug '20

12 Aug '20

commit a2e64b3e85da186eaaf87fff939cc1c5b36a2f5e Author: Nick Mathewson <nickm(a)torproject.org> Date: Wed Aug 12 19:53:27 2020 -0400 small edits to 0444-rc changelog --- ChangeLog | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/ChangeLog b/ChangeLog index 7ef8c543c3..4e584e8fad 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,4 +1,4 @@ -Changes in version 0.4.4.4-rc - 2020-08-12 +Changes in version 0.4.4.4-rc - 2020-08-13 Tor 0.4.4.4-rc is the first release candidate in its series. It fixes several bugs in previous versions, including some that caused annoying behavior for relay and bridge operators. @@ -45,7 +45,7 @@ Changes in version 0.4.4.4-rc - 2020-08-12 on 0.2.8.1-alpha. o Minor bugfixes (testing): - - When running the subsystem order check, use the python binary + - When running the subsystem order check, use the Python binary configured with the PYTHON environment variable. Fixes bug 40095; bugfix on 0.4.4.1-alpha.

1 0

[stem/master] Present a safer pkill command
by atagar＠torproject.org 12 Aug '20

12 Aug '20

commit daed7bca9f930e03a89ce8a4254e02a5310a0fc1 Author: Damian Johnson <atagar(a)torproject.org> Date: Wed Aug 12 15:41:46 2020 -0700 Present a safer pkill command pkill's '-x' flag picks a process via an exact match rather than a regex. Caught thanks to uokf... 17:16 < uokf> I'd like to make a suggestion to change https://stem.torproject.org/faq.html#how-do-i-reload-my-torrc 17:17 < uokf> it says pkill -sighup tor 17:17 < uokf> but it should say pkill -x -sighup tor 17:17 < uokf> pkill sends the signal to all matching pids 17:18 <@arma> so if you have a process called 'extractor' then it'll hit that one too? 17:18 <@arma> that seems like a good bug. can you open a ticket for stem? i think atagar likes his tickets in github but i'm not sure. --- docs/faq.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/faq.rst b/docs/faq.rst index a7333fc7..50c9786a 100644 --- a/docs/faq.rst +++ b/docs/faq.rst @@ -444,7 +444,7 @@ Tor is configured through its `torrc you need to either restart Tor or issue a **HUP** for the changes to be reflected. To issue a HUP you can either... - * Run **pkill -sighup tor**. + * Run **pkill -x -sighup tor**. * Send Tor a **HUP** signal through its control port... ::

1 0

[translation/support-portal] https://gitweb.torproject.org/translation.git/commit/?h=support-portal
by translation＠torproject.org 12 Aug '20

12 Aug '20

commit cc219f7e1201432e9c140402d2e081f8efe12e3e Author: Translation commit bot <translation(a)torproject.org> Date: Wed Aug 12 21:48:09 2020 +0000 https://gitweb.torproject.org/translation.git/commit/?h=support-portal --- contents+ar.po | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/contents+ar.po b/contents+ar.po index 3d6ed391d7..331f1521d6 100644 --- a/contents+ar.po +++ b/contents+ar.po @@ -8901,6 +8901,12 @@ msgid "" "[Facebook](https://www.facebook.com/notes/protect-the-graph/making-" "connections-to-facebook-more-secure/1526085754298237/)." msgstr "" +"يتم الاعتماد أيضًا على خدمات Onion للدردشة الخالية من البيانات الوصفية " +"ومشاركة الملفات ، والتفاعل الآمن بين الصحفيين ومصادرهم مثل [SecureDrop] " +"(https://securedrop.org/) أو [OnionShare] (https://onionshare.org/ ) " +"وتحديثات البرامج الأكثر أمانًا وطرق أكثر أمانًا للوصول إلى مواقع الويب " +"الشهيرة مثل [Facebook] (https://www.facebook.com/notes/protect-the-graph" +"/making-connections-to-facebook-more-secure/1526085754298237 /)." #: https//support.torproject.org/onionservices/onionservices-2/ #: (content/onionservices/onionservices-2/contents+en.lrquestion.description)

1 0

[translation/support-portal] https://gitweb.torproject.org/translation.git/commit/?h=support-portal
by translation＠torproject.org 12 Aug '20

12 Aug '20

commit 4e269312f72252b3e52ba382828bdf73533885f1 Author: Translation commit bot <translation(a)torproject.org> Date: Wed Aug 12 18:48:11 2020 +0000 https://gitweb.torproject.org/translation.git/commit/?h=support-portal --- contents+ar.po | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/contents+ar.po b/contents+ar.po index d829b7fa39..3d6ed391d7 100644 --- a/contents+ar.po +++ b/contents+ar.po @@ -8850,6 +8850,8 @@ msgid "" "Websites that are only accessible over Tor are called \"onions\" and end in " "the TLD .onion." msgstr "" +"تسمى مواقع الويب التي يمكن الوصول إليها عبر Tor \"onions\" وتنتهي بـTLD " +".onion." #: https//support.torproject.org/onionservices/onionservices-1/ #: (content/onionservices/onionservices-1/contents+en.lrquestion.description) @@ -8857,11 +8859,13 @@ msgid "" "For example, the DuckDuckGo onion is " "[https://3g2upl4pq6kufc4m.onion](https://3g2upl4pq6kufc4m.onion)." msgstr "" +"على سبيل المثال ، onion DuckDuckGo هو [https://3g2upl4pq6kufc4m.onion] " +"(https://3g2upl4pq6kufc4m.onion)" #: https//support.torproject.org/onionservices/onionservices-1/ #: (content/onionservices/onionservices-1/contents+en.lrquestion.description) msgid "You can access these websites by using Tor Browser." -msgstr "" +msgstr "يمكنك الوصول إلى هذه المواقع باستخدام متصفح Tor." #: https//support.torproject.org/onionservices/onionservices-1/ #: (content/onionservices/onionservices-1/contents+en.lrquestion.description) @@ -8869,6 +8873,8 @@ msgid "" "The addresses must be shared with you by the website host, as onions are not" " indexed in search engines in the typical way that vanilla websites are." msgstr "" +"يجب مشاركة العناوين معك من قبل مضيف الموقع ، حيث لا تتم فهرسة onion في " +"محركات البحث بالطريقة المعتادة مثل مواقع الفانيليا." #: https//support.torproject.org/onionservices/onionservices-2/ #: (content/onionservices/onionservices-2/contents+en.lrquestion.title) @@ -8881,6 +8887,8 @@ msgid "" "Onion services allow people to browse but also to publish anonymously, " "including publishing anonymous websites." msgstr "" +"تسمح خدمات Onion للأشخاص بالتصفح ولكن أيضًا للنشر دون الكشف عن هويتهم ، بما " +"في ذلك نشر مواقع الويب المجهولة." #: https//support.torproject.org/onionservices/onionservices-2/ #: (content/onionservices/onionservices-2/contents+en.lrquestion.description)

1 0

[tor/master] Add support for patterns on %include #25140
by nickm＠torproject.org 12 Aug '20

12 Aug '20

commit 34fa2c4d0d8117b75c5c52a7c825486eb0284ae0 Author: Daniel Pinto <danielpinto52(a)gmail.com> Date: Wed Jun 3 22:09:42 2020 +0100 Add support for patterns on %include #25140 Also adds generic tor_glob function to expand globs. --- configure.ac | 5 +- doc/man/tor.1.txt | 12 +- scripts/codegen/fuzzing_include_am.py | 2 +- src/app/include.am | 4 +- src/config/torrc.sample.in | 13 +- src/lib/fs/conffile.c | 141 +++++++---- src/lib/fs/files.c | 16 ++ src/lib/fs/files.h | 2 + src/lib/fs/path.c | 296 +++++++++++++++++++++++ src/lib/fs/path.h | 7 + src/lib/sandbox/sandbox.c | 2 + src/lib/string/util_string.c | 9 + src/lib/string/util_string.h | 1 + src/test/fuzz/include.am | 2 +- src/test/include.am | 21 +- src/test/test_config.c | 346 ++++++++++++++++++++++++++- src/test/test_helpers.c | 83 +++++++ src/test/test_helpers.h | 2 + src/test/test_util.c | 436 ++++++++++++++++++++++++++++++++++ src/tools/include.am | 8 +- 20 files changed, 1330 insertions(+), 78 deletions(-) diff --git a/configure.ac b/configure.ac index 3076f2f1ff..b53df59148 100644 --- a/configure.ac +++ b/configure.ac @@ -866,6 +866,7 @@ dnl Where do you live, libevent? And how do we call you? if test "$bwin32" = "true"; then TOR_LIB_WS32=-lws2_32 TOR_LIB_IPHLPAPI=-liphlpapi + TOR_LIB_SHLWAPI=-lshlwapi # Some of the cargo-cults recommend -lwsock32 as well, but I don't # think it's actually necessary. TOR_LIB_GDI=-lgdi32 @@ -878,6 +879,7 @@ fi AC_SUBST(TOR_LIB_WS32) AC_SUBST(TOR_LIB_GDI) AC_SUBST(TOR_LIB_IPHLPAPI) +AC_SUBST(TOR_LIB_SHLWAPI) AC_SUBST(TOR_LIB_USERENV) tor_libevent_pkg_redhat="libevent" @@ -1646,7 +1648,8 @@ AC_CHECK_HEADERS([errno.h \ sys/utime.h \ sys/wait.h \ syslog.h \ - utime.h]) + utime.h \ + glob.h]) AC_CHECK_HEADERS(sys/param.h) diff --git a/doc/man/tor.1.txt b/doc/man/tor.1.txt index ca54fa125b..bb01315d46 100644 --- a/doc/man/tor.1.txt +++ b/doc/man/tor.1.txt @@ -205,14 +205,22 @@ backslash character (\) before the end of the line. Comments can be used in such multiline entries, but they must start at the beginning of a line. Configuration options can be imported from files or folders using the %include -option with the value being a path. If the path is a file, the options from the +option with the value being a path. This path can have wildcards. Wildcards are +expanded first, using lexical order. Then, for each matching file or folder, the +following rules are followed: if the path is a file, the options from the file will be parsed as if they were written where the %include option is. If the path is a folder, all files on that folder will be parsed following lexical -order. Files starting with a dot are ignored. Files on subfolders are ignored. +order. Files starting with a dot are ignored. Files in subfolders are ignored. The %include option can be used recursively. New configuration files or directories cannot be added to already running Tor instance if **Sandbox** is enabled. +The supported wildcards are * meaning any number of characters including none +and ? meaning exactly one character. These characters can be escaped by preceding +them with a backslash, except on Windows. Files starting with a dot are not matched +when expanding wildcards unless the starting dot is explicitly in the pattern, except +on Windows. + By default, an option on the command line overrides an option found in the configuration file, and an option in a configuration file overrides one in the defaults file. diff --git a/scripts/codegen/fuzzing_include_am.py b/scripts/codegen/fuzzing_include_am.py index ae50563074..b3892b6fd3 100755 --- a/scripts/codegen/fuzzing_include_am.py +++ b/scripts/codegen/fuzzing_include_am.py @@ -35,7 +35,7 @@ FUZZING_LIBS = \ $(rust_ldadd) \ @TOR_ZLIB_LIBS@ @TOR_LIB_MATH@ \ @TOR_LIBEVENT_LIBS@ $(TOR_LIBS_CRYPTLIB) \ - @TOR_LIB_WS32@ @TOR_LIB_IPHLPAPI@ @TOR_LIB_GDI@ @TOR_LIB_USERENV@ @CURVE25519_LIBS@ \ + @TOR_LIB_WS32@ @TOR_LIB_IPHLPAPI@ @TOR_LIB_SHLWAPI@ @TOR_LIB_GDI@ @TOR_LIB_USERENV@ @CURVE25519_LIBS@ \ @TOR_SYSTEMD_LIBS@ \ @TOR_LZMA_LIBS@ \ @TOR_ZSTD_LIBS@ diff --git a/src/app/include.am b/src/app/include.am index 3caa0bab1c..8488a1bf19 100644 --- a/src/app/include.am +++ b/src/app/include.am @@ -18,7 +18,7 @@ src_app_tor_LDFLAGS = @TOR_LDFLAGS_zlib@ $(TOR_LDFLAGS_CRYPTLIB) @TOR_LDFLAGS_li src_app_tor_LDADD = $(TOR_INTERNAL_LIBS) \ $(rust_ldadd) \ @TOR_ZLIB_LIBS@ @TOR_LIB_MATH@ @TOR_LIBEVENT_LIBS@ $(TOR_LIBS_CRYPTLIB) \ - @TOR_LIB_WS32@ @TOR_LIB_IPHLPAPI@ @TOR_LIB_GDI@ @TOR_LIB_USERENV@ \ + @TOR_LIB_WS32@ @TOR_LIB_IPHLPAPI@ @TOR_LIB_SHLWAPI@ @TOR_LIB_GDI@ @TOR_LIB_USERENV@ \ @CURVE25519_LIBS@ @TOR_SYSTEMD_LIBS@ \ @TOR_LZMA_LIBS@ @TOR_ZSTD_LIBS@ @TOR_TRACE_LIBS@ @@ -29,7 +29,7 @@ src_app_tor_cov_CFLAGS = $(AM_CFLAGS) $(TEST_CFLAGS) src_app_tor_cov_LDFLAGS = @TOR_LDFLAGS_zlib@ $(TOR_LDFLAGS_CRYPTLIB) @TOR_LDFLAGS_libevent@ src_app_tor_cov_LDADD = $(TOR_INTERNAL_TESTING_LIBS) \ @TOR_ZLIB_LIBS@ @TOR_LIB_MATH@ @TOR_LIBEVENT_LIBS@ $(TOR_LIBS_CRYPTLIB) \ - @TOR_LIB_WS32@ @TOR_LIB_IPHLPAPI@ @TOR_LIB_GDI@ \ + @TOR_LIB_WS32@ @TOR_LIB_IPHLPAPI@ @TOR_LIB_SHLWAPI@ @TOR_LIB_GDI@ \ @CURVE25519_LIBS@ @TOR_SYSTEMD_LIBS@ \ @TOR_LZMA_LIBS@ @TOR_ZSTD_LIBS@ @TOR_TRACE_LIBS@ endif diff --git a/src/config/torrc.sample.in b/src/config/torrc.sample.in index 51e1c3af4b..0690cbb931 100644 --- a/src/config/torrc.sample.in +++ b/src/config/torrc.sample.in @@ -242,11 +242,12 @@ #PublishServerDescriptor 0 ## Configuration options can be imported from files or folders using the %include -## option with the value being a path. If the path is a file, the options from the -## file will be parsed as if they were written where the %include option is. If -## the path is a folder, all files on that folder will be parsed following lexical -## order. Files starting with a dot are ignored. Files on subfolders are ignored. +## option with the value being a path. This path can have wildcards. Wildcards are +## expanded first, using lexical order. Then, for each matching file or folder, the following +## rules are followed: if the path is a file, the options from the file will be parsed as if +## they were written where the %include option is. If the path is a folder, all files on that +## folder will be parsed following lexical order. Files starting with a dot are ignored. Files +## on subfolders are ignored. ## The %include option can be used recursively. -#%include /etc/torrc.d/ -#%include /etc/torrc.custom +#%include /etc/torrc.d/*.conf diff --git a/src/lib/fs/conffile.c b/src/lib/fs/conffile.c index 9583093c12..f1f6d8ae5f 100644 --- a/src/lib/fs/conffile.c +++ b/src/lib/fs/conffile.c @@ -21,6 +21,8 @@ #include "lib/malloc/malloc.h" #include "lib/string/printf.h" +#include <stdbool.h> + static smartlist_t *config_get_file_list(const char *path, smartlist_t *opened_files); static int config_get_included_config(const char *path, int recursion_level, @@ -50,62 +52,109 @@ config_get_lines_include(const char *string, config_line_t **result, opened_lst, 1, NULL, config_process_include); } -/** Adds a list of configuration files present on path to - * file_list. path can be a file or a directory. If it is a file, - * only that file will be added to file_list. If it is a directory, - * all paths for files on that directory root (no recursion) except for files - * whose name starts with a dot will be added to file_list. - * opened_files will have a list of files opened by this function - * if provided. Return 0 on success, -1 on failure. Ignores empty files. - */ +/** Returns a list of paths obtained when expading globs in pattern. If + * pattern has no globs, returns a list with pattern if it is an + * existing path or NULL otherwise. If opened_files is provided, adds + * paths opened by glob to it. Returns NULL on failure. */ static smartlist_t * -config_get_file_list(const char *path, smartlist_t *opened_files) +expand_glob(const char *pattern, smartlist_t *opened_files) { - smartlist_t *file_list = smartlist_new(); + smartlist_t *matches = tor_glob(pattern); + if (!matches) { + return NULL; + } - if (opened_files) { - smartlist_add_strdup(opened_files, path); + // if it is not a glob, return error when the path is missing + if (!has_glob(pattern) && smartlist_len(matches) == 0) { + smartlist_free(matches); + return NULL; } - file_status_t file_type = file_status(path); - if (file_type == FN_FILE) { - smartlist_add_strdup(file_list, path); - return file_list; - } else if (file_type == FN_DIR) { - smartlist_t *all_files = tor_listdir(path); - if (!all_files) { - smartlist_free(file_list); + if (opened_files) { + smartlist_t *glob_opened = get_glob_opened_files(pattern); + if (!glob_opened) { + SMARTLIST_FOREACH(matches, char *, f, tor_free(f)); + smartlist_free(matches); return NULL; } - smartlist_sort_strings(all_files); - SMARTLIST_FOREACH_BEGIN(all_files, char *, f) { - if (f[0] == '.') { - tor_free(f); - continue; - } + smartlist_add_all(opened_files, glob_opened); + smartlist_free(glob_opened); + } + smartlist_sort_strings(matches); + return matches; +} + +/** Returns a list of configuration files present on paths that match + * pattern. The pattern is expanded and then all the paths are + * processed. A path can be a file or a directory. If it is a file, that file + * will be added to the list to be returned. If it is a directory, + * all paths for files on that directory root (no recursion) except for files + * whose name starts with a dot will be added to the list to be returned. + * opened_files will have a list of files opened by this function + * if provided. Return NULL on failure. Ignores empty files. + */ +static smartlist_t * +config_get_file_list(const char *pattern, smartlist_t *opened_files) +{ + smartlist_t *glob_matches = expand_glob(pattern, opened_files); + if (!glob_matches) { + return NULL; + } - char *fullname; - tor_asprintf(&fullname, "%s"PATH_SEPARATOR"%s", path, f); - tor_free(f); + bool error_found = false; + smartlist_t *file_list = smartlist_new(); + SMARTLIST_FOREACH_BEGIN(glob_matches, char *, path) { + if (opened_files) { + smartlist_add_strdup(opened_files, path); + } - if (opened_files) { - smartlist_add_strdup(opened_files, fullname); + file_status_t file_type = file_status(path); + if (file_type == FN_FILE) { + smartlist_add_strdup(file_list, path); + } else if (file_type == FN_DIR) { + smartlist_t *all_files = tor_listdir(path); + if (!all_files) { + error_found = true; + break; } - - if (file_status(fullname) != FN_FILE) { - tor_free(fullname); + smartlist_sort_strings(all_files); + SMARTLIST_FOREACH_BEGIN(all_files, char *, f) { + if (f[0] == '.') { + continue; + } + + char *fullname; + tor_asprintf(&fullname, "%s"PATH_SEPARATOR"%s", path, f); + + if (opened_files) { + smartlist_add_strdup(opened_files, fullname); + } + + if (file_status(fullname) != FN_FILE) { + tor_free(fullname); + continue; + } + smartlist_add(file_list, fullname); + } SMARTLIST_FOREACH_END(f); + SMARTLIST_FOREACH(all_files, char *, f, tor_free(f)); + smartlist_free(all_files); + } else if (file_type == FN_EMPTY) { continue; - } - smartlist_add(file_list, fullname); - } SMARTLIST_FOREACH_END(f); - smartlist_free(all_files); - return file_list; - } else if (file_type == FN_EMPTY) { - return file_list; - } else { + } else { + error_found = true; + break; + } + } SMARTLIST_FOREACH_END(path); + SMARTLIST_FOREACH(glob_matches, char *, f, tor_free(f)); + smartlist_free(glob_matches); + + if (error_found) { + SMARTLIST_FOREACH(file_list, char *, f, tor_free(f)); smartlist_free(file_list); - return NULL; + file_list = NULL; } + + return file_list; } /** Creates a list of config lines present on included path. @@ -133,19 +182,19 @@ config_get_included_config(const char *path, int recursion_level, int extended, return 0; } -/** Process an %include path in a config file. Set list to the +/** Process an %include pattern in a config file. Set list to the * list of configuration settings obtained and list_last to the last * element of the same list. opened_lst will have a list of opened * files if provided. Return 0 on success, -1 on failure. */ static int -config_process_include(const char *path, int recursion_level, int extended, +config_process_include(const char *pattern, int recursion_level, int extended, config_line_t **list, config_line_t **list_last, smartlist_t *opened_lst) { config_line_t *ret_list = NULL; config_line_t **next = &ret_list; - smartlist_t *config_files = config_get_file_list(path, opened_lst); + smartlist_t *config_files = config_get_file_list(pattern, opened_lst); if (!config_files) { return -1; } diff --git a/src/lib/fs/files.c b/src/lib/fs/files.c index a0b5a40aac..b4a432701f 100644 --- a/src/lib/fs/files.c +++ b/src/lib/fs/files.c @@ -247,6 +247,22 @@ file_status(const char *fname) } } +/** Returns true if file_type represents an existing file (even if + * empty). Returns false otherwise. */ +bool +is_file(file_status_t file_type) +{ + return file_type != FN_ERROR && file_type != FN_NOENT && file_type != FN_DIR; +} + +/** Returns true if file_type represents an existing directory. Returns + * false otherwise. */ +bool +is_dir(file_status_t file_type) +{ + return file_type == FN_DIR; +} + /** Create a file named fname with the contents str. Overwrite * the previous fname if possible. Return 0 on success, -1 on failure. * diff --git a/src/lib/fs/files.h b/src/lib/fs/files.h index a109cd6248..6eeba85e89 100644 --- a/src/lib/fs/files.h +++ b/src/lib/fs/files.h @@ -55,6 +55,8 @@ MOCK_DECL(int,tor_unlink,(const char *pathname)); typedef enum { FN_ERROR, FN_NOENT, FN_FILE, FN_DIR, FN_EMPTY } file_status_t; file_status_t file_status(const char *filename); +bool is_file(file_status_t file_type); +bool is_dir(file_status_t file_type); int64_t tor_get_avail_disk_space(const char *path); diff --git a/src/lib/fs/path.c b/src/lib/fs/path.c index 0d57be4b06..4532bfb7d1 100644 --- a/src/lib/fs/path.c +++ b/src/lib/fs/path.c @@ -13,18 +13,44 @@ #include "lib/malloc/malloc.h" #include "lib/log/log.h" #include "lib/log/util_bug.h" +#include "lib/container/smartlist.h" +#include "lib/sandbox/sandbox.h" #include "lib/string/printf.h" #include "lib/string/util_string.h" #include "lib/string/compat_ctype.h" +#include "lib/string/compat_string.h" +#include "lib/fs/files.h" +#include "lib/fs/dir.h" #include "lib/fs/userdb.h" +#ifdef HAVE_SYS_TYPES_H +#include <sys/types.h> +#endif +#ifdef HAVE_SYS_STAT_H +#include <sys/stat.h> +#endif #ifdef HAVE_UNISTD_H #include <unistd.h> #endif +#ifdef _WIN32 +#include <windows.h> +#include <shlwapi.h> +#else /* !(defined(_WIN32)) */ +#include <dirent.h> +#include <glob.h> +#endif /* defined(_WIN32) */ + #include <errno.h> #include <string.h> +#ifdef _WIN32 +#define IS_GLOB_CHAR(s,i) (((s)[(i)]) == '*' || ((s)[(i)]) == '?') +#else +#define IS_GLOB_CHAR(s,i) ((((s)[(i)]) == '*' || ((s)[(i)]) == '?') &&\ + ((i) == 0 || (s)[(i)-1] != '\\')) /* check escape */ +#endif + /** Removes enclosing quotes from path and unescapes quotes between the * enclosing quotes. Backslashes are not unescaped. Return the unquoted * path on success or 0 if path is not quoted correctly. */ @@ -294,3 +320,273 @@ make_path_absolute(const char *fname) return absfname; #endif /* defined(_WIN32) */ } + +/** Expands globs in pattern for the path fragment between + * prev_sep and next_sep. Returns NULL on failure. */ +typedef struct smartlist_t * unglob_fn(const char *pattern, int prev_sep, + int next_sep); + +/** Adds path to result if it exists and is a file type we can + * handle. Returns false if path is a file type we cannot handle, + * returns true otherwise. Used on tor_glob for WIN32. */ +static bool +add_non_glob_path(const char *path, struct smartlist_t *result) +{ + file_status_t file_type = file_status(path); + if (file_type == FN_ERROR) { + return false; + } else if (file_type != FN_NOENT) { + char *to_add = tor_strdup(path); + clean_fname_for_stat(to_add); + smartlist_add(result, to_add); + } + /* If WIN32 tor_glob is called with a non-existing path, we want it to + * return an empty list instead of error to match the regular version */ + return true; +} + +/** Auxiliary function used by get_glob_opened_files and WIN32 tor_glob. + * Returns a list of paths obtained from pattern using unglob to + * expand each path fragment. If final is true, the paths are the result + * of the glob expansion of pattern (implements tor_glob). Otherwise, + * the paths are the paths opened by glob while expanding pattern + * (implements get_glb_opened_files). Returns NULL on failure. */ +static struct smartlist_t * +get_glob_paths(const char *pattern, unglob_fn unglob, bool final) +{ + smartlist_t *result = smartlist_new(); + int i, prev_sep = -1, next_sep = -1; + bool is_glob = false, error_found = false, is_sep = false, is_last = false; + + // find first path fragment with globs + for (i = 0; pattern[i]; i++) { + is_glob = is_glob || IS_GLOB_CHAR(pattern, i); + is_last = !pattern[i+1]; + is_sep = pattern[i] == *PATH_SEPARATOR || pattern[i] == '/'; + if (is_sep || is_last) { + prev_sep = next_sep; + next_sep = i; // next_sep+1 is start of next fragment or end of string + if (is_glob) { + break; + } + } + } + + if (!is_glob) { // pattern fully expanded or no glob in pattern + if (final && !add_non_glob_path(pattern, result)) { + error_found = true; + goto end; + } + return result; + } + + if (!final) { + // add path before the glob to result + int len = prev_sep < 1 ? prev_sep + 1 : prev_sep; // handle /* + char *path_until_glob = tor_strndup(pattern, len); + smartlist_add(result, path_until_glob); + } + + smartlist_t *unglobbed_paths = unglob(pattern, prev_sep, next_sep); + if (!unglobbed_paths) { + error_found = true; + } else { + // for each path for current fragment, add the rest of the pattern + // and call recursively to get all expanded paths + SMARTLIST_FOREACH_BEGIN(unglobbed_paths, char *, current_path) { + char *next_path; + tor_asprintf(&next_path, "%s"PATH_SEPARATOR"%s", current_path, + &pattern[next_sep+1]); + smartlist_t *opened_next = get_glob_paths(next_path, unglob, final); + tor_free(next_path); + if (!opened_next) { + error_found = true; + break; + } + smartlist_add_all(result, opened_next); + smartlist_free(opened_next); + } SMARTLIST_FOREACH_END(current_path); + SMARTLIST_FOREACH(unglobbed_paths, char *, p, tor_free(p)); + smartlist_free(unglobbed_paths); + } + +end: + if (error_found) { + SMARTLIST_FOREACH(result, char *, p, tor_free(p)); + smartlist_free(result); + result = NULL; + } + return result; +} + +#ifdef _WIN32 +/** Expands globs in pattern for the path fragment between + * prev_sep and next_sep using the WIN32 API. Returns NULL on + * failure. Used by the WIN32 implementation of tor_glob. */ +static struct smartlist_t * +unglob_win32(const char *pattern, int prev_sep, int next_sep) +{ + smartlist_t *result = smartlist_new(); + int len = prev_sep < 1 ? prev_sep + 1 : prev_sep; // handle /* + char *path_until_glob = tor_strndup(pattern, len); + + if (!is_file(file_status(path_until_glob))) { + smartlist_t *filenames = tor_listdir(path_until_glob); + if (!filenames) { + smartlist_free(result); + result = NULL; + } else { + SMARTLIST_FOREACH_BEGIN(filenames, char *, filename) { + TCHAR tpattern[MAX_PATH] = {0}; + TCHAR tfile[MAX_PATH] = {0}; + char *full_path; + tor_asprintf(&full_path, "%s"PATH_SEPARATOR"%s", + path_until_glob, filename); + char *path_curr_glob = tor_strndup(pattern, next_sep + 1); + // *\ must return only dirs, remove \ from the pattern so it matches + if (is_dir(file_status(full_path))) { + clean_fname_for_stat(path_curr_glob); + } +#ifdef UNICODE + mbstowcs(tpattern, path_curr_glob, MAX_PATH); + mbstowcs(tfile, full_path, MAX_PATH); +#else + strlcpy(tpattern, path_curr_glob, MAX_PATH); + strlcpy(tfile, full_path, MAX_PATH); +#endif + if (PathMatchSpec(tfile, tpattern)) { + smartlist_add(result, full_path); + } else { + tor_free(full_path); + } + tor_free(path_curr_glob); + } SMARTLIST_FOREACH_END(filename); + SMARTLIST_FOREACH(filenames, char *, p, tor_free(p)); + smartlist_free(filenames); + } + } + tor_free(path_until_glob); + return result; +} +#else /* !defined(_WIN32) */ +/** Same as opendir but calls sandbox_intern_string before */ +static DIR * +prot_opendir(const char *name) +{ + return opendir(sandbox_intern_string(name)); +} + +/** Same as stat but calls sandbox_intern_string before */ +static int +prot_stat(const char *pathname, struct stat *buf) +{ + return stat(sandbox_intern_string(pathname), buf); +} + +/** Same as lstat but calls sandbox_intern_string before */ +static int +prot_lstat(const char *pathname, struct stat *buf) +{ + return lstat(sandbox_intern_string(pathname), buf); +} +#endif /* defined(_WIN32) */ + +/** Return a new list containing the paths that match the pattern + * pattern. Return NULL on error. + */ +struct smartlist_t * +tor_glob(const char *pattern) +{ + smartlist_t *result; +#ifdef _WIN32 + // PathMatchSpec does not support forward slashes, change them to backslashes + char *pattern_normalized = tor_strdup(pattern); + tor_strreplacechar(pattern_normalized, '/', *PATH_SEPARATOR); + result = get_glob_paths(pattern_normalized, unglob_win32, true); + tor_free(pattern_normalized); +#else /* !(defined(_WIN32)) */ + glob_t matches; + int flags = GLOB_ERR | GLOB_NOSORT; +#ifdef GLOB_ALTDIRFUNC + /* use functions that call sandbox_intern_string */ + flags |= GLOB_ALTDIRFUNC; + typedef void *(*gl_opendir)(const char * name); + typedef struct dirent *(*gl_readdir)(void *); + typedef void (*gl_closedir)(void *); + matches.gl_opendir = (gl_opendir) &prot_opendir; + matches.gl_readdir = (gl_readdir) &readdir; + matches.gl_closedir = (gl_closedir) &closedir; + matches.gl_stat = &prot_stat; + matches.gl_lstat = &prot_lstat; +#endif /* defined(GLOB_ALTDIRFUNC) */ + int ret = glob(pattern, flags, NULL, &matches); + if (ret == GLOB_NOMATCH) { + return smartlist_new(); + } else if (ret != 0) { + return NULL; + } + + result = smartlist_new(); + size_t i; + for (i = 0; i < matches.gl_pathc; i++) { + char *match = tor_strdup(matches.gl_pathv[i]); + size_t len = strlen(match); + if (len > 0 && match[len-1] == *PATH_SEPARATOR) { + match[len-1] = '\0'; + } + smartlist_add(result, match); + } + globfree(&matches); +#endif /* defined(_WIN32) */ + return result; +} + +/** Returns true if s contains characters that can be globbed. + * Returns false otherwise. */ +bool +has_glob(const char *s) +{ + int i; + for (i = 0; s[i]; i++) { + if (IS_GLOB_CHAR(s, i)) { + return true; + } + } + return false; +} + +/** Expands globs in pattern for the path fragment between + * prev_sep and next_sep using tor_glob. Returns NULL on + * failure. Used by get_glob_opened_files. */ +static struct smartlist_t * +unglob_opened_files(const char *pattern, int prev_sep, int next_sep) +{ + (void)prev_sep; + smartlist_t *result = smartlist_new(); + // if the following fragments have no globs, we're done + if (has_glob(&pattern[next_sep+1])) { + // if there is a glob after next_sep, we know it is a separator and not the + // last char and glob_path will have the path without the separator + char *glob_path = tor_strndup(pattern, next_sep); + smartlist_t *child_paths = tor_glob(glob_path); + tor_free(glob_path); + if (!child_paths) { + smartlist_free(result); + result = NULL; + } else { + smartlist_add_all(result, child_paths); + smartlist_free(child_paths); + } + } + return result; +} + +/** Returns a list of files that are opened by the tor_glob function when + * called with pattern. Returns NULL on error. The purpose of this + * function is to create a list of files to be added to the sandbox white list + * before the sandbox is enabled. */ +struct smartlist_t * +get_glob_opened_files(const char *pattern) +{ + return get_glob_paths(pattern, unglob_opened_files, false); +} diff --git a/src/lib/fs/path.h b/src/lib/fs/path.h index f0e253c556..425bd12516 100644 --- a/src/lib/fs/path.h +++ b/src/lib/fs/path.h @@ -12,6 +12,10 @@ #ifndef TOR_PATH_H #define TOR_PATH_H +#include <stdbool.h> +#ifdef _WIN32 +#include <windows.h> +#endif #include "lib/cc/compat_compiler.h" #ifdef _WIN32 @@ -26,5 +30,8 @@ int path_is_relative(const char *filename); void clean_fname_for_stat(char *name); int get_parent_directory(char *fname); char *make_path_absolute(const char *fname); +struct smartlist_t *tor_glob(const char *pattern); +bool has_glob(const char *s); +struct smartlist_t *get_glob_opened_files(const char *pattern); #endif /* !defined(TOR_PATH_H) */ diff --git a/src/lib/sandbox/sandbox.c b/src/lib/sandbox/sandbox.c index 820e4fd1a5..81240bcfdb 100644 --- a/src/lib/sandbox/sandbox.c +++ b/src/lib/sandbox/sandbox.c @@ -200,6 +200,8 @@ static int filter_nopar_gen[] = { #ifdef __NR__llseek SCMP_SYS(_llseek), #endif + // glob uses this.. + SCMP_SYS(lstat), SCMP_SYS(mkdir), SCMP_SYS(mlockall), #ifdef __NR_mmap diff --git a/src/lib/string/util_string.c b/src/lib/string/util_string.c index c8f12d780e..ba5f9f2203 100644 --- a/src/lib/string/util_string.c +++ b/src/lib/string/util_string.c @@ -143,6 +143,15 @@ tor_strupper(char *s) } } +/** Replaces old with replacement in s */ +void +tor_strreplacechar(char *s, char find, char replacement) +{ + for (s = strchr(s, find); s; s = strchr(s + 1, find)) { + *s = replacement; + } +} + /** Return 1 if every character in s is printable, else return 0. */ int diff --git a/src/lib/string/util_string.h b/src/lib/string/util_string.h index e89233df88..15d35415fe 100644 --- a/src/lib/string/util_string.h +++ b/src/lib/string/util_string.h @@ -31,6 +31,7 @@ int tor_digest256_is_zero(const char *digest); #define HEX_CHARACTERS "0123456789ABCDEFabcdef" void tor_strlower(char *s); void tor_strupper(char *s); +void tor_strreplacechar(char *s, char find, char replacement); int tor_strisprint(const char *s); int tor_strisnonupper(const char *s); int tor_strisspace(const char *s); diff --git a/src/test/fuzz/include.am b/src/test/fuzz/include.am index f3f7202ce2..510ff35a3c 100644 --- a/src/test/fuzz/include.am +++ b/src/test/fuzz/include.am @@ -11,7 +11,7 @@ FUZZING_LIBS = \ $(rust_ldadd) \ @TOR_ZLIB_LIBS@ @TOR_LIB_MATH@ \ @TOR_LIBEVENT_LIBS@ $(TOR_LIBS_CRYPTLIB) \ - @TOR_LIB_WS32@ @TOR_LIB_IPHLPAPI@ @TOR_LIB_GDI@ @TOR_LIB_USERENV@ @CURVE25519_LIBS@ \ + @TOR_LIB_WS32@ @TOR_LIB_IPHLPAPI@ @TOR_LIB_SHLWAPI@ @TOR_LIB_GDI@ @TOR_LIB_USERENV@ @CURVE25519_LIBS@ \ @TOR_SYSTEMD_LIBS@ \ @TOR_LZMA_LIBS@ \ @TOR_ZSTD_LIBS@ @TOR_TRACE_LIBS@ diff --git a/src/test/include.am b/src/test/include.am index d7be1a5f77..c049053438 100644 --- a/src/test/include.am +++ b/src/test/include.am @@ -296,16 +296,15 @@ src_test_test_switch_id_LDADD = \ $(TOR_UTIL_TESTING_LIBS) \ $(rust_ldadd) \ @TOR_ZLIB_LIBS@ @TOR_LIB_MATH@ \ - @TOR_LIB_WS32@ @TOR_LIB_IPHLPAPI@ @TOR_LIB_USERENV@ \ + @TOR_LIB_WS32@ @TOR_LIB_IPHLPAPI@ @TOR_LIB_SHLWAPI@ @TOR_LIB_USERENV@ \ @TOR_LZMA_LIBS@ @TOR_ZSTD_LIBS@ @TOR_TRACE_LIBS@ - src_test_test_LDFLAGS = @TOR_LDFLAGS_zlib@ $(TOR_LDFLAGS_CRYPTLIB) \ @TOR_LDFLAGS_libevent@ src_test_test_LDADD = \ $(TOR_INTERNAL_TESTING_LIBS) \ $(rust_ldadd) \ @TOR_ZLIB_LIBS@ @TOR_LIB_MATH@ @TOR_LIBEVENT_LIBS@ \ - $(TOR_LIBS_CRYPTLIB) @TOR_LIB_WS32@ @TOR_LIB_IPHLPAPI@ @TOR_LIB_GDI@ @TOR_LIB_USERENV@ \ + $(TOR_LIBS_CRYPTLIB) @TOR_LIB_WS32@ @TOR_LIB_IPHLPAPI@ @TOR_LIB_SHLWAPI@ @TOR_LIB_GDI@ @TOR_LIB_USERENV@ \ @CURVE25519_LIBS@ \ @TOR_SYSTEMD_LIBS@ @TOR_LZMA_LIBS@ @TOR_ZSTD_LIBS@ @TOR_TRACE_LIBS@ @@ -334,7 +333,7 @@ src_test_bench_LDADD = \ $(TOR_INTERNAL_LIBS) \ $(rust_ldadd) \ @TOR_ZLIB_LIBS@ @TOR_LIB_MATH@ @TOR_LIBEVENT_LIBS@ \ - $(TOR_LIBS_CRYPTLIB) @TOR_LIB_WS32@ @TOR_LIB_IPHLPAPI@ @TOR_LIB_GDI@ @TOR_LIB_USERENV@ \ + $(TOR_LIBS_CRYPTLIB) @TOR_LIB_WS32@ @TOR_LIB_IPHLPAPI@ @TOR_LIB_SHLWAPI@ @TOR_LIB_GDI@ @TOR_LIB_USERENV@ \ @CURVE25519_LIBS@ \ @TOR_SYSTEMD_LIBS@ @TOR_LZMA_LIBS@ @TOR_ZSTD_LIBS@ @TOR_TRACE_LIBS@ @@ -344,7 +343,7 @@ src_test_test_workqueue_LDADD = \ $(TOR_INTERNAL_TESTING_LIBS) \ $(rust_ldadd) \ @TOR_ZLIB_LIBS@ @TOR_LIB_MATH@ @TOR_LIBEVENT_LIBS@ \ - $(TOR_LIBS_CRYPTLIB) @TOR_LIB_WS32@ @TOR_LIB_IPHLPAPI@ @TOR_LIB_GDI@ @TOR_LIB_USERENV@ \ + $(TOR_LIBS_CRYPTLIB) @TOR_LIB_WS32@ @TOR_LIB_IPHLPAPI@ @TOR_LIB_SHLWAPI@ @TOR_LIB_GDI@ @TOR_LIB_USERENV@ \ @CURVE25519_LIBS@ \ @TOR_LZMA_LIBS@ @TOR_ZSTD_LIBS@ @TOR_TRACE_LIBS@ @@ -356,7 +355,7 @@ src_test_test_timers_LDADD = \ $(TOR_UTIL_TESTING_LIBS) \ $(rust_ldadd) \ @TOR_ZLIB_LIBS@ @TOR_LIB_MATH@ @TOR_LIBEVENT_LIBS@ \ - $(TOR_LIBS_CRYPTLIB) @TOR_LIB_WS32@ @TOR_LIB_IPHLPAPI@ @TOR_LIB_GDI@ @TOR_LIB_USERENV@ \ + $(TOR_LIBS_CRYPTLIB) @TOR_LIB_WS32@ @TOR_LIB_IPHLPAPI@ @TOR_LIB_SHLWAPI@ @TOR_LIB_GDI@ @TOR_LIB_USERENV@ \ @CURVE25519_LIBS@ \ @TOR_LZMA_LIBS@ @TOR_TRACE_LIBS@ src_test_test_timers_LDFLAGS = $(src_test_test_LDFLAGS) @@ -393,7 +392,7 @@ src_test_test_ntor_cl_LDADD = \ $(TOR_INTERNAL_LIBS) \ $(rust_ldadd) \ @TOR_ZLIB_LIBS@ @TOR_LIB_MATH@ \ - $(TOR_LIBS_CRYPTLIB) @TOR_LIB_WS32@ @TOR_LIB_IPHLPAPI@ @TOR_LIB_GDI@ @TOR_LIB_USERENV@ \ + $(TOR_LIBS_CRYPTLIB) @TOR_LIB_WS32@ @TOR_LIB_IPHLPAPI@ @TOR_LIB_SHLWAPI@ @TOR_LIB_GDI@ @TOR_LIB_USERENV@ \ @CURVE25519_LIBS@ @TOR_LZMA_LIBS@ @TOR_TRACE_LIBS@ src_test_test_ntor_cl_AM_CPPFLAGS = \ $(AM_CPPFLAGS) @@ -403,8 +402,8 @@ src_test_test_hs_ntor_cl_LDFLAGS = @TOR_LDFLAGS_zlib@ $(TOR_LDFLAGS_CRYPTLIB) src_test_test_hs_ntor_cl_LDADD = \ $(TOR_INTERNAL_LIBS) \ @TOR_ZLIB_LIBS@ @TOR_LIB_MATH@ \ - $(TOR_LIBS_CRYPTLIB) @TOR_LIB_WS32@ @TOR_LIB_IPHLPAPI@ @TOR_LIB_GDI@ \ - @CURVE25519_LIBS@ @TOR_TRACE_LIBS@ + $(TOR_LIBS_CRYPTLIB) @TOR_LIB_WS32@ @TOR_LIB_IPHLPAPI@ @TOR_LIB_SHLWAPI@ @TOR_LIB_GDI@ \ + @CURVE25519_LIBS@ @TOR_TRACE_LIBS@ src_test_test_hs_ntor_cl_AM_CPPFLAGS = \ $(AM_CPPFLAGS) @@ -416,8 +415,8 @@ src_test_test_bt_cl_LDADD = \ $(TOR_UTIL_TESTING_LIBS) \ $(rust_ldadd) \ @TOR_LIB_MATH@ \ - @TOR_LIB_WS32@ @TOR_LIB_IPHLPAPI@ @TOR_LIB_GDI@ @TOR_LIB_USERENV@ \ - @TOR_TRACE_LIBS@ + @TOR_LIB_WS32@ @TOR_LIB_IPHLPAPI@ @TOR_LIB_SHLWAPI@ @TOR_LIB_GDI@ @TOR_LIB_USERENV@ \ + @TOR_TRACE_LIBS@ src_test_test_bt_cl_CFLAGS = $(AM_CFLAGS) $(TEST_CFLAGS) src_test_test_bt_cl_CPPFLAGS= $(src_test_AM_CPPFLAGS) $(TEST_CPPFLAGS) endif diff --git a/src/test/test_config.c b/src/test/test_config.c index cdf668be49..617a52b8ac 100644 --- a/src/test/test_config.c +++ b/src/test/test_config.c @@ -5957,6 +5957,286 @@ test_config_include_flag_defaults_only(void *data) tor_free(dir); } +static void +test_config_include_wildcards(void *data) +{ + (void)data; + + char *temp = NULL, *folder = NULL; + config_line_t *result = NULL; + char *dir = tor_strdup(get_fname("test_include_wildcards")); + tt_ptr_op(dir, OP_NE, NULL); + +#ifdef _WIN32 + tt_int_op(mkdir(dir), OP_EQ, 0); +#else + tt_int_op(mkdir(dir, 0700), OP_EQ, 0); +#endif + + tor_asprintf(&temp, "%s"PATH_SEPARATOR"%s", dir, "01_one.conf"); + tt_int_op(write_str_to_file(temp, "Test 1\n", 0), OP_EQ, 0); + tor_free(temp); + + tor_asprintf(&temp, "%s"PATH_SEPARATOR"%s", dir, "02_two.conf"); + tt_int_op(write_str_to_file(temp, "Test 2\n", 0), OP_EQ, 0); + tor_free(temp); + + tor_asprintf(&temp, "%s"PATH_SEPARATOR"%s", dir, "aa_three.conf"); + tt_int_op(write_str_to_file(temp, "Test 3\n", 0), OP_EQ, 0); + tor_free(temp); + + tor_asprintf(&temp, "%s"PATH_SEPARATOR"%s", dir, "foo"); + tt_int_op(write_str_to_file(temp, "Test 6\n", 0), OP_EQ, 0); + tor_free(temp); + + tor_asprintf(&folder, "%s"PATH_SEPARATOR"%s", dir, "folder"); + +#ifdef _WIN32 + tt_int_op(mkdir(folder), OP_EQ, 0); +#else + tt_int_op(mkdir(folder, 0700), OP_EQ, 0); +#endif + + tor_asprintf(&temp, "%s"PATH_SEPARATOR"%s", folder, "04_four.conf"); + tt_int_op(write_str_to_file(temp, "Test 4\n", 0), OP_EQ, 0); + tor_free(temp); + + tor_asprintf(&temp, "%s"PATH_SEPARATOR"%s", folder, "05_five.conf"); + tt_int_op(write_str_to_file(temp, "Test 5\n", 0), OP_EQ, 0); + tor_free(temp); + + char torrc_contents[1000]; + int include_used; + + // test pattern that matches no file + tor_snprintf(torrc_contents, sizeof(torrc_contents), + "%%include %s"PATH_SEPARATOR"not-exist*\n", + dir); + tt_int_op(config_get_lines_include(torrc_contents, &result, 0, &include_used, + NULL), OP_EQ, 0); + tt_ptr_op(result, OP_EQ, NULL); + tt_int_op(include_used, OP_EQ, 1); + config_free_lines(result); + +#ifndef _WIN32 + // test wildcard escaping + tor_snprintf(torrc_contents, sizeof(torrc_contents), + "%%include %s"PATH_SEPARATOR"\\*\n", + dir); + tt_int_op(config_get_lines_include(torrc_contents, &result, 0, &include_used, + NULL), OP_EQ, -1); + tt_ptr_op(result, OP_EQ, NULL); + tt_int_op(include_used, OP_EQ, 1); + config_free_lines(result); +#endif + + // test pattern *.conf + tor_snprintf(torrc_contents, sizeof(torrc_contents), + "%%include %s"PATH_SEPARATOR"*.conf\n", + dir); + tt_int_op(config_get_lines_include(torrc_contents, &result, 0, &include_used, + NULL), OP_EQ, 0); + tt_ptr_op(result, OP_NE, NULL); + tt_int_op(include_used, OP_EQ, 1); + + int len = 0; + config_line_t *next; + char expected[10]; + for (next = result; next != NULL; next = next->next) { + tor_snprintf(expected, sizeof(expected), "%d", len + 1); + tt_str_op(next->key, OP_EQ, "Test"); + tt_str_op(next->value, OP_EQ, expected); + len++; + } + tt_int_op(len, OP_EQ, 3); + config_free_lines(result); + + // test pattern that matches folder and files + tor_snprintf(torrc_contents, sizeof(torrc_contents), + "%%include %s"PATH_SEPARATOR"*\n", + dir); + tt_int_op(config_get_lines_include(torrc_contents, &result, 0, &include_used, + NULL), OP_EQ, 0); + tt_ptr_op(result, OP_NE, NULL); + tt_int_op(include_used, OP_EQ, 1); + + len = 0; + for (next = result; next != NULL; next = next->next) { + tor_snprintf(expected, sizeof(expected), "%d", len + 1); + tt_str_op(next->key, OP_EQ, "Test"); + tt_str_op(next->value, OP_EQ, expected); + len++; + } + tt_int_op(len, OP_EQ, 6); + config_free_lines(result); + + // test pattern ending in PATH_SEPARATOR, test linux path separator + tor_snprintf(torrc_contents, sizeof(torrc_contents), + "%%include %s/f*/\n", + dir); + tt_int_op(config_get_lines_include(torrc_contents, &result, 0, &include_used, + NULL), OP_EQ, 0); + tt_ptr_op(result, OP_NE, NULL); + tt_int_op(include_used, OP_EQ, 1); + + len = 0; + for (next = result; next != NULL; next = next->next) { + tor_snprintf(expected, sizeof(expected), "%d", len + 1 + 3); + tt_str_op(next->key, OP_EQ, "Test"); + tt_str_op(next->value, OP_EQ, expected); + len++; + } + tt_int_op(len, OP_EQ, 2); + config_free_lines(result); + + // test pattern with wildcards in folder and file + tor_snprintf(torrc_contents, sizeof(torrc_contents), + "%%include %s"PATH_SEPARATOR"*"PATH_SEPARATOR"*.conf\n", + dir); + tt_int_op(config_get_lines_include(torrc_contents, &result, 0, &include_used, + NULL), OP_EQ, 0); + tt_ptr_op(result, OP_NE, NULL); + tt_int_op(include_used, OP_EQ, 1); + + len = 0; + for (next = result; next != NULL; next = next->next) { + tor_snprintf(expected, sizeof(expected), "%d", len + 1 + 3); + tt_str_op(next->key, OP_EQ, "Test"); + tt_str_op(next->value, OP_EQ, expected); + len++; + } + tt_int_op(len, OP_EQ, 2); + config_free_lines(result); + + done: + config_free_lines(result); + tor_free(folder); + tor_free(temp); + tor_free(dir); +} + +static void +test_config_include_hidden(void *data) +{ + (void)data; + + char *temp = NULL, *folder = NULL; + config_line_t *result = NULL; + char *dir = tor_strdup(get_fname("test_include_hidden")); + tt_ptr_op(dir, OP_NE, NULL); + +#ifdef _WIN32 + tt_int_op(mkdir(dir), OP_EQ, 0); +#else + tt_int_op(mkdir(dir, 0700), OP_EQ, 0); +#endif + + tor_asprintf(&folder, "%s"PATH_SEPARATOR"%s", dir, ".dotdir"); + +#ifdef _WIN32 + tt_int_op(mkdir(folder), OP_EQ, 0); +#else + tt_int_op(mkdir(folder, 0700), OP_EQ, 0); +#endif + + tor_asprintf(&temp, "%s"PATH_SEPARATOR"%s", folder, ".dotfile"); + tt_int_op(write_str_to_file(temp, "Test 1\n", 0), OP_EQ, 0); + tor_free(temp); + + tor_asprintf(&temp, "%s"PATH_SEPARATOR"%s", folder, "file"); + tt_int_op(write_str_to_file(temp, "Test 2\n", 0), OP_EQ, 0); + tor_free(temp); + + char torrc_contents[1000]; + int include_used; + int len = 0; + config_line_t *next; + char expected[10]; + + // test wildcards do not expand to dot folders (except for windows) + tor_snprintf(torrc_contents, sizeof(torrc_contents), + "%%include %s"PATH_SEPARATOR"*\n", + dir); + tt_int_op(config_get_lines_include(torrc_contents, &result, 0, &include_used, + NULL), OP_EQ, 0); + tt_int_op(include_used, OP_EQ, 1); +#ifdef _WIN32 // wildcard expansion includes dot files on Windows + for (next = result; next != NULL; next = next->next) { + tor_snprintf(expected, sizeof(expected), "%d", len + 2); + tt_str_op(next->key, OP_EQ, "Test"); + tt_str_op(next->value, OP_EQ, expected); + len++; + } + tt_int_op(len, OP_EQ, 1); +#else + tt_ptr_op(result, OP_EQ, NULL); +#endif + config_free_lines(result); + + // test wildcards match hidden folders when explicitly in the pattern + tor_snprintf(torrc_contents, sizeof(torrc_contents), + "%%include %s"PATH_SEPARATOR".*\n", + dir); + tt_int_op(config_get_lines_include(torrc_contents, &result, 0, &include_used, + NULL), OP_EQ, 0); + tt_ptr_op(result, OP_NE, NULL); + tt_int_op(include_used, OP_EQ, 1); + + len = 0; + for (next = result; next != NULL; next = next->next) { + tor_snprintf(expected, sizeof(expected), "%d", len + 2); + tt_str_op(next->key, OP_EQ, "Test"); + tt_str_op(next->value, OP_EQ, expected); + len++; + } + tt_int_op(len, OP_EQ, 1); + config_free_lines(result); + + // test hidden dir when explicitly included + tor_snprintf(torrc_contents, sizeof(torrc_contents), + "%%include %s"PATH_SEPARATOR".dotdir\n", + dir); + tt_int_op(config_get_lines_include(torrc_contents, &result, 0, &include_used, + NULL), OP_EQ, 0); + tt_ptr_op(result, OP_NE, NULL); + tt_int_op(include_used, OP_EQ, 1); + + len = 0; + for (next = result; next != NULL; next = next->next) { + tor_snprintf(expected, sizeof(expected), "%d", len + 2); + tt_str_op(next->key, OP_EQ, "Test"); + tt_str_op(next->value, OP_EQ, expected); + len++; + } + tt_int_op(len, OP_EQ, 1); + config_free_lines(result); + + // test hidden file when explicitly included + tor_snprintf(torrc_contents, sizeof(torrc_contents), + "%%include %s"PATH_SEPARATOR".dotdir"PATH_SEPARATOR".dotfile\n", + dir); + tt_int_op(config_get_lines_include(torrc_contents, &result, 0, &include_used, + NULL), OP_EQ, 0); + tt_ptr_op(result, OP_NE, NULL); + tt_int_op(include_used, OP_EQ, 1); + + len = 0; + for (next = result; next != NULL; next = next->next) { + tor_snprintf(expected, sizeof(expected), "%d", len + 1); + tt_str_op(next->key, OP_EQ, "Test"); + tt_str_op(next->value, OP_EQ, expected); + len++; + } + tt_int_op(len, OP_EQ, 1); + config_free_lines(result); + + done: + config_free_lines(result); + tor_free(folder); + tor_free(temp); + tor_free(dir); +} + static void test_config_dup_and_filter(void *arg) { @@ -6086,7 +6366,7 @@ test_config_include_opened_file_list(void *data) smartlist_t *opened_files = smartlist_new(); char *torrcd = NULL; char *subfolder = NULL; - char *path = NULL; + char *in_subfolder = NULL; char *empty = NULL; char *file = NULL; char *dot = NULL; @@ -6115,9 +6395,9 @@ test_config_include_opened_file_list(void *data) tt_int_op(mkdir(subfolder, 0700), OP_EQ, 0); #endif - tor_asprintf(&path, "%s"PATH_SEPARATOR"%s", subfolder, + tor_asprintf(&in_subfolder, "%s"PATH_SEPARATOR"%s", subfolder, "01_file_in_subfolder"); - tt_int_op(write_str_to_file(path, "Test 1\n", 0), OP_EQ, 0); + tt_int_op(write_str_to_file(in_subfolder, "Test 1\n", 0), OP_EQ, 0); tor_asprintf(&empty, "%s"PATH_SEPARATOR"%s", torrcd, "empty"); tt_int_op(write_str_to_file(empty, "", 0), OP_EQ, 0); @@ -6148,13 +6428,69 @@ test_config_include_opened_file_list(void *data) // dot files are not opened as we ignore them when we get their name from // their parent folder + // test with wildcards + SMARTLIST_FOREACH(opened_files, char *, f, tor_free(f)); + smartlist_clear(opened_files); + config_free_lines(result); + tor_snprintf(torrc_contents, sizeof(torrc_contents), + "%%include %s"PATH_SEPARATOR"*\n", + torrcd); + tt_int_op(config_get_lines_include(torrc_contents, &result, 0, &include_used, + opened_files), OP_EQ, 0); + tt_ptr_op(result, OP_NE, NULL); + tt_int_op(include_used, OP_EQ, 1); + +#ifdef _WIN32 + tt_int_op(smartlist_len(opened_files), OP_EQ, 6); +#else + tt_int_op(smartlist_len(opened_files), OP_EQ, 5); +#endif + tt_int_op(smartlist_contains_string(opened_files, torrcd), OP_EQ, 1); + tt_int_op(smartlist_contains_string(opened_files, subfolder), OP_EQ, 1); + // * will match the subfolder inside torrc.d, so it will be included + tt_int_op(smartlist_contains_string(opened_files, in_subfolder), OP_EQ, 1); + tt_int_op(smartlist_contains_string(opened_files, empty), OP_EQ, 1); + tt_int_op(smartlist_contains_string(opened_files, file), OP_EQ, 1); +#ifdef _WIN32 + // * matches the dot file on Windows + tt_int_op(smartlist_contains_string(opened_files, dot), OP_EQ, 1); +#endif + + // test with wildcards in folder and file + SMARTLIST_FOREACH(opened_files, char *, f, tor_free(f)); + smartlist_clear(opened_files); + config_free_lines(result); + tor_snprintf(torrc_contents, sizeof(torrc_contents), + "%%include %s"PATH_SEPARATOR"*"PATH_SEPARATOR"*\n", + torrcd); + tt_int_op(config_get_lines_include(torrc_contents, &result, 0, &include_used, + opened_files), OP_EQ, 0); + tt_ptr_op(result, OP_NE, NULL); + tt_int_op(include_used, OP_EQ, 1); + +#ifdef _WIN32 + tt_int_op(smartlist_len(opened_files), OP_EQ, 6); +#else + tt_int_op(smartlist_len(opened_files), OP_EQ, 5); +#endif + tt_int_op(smartlist_contains_string(opened_files, torrcd), OP_EQ, 1); + tt_int_op(smartlist_contains_string(opened_files, subfolder), OP_EQ, 1); + tt_int_op(smartlist_contains_string(opened_files, in_subfolder), OP_EQ, 1); + // stat is called on the following files, so they count as opened + tt_int_op(smartlist_contains_string(opened_files, empty), OP_EQ, 1); + tt_int_op(smartlist_contains_string(opened_files, file), OP_EQ, 1); +#ifdef _WIN32 + // * matches the dot file on Windows + tt_int_op(smartlist_contains_string(opened_files, dot), OP_EQ, 1); +#endif + done: SMARTLIST_FOREACH(opened_files, char *, f, tor_free(f)); smartlist_free(opened_files); config_free_lines(result); tor_free(torrcd); tor_free(subfolder); - tor_free(path); + tor_free(in_subfolder); tor_free(empty); tor_free(file); tor_free(dot); @@ -6538,6 +6874,8 @@ struct testcase_t config_tests[] = { CONFIG_TEST(include_flag_both_without, TT_FORK), CONFIG_TEST(include_flag_torrc_only, TT_FORK), CONFIG_TEST(include_flag_defaults_only, TT_FORK), + CONFIG_TEST(include_wildcards, 0), + CONFIG_TEST(include_hidden, 0), CONFIG_TEST(dup_and_filter, 0), CONFIG_TEST(check_bridge_distribution_setting_not_a_bridge, TT_FORK), CONFIG_TEST(check_bridge_distribution_setting_valid, 0), diff --git a/src/test/test_helpers.c b/src/test/test_helpers.c index 14913b4b40..851946931c 100644 --- a/src/test/test_helpers.c +++ b/src/test/test_helpers.c @@ -47,6 +47,17 @@ #include "feature/nodelist/node_st.h" #include "feature/nodelist/routerlist_st.h" +#ifdef HAVE_SYS_STAT_H +#include <sys/stat.h> +#endif + +#ifdef _WIN32 +/* For mkdir() */ +#include <direct.h> +#else +#include <dirent.h> +#endif /* defined(_WIN32) */ + #include "test/test.h" #include "test/test_helpers.h" #include "test/test_connection.h" @@ -194,6 +205,78 @@ mock_tor_addr_lookup__fail_on_bad_addrs(const char *name, return tor_addr_lookup__real(name, family, out); } +static char * +create_directory(const char *parent_dir, const char *name) +{ + char *dir = NULL; + tor_asprintf(&dir, "%s"PATH_SEPARATOR"%s", parent_dir, name); +#ifdef _WIN32 + tt_int_op(mkdir(dir), OP_EQ, 0); +#else + tt_int_op(mkdir(dir, 0700), OP_EQ, 0); +#endif + return dir; + + done: + tor_free(dir); + return NULL; +} + +static char * +create_file(const char *parent_dir, const char *name, const char *contents) +{ + char *path = NULL; + tor_asprintf(&path, "%s"PATH_SEPARATOR"%s", parent_dir, name); + contents = contents == NULL ? "" : contents; + tt_int_op(write_str_to_file(path, contents, 0), OP_EQ, 0); + return path; + + done: + tor_free(path); + return NULL; +} + +int +create_test_directory_structure(const char *parent_dir) +{ + int ret = -1; + char *dir1 = NULL; + char *dir2 = NULL; + char *file1 = NULL; + char *file2 = NULL; + char *dot = NULL; + char *empty = NULL; + char *forbidden = NULL; + + dir1 = create_directory(parent_dir, "dir1"); + tt_assert(dir1); + dir2 = create_directory(parent_dir, "dir2"); + tt_assert(dir2); + file1 = create_file(parent_dir, "file1", "Test 1"); + tt_assert(file1); + file2 = create_file(parent_dir, "file2", "Test 2"); + tt_assert(file2); + dot = create_file(parent_dir, ".test-hidden", "Test ."); + tt_assert(dot); + empty = create_file(parent_dir, "empty", NULL); + tt_assert(empty); + forbidden = create_directory(parent_dir, "forbidden"); + tt_assert(forbidden); +#ifndef _WIN32 + tt_int_op(chmod(forbidden, 0), OP_EQ, 0); +#endif + ret = 0; + done: + tor_free(dir1); + tor_free(dir2); + tor_free(file1); + tor_free(file2); + tor_free(dot); + tor_free(empty); + tor_free(forbidden); + return ret; +} + /*********** Helper funcs for making new connections/streams *****************/ /* Helper for test_conn_get_connection() */ diff --git a/src/test/test_helpers.h b/src/test/test_helpers.h index 66007873d1..f02ecbb0ac 100644 --- a/src/test/test_helpers.h +++ b/src/test/test_helpers.h @@ -33,6 +33,8 @@ connection_t *test_conn_get_connection(uint8_t state, uint8_t type, uint8_t purpose); or_options_t *helper_parse_options(const char *conf); +int create_test_directory_structure(const char *parent_dir); + extern const char TEST_DESCRIPTORS[]; void *helper_setup_pubsub(const struct testcase_t *); diff --git a/src/test/test_util.c b/src/test/test_util.c index 2aee07a26a..4f54d45468 100644 --- a/src/test/test_util.c +++ b/src/test/test_util.c @@ -18,6 +18,7 @@ #include "lib/crypt_ops/crypto_rand.h" #include "lib/defs/time.h" #include "test/test.h" +#include "test/test_helpers.h" #include "lib/memarea/memarea.h" #include "lib/process/waitpid.h" #include "lib/process/process_win32.h" @@ -4132,6 +4133,31 @@ test_util_find_str_at_start_of_line(void *ptr) ; } +static void +test_util_tor_strreplacechar(void *ptr) +{ + (void)ptr; + char empty[] = ""; + char not_contain[] = "bbb"; + char contains[] = "bab"; + char contains_all[] = "aaa"; + + tor_strreplacechar(empty, 'a', 'b'); + tt_str_op(empty, OP_EQ, ""); + + tor_strreplacechar(not_contain, 'a', 'b'); + tt_str_op(not_contain, OP_EQ, "bbb"); + + tor_strreplacechar(contains, 'a', 'b'); + tt_str_op(contains, OP_EQ, "bbb"); + + tor_strreplacechar(contains_all, 'a', 'b'); + tt_str_op(contains_all, OP_EQ, "bbb"); + + done: + ; +} + static void test_util_string_is_C_identifier(void *ptr) { @@ -4358,6 +4384,413 @@ test_util_listdir(void *ptr) } } +static void +test_util_glob(void *ptr) +{ + (void)ptr; + + smartlist_t *results = NULL; + int r, i; + char *dir1 = NULL, *dir2 = NULL, *forbidden = NULL, *dirname = NULL; + char *expected = NULL, *pattern = NULL; + // used for cleanup + char *dir1_forbidden = NULL, *dir2_forbidden = NULL; + char *forbidden_forbidden = NULL; + + dirname = tor_strdup(get_fname("test_glob")); + tt_ptr_op(dirname, OP_NE, NULL); + +#ifdef _WIN32 + r = mkdir(dirname); +#else + r = mkdir(dirname, 0700); +#endif + if (r) { + fprintf(stderr, "Can't create directory %s:", dirname); + perror(""); + exit(1); + } + + tt_int_op(0, OP_EQ, create_test_directory_structure(dirname)); + tor_asprintf(&dir1, "%s"PATH_SEPARATOR"dir1", dirname); + tor_asprintf(&dir1_forbidden, + "%s"PATH_SEPARATOR"dir1"PATH_SEPARATOR"forbidden", dirname); + tt_int_op(0, OP_EQ, create_test_directory_structure(dir1)); + tor_asprintf(&dir2, "%s"PATH_SEPARATOR"dir2", dirname); + tor_asprintf(&dir2_forbidden, + "%s"PATH_SEPARATOR"dir2"PATH_SEPARATOR"forbidden", dirname); + tt_int_op(0, OP_EQ, create_test_directory_structure(dir2)); + tor_asprintf(&forbidden, "%s"PATH_SEPARATOR"forbidden", dirname); + tor_asprintf(&forbidden_forbidden, + "%s"PATH_SEPARATOR"forbidden"PATH_SEPARATOR"forbidden",dirname); +#ifndef _WIN32 + chmod(forbidden, 0700); +#endif + tt_int_op(0, OP_EQ, create_test_directory_structure(forbidden)); +#ifndef _WIN32 + chmod(forbidden, 0); +#endif + +#define TEST(input) \ + do { \ + tor_asprintf(&pattern, "%s"PATH_SEPARATOR"%s", dirname, input); \ + results = tor_glob(pattern); \ + tor_free(pattern); \ + tt_assert(results); \ + smartlist_sort_strings(results); \ + } while (0); + +#define EXPECT(result) \ + do { \ + tt_int_op(smartlist_len(results), OP_EQ, \ + sizeof(result)/sizeof(*result)); \ + i = 0; \ + SMARTLIST_FOREACH_BEGIN(results, const char *, f) { \ + tor_asprintf(&expected, "%s"PATH_SEPARATOR"%s", dirname, result[i]); \ + tt_str_op(f, OP_EQ, expected); \ + i++; \ + tor_free(expected); \ + } SMARTLIST_FOREACH_END(f); \ + SMARTLIST_FOREACH(results, char *, f, tor_free(f)); \ + smartlist_free(results); \ + } while (0); + +#define EXPECT_EMPTY() \ + do { \ + tt_int_op(smartlist_len(results), OP_EQ, 0); \ + SMARTLIST_FOREACH(results, char *, f, tor_free(f)); \ + smartlist_free(results); \ + } while (0); + + // wilcards at beginning + const char *results_test1[] = {"dir2", "file2"}; + TEST("*2"); + EXPECT(results_test1); + + // wildcards at end + const char *results_test2[] = {"dir1", "dir2"}; + TEST("d*"); + EXPECT(results_test2); + + // wildcards at beginning and end +#ifdef _WIN32 + // dot files are not ignored on Windows + const char *results_test3[] = {".test-hidden", "dir1", "dir2", "file1", + "file2", "forbidden"}; +#else + const char *results_test3[] = {"dir1", "dir2", "file1", "file2", + "forbidden"}; +#endif + TEST("*i*"); + EXPECT(results_test3); + + // wildcards in middle + const char *results_test4[] = {"dir1", "dir2"}; + TEST("d?r*"); + EXPECT(results_test4); + + // test file that does not exist + TEST("not-exist"); + EXPECT_EMPTY(); + + // test wildcard that matches nothing + TEST("*not-exist*"); + EXPECT_EMPTY(); + + // test path separator at end - no wildcards + const char *results_test7[] = {"dir1"}; + TEST("dir1"); + EXPECT(results_test7); + + const char *results_test8[] = {"dir1"}; + TEST("dir1"PATH_SEPARATOR); + EXPECT(results_test8); + + const char *results_test9[] = {"file1"}; + TEST("file1"); + EXPECT(results_test9); + +#if defined(__APPLE__) || defined(__darwin__) || \ + defined(__FreeBSD__) || defined(__NetBSD__) || defined(OpenBSD) + TEST("file1"PATH_SEPARATOR); + EXPECT_EMPTY(); +#else + const char *results_test10[] = {"file1"}; + TEST("file1"PATH_SEPARATOR); + EXPECT(results_test10); +#endif + + // test path separator at end - with wildcards and linux path separator + const char *results_test11[] = {"dir1", "dir2", "forbidden"}; + TEST("*/"); + EXPECT(results_test11); + +#ifdef _WIN32 + // dot files are not ignored on Windows + const char *results_test12[] = {".test-hidden", "dir1", "dir2", "empty", + "file1", "file2", "forbidden"}; +#else + const char *results_test12[] = {"dir1", "dir2", "empty", "file1", "file2", + "forbidden"}; +#endif + TEST("*"); + EXPECT(results_test12); + + // wildcards on folder and file and linux path separator + const char *results_test13[] = {"dir1"PATH_SEPARATOR"dir1", + "dir1"PATH_SEPARATOR"dir2", + "dir1"PATH_SEPARATOR"file1", + "dir1"PATH_SEPARATOR"file2", + "dir2"PATH_SEPARATOR"dir1", + "dir2"PATH_SEPARATOR"dir2", + "dir2"PATH_SEPARATOR"file1", + "dir2"PATH_SEPARATOR"file2"}; + TEST("?i*/?i*"); + EXPECT(results_test13); + + // wildcards on file only + const char *results_test14[] = {"dir1"PATH_SEPARATOR"dir1", + "dir1"PATH_SEPARATOR"dir2", + "dir1"PATH_SEPARATOR"file1", + "dir1"PATH_SEPARATOR"file2"}; + TEST("dir1"PATH_SEPARATOR"?i*"); + EXPECT(results_test14); + + // wildcards on folder only + const char *results_test15[] = {"dir1"PATH_SEPARATOR"file1", + "dir2"PATH_SEPARATOR"file1"}; + TEST("?i*"PATH_SEPARATOR"file1"); + EXPECT(results_test15); + + // wildcards after file name + TEST("file1"PATH_SEPARATOR"*"); + EXPECT_EMPTY(); + +#ifndef _WIN32 + // test wildcard escaping + TEST("\\*"); + EXPECT_EMPTY(); + + // test forbidden directory + tor_asprintf(&pattern, "%s"PATH_SEPARATOR"*"PATH_SEPARATOR"*", dirname); + results = tor_glob(pattern); + tor_free(pattern); + tt_assert(!results); +#endif + +#undef TEST +#undef EXPECT +#undef EXPECT_EMPTY + + done: +#ifndef _WIN32 + chmod(forbidden, 0700); + chmod(dir1_forbidden, 0700); + chmod(dir2_forbidden, 0700); + chmod(forbidden_forbidden, 0700); +#endif + tor_free(dir1); + tor_free(dir2); + tor_free(forbidden); + tor_free(dirname); + tor_free(dir1_forbidden); + tor_free(dir2_forbidden); + tor_free(forbidden_forbidden); + tor_free(expected); + tor_free(pattern); + if (results) { + SMARTLIST_FOREACH(results, char *, f, tor_free(f)); + smartlist_free(results); + } +} + +static void +test_util_get_glob_opened_files(void *ptr) +{ + (void)ptr; + + smartlist_t *results = NULL; + int r, i; + char *dir1 = NULL, *dir2 = NULL, *forbidden = NULL, *dirname = NULL; + char *expected = NULL, *pattern = NULL; + // used for cleanup + char *dir1_forbidden = NULL, *dir2_forbidden = NULL; + char *forbidden_forbidden = NULL; + + dirname = tor_strdup(get_fname("test_get_glob_opened_files")); + tt_ptr_op(dirname, OP_NE, NULL); + +#ifdef _WIN32 + r = mkdir(dirname); +#else + r = mkdir(dirname, 0700); +#endif + if (r) { + fprintf(stderr, "Can't create directory %s:", dirname); + perror(""); + exit(1); + } + + tt_int_op(0, OP_EQ, create_test_directory_structure(dirname)); + tor_asprintf(&dir1, "%s"PATH_SEPARATOR"dir1", dirname); + tor_asprintf(&dir1_forbidden, + "%s"PATH_SEPARATOR"dir1"PATH_SEPARATOR"forbidden", dirname); + tt_int_op(0, OP_EQ, create_test_directory_structure(dir1)); + tor_asprintf(&dir2, "%s"PATH_SEPARATOR"dir2", dirname); + tor_asprintf(&dir2_forbidden, + "%s"PATH_SEPARATOR"dir2"PATH_SEPARATOR"forbidden", dirname); + tt_int_op(0, OP_EQ, create_test_directory_structure(dir2)); + tor_asprintf(&forbidden, "%s"PATH_SEPARATOR"forbidden", dirname); + tor_asprintf(&forbidden_forbidden, + "%s"PATH_SEPARATOR"forbidden"PATH_SEPARATOR"forbidden",dirname); +#ifndef _WIN32 + chmod(forbidden, 0700); +#endif + tt_int_op(0, OP_EQ, create_test_directory_structure(forbidden)); +#ifndef _WIN32 + chmod(forbidden, 0); +#endif + +#define TEST(input) \ + do { \ + if (*input) { \ + tor_asprintf(&pattern, "%s"PATH_SEPARATOR"%s", dirname, input); \ + } else { /* do not add path separator if empty string */ \ + tor_asprintf(&pattern, "%s", dirname); \ + } \ + results = get_glob_opened_files(pattern); \ + tor_free(pattern); \ + tt_assert(results); \ + smartlist_sort_strings(results); \ + } while (0); + +#define EXPECT(result) \ + do { \ + tt_int_op(smartlist_len(results), OP_EQ, \ + sizeof(result)/sizeof(*result)); \ + i = 0; \ + SMARTLIST_FOREACH_BEGIN(results, const char *, f) { \ + if (*result[i]) { \ + tor_asprintf(&expected, "%s"PATH_SEPARATOR"%s", dirname, result[i]); \ + } else { /* do not add path separator if empty string */ \ + tor_asprintf(&expected, "%s", dirname); \ + } \ + tt_str_op(f, OP_EQ, expected); \ + i++; \ + tor_free(expected); \ + } SMARTLIST_FOREACH_END(f); \ + SMARTLIST_FOREACH(results, char *, f, tor_free(f)); \ + smartlist_free(results); \ + } while (0); + +#define EXPECT_EMPTY() \ + do { \ + tt_int_op(smartlist_len(results), OP_EQ, 0); \ + SMARTLIST_FOREACH(results, char *, f, tor_free(f)); \ + smartlist_free(results); \ + } while (0); + + // all files on folder + const char *results_test1[] = {""}; // only the folder is read + TEST("*"); + EXPECT(results_test1); + + // same as before but ending in path separator + const char *results_test2[] = {""}; // only the folder is read + TEST("*"PATH_SEPARATOR); + EXPECT(results_test2); + + // wilcards in multiple path components +#ifndef _WIN32 + const char *results_test3[] = {"", "dir1", "dir2", "empty", "file1", "file2", + "forbidden"}; +#else + // dot files are not special on windows + const char *results_test3[] = {"", ".test-hidden", "dir1", "dir2", "empty", + "file1", "file2", "forbidden"}; +#endif + TEST("*"PATH_SEPARATOR"*"); + EXPECT(results_test3); + + // same as before but ending in path separator +#ifndef _WIN32 + const char *results_test4[] = {"", "dir1", "dir2", "empty", "file1", "file2", + "forbidden"}; +#else + // dot files are not special on windows + const char *results_test4[] = {"", ".test-hidden", "dir1", "dir2", "empty", + "file1", "file2", "forbidden"}; +#endif + TEST("*"PATH_SEPARATOR"*"PATH_SEPARATOR); + EXPECT(results_test4); + + // no glob - folder + TEST(""); + EXPECT_EMPTY(); + + // same as before but ending in path separator + TEST(PATH_SEPARATOR); + EXPECT_EMPTY(); + + // no glob - file + TEST("file1"); + EXPECT_EMPTY(); + + // same as before but ending in path separator and linux path separator + TEST("file1/"); + EXPECT_EMPTY(); + + // file but with wildcard after + const char *results_test9[] = {"file1"}; + TEST("file1"PATH_SEPARATOR"*"); + EXPECT(results_test9); + + // dir inside dir and linux path separator + TEST("dir1/dir1"); + EXPECT_EMPTY(); + + // same as before but ending in path separator + TEST("dir1"PATH_SEPARATOR"dir1"PATH_SEPARATOR); + EXPECT_EMPTY(); + + // no glob - empty + TEST("empty"); + EXPECT_EMPTY(); + + // same as before but ending in path separator + TEST("empty"PATH_SEPARATOR); + EXPECT_EMPTY(); + + // no glob - does not exist + TEST("not_exist"); + EXPECT_EMPTY(); + +#undef TEST +#undef EXPECT +#undef EXPECT_EMPTY + + done: +#ifndef _WIN32 + chmod(forbidden, 0700); + chmod(dir1_forbidden, 0700); + chmod(dir2_forbidden, 0700); + chmod(forbidden_forbidden, 0700); +#endif + tor_free(dir1); + tor_free(dir2); + tor_free(forbidden); + tor_free(dirname); + tor_free(dir1_forbidden); + tor_free(dir2_forbidden); + tor_free(forbidden_forbidden); + tor_free(expected); + tor_free(pattern); + if (results) { + SMARTLIST_FOREACH(results, char *, f, tor_free(f)); + smartlist_free(results); + } +} + static void test_util_parent_dir(void *ptr) { @@ -6520,10 +6953,13 @@ struct testcase_t util_tests[] = { UTIL_TEST(laplace, 0), UTIL_TEST(clamp_double_to_int64, 0), UTIL_TEST(find_str_at_start_of_line, 0), + UTIL_TEST(tor_strreplacechar, 0), UTIL_TEST(string_is_C_identifier, 0), UTIL_TEST(string_is_utf8, 0), UTIL_TEST(asprintf, 0), UTIL_TEST(listdir, 0), + UTIL_TEST(glob, 0), + UTIL_TEST(get_glob_opened_files, 0), UTIL_TEST(parent_dir, 0), UTIL_TEST(ftruncate, 0), UTIL_TEST(nowrap_math, 0), diff --git a/src/tools/include.am b/src/tools/include.am index 72dfe6017c..6daa27f6de 100644 --- a/src/tools/include.am +++ b/src/tools/include.am @@ -11,7 +11,7 @@ src_tools_tor_resolve_LDADD = \ $(TOR_UTIL_LIBS) \ $(TOR_CRYPTO_LIBS) $(TOR_LIBS_CRYPTLIB)\ $(rust_ldadd) \ - @TOR_LIB_MATH@ @TOR_LIB_WS32@ @TOR_LIB_IPHLPAPI@ @TOR_LIB_USERENV@ + @TOR_LIB_MATH@ @TOR_LIB_WS32@ @TOR_LIB_IPHLPAPI@ @TOR_LIB_SHLWAPI@ @TOR_LIB_USERENV@ if COVERAGE_ENABLED src_tools_tor_cov_resolve_SOURCES = src/tools/tor-resolve.c @@ -36,7 +36,7 @@ src_tools_tor_gencert_LDADD = \ $(TOR_UTIL_LIBS) \ $(rust_ldadd) \ @TOR_LIB_MATH@ @TOR_ZLIB_LIBS@ $(TOR_LIBS_CRYPTLIB) \ - @TOR_LIB_WS32@ @TOR_LIB_IPHLPAPI@ @TOR_LIB_GDI@ @TOR_LIB_USERENV@ @CURVE25519_LIBS@ + @TOR_LIB_WS32@ @TOR_LIB_IPHLPAPI@ @TOR_LIB_SHLWAPI@ @TOR_LIB_GDI@ @TOR_LIB_USERENV@ @CURVE25519_LIBS@ endif src_tools_tor_print_ed_signing_cert_SOURCES = src/tools/tor-print-ed-signing-cert.c @@ -46,7 +46,7 @@ src_tools_tor_print_ed_signing_cert_LDADD = \ $(TOR_CRYPTO_LIBS) \ $(TOR_UTIL_LIBS) \ @TOR_LIB_MATH@ $(TOR_LIBS_CRYPTLIB) \ - @TOR_LIB_WS32@ @TOR_LIB_USERENV@ @TOR_LIB_GDI@ + @TOR_LIB_WS32@ @TOR_LIB_USERENV@ @TOR_LIB_SHLWAPI@ @TOR_LIB_GDI@ if USE_NSS # ... @@ -61,7 +61,7 @@ src_tools_tor_cov_gencert_LDADD = \ $(TOR_CRYPTO_TESTING_LIBS) \ $(TOR_UTIL_TESTING_LIBS) \ @TOR_LIB_MATH@ @TOR_ZLIB_LIBS@ $(TOR_LIBS_CRYPTLIB) \ - @TOR_LIB_WS32@ @TOR_LIB_IPHLPAPI@ @TOR_LIB_GDI@ @CURVE25519_LIBS@ + @TOR_LIB_WS32@ @TOR_LIB_IPHLPAPI@ @TOR_LIB_SHLWAPI@ @TOR_LIB_GDI@ @CURVE25519_LIBS@ endif endif

1 0

[tor/master] Changes file for #25140
by nickm＠torproject.org 12 Aug '20

12 Aug '20

commit ac2c4b1e4ae297b33be6c971fd0e4aca1198b282 Author: Daniel Pinto <danielpinto52(a)gmail.com> Date: Wed Jun 3 22:09:58 2020 +0100 Changes file for #25140 --- changes/feature25140 | 3 +++ 1 file changed, 3 insertions(+) diff --git a/changes/feature25140 b/changes/feature25140 new file mode 100644 index 0000000000..5202fa11ce --- /dev/null +++ b/changes/feature25140 @@ -0,0 +1,3 @@ + o Minor feature (configuration): + - Allow the using wildcards (* and ?) with the %include option on + configuration files. Closes ticket 25140. Patch by Daniel Pinto.

1 0

[tor/master] Fix small memory leak in config include tests
by nickm＠torproject.org 12 Aug '20

12 Aug '20

commit 3c70f26dc07cea98e009dc8b15009ca8e72ba226 Author: Daniel Pinto <danielpinto52(a)gmail.com> Date: Wed Jun 3 22:09:54 2020 +0100 Fix small memory leak in config include tests Tests that used options_init_from_string were not freeing calling config_free_all(). --- src/test/test_config.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/src/test/test_config.c b/src/test/test_config.c index 617a52b8ac..cb88b95761 100644 --- a/src/test/test_config.c +++ b/src/test/test_config.c @@ -5875,6 +5875,7 @@ test_config_include_flag_both_without(void *data) done: tor_free(errmsg); + config_free_all(); } static void @@ -5915,6 +5916,7 @@ test_config_include_flag_torrc_only(void *data) tor_free(errmsg); tor_free(path); tor_free(dir); + config_free_all(); } static void @@ -5955,6 +5957,7 @@ test_config_include_flag_defaults_only(void *data) tor_free(errmsg); tor_free(path); tor_free(dir); + config_free_all(); } static void

1 0

[tor/master] Improved documentation and comments #25140
by nickm＠torproject.org 12 Aug '20

12 Aug '20

commit 0b633b1f6d022a359806a705bd4a6082d92bd3f3 Author: Daniel Pinto <danielpinto52(a)gmail.com> Date: Tue Aug 11 18:26:41 2020 +0100 Improved documentation and comments #25140 --- doc/man/tor.1.txt | 6 ++-- src/lib/fs/path.c | 104 ++++++++++++++++++++++++++++++++++++++++++++---------- 2 files changed, 88 insertions(+), 22 deletions(-) diff --git a/doc/man/tor.1.txt b/doc/man/tor.1.txt index bb01315d46..3febf43513 100644 --- a/doc/man/tor.1.txt +++ b/doc/man/tor.1.txt @@ -206,9 +206,9 @@ such multiline entries, but they must start at the beginning of a line. Configuration options can be imported from files or folders using the %include option with the value being a path. This path can have wildcards. Wildcards are -expanded first, using lexical order. Then, for each matching file or folder, the -following rules are followed: if the path is a file, the options from the -file will be parsed as if they were written where the %include option is. If +expanded first, then sorted using lexical order. Then, for each matching file or +folder, the following rules are followed: if the path is a file, the options from +the file will be parsed as if they were written where the %include option is. If the path is a folder, all files on that folder will be parsed following lexical order. Files starting with a dot are ignored. Files in subfolders are ignored. The %include option can be used recursively. diff --git a/src/lib/fs/path.c b/src/lib/fs/path.c index 4532bfb7d1..1a60ca70d4 100644 --- a/src/lib/fs/path.c +++ b/src/lib/fs/path.c @@ -44,13 +44,6 @@ #include <errno.h> #include <string.h> -#ifdef _WIN32 -#define IS_GLOB_CHAR(s,i) (((s)[(i)]) == '*' || ((s)[(i)]) == '?') -#else -#define IS_GLOB_CHAR(s,i) ((((s)[(i)]) == '*' || ((s)[(i)]) == '?') &&\ - ((i) == 0 || (s)[(i)-1] != '\\')) /* check escape */ -#endif - /** Removes enclosing quotes from path and unescapes quotes between the * enclosing quotes. Backslashes are not unescaped. Return the unquoted * path on success or 0 if path is not quoted correctly. */ @@ -321,8 +314,78 @@ make_path_absolute(const char *fname) #endif /* defined(_WIN32) */ } -/** Expands globs in pattern for the path fragment between - * prev_sep and next_sep. Returns NULL on failure. */ +/* The code below implements tor_glob and get_glob_opened_files. Because it is + * not easy to understand it by looking at individual functions, the big + * picture explanation here should be read first. + * + * Purpose of the functions: + * - tor_glob - recevies a pattern and returns all the paths that result from + * its glob expansion, globs can be present on all path components. + * - get_glob_opened_files - receives a pattern and returns all the paths that + * are opened during its expansion (the paths before any path fragment that + * contains a glob as they have to be opened to check for glob matches). This + * is used to get the paths that have to be added to the seccomp sandbox + * allowed list. + * + * Due to OS API differences explained below, the implementation of tor_glob is + * completly different for Windows and POSIX systems, so we ended up with three + * different implementations: + * - tor_glob for POSIX - as POSIX glob does everything we need, we simply call + * it and process the results. This is completly implemented in tor_glob. + * - tor_glob for WIN32 - because the WIN32 API only supports expanding globs + * in the last path fragment, we need to expand the globs in each path + * fragment manually and call recursively to get the same behaviour as POSIX + * glob. When there are no globs in pattern, we know we are on the last path + * fragment and collect the full path. + * - get_glob_opened_files - because the paths before any path fragment with a + * glob will be opened to check for matches, we need to collect them and we + * need to expand the globs in each path fragments and call recursively until + * we find no more globs. + * + * As seen from the description above, both tor_glob for WIN32 and + * get_glob_opened_files receive a pattern and return a list of paths and have + * to expand all path fragments that contain globs and call themselves + * recursively. The differences are: + * - get_glob_opened_files collects paths before path fragments with globs + * while tor_glob for WIN32 collects full paths resulting from the expansion + * of all globs. + * - get_glob_opened_files can call tor_glob to expand path fragments with + * globs while tor_glob for WIN32 cannot because it IS tor_glob. For tor_glob + * for WIN32, an auxiliary function has to be used for this purpose. + * + * To avoid code duplication, the logic of tor_glob for WIN32 and + * get_glob_opened_files is implemented in get_glob_paths. The differences are + * configured by the extra function parameters: + * - final - if true, returns a list of paths obtained from expanding pattern + * (implements tor_glob). Otherwise, returns the paths before path fragments + * with globs (implements get_glob_opened_files). + * - unglob - function used to expand a path fragment. The function signature + * is defined by the unglob_fn typedef. Two implementations are available: + * - unglob_win32 - uses tor_listdir and PathMatchSpec (for tor_glob WIN32) + * - unglob_opened_files - uses tor_glob (for get_glob_opened_files) + */ + +/** Returns true if the character at position pos in pattern is + * considered a glob. Returns false otherwise. Takes escaping into account on + * systems where escaping globs is supported. */ +static inline bool +is_glob_char(const char *pattern, int pos) +{ + bool is_glob = pattern[pos] == '*' || pattern[pos] == '?'; +#ifdef _WIN32 + return is_glob; +#else /* !defined(_WIN32) */ + bool is_escaped = pos > 0 && pattern[pos-1] == '\\'; + return is_glob && !is_escaped; +#endif /* defined(_WIN32) */ +} + +/** Expands the first path fragment of pattern that contains globs. The + * path fragment is between prev_sep and next_sep. If the path + * fragment is the last fragment of pattern, next_sep will be the + * index of the last char. Returns a list of paths resulting from the glob + * expansion of the path fragment. Anything after next_sep is not + * included in the returned list. Returns NULL on failure. */ typedef struct smartlist_t * unglob_fn(const char *pattern, int prev_sep, int next_sep); @@ -350,7 +413,7 @@ add_non_glob_path(const char *path, struct smartlist_t *result) * expand each path fragment. If final is true, the paths are the result * of the glob expansion of pattern (implements tor_glob). Otherwise, * the paths are the paths opened by glob while expanding pattern - * (implements get_glb_opened_files). Returns NULL on failure. */ + * (implements get_glob_opened_files). Returns NULL on failure. */ static struct smartlist_t * get_glob_paths(const char *pattern, unglob_fn unglob, bool final) { @@ -360,7 +423,7 @@ get_glob_paths(const char *pattern, unglob_fn unglob, bool final) // find first path fragment with globs for (i = 0; pattern[i]; i++) { - is_glob = is_glob || IS_GLOB_CHAR(pattern, i); + is_glob = is_glob || is_glob_char(pattern, i); is_last = !pattern[i+1]; is_sep = pattern[i] == *PATH_SEPARATOR || pattern[i] == '/'; if (is_sep || is_last) { @@ -422,7 +485,8 @@ end: #ifdef _WIN32 /** Expands globs in pattern for the path fragment between * prev_sep and next_sep using the WIN32 API. Returns NULL on - * failure. Used by the WIN32 implementation of tor_glob. */ + * failure. Used by the WIN32 implementation of tor_glob. Implements unglob_fn, + * see its description for more details. */ static struct smartlist_t * unglob_win32(const char *pattern, int prev_sep, int next_sep) { @@ -450,10 +514,10 @@ unglob_win32(const char *pattern, int prev_sep, int next_sep) #ifdef UNICODE mbstowcs(tpattern, path_curr_glob, MAX_PATH); mbstowcs(tfile, full_path, MAX_PATH); -#else +#else /* !defined(UNICODE) */ strlcpy(tpattern, path_curr_glob, MAX_PATH); strlcpy(tfile, full_path, MAX_PATH); -#endif +#endif /* defined(UNICODE) */ if (PathMatchSpec(tfile, tpattern)) { smartlist_add(result, full_path); } else { @@ -492,7 +556,8 @@ prot_lstat(const char *pathname, struct stat *buf) #endif /* defined(_WIN32) */ /** Return a new list containing the paths that match the pattern - * pattern. Return NULL on error. + * pattern. Return NULL on error. On POSIX systems, errno is set by the + * glob function. */ struct smartlist_t * tor_glob(const char *pattern) @@ -548,7 +613,7 @@ has_glob(const char *s) { int i; for (i = 0; s[i]; i++) { - if (IS_GLOB_CHAR(s, i)) { + if (is_glob_char(s, i)) { return true; } } @@ -557,7 +622,8 @@ has_glob(const char *s) /** Expands globs in pattern for the path fragment between * prev_sep and next_sep using tor_glob. Returns NULL on - * failure. Used by get_glob_opened_files. */ + * failure. Used by get_glob_opened_files. Implements unglob_fn, see its + * description for more details. */ static struct smartlist_t * unglob_opened_files(const char *pattern, int prev_sep, int next_sep) { @@ -565,8 +631,8 @@ unglob_opened_files(const char *pattern, int prev_sep, int next_sep) smartlist_t *result = smartlist_new(); // if the following fragments have no globs, we're done if (has_glob(&pattern[next_sep+1])) { - // if there is a glob after next_sep, we know it is a separator and not the - // last char and glob_path will have the path without the separator + // if there is a glob after next_sep, we know next_sep is a separator and + // not the last char and glob_path will have the path without the separator char *glob_path = tor_strndup(pattern, next_sep); smartlist_t *child_paths = tor_glob(glob_path); tor_free(glob_path);

1 0

[tor/master] Add test for torrc %include functionality and seccomp sandbox
by nickm＠torproject.org 12 Aug '20

12 Aug '20

commit 49dab42782e06e51c218cf40b8cbb98b80d9803f Author: Daniel Pinto <danielpinto52(a)gmail.com> Date: Wed Jun 3 22:09:51 2020 +0100 Add test for torrc %include functionality and seccomp sandbox Tests %include with files and folders, modifying and reloading the config file with sandbox enabled and reponse of SAVECONF and getinfo config-can-saveconf control commmands. --- .travis.yml | 4 +- src/test/include.am | 3 + src/test/test_include.py | 196 +++++++++++++++++++++++++++++++++++++++++++++++ src/test/test_include.sh | 111 +++++++++++++++++++++++++++ 4 files changed, 312 insertions(+), 2 deletions(-) diff --git a/.travis.yml b/.travis.yml index 01343e65d9..aaca3a7368 100644 --- a/.travis.yml +++ b/.travis.yml @@ -237,8 +237,8 @@ install: - dd ibs=1 count=1024 if=/dev/urandom > ~/.torrc script: - # Skip test_rebind on macOS - - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then export TOR_SKIP_TEST_REBIND=true; fi + # Skip test_rebind and test_include on macOS + - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then export TOR_SKIP_TEST_REBIND=true; export TOR_SKIP_TEST_INCLUDE=true; fi - ./autogen.sh - CONFIGURE_FLAGS="$ASCIIDOC_OPTIONS $COVERAGE_OPTIONS $HARDENING_OPTIONS $MODULES_OPTIONS $NSS_OPTIONS $OPENSSL_OPTIONS $RUST_OPTIONS --enable-fatal-warnings --disable-silent-rules" - echo "Configure flags are $CONFIGURE_FLAGS CC=\"$CC $C_DIALECT_OPTIONS\"" diff --git a/src/test/include.am b/src/test/include.am index c049053438..478547d655 100644 --- a/src/test/include.am +++ b/src/test/include.am @@ -46,6 +46,7 @@ else # Only do this when coverage is not on, since it invokes lots of code # in a kind of unpredictable way. TESTSCRIPTS += src/test/test_rebind.sh +TESTSCRIPTS += src/test/test_include.sh endif endif @@ -431,6 +432,8 @@ EXTRA_DIST += \ src/test/slownacl_curve25519.py \ src/test/test_rebind.sh \ src/test/test_rebind.py \ + src/test/test_include.sh \ + src/test/test_include.py \ src/test/zero_length_keys.sh \ scripts/maint/run_check_subsystem_order.sh \ src/test/rust_supp.txt \ diff --git a/src/test/test_include.py b/src/test/test_include.py new file mode 100644 index 0000000000..0ac87ee8fb --- /dev/null +++ b/src/test/test_include.py @@ -0,0 +1,196 @@ +# Future imports for Python 2.7, mandatory in 3.0 +from __future__ import division +from __future__ import print_function +from __future__ import unicode_literals + +import errno +import logging +import os +import random +import socket +import subprocess +import sys +import time +import re + +CONTROL_SOCK_TIMEOUT = 10.0 +LOG_TIMEOUT = 60.0 +LOG_WAIT = 0.1 + +def fail(msg): + logging.error('FAIL') + sys.exit(msg) + +def skip(msg): + logging.warning('SKIP: {}'.format(msg)) + sys.exit(77) + +def wait_for_log(s): + cutoff = time.time() + LOG_TIMEOUT + while time.time() < cutoff: + l = tor_process.stdout.readline() + l = l.decode('utf8', 'backslashreplace') + if s in l: + logging.info('Tor logged: "{}"'.format(l.strip())) + return + # readline() returns a blank string when there is no output + # avoid busy-waiting + if len(l) == 0: + logging.debug('Tor has not logged anything, waiting for "{}"'.format(s)) + time.sleep(LOG_WAIT) + else: + logging.info('Tor logged: "{}", waiting for "{}"'.format(l.strip(), s)) + fail('Could not find "{}" in logs after {} seconds'.format(s, LOG_TIMEOUT)) + +def pick_random_port(): + port = 0 + random.seed() + + for i in range(8): + port = random.randint(10000, 60000) + s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + if s.connect_ex(('127.0.0.1', port)) == 0: + s.close() + else: + break + + if port == 0: + fail('Could not find a random free port between 10000 and 60000') + + return port + +def check_control_list(control_out_file, expected, value_name): + received_count = 0 + for e in expected: + received = control_out_file.readline().strip() + received_count += 1 + parts = re.split('[ =-]', received.strip()) + if len(parts) != 3 or parts[0] != '250' or parts[1] != value_name or parts[2] != e: + fail('Unexpected value in response line "{}". Expected {} for value {}'.format(received, e, value_name)) + if received.startswith('250 '): + break + + if received_count != len(expected): + fail('Expected response with {} lines but received {} lines'.format(len(expected), received_count)) + + +logging.basicConfig(level=logging.DEBUG, + format='%(asctime)s.%(msecs)03d %(message)s', + datefmt='%Y-%m-%d %H:%M:%S') + +if sys.hexversion < 0x02070000: + fail("ERROR: unsupported Python version (should be >= 2.7)") + +if sys.hexversion > 0x03000000 and sys.hexversion < 0x03010000: + fail("ERROR: unsupported Python3 version (should be >= 3.1)") + +if 'TOR_SKIP_TEST_INCLUDE' in os.environ: + skip('$TOR_SKIP_TEST_INCLUDE is set') + +control_port = pick_random_port() + +assert control_port != 0 + +if len(sys.argv) < 4: + fail('Usage: %s <path-to-tor> <data-dir> <torrc>' % sys.argv[0]) + +if not os.path.exists(sys.argv[1]): + fail('ERROR: cannot find tor at %s' % sys.argv[1]) +if not os.path.exists(sys.argv[2]): + fail('ERROR: cannot find datadir at %s' % sys.argv[2]) +if not os.path.exists(sys.argv[3]): + fail('ERROR: cannot find torrcdir at %s' % sys.argv[3]) + +tor_path = sys.argv[1] +data_dir = sys.argv[2] +torrc_dir = sys.argv[3] + +empty_torrc_path = os.path.join(data_dir, 'empty_torrc') +open(empty_torrc_path, 'w').close() +empty_defaults_torrc_path = os.path.join(data_dir, 'empty_defaults_torrc') +open(empty_defaults_torrc_path, 'w').close() +torrc = os.path.join(torrc_dir, 'torrc') + +tor_process = subprocess.Popen([tor_path, + '-DataDirectory', data_dir, + '-ControlPort', '127.0.0.1:{}'.format(control_port), + '-Log', 'info stdout', + '-LogTimeGranularity', '1', + '-FetchServerDescriptors', '0', + '-DisableNetwork', '1', + '-f', torrc, + '--defaults-torrc', empty_defaults_torrc_path, + ], + stdout=subprocess.PIPE, + stderr=subprocess.PIPE) + +if tor_process == None: + fail('ERROR: running tor failed') + +wait_for_log('Opened Control listener on') + +control_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) +if control_socket.connect_ex(('127.0.0.1', control_port)): + tor_process.terminate() + fail('Cannot connect to ControlPort') +control_socket.settimeout(CONTROL_SOCK_TIMEOUT) +control_out_file = control_socket.makefile('r') + +control_socket.sendall('AUTHENTICATE \r\n'.encode('ascii')) +res = control_out_file.readline().strip() +if res != '250 OK': + tor_process.terminate() + fail('Cannot authenticate. Response was: {}'.format(res)) + +# test configuration file values and order +control_socket.sendall('GETCONF NodeFamily\r\n'.encode('ascii')) +check_control_list(control_out_file, ['1', '2', '3', '4', '5', '6', '4' , '5'], 'NodeFamily') + +# test reloading the configuration file with seccomp sandbox enabled +foo_path = os.path.join(torrc_dir, 'torrc.d', 'foo') +with open(foo_path, 'a') as foo: + foo.write('NodeFamily 7') + +control_socket.sendall('SIGNAL RELOAD\r\n'.encode('ascii')) +wait_for_log('Reloading config and resetting internal state.') +res = control_out_file.readline().strip() +if res != '250 OK': + tor_process.terminate() + fail('Cannot reload configuration. Response was: {}'.format(res)) + + +control_socket.sendall('GETCONF NodeFamily\r\n'.encode('ascii')) +check_control_list(control_out_file, ['1', '2', '3', '4', '5', '6', '7', '4' , '5'], 'NodeFamily') + +# test that config-can-saveconf is 0 because we have a %include +control_socket.sendall('getinfo config-can-saveconf\r\n'.encode('ascii')) +res = control_out_file.readline().strip() +if res != '250-config-can-saveconf=0': + tor_process.terminate() + fail('getinfo config-can-saveconf returned wrong response: {}'.format(res)) +else: + res = control_out_file.readline().strip() + if res != '250 OK': + tor_process.terminate() + fail('getinfo failed. Response was: {}'.format(res)) + +# test that saveconf returns error because we have a %include +control_socket.sendall('SAVECONF\r\n'.encode('ascii')) +res = control_out_file.readline().strip() +if res != '551 Unable to write configuration to disk.': + tor_process.terminate() + fail('SAVECONF returned wrong response. Response was: {}'.format(res)) + +control_socket.sendall('SIGNAL HALT\r\n'.encode('ascii')) + +wait_for_log('exiting cleanly') +logging.info('OK') + +try: + tor_process.terminate() +except OSError as e: + if e.errno == errno.ESRCH: # errno 3: No such process + # assume tor has already exited due to SIGNAL HALT + logging.warn("Tor has already exited") + else: + raise diff --git a/src/test/test_include.sh b/src/test/test_include.sh new file mode 100755 index 0000000000..6cf695fe44 --- /dev/null +++ b/src/test/test_include.sh @@ -0,0 +1,111 @@ +#!/bin/sh + +umask 077 +set -e +set -x + +# emulate realpath(), in case coreutils or equivalent is not installed. +abspath() { + f="$*" + if [ -d "$f" ]; then + dir="$f" + base="" + else + dir="$(dirname "$f")" + base="/$(basename "$f")" + fi + dir="$(cd "$dir" && pwd)" + echo "$dir$base" +} + +UNAME_OS=$(uname -s | cut -d_ -f1) +if test "$UNAME_OS" = 'CYGWIN' || \ + test "$UNAME_OS" = 'MSYS' || \ + test "$UNAME_OS" = 'MINGW' || \ + test "$UNAME_OS" = 'MINGW32' || \ + test "$UNAME_OS" = 'MINGW64'; then + if test "$APPVEYOR" = 'True'; then + echo "This test is disabled on Windows CI, as it requires firewall exemptions. Skipping." >&2 + exit 77 + fi +fi + +# find the tor binary +if [ $# -ge 1 ]; then + TOR_BINARY="${1}" + shift +else + TOR_BINARY="${TESTING_TOR_BINARY:-./src/app/tor}" +fi + +TOR_BINARY="$(abspath "$TOR_BINARY")" + +echo "TOR BINARY IS ${TOR_BINARY}" + +if "${TOR_BINARY}" --list-modules | grep -q "relay: no"; then + echo "This test requires the relay module. Skipping." >&2 + exit 77 +fi + +tmpdir= +clean () { + if [ -n "$tmpdir" ] && [ -d "$tmpdir" ]; then + rm -rf "$tmpdir" + fi +} + +trap clean EXIT HUP INT TERM + +tmpdir="$(mktemp -d -t tor_include_test.XXXXXX)" +if [ -z "$tmpdir" ]; then + echo >&2 mktemp failed + exit 2 +elif [ ! -d "$tmpdir" ]; then + echo >&2 mktemp failed to make a directory + exit 3 +fi + +datadir="$tmpdir/data" +mkdir "$datadir" + +configdir="$tmpdir/config" +mkdir "$configdir" + +# translate paths to windows format +if test "$UNAME_OS" = 'CYGWIN' || \ + test "$UNAME_OS" = 'MSYS' || \ + test "$UNAME_OS" = 'MINGW' || \ + test "$UNAME_OS" = 'MINGW32' || \ + test "$UNAME_OS" = 'MINGW64'; then + datadir=$(cygpath --windows "$datadir") + configdir=$(cygpath --windows "$configdir") +fi + +# create test folder structure in configdir +torrcd="$configdir/torrc.d" +mkdir "$torrcd" +mkdir "$torrcd/folder" +mkdir "$torrcd/empty_folder" +echo "NodeFamily 1" > "$torrcd/01_one.conf" +echo "NodeFamily 2" > "$torrcd/02_two.conf" +echo "NodeFamily 3" > "$torrcd/aa_three.conf" +echo "NodeFamily 42" > "$torrcd/.hidden.conf" +echo "NodeFamily 6" > "$torrcd/foo" +touch "$torrcd/empty.conf" +echo "# comment" > "$torrcd/comment.conf" +echo "NodeFamily 4" > "$torrcd/folder/04_four.conf" +echo "NodeFamily 5" > "$torrcd/folder/05_five.conf" +torrc="$configdir/torrc" +echo "Sandbox 1" > "$torrc" +echo " +%include $torrcd/*.conf +%include $torrcd/f* +%include $torrcd/*/* +%include $torrcd/empty_folder +%include $torrcd/empty.conf +%include $torrcd/comment.conf +" >> "$torrc" + +"${PYTHON:-python}" "${abs_top_srcdir:-.}/src/test/test_include.py" "${TOR_BINARY}" "$datadir" "$configdir" + +exit $?

1 0