January 2021 - tor-commits - lists.torproject.org

[tor/maint-0.4.5] Limit the number of items in the consdiffmgr on Windows.
by nickm＠torproject.org 21 Jan '21

21 Jan '21

commit c38c36e5f1c65b817a2f7e4d54e81f17b97228b6 Author: Alexander Færøy <ahf(a)torproject.org> Date: Wed Jan 20 16:33:17 2021 +0000 Limit the number of items in the consdiffmgr on Windows. This patch limits the number of items in the consensus diff cache to 64 on the Windows platform. Hopefully, this will allow us to investigate a smarter fix while avoiding the situation reported in tor#24857 where Windows relay operators report Tor using 100% CPU. See: tor#24857 --- changes/bug24857 | 6 ++++++ src/feature/dircache/consdiffmgr.c | 12 +++++++++++- 2 files changed, 17 insertions(+), 1 deletion(-) diff --git a/changes/bug24857 b/changes/bug24857 new file mode 100644 index 0000000000..fb2a7ab6dd --- /dev/null +++ b/changes/bug24857 @@ -0,0 +1,6 @@ + o Minor bugfixes (directory cache, performance, windows): + - Limit the number of items in the consensus diff cache to 64 on Windows. + We hope this will resolve an issue where Windows relay operators reported + Tor using 100% CPU while we investigate better solutions. Fixes bug 24857; + bugfix on tor-0.3.1.1-alpha. + diff --git a/src/feature/dircache/consdiffmgr.c b/src/feature/dircache/consdiffmgr.c index 10590cd6d2..21f536432c 100644 --- a/src/feature/dircache/consdiffmgr.c +++ b/src/feature/dircache/consdiffmgr.c @@ -177,6 +177,16 @@ typedef struct cdm_diff_t { /** Hashtable mapping flavor and source consensus digest to status. */ static HT_HEAD(cdm_diff_ht, cdm_diff_t) cdm_diff_ht = HT_INITIALIZER(); +#ifdef _WIN32 + // XXX(ahf): For tor#24857, a contributor suggested that on Windows, the CPU + // begins to spike at 100% once the number of files handled by the consensus + // diff manager becomes larger than 64. To see if the issue goes away, we + // hardcode this value to 64 now while we investigate a better solution. +# define CACHE_MAX_NUM 64 +#else +# define CACHE_MAX_NUM 128 +#endif + /** * Configuration for this module */ @@ -184,7 +194,7 @@ static consdiff_cfg_t consdiff_cfg = { // XXXX I'd like to make this number bigger, but it interferes with the // XXXX seccomp2 syscall filter, which tops out at BPF_MAXINS (4096) // XXXX rules. - /* .cache_max_num = */ 128 + /* .cache_max_num = */ CACHE_MAX_NUM }; static int consdiffmgr_ensure_space_for_files(int n);

1 0

[tor/maint-0.4.5] Merge remote-tracking branch 'tor-gitlab/mr/269' into maint-0.4.5
by nickm＠torproject.org 21 Jan '21

21 Jan '21

commit 3d952b461dd305951a1a566eafd87874e1b2849a Merge: 71fd30b75a c38c36e5f1 Author: Nick Mathewson <nickm(a)torproject.org> Date: Thu Jan 21 13:37:21 2021 -0500 Merge remote-tracking branch 'tor-gitlab/mr/269' into maint-0.4.5 changes/bug24857 | 6 ++++++ src/feature/dircache/consdiffmgr.c | 12 +++++++++++- 2 files changed, 17 insertions(+), 1 deletion(-)

1 0

[tor/maint-0.4.5] fix changes file
by nickm＠torproject.org 21 Jan '21

21 Jan '21

commit 08c0ef61d8aa9490f78c3a77c1fbb86de09b1872 Author: Nick Mathewson <nickm(a)torproject.org> Date: Thu Jan 21 13:39:40 2021 -0500 fix changes file --- changes/bug24857 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/changes/bug24857 b/changes/bug24857 index fb2a7ab6dd..ea9c763332 100644 --- a/changes/bug24857 +++ b/changes/bug24857 @@ -2,5 +2,5 @@ - Limit the number of items in the consensus diff cache to 64 on Windows. We hope this will resolve an issue where Windows relay operators reported Tor using 100% CPU while we investigate better solutions. Fixes bug 24857; - bugfix on tor-0.3.1.1-alpha. + bugfix on 0.3.1.1-alpha.

1 0

[tor/master] Limit the number of items in the consdiffmgr on Windows.
by nickm＠torproject.org 21 Jan '21

21 Jan '21

commit c38c36e5f1c65b817a2f7e4d54e81f17b97228b6 Author: Alexander Færøy <ahf(a)torproject.org> Date: Wed Jan 20 16:33:17 2021 +0000 Limit the number of items in the consdiffmgr on Windows. This patch limits the number of items in the consensus diff cache to 64 on the Windows platform. Hopefully, this will allow us to investigate a smarter fix while avoiding the situation reported in tor#24857 where Windows relay operators report Tor using 100% CPU. See: tor#24857 --- changes/bug24857 | 6 ++++++ src/feature/dircache/consdiffmgr.c | 12 +++++++++++- 2 files changed, 17 insertions(+), 1 deletion(-) diff --git a/changes/bug24857 b/changes/bug24857 new file mode 100644 index 0000000000..fb2a7ab6dd --- /dev/null +++ b/changes/bug24857 @@ -0,0 +1,6 @@ + o Minor bugfixes (directory cache, performance, windows): + - Limit the number of items in the consensus diff cache to 64 on Windows. + We hope this will resolve an issue where Windows relay operators reported + Tor using 100% CPU while we investigate better solutions. Fixes bug 24857; + bugfix on tor-0.3.1.1-alpha. + diff --git a/src/feature/dircache/consdiffmgr.c b/src/feature/dircache/consdiffmgr.c index 10590cd6d2..21f536432c 100644 --- a/src/feature/dircache/consdiffmgr.c +++ b/src/feature/dircache/consdiffmgr.c @@ -177,6 +177,16 @@ typedef struct cdm_diff_t { /** Hashtable mapping flavor and source consensus digest to status. */ static HT_HEAD(cdm_diff_ht, cdm_diff_t) cdm_diff_ht = HT_INITIALIZER(); +#ifdef _WIN32 + // XXX(ahf): For tor#24857, a contributor suggested that on Windows, the CPU + // begins to spike at 100% once the number of files handled by the consensus + // diff manager becomes larger than 64. To see if the issue goes away, we + // hardcode this value to 64 now while we investigate a better solution. +# define CACHE_MAX_NUM 64 +#else +# define CACHE_MAX_NUM 128 +#endif + /** * Configuration for this module */ @@ -184,7 +194,7 @@ static consdiff_cfg_t consdiff_cfg = { // XXXX I'd like to make this number bigger, but it interferes with the // XXXX seccomp2 syscall filter, which tops out at BPF_MAXINS (4096) // XXXX rules. - /* .cache_max_num = */ 128 + /* .cache_max_num = */ CACHE_MAX_NUM }; static int consdiffmgr_ensure_space_for_files(int n);

1 0

[tor/master] Merge remote-tracking branch 'tor-gitlab/mr/269' into maint-0.4.5
by nickm＠torproject.org 21 Jan '21

21 Jan '21

commit 3d952b461dd305951a1a566eafd87874e1b2849a Merge: 71fd30b75a c38c36e5f1 Author: Nick Mathewson <nickm(a)torproject.org> Date: Thu Jan 21 13:37:21 2021 -0500 Merge remote-tracking branch 'tor-gitlab/mr/269' into maint-0.4.5 changes/bug24857 | 6 ++++++ src/feature/dircache/consdiffmgr.c | 12 +++++++++++- 2 files changed, 17 insertions(+), 1 deletion(-)

1 0

[tor/master] Merge branch 'maint-0.4.5'
by nickm＠torproject.org 21 Jan '21

21 Jan '21

commit b2536c97f941ad2e4e068c43f8ce18469c63585c Merge: 2243fc3ad1 08c0ef61d8 Author: Nick Mathewson <nickm(a)torproject.org> Date: Thu Jan 21 13:40:46 2021 -0500 Merge branch 'maint-0.4.5' changes/bug24857 | 6 ++++++ src/feature/dircache/consdiffmgr.c | 12 +++++++++++- 2 files changed, 17 insertions(+), 1 deletion(-)

1 0

[tor/master] fix changes file
by nickm＠torproject.org 21 Jan '21

21 Jan '21

commit 08c0ef61d8aa9490f78c3a77c1fbb86de09b1872 Author: Nick Mathewson <nickm(a)torproject.org> Date: Thu Jan 21 13:39:40 2021 -0500 fix changes file --- changes/bug24857 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/changes/bug24857 b/changes/bug24857 index fb2a7ab6dd..ea9c763332 100644 --- a/changes/bug24857 +++ b/changes/bug24857 @@ -2,5 +2,5 @@ - Limit the number of items in the consensus diff cache to 64 on Windows. We hope this will resolve an issue where Windows relay operators reported Tor using 100% CPU while we investigate better solutions. Fixes bug 24857; - bugfix on tor-0.3.1.1-alpha. + bugfix on 0.3.1.1-alpha.

1 0

[tor/release-0.4.5] Merge remote-tracking branch 'tor-gitlab/mr/269' into maint-0.4.5
by nickm＠torproject.org 21 Jan '21

21 Jan '21

commit 3d952b461dd305951a1a566eafd87874e1b2849a Merge: 71fd30b75a c38c36e5f1 Author: Nick Mathewson <nickm(a)torproject.org> Date: Thu Jan 21 13:37:21 2021 -0500 Merge remote-tracking branch 'tor-gitlab/mr/269' into maint-0.4.5 changes/bug24857 | 6 ++++++ src/feature/dircache/consdiffmgr.c | 12 +++++++++++- 2 files changed, 17 insertions(+), 1 deletion(-)

1 0

[tor/release-0.4.5] fix changes file
by nickm＠torproject.org 21 Jan '21

21 Jan '21

commit 08c0ef61d8aa9490f78c3a77c1fbb86de09b1872 Author: Nick Mathewson <nickm(a)torproject.org> Date: Thu Jan 21 13:39:40 2021 -0500 fix changes file --- changes/bug24857 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/changes/bug24857 b/changes/bug24857 index fb2a7ab6dd..ea9c763332 100644 --- a/changes/bug24857 +++ b/changes/bug24857 @@ -2,5 +2,5 @@ - Limit the number of items in the consensus diff cache to 64 on Windows. We hope this will resolve an issue where Windows relay operators reported Tor using 100% CPU while we investigate better solutions. Fixes bug 24857; - bugfix on tor-0.3.1.1-alpha. + bugfix on 0.3.1.1-alpha.

1 0

[tor/release-0.4.5] Limit the number of items in the consdiffmgr on Windows.
by nickm＠torproject.org 21 Jan '21

21 Jan '21

commit c38c36e5f1c65b817a2f7e4d54e81f17b97228b6 Author: Alexander Færøy <ahf(a)torproject.org> Date: Wed Jan 20 16:33:17 2021 +0000 Limit the number of items in the consdiffmgr on Windows. This patch limits the number of items in the consensus diff cache to 64 on the Windows platform. Hopefully, this will allow us to investigate a smarter fix while avoiding the situation reported in tor#24857 where Windows relay operators report Tor using 100% CPU. See: tor#24857 --- changes/bug24857 | 6 ++++++ src/feature/dircache/consdiffmgr.c | 12 +++++++++++- 2 files changed, 17 insertions(+), 1 deletion(-) diff --git a/changes/bug24857 b/changes/bug24857 new file mode 100644 index 0000000000..fb2a7ab6dd --- /dev/null +++ b/changes/bug24857 @@ -0,0 +1,6 @@ + o Minor bugfixes (directory cache, performance, windows): + - Limit the number of items in the consensus diff cache to 64 on Windows. + We hope this will resolve an issue where Windows relay operators reported + Tor using 100% CPU while we investigate better solutions. Fixes bug 24857; + bugfix on tor-0.3.1.1-alpha. + diff --git a/src/feature/dircache/consdiffmgr.c b/src/feature/dircache/consdiffmgr.c index 10590cd6d2..21f536432c 100644 --- a/src/feature/dircache/consdiffmgr.c +++ b/src/feature/dircache/consdiffmgr.c @@ -177,6 +177,16 @@ typedef struct cdm_diff_t { /** Hashtable mapping flavor and source consensus digest to status. */ static HT_HEAD(cdm_diff_ht, cdm_diff_t) cdm_diff_ht = HT_INITIALIZER(); +#ifdef _WIN32 + // XXX(ahf): For tor#24857, a contributor suggested that on Windows, the CPU + // begins to spike at 100% once the number of files handled by the consensus + // diff manager becomes larger than 64. To see if the issue goes away, we + // hardcode this value to 64 now while we investigate a better solution. +# define CACHE_MAX_NUM 64 +#else +# define CACHE_MAX_NUM 128 +#endif + /** * Configuration for this module */ @@ -184,7 +194,7 @@ static consdiff_cfg_t consdiff_cfg = { // XXXX I'd like to make this number bigger, but it interferes with the // XXXX seccomp2 syscall filter, which tops out at BPF_MAXINS (4096) // XXXX rules. - /* .cache_max_num = */ 128 + /* .cache_max_num = */ CACHE_MAX_NUM }; static int consdiffmgr_ensure_space_for_files(int n);

1 0