December 2018 - tor-commits - lists.torproject.org

[translation/support-portal_completed] Update translations for support-portal_completed
by translation＠torproject.org 19 Dec '18

19 Dec '18

commit 20cf3f8888ee455cc9c5af0736abac61565a7a5a Author: Translation commit bot <translation(a)torproject.org> Date: Wed Dec 19 06:50:05 2018 +0000 Update translations for support-portal_completed --- contents+es.po | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/contents+es.po b/contents+es.po index 324360931..29e4982f4 100644 --- a/contents+es.po +++ b/contents+es.po @@ -681,7 +681,7 @@ msgstr "instalar-anadiendo-extensiones-al-navegador-tor" #: http//localhost/tbb/download-tor-browser-chromeos/ #: (content/tbb/tbb-15/contents+en.lrquestion.title) msgid "Can I download Tor Browser for ChromeOS?" -msgstr "¿Puedo descargar el Tor Browser, para ChromeOS?" +msgstr "¿Puedo descargar Tor Browser para Chrome OS?" #: http//localhost/tbb/download-tor-browser-chromeos/ #: (content/tbb/tbb-15/contents+en.lrquestion.description)

1 0

[translation/support-portal] Update translations for support-portal
by translation＠torproject.org 19 Dec '18

19 Dec '18

commit a8843d766724e0c486e6109cc57c961079629130 Author: Translation commit bot <translation(a)torproject.org> Date: Wed Dec 19 06:49:55 2018 +0000 Update translations for support-portal --- contents+es.po | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/contents+es.po b/contents+es.po index 324360931..29e4982f4 100644 --- a/contents+es.po +++ b/contents+es.po @@ -681,7 +681,7 @@ msgstr "instalar-anadiendo-extensiones-al-navegador-tor" #: http//localhost/tbb/download-tor-browser-chromeos/ #: (content/tbb/tbb-15/contents+en.lrquestion.title) msgid "Can I download Tor Browser for ChromeOS?" -msgstr "¿Puedo descargar el Tor Browser, para ChromeOS?" +msgstr "¿Puedo descargar Tor Browser para Chrome OS?" #: http//localhost/tbb/download-tor-browser-chromeos/ #: (content/tbb/tbb-15/contents+en.lrquestion.description)

1 0

[snowflake/master] Link to #26360 for why we blackhole logs without -log.
by dcf＠torproject.org 19 Dec '18

19 Dec '18

commit a55443937037a8c1f1655aaefc1ca8bb4c0a0f34 Author: David Fifield <david(a)bamsoftware.com> Date: Tue Dec 18 21:24:32 2018 -0700 Link to #26360 for why we blackhole logs without -log. Maybe we'll able to remove this in the far future. --- client/snowflake.go | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/client/snowflake.go b/client/snowflake.go index b2dea5c..006de6b 100644 --- a/client/snowflake.go +++ b/client/snowflake.go @@ -93,6 +93,11 @@ func main() { defer logFile.Close() log.SetOutput(logFile) } else { + // Don't write to stderr; versions of tor earlier than about + // 0.3.5.6 do not read from the pipe, and eventually we will + // deadlock because the buffer is full. + // https://bugs.torproject.org/26360 + // https://bugs.torproject.org/25600#comment:14 log.SetOutput(ioutil.Discard) }

1 0

[translation/tbmanual-contentspot_completed] Update translations for tbmanual-contentspot_completed
by translation＠torproject.org 19 Dec '18

19 Dec '18

commit 41f83b8bab55af92d96a39a74d0f88e2e7686396 Author: Translation commit bot <translation(a)torproject.org> Date: Wed Dec 19 00:47:23 2018 +0000 Update translations for tbmanual-contentspot_completed --- contents+zh-CN.po | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/contents+zh-CN.po b/contents+zh-CN.po index 9643e8f29..d86aaad20 100644 --- a/contents+zh-CN.po +++ b/contents+zh-CN.po @@ -557,7 +557,7 @@ msgid "" "screen to configure a pluggable transport." msgstr "" "在第一个画面中您必须回答您所处的网络环境是否会过滤阻挡通往洋葱路由网络的连接，若您认为不会的话，请选择“否”。否则若您的网络有被过滤阻挡，或您已经尝试过各种连上洋葱路由网络的方法但都失败的话，请选择“是”，本程序将会带领您前往<a" -" href=\"circumvention\">审查规避</a>画面进行可插拔传输层的相关设置。" +" href=\"circumvention\">审查规避</a>画面进行可插拔传输层的相关设置。" #: https//tb-manual.torproject.org/en-US/running-tor-browser/ #: (content/running-tor-browser/contents+en-US.lrtopic.body) @@ -595,7 +595,7 @@ msgstr "网桥" msgid "" "Most Pluggable Transports, such as obfs3 and obfs4, rely on the use of " "“bridge” relays." -msgstr "大多数可插拔传输（诸如 obfs3 和 obfs4）依赖于使用“网桥”中继。" +msgstr "大多数可插拔传输（如 obfs3 和 obfs4）依赖于使用“网桥”中继。" #: https//tb-manual.torproject.org/en-US/bridges/ #: (content/bridges/contents+en-US.lrtopic.body)

1 0

[translation/tbmanual-contentspot] Update translations for tbmanual-contentspot
by translation＠torproject.org 19 Dec '18

19 Dec '18

commit cb99b9444522cefc42f7fcdd1d486a36ec5fe5db Author: Translation commit bot <translation(a)torproject.org> Date: Wed Dec 19 00:47:17 2018 +0000 Update translations for tbmanual-contentspot --- contents+zh-CN.po | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/contents+zh-CN.po b/contents+zh-CN.po index 9643e8f29..d86aaad20 100644 --- a/contents+zh-CN.po +++ b/contents+zh-CN.po @@ -557,7 +557,7 @@ msgid "" "screen to configure a pluggable transport." msgstr "" "在第一个画面中您必须回答您所处的网络环境是否会过滤阻挡通往洋葱路由网络的连接，若您认为不会的话，请选择“否”。否则若您的网络有被过滤阻挡，或您已经尝试过各种连上洋葱路由网络的方法但都失败的话，请选择“是”，本程序将会带领您前往<a" -" href=\"circumvention\">审查规避</a>画面进行可插拔传输层的相关设置。" +" href=\"circumvention\">审查规避</a>画面进行可插拔传输层的相关设置。" #: https//tb-manual.torproject.org/en-US/running-tor-browser/ #: (content/running-tor-browser/contents+en-US.lrtopic.body) @@ -595,7 +595,7 @@ msgstr "网桥" msgid "" "Most Pluggable Transports, such as obfs3 and obfs4, rely on the use of " "“bridge” relays." -msgstr "大多数可插拔传输（诸如 obfs3 和 obfs4）依赖于使用“网桥”中继。" +msgstr "大多数可插拔传输（如 obfs3 和 obfs4）依赖于使用“网桥”中继。" #: https//tb-manual.torproject.org/en-US/bridges/ #: (content/bridges/contents+en-US.lrtopic.body)

1 0

[tor/master] Rewrite the core of parse_short_policy() to be faster.
by nickm＠torproject.org 19 Dec '18

19 Dec '18

commit 3dd1f064a7d5708585f88beffaf3897ba6555208 Author: Nick Mathewson <nickm(a)torproject.org> Date: Fri Dec 14 16:07:10 2018 -0500 Rewrite the core of parse_short_policy() to be faster. The old implementation did some funky out-of-order lexing, and tended to parse every port twice if the %d-%d pattern didn't match. Closes ticket 28853. --- changes/ticket28853 | 3 ++ src/core/or/policies.c | 80 ++++++++++++++++++++++++++------------------------ 2 files changed, 45 insertions(+), 38 deletions(-) diff --git a/changes/ticket28853 b/changes/ticket28853 new file mode 100644 index 000000000..e76f6bd8c --- /dev/null +++ b/changes/ticket28853 @@ -0,0 +1,3 @@ + o Minor features (performance): + - Replace parse_short_policy() with a faster implementation, to improve + microdescriptor parsing time. Closes ticket 28853. diff --git a/src/core/or/policies.c b/src/core/or/policies.c index 123fc8566..bffdb1fdd 100644 --- a/src/core/or/policies.c +++ b/src/core/or/policies.c @@ -2720,7 +2720,7 @@ parse_short_policy(const char *summary) int is_accept; int n_entries; short_policy_entry_t entries[MAX_EXITPOLICY_SUMMARY_LEN]; /* overkill */ - const char *next; + char *next; if (!strcmpstart(summary, "accept ")) { is_accept = 1; @@ -2735,57 +2735,56 @@ parse_short_policy(const char *summary) n_entries = 0; for ( ; *summary; summary = next) { - const char *comma = strchr(summary, ','); - unsigned low, high; - char dummy; - char ent_buf[32]; - size_t len; - - next = comma ? comma+1 : strchr(summary, '\0'); - len = comma ? (size_t)(comma - summary) : strlen(summary); - if (n_entries == MAX_EXITPOLICY_SUMMARY_LEN) { log_fn(LOG_PROTOCOL_WARN, LD_DIR, "Impossibly long policy summary %s", escaped(orig_summary)); return NULL; } - if (! TOR_ISDIGIT(*summary) || len > (sizeof(ent_buf)-1)) { - /* unrecognized entry format. skip it. */ - continue; - } - if (len < 1) { - /* empty; skip it. */ - /* XXX This happens to be unreachable, since if len==0, then *summary is - * ',' or '\0', and the TOR_ISDIGIT test above would have failed. */ - continue; + unsigned low, high; + int ok; + low = (unsigned) tor_parse_ulong(summary, 10, 1, 65535, &ok, &next); + if (!ok) { + if (! TOR_ISDIGIT(*summary) || *summary == ',') { + /* Unrecognized format: skip it. */ + goto skip_ent; + } else { + goto bad_ent; + } } - memcpy(ent_buf, summary, len); - ent_buf[len] = '\0'; + switch (*next) { + case ',': + ++next; + /* fall through */ + case '\0': + high = low; + break; + case '-': + high = (unsigned) tor_parse_ulong(next+1, 10, low, 65535, &ok, &next); + if (!ok) + goto bad_ent; + + if (*next == ',') + ++next; + else if (*next != '\0') + goto bad_ent; - if (tor_sscanf(ent_buf, "%u-%u%c", &low, &high, &dummy) == 2) { - if (low<1 || low>65535 || high<1 || high>65535 || low>high) { - log_fn(LOG_PROTOCOL_WARN, LD_DIR, - "Found bad entry in policy summary %s", escaped(orig_summary)); - return NULL; - } - } else if (tor_sscanf(ent_buf, "%u%c", &low, &dummy) == 1) { - if (low<1 || low>65535) { - log_fn(LOG_PROTOCOL_WARN, LD_DIR, - "Found bad entry in policy summary %s", escaped(orig_summary)); - return NULL; - } - high = low; - } else { - log_fn(LOG_PROTOCOL_WARN, LD_DIR,"Found bad entry in policy summary %s", - escaped(orig_summary)); - return NULL; + break; + default: + goto bad_ent; } entries[n_entries].min_port = low; entries[n_entries].max_port = high; n_entries++; + + continue; + skip_ent: + next = strchr(next, ','); + if (!next) + break; + ++next; } if (n_entries == 0) { @@ -2806,6 +2805,11 @@ parse_short_policy(const char *summary) result->n_entries = n_entries; memcpy(result->entries, entries, sizeof(short_policy_entry_t)*n_entries); return result; + + bad_ent: + log_fn(LOG_PROTOCOL_WARN, LD_DIR,"Found bad entry in policy summary %s", + escaped(orig_summary)); + return NULL; } /** Write <b>policy</b> back out into a string. */

1 0

[tor/master] Merge branch 'ticket28853'
by nickm＠torproject.org 19 Dec '18

19 Dec '18

commit ed0bc85ed0ac765c91def249afa1390bd03cfe85 Merge: bb091da1e 3dd1f064a Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Dec 18 18:59:56 2018 -0500 Merge branch 'ticket28853' changes/ticket28853 | 3 ++ src/core/or/policies.c | 80 ++++++++++++++++++++++++++------------------------ 2 files changed, 45 insertions(+), 38 deletions(-)

1 0

[tor/master] Use 25% less RAM for base64-encoded directory objects
by nickm＠torproject.org 18 Dec '18

18 Dec '18

commit 6dc90d290daa29b4ff2c7692be3a2ed64f25dfc1 Author: Nick Mathewson <nickm(a)torproject.org> Date: Fri Dec 14 13:11:58 2018 -0500 Use 25% less RAM for base64-encoded directory objects We were allocating N bytes to decode an N-byte base64 encoding, when 3N/4 would have been enough. --- src/feature/dirparse/parsecommon.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/src/feature/dirparse/parsecommon.c b/src/feature/dirparse/parsecommon.c index e00af0eea..91b775533 100644 --- a/src/feature/dirparse/parsecommon.c +++ b/src/feature/dirparse/parsecommon.c @@ -393,8 +393,9 @@ get_next_token(memarea_t *area, RET_ERR("Couldn't parse private key."); } else { /* If it's something else, try to base64-decode it */ int r; - tok->object_body = ALLOC(next-*s); /* really, this is too much RAM. */ - r = base64_decode(tok->object_body, next-*s, *s, next-*s); + size_t maxsize = base64_decode_maxsize(next-*s); + tok->object_body = ALLOC(maxsize); + r = base64_decode(tok->object_body, maxsize, *s, next-*s); if (r<0) RET_ERR("Malformed object: bad base64-encoded data"); tok->object_size = r;

1 0

[tor/master] Add a benchmark for parsing a microdescriptor
by nickm＠torproject.org 18 Dec '18

18 Dec '18

commit cf7342ab6f653c2dc49134024d668b06bac2c96c Author: Nick Mathewson <nickm(a)torproject.org> Date: Fri Dec 14 13:45:22 2018 -0500 Add a benchmark for parsing a microdescriptor --- src/test/bench.c | 40 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/src/test/bench.c b/src/test/bench.c index ff8707d41..f8680c3ab 100644 --- a/src/test/bench.c +++ b/src/test/bench.c @@ -39,6 +39,9 @@ #include "lib/crypt_ops/digestset.h" #include "lib/crypt_ops/crypto_init.h" +#include "feature/dirparse/microdesc_parse.h" +#include "feature/nodelist/microdesc.h" + #if defined(HAVE_CLOCK_GETTIME) && defined(CLOCK_PROCESS_CPUTIME_ID) static uint64_t nanostart; static inline uint64_t @@ -639,6 +642,41 @@ bench_ecdh_p224(void) } #endif +static void +bench_md_parse(void) +{ + uint64_t start, end; + const int N = 100000; + // selected arbitrarily + const char md_text[] = + "@last-listed 2018-12-14 18:14:14\n" + "onion-key\n" + "-----BEGIN RSA PUBLIC KEY-----\n" + "MIGJAoGBAMHkZeXNDX/49JqM2BVLmh1Fnb5iMVnatvZZTLJyedqDLkbXZ1WKP5oh\n" + "7ec14dj/k3ntpwHD4s2o3Lb6nfagWbug4+F/rNJ7JuFru/PSyOvDyHGNAuegOXph\n" + "3gTGjdDpv/yPoiadGebbVe8E7n6hO+XxM2W/4dqheKimF0/s9B7HAgMBAAE=\n" + "-----END RSA PUBLIC KEY-----\n" + "ntor-onion-key QgF/EjqlNG1wRHLIop/nCekEH+ETGZSgYOhu26eiTF4=\n" + "family $00E9A86E7733240E60D8435A7BBD634A23894098 " + "$329BD7545DEEEBBDC8C4285F243916F248972102 " + "$69E06EBB2573A4F89330BDF8BC869794A3E10E4D " + "$DCA2A3FAE50B3729DAA15BC95FB21AF03389818B\n" + "p accept 53,80,443,5222-5223,25565\n" + "id ed25519 BzffzY99z6Q8KltcFlUTLWjNTBU7yKK+uQhyi1Ivb3A\n"; + + reset_perftime(); + start = perftime(); + for (int i = 0; i < N; ++i) { + smartlist_t *s = microdescs_parse_from_string(md_text, NULL, 1, + SAVED_IN_CACHE, NULL); + SMARTLIST_FOREACH(s, microdesc_t *, md, microdesc_free(md)); + smartlist_free(s); + } + + end = perftime(); + printf("Microdesc parse: %f nsec\n", NANOCOUNT(start, end, N)); +} + typedef void (*bench_fn)(void); typedef struct benchmark_t { @@ -666,6 +704,8 @@ static struct benchmark_t benchmarks[] = { ENT(ecdh_p256), ENT(ecdh_p224), #endif + + ENT(md_parse), {NULL,NULL,0} };

1 0

[tor/master] Use a single path for all PEM-like objects in get_next_token()
by nickm＠torproject.org 18 Dec '18

18 Dec '18

commit 0556942284d7dcdf0a5e7a31e94b925378a338a8 Author: Nick Mathewson <nickm(a)torproject.org> Date: Fri Dec 14 13:17:17 2018 -0500 Use a single path for all PEM-like objects in get_next_token() Previously, we would decode the PEM wrapper for keys twice: once in get_next_token, and once later in PEM decode. Now we just do all of the wrapper and base64 stuff in get_next_token, and store the base64-decoded part in the token object for keys and non-keys alike. This change should speed up parsing slightly by letting us skip a bunch of stuff in crypto_pk_read_*from_string(), including the tag detection parts of pem_decode(), and an extra key allocation and deallocation pair. Retaining the base64-decoded part in the token object will allow us to speed up our microdesc parsing, since it is the asn1 portion that we actually want to retain. --- src/feature/dirparse/parsecommon.c | 24 +++++++++++++----------- src/test/test_parsecommon.c | 8 ++++---- 2 files changed, 17 insertions(+), 15 deletions(-) diff --git a/src/feature/dirparse/parsecommon.c b/src/feature/dirparse/parsecommon.c index 91b775533..2e7cea816 100644 --- a/src/feature/dirparse/parsecommon.c +++ b/src/feature/dirparse/parsecommon.c @@ -266,7 +266,7 @@ get_next_token(memarea_t *area, * attack, a bug, or some other nonsense. */ #define MAX_LINE_LENGTH (128*1024) - const char *next, *eol, *obstart; + const char *next, *eol; size_t obname_len; int i; directory_token_t *tok; @@ -352,7 +352,6 @@ get_next_token(memarea_t *area, if (!eol || eol-*s<11 || strcmpstart(*s, "-----BEGIN ")) /* No object. */ goto check_object; - obstart = *s; /* Set obstart to start of object spec */ if (eol - *s <= 16 || memchr(*s+11,'\0',eol-*s-16) || /* no short lines, */ strcmp_len(eol-5, "-----", 5) || /* nuls or invalid endings */ (eol-*s) > MAX_UNPARSED_OBJECT_SIZE) { /* name too long */ @@ -383,15 +382,7 @@ get_next_token(memarea_t *area, if (next - *s > MAX_UNPARSED_OBJECT_SIZE) RET_ERR("Couldn't parse object: missing footer or object much too big."); - if (!strcmp(tok->object_type, "RSA PUBLIC KEY")) { /* If it's a public key */ - tok->key = crypto_pk_new(); - if (crypto_pk_read_public_key_from_string(tok->key, obstart, eol-obstart)) - RET_ERR("Couldn't parse public key."); - } else if (!strcmp(tok->object_type, "RSA PRIVATE KEY")) { /* private key */ - tok->key = crypto_pk_new(); - if (crypto_pk_read_private_key_from_string(tok->key, obstart, eol-obstart)) - RET_ERR("Couldn't parse private key."); - } else { /* If it's something else, try to base64-decode it */ + { int r; size_t maxsize = base64_decode_maxsize(next-*s); tok->object_body = ALLOC(maxsize); @@ -400,6 +391,17 @@ get_next_token(memarea_t *area, RET_ERR("Malformed object: bad base64-encoded data"); tok->object_size = r; } + + if (!strcmp(tok->object_type, "RSA PUBLIC KEY")) { /* If it's a public key */ + tok->key = crypto_pk_asn1_decode(tok->object_body, tok->object_size); + if (! tok->key) + RET_ERR("Couldn't parse public key."); + } else if (!strcmp(tok->object_type, "RSA PRIVATE KEY")) { /* private key */ + tok->key = crypto_pk_asn1_decode_private(tok->object_body, + tok->object_size); + if (! tok->key) + RET_ERR("Couldn't parse private key."); + } *s = eol; check_object: diff --git a/src/test/test_parsecommon.c b/src/test/test_parsecommon.c index 6da125dd0..8e74fcdb4 100644 --- a/src/test/test_parsecommon.c +++ b/src/test/test_parsecommon.c @@ -300,8 +300,8 @@ test_parsecommon_get_next_token_parse_keys(void *arg) tt_int_op(token->tp, OP_EQ, R_IPO_ONION_KEY); tt_int_op(token->n_args, OP_EQ, 0); tt_str_op(token->object_type, OP_EQ, "RSA PUBLIC KEY"); - tt_int_op(token->object_size, OP_EQ, 0); - tt_assert(!token->object_body); + tt_int_op(token->object_size, OP_EQ, 140); + tt_assert(token->object_body); tt_assert(token->key); tt_assert(!token->error); @@ -335,8 +335,8 @@ test_parsecommon_get_next_token_parse_keys(void *arg) tt_int_op(token2->tp, OP_EQ, C_CLIENT_KEY); tt_int_op(token2->n_args, OP_EQ, 0); tt_str_op(token2->object_type, OP_EQ, "RSA PRIVATE KEY"); - tt_int_op(token2->object_size, OP_EQ, 0); - tt_assert(!token2->object_body); + tt_int_op(token2->object_size, OP_EQ, 608); + tt_assert(token2->object_body); tt_assert(token2->key); tt_assert(!token->error);

1 0