June 2016 - tor-commits - lists.torproject.org

[translation/https_everywhere_completed] Update translations for https_everywhere_completed
by translation＠torproject.org 23 Jun '16

23 Jun '16

commit 3b143c3986443ced5f21aa02578abe30b424180c Author: Translation commit bot <translation(a)torproject.org> Date: Thu Jun 23 16:45:34 2016 +0000 Update translations for https_everywhere_completed --- fi/https-everywhere.dtd | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/fi/https-everywhere.dtd b/fi/https-everywhere.dtd index 8c9a8ea..b809f85 100644 --- a/fi/https-everywhere.dtd +++ b/fi/https-everywhere.dtd @@ -5,6 +5,7 @@ <!ENTITY https-everywhere.about.created_by "Tekijät:"> <!ENTITY https-everywhere.about.and ", ja"> <!ENTITY https-everywhere.about.librarians "Sääntöjen hallinnoijat"> +<!ENTITY https-everywhere.about.add_new_rule "Lisää Uusi Sääntö"> <!ENTITY https-everywhere.about.thanks "Kiitokset"> <!ENTITY https-everywhere.about.many_contributors "Monia myötävaikuttajia, mukaanlukien"> <!ENTITY https-everywhere.about.noscript "Lisäksi, osia HTTPS Everywhere:stä perustuu Giorgio Maonen ja muiden koodiin NoScriptistä. Olemme kiitollisia heidän mahtavasta työstään!"> @@ -14,6 +15,7 @@ <!ENTITY https-everywhere.about.or "tai"> <!ENTITY https-everywhere.about.donate_eff "Lahjoita EFFille"> +<!ENTITY https-everywhere.menu.donate_eff_imperative "Lahjoita EFF:lle"> <!ENTITY https-everywhere.menu.about "Tietoja HTTPS Everywheresta"> <!ENTITY https-everywhere.menu.observatory "SSL Observatoryn asetukset"> <!ENTITY https-everywhere.menu.globalEnable "Ota HTTPS Everywhere käyttöön"> @@ -41,3 +43,15 @@ <!ENTITY https-everywhere.chrome.stable_rules_description "Pakota salatut yhteydet näihin nettisivustoihin:"> <!ENTITY https-everywhere.chrome.experimental_rules "Kokeelliset käytännöt"> <!ENTITY https-everywhere.chrome.experimental_rules_description "Voi aiheuttaa varoituksia tai hajoamisen. Pois käytöstä oletuksena."> +<!ENTITY https-everywhere.chrome.add_rule "Lisää sääntö sivulle"> +<!ENTITY https-everywhere.chrome.add_new_rule "Lisää uusi sääntö sivulle"> +<!ENTITY https-everywhere.chrome.always_https_for_host "Käytä aina https tälle isännälle"> +<!ENTITY https-everywhere.chrome.host "Isäntä:"> +<!ENTITY https-everywhere.chrome.show_advanced "Näytä lisäasetukset"> +<!ENTITY https-everywhere.chrome.hide_advanced "Piillota lisäasetukset"> +<!ENTITY https-everywhere.chrome.rule_name "Säännön nimi"> +<!ENTITY https-everywhere.chrome.regex "Sopiva säännöllinen lauseke"> +<!ENTITY https-everywhere.chrome.redirect_to "Uudelleenohjaa kohteeseen"> +<!ENTITY https-everywhere.chrome.status_cancel_button "Peruuta"> + +

1 0

[translation/https_everywhere] Update translations for https_everywhere
by translation＠torproject.org 23 Jun '16

23 Jun '16

commit 7f8f9fec601df5c54e4503e8f6f23f07f4352b01 Author: Translation commit bot <translation(a)torproject.org> Date: Thu Jun 23 16:45:27 2016 +0000 Update translations for https_everywhere --- fi/https-everywhere.dtd | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/fi/https-everywhere.dtd b/fi/https-everywhere.dtd index a7ffdd1..b809f85 100644 --- a/fi/https-everywhere.dtd +++ b/fi/https-everywhere.dtd @@ -5,7 +5,7 @@ <!ENTITY https-everywhere.about.created_by "Tekijät:"> <!ENTITY https-everywhere.about.and ", ja"> <!ENTITY https-everywhere.about.librarians "Sääntöjen hallinnoijat"> -<!ENTITY https-everywhere.about.add_new_rule "Add New Rule"> +<!ENTITY https-everywhere.about.add_new_rule "Lisää Uusi Sääntö"> <!ENTITY https-everywhere.about.thanks "Kiitokset"> <!ENTITY https-everywhere.about.many_contributors "Monia myötävaikuttajia, mukaanlukien"> <!ENTITY https-everywhere.about.noscript "Lisäksi, osia HTTPS Everywhere:stä perustuu Giorgio Maonen ja muiden koodiin NoScriptistä. Olemme kiitollisia heidän mahtavasta työstään!"> @@ -43,14 +43,14 @@ <!ENTITY https-everywhere.chrome.stable_rules_description "Pakota salatut yhteydet näihin nettisivustoihin:"> <!ENTITY https-everywhere.chrome.experimental_rules "Kokeelliset käytännöt"> <!ENTITY https-everywhere.chrome.experimental_rules_description "Voi aiheuttaa varoituksia tai hajoamisen. Pois käytöstä oletuksena."> -<!ENTITY https-everywhere.chrome.add_rule "Add a rule for this site"> -<!ENTITY https-everywhere.chrome.add_new_rule "Add a new rule for this site"> +<!ENTITY https-everywhere.chrome.add_rule "Lisää sääntö sivulle"> +<!ENTITY https-everywhere.chrome.add_new_rule "Lisää uusi sääntö sivulle"> <!ENTITY https-everywhere.chrome.always_https_for_host "Käytä aina https tälle isännälle"> <!ENTITY https-everywhere.chrome.host "Isäntä:"> -<!ENTITY https-everywhere.chrome.show_advanced "Show advanced"> -<!ENTITY https-everywhere.chrome.hide_advanced "Hide advanced"> -<!ENTITY https-everywhere.chrome.rule_name "Rule name"> -<!ENTITY https-everywhere.chrome.regex "Matching regex"> +<!ENTITY https-everywhere.chrome.show_advanced "Näytä lisäasetukset"> +<!ENTITY https-everywhere.chrome.hide_advanced "Piillota lisäasetukset"> +<!ENTITY https-everywhere.chrome.rule_name "Säännön nimi"> +<!ENTITY https-everywhere.chrome.regex "Sopiva säännöllinen lauseke"> <!ENTITY https-everywhere.chrome.redirect_to "Uudelleenohjaa kohteeseen"> <!ENTITY https-everywhere.chrome.status_cancel_button "Peruuta">

1 0

[torsocks/master] Fix typo: catched -> caught
by dgoulet＠torproject.org 23 Jun '16

23 Jun '16

commit 1964b72e5592fea857aa7a5440babfc8805b5be7 Author: intrigeri <intrigeri(a)boum.org> Date: Thu Jun 23 11:11:07 2016 -0400 Fix typo: catched -> caught Signed-off-by: David Goulet <dgoulet(a)ev0ke.net> --- src/common/compat.h | 2 +- src/lib/close.c | 2 +- src/lib/connect.c | 2 +- src/lib/fclose.c | 2 +- src/lib/sendto.c | 2 +- 5 files changed, 5 insertions(+), 5 deletions(-) diff --git a/src/common/compat.h b/src/common/compat.h index b14df7f..41ab8a8 100644 --- a/src/common/compat.h +++ b/src/common/compat.h @@ -65,7 +65,7 @@ void tsocks_once(tsocks_once_t *o, void (*init_routine)(void)); /* * Some old system requires kernel headers for those values. If they are not * defined, set them to a bad syscall value. Just to be clear, if the value is - * undefined, tsocks syscall() will DENY the real syscall if catched. + * undefined, tsocks syscall() will DENY the real syscall if caught. * * The values are not the same per syscall here so we don't end up with * duplicates in the switch case in the tsocks sycall wrapper. diff --git a/src/lib/close.c b/src/lib/close.c index 7fb9e53..23c80e6 100644 --- a/src/lib/close.c +++ b/src/lib/close.c @@ -30,7 +30,7 @@ LIBC_CLOSE_RET_TYPE tsocks_close(LIBC_CLOSE_SIG) { struct connection *conn; - DBG("Close catched for fd %d", fd); + DBG("Close caught for fd %d", fd); connection_registry_lock(); conn = connection_find(fd); diff --git a/src/lib/connect.c b/src/lib/connect.c index 71c5886..4f18289 100644 --- a/src/lib/connect.c +++ b/src/lib/connect.c @@ -115,7 +115,7 @@ LIBC_CONNECT_RET_TYPE tsocks_connect(LIBC_CONNECT_SIG) struct connection *new_conn; struct onion_entry *on_entry; - DBG("Connect catched on fd %d", sockfd); + DBG("Connect caught on fd %d", sockfd); /* * Validate socket values in order to see if we can handle this connect diff --git a/src/lib/fclose.c b/src/lib/fclose.c index b6f3251..6bb6a2c 100644 --- a/src/lib/fclose.c +++ b/src/lib/fclose.c @@ -42,7 +42,7 @@ LIBC_FCLOSE_RET_TYPE tsocks_fclose(LIBC_FCLOSE_SIG) goto error; } - DBG("[fclose] Close catched for fd %d", fd); + DBG("[fclose] Close caught for fd %d", fd); connection_registry_lock(); conn = connection_find(fd); diff --git a/src/lib/sendto.c b/src/lib/sendto.c index 75994f4..c8461bc 100644 --- a/src/lib/sendto.c +++ b/src/lib/sendto.c @@ -51,7 +51,7 @@ LIBC_SENDTO_RET_TYPE tsocks_sendto(LIBC_SENDTO_SIG) goto libc_sendto; } - DBG("[sendto] TCP fast open catched on fd %d", sockfd); + DBG("[sendto] TCP fast open caught on fd %d", sockfd); ret = connect(sockfd, dest_addr, addrlen); if (ret == 0) {

1 0

[torsocks/master] syscall: Whitelist sched_getaffinity(2)
by dgoulet＠torproject.org 23 Jun '16

23 Jun '16

commit e389c99937a35815e2752952634cc80c3aaf1d4e Author: David Goulet <dgoulet(a)ev0ke.net> Date: Thu Jun 23 09:51:27 2016 -0400 syscall: Whitelist sched_getaffinity(2) ffmepg needed this syscall else it would end up in an infinite loop. Reported in Debian: https://bugs.debian.org/805741 Signed-off-by: David Goulet <dgoulet(a)ev0ke.net> --- src/common/compat.h | 5 +++++ src/lib/syscall.c | 22 ++++++++++++++++++++++ 2 files changed, 27 insertions(+) diff --git a/src/common/compat.h b/src/common/compat.h index 38bf191..b14df7f 100644 --- a/src/common/compat.h +++ b/src/common/compat.h @@ -109,6 +109,9 @@ void tsocks_once(tsocks_once_t *o, void (*init_routine)(void)); #ifndef __NR_accept4 #define __NR_accept4 -13 #endif +#ifndef __NR_sched_getaffinity +#define __NR_sched_getaffinity -14 +#endif #define TSOCKS_NR_SOCKET __NR_socket #define TSOCKS_NR_CONNECT __NR_connect @@ -123,6 +126,7 @@ void tsocks_once(tsocks_once_t *o, void (*init_routine)(void)); #define TSOCKS_NR_GETRANDOM __NR_getrandom #define TSOCKS_NR_FUTEX __NR_futex #define TSOCKS_NR_ACCEPT4 __NR_accept4 +#define TSOCKS_NR_SCHED_GETAFFINITY __NR_sched_getaffinity /* * Despite glibc providing wrappers for these calls for a long time @@ -188,6 +192,7 @@ void tsocks_once(tsocks_once_t *o, void (*init_routine)(void)); #define TSOCKS_NR_GETPEERNAME SYS_getpeername #define TSOCKS_NR_LISTEN SYS_listen #define TSOCKS_NR_RECVMSG SYS_recvmsg +#define TSOCKS_NR_SCHED_GETAFFINITY SYS_sched_getaffinity #endif /* __FreeBSD__, __FreeBSD_kernel__, __darwin__, __NetBSD__ */ diff --git a/src/lib/syscall.c b/src/lib/syscall.c index 8b24f4c..9495a64 100644 --- a/src/lib/syscall.c +++ b/src/lib/syscall.c @@ -164,6 +164,25 @@ static LIBC_RECVMSG_RET_TYPE handle_recvmsg(va_list args) return tsocks_recvmsg(sockfd, msg, flags); } +/* + * Handle sched_getaffinity(2) syscall. + * NOTE: ffmpeg is one of the application that needs this one on the + * whitelist. + */ +static LIBC_SYSCALL_RET_TYPE handle_sched_getaffinity(va_list args) +{ + pid_t pid; + size_t cpusetsize; + cpu_set_t *mask; + + pid = va_arg(args, __typeof__(pid)); + cpusetsize = va_arg(args, __typeof__(cpusetsize)); + mask = va_arg(args, __typeof__(mask)); + + return tsocks_libc_syscall(TSOCKS_NR_SCHED_GETAFFINITY, pid, cpusetsize, + mask); +} + #if defined(__linux__) /* * Handle gettid(2) syscall. @@ -455,6 +474,9 @@ LIBC_SYSCALL_RET_TYPE tsocks_syscall(long int number, va_list args) case TSOCKS_NR_INOTIFY_RM_WATCH: ret = handle_inotify_rm_watch(args); break; + case TSOCKS_NR_SCHED_GETAFFINITY: + ret = handle_sched_getaffinity(args); + break; #endif /* __linux__ */ default: /*

1 0

[torspec/master] Fixed a typo in EXTENSIONS field ( Length -> ExtLength )
by nickm＠torproject.org 23 Jun '16

23 Jun '16

commit 92b465ba910c21853d4524b8684d1079b1ca0dfb Author: Ivan Markin <twim(a)riseup.net> Date: Thu Jun 23 10:08:54 2016 +0000 Fixed a typo in EXTENSIONS field ( Length -> ExtLength ) --- cert-spec.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cert-spec.txt b/cert-spec.txt index c54fd86..b242022 100644 --- a/cert-spec.txt +++ b/cert-spec.txt @@ -55,7 +55,7 @@ ExtLength [2 bytes] ExtType [1 byte] ExtFlags [1 byte] - ExtData [Length bytes] + ExtData [ExtLength bytes] The meaning of the ExtData field in an extension is type-dependent.

1 0

[tor/master] Implement DL_SCHED_RANDOM_EXPONENTIAL support for download_status_t
by nickm＠torproject.org 23 Jun '16

23 Jun '16

commit 695b0bd1d5aca52a05df1a697a6b23a20be529d4 Author: Andrea Shepard <andrea(a)torproject.org> Date: Sun Jun 12 19:07:11 2016 +0000 Implement DL_SCHED_RANDOM_EXPONENTIAL support for download_status_t --- src/or/directory.c | 112 +++++++++++++++++++++++++++++++++++++++++++++---- src/or/directory.h | 4 ++ src/or/networkstatus.c | 8 ++-- src/or/or.h | 16 +++++++ src/or/routerlist.c | 3 ++ 5 files changed, 130 insertions(+), 13 deletions(-) diff --git a/src/or/directory.c b/src/or/directory.c index 6caca11..5b890ca 100644 --- a/src/or/directory.c +++ b/src/or/directory.c @@ -3762,6 +3762,28 @@ find_dl_schedule(download_status_t *dls, const or_options_t *options) return NULL; } +/** Decide which minimum and maximum delay step we want to use based on + * descriptor type in <b>dls</b> and <b>options</b>. + * Helper function for download_status_schedule_get_delay(). */ +STATIC void +find_dl_min_and_max_delay(download_status_t *dls, const or_options_t *options, + int *min, int *max) +{ + tor_assert(dls); + tor_assert(options); + tor_assert(min); + tor_assert(max); + + /* + * For now, just use the existing schedule config stuff and pick the + * first/last entries off to get min/max delay for backoff purposes + */ + const smartlist_t *schedule = find_dl_schedule(dls, options); + tor_assert(schedule != NULL && smartlist_len(schedule) >= 2); + *min = *((int *)(smartlist_get(schedule, 0))); + *max = *((int *)((smartlist_get(schedule, smartlist_len(schedule) - 1)))); +} + /* Find the current delay for dls based on schedule. * Set dls->next_attempt_at based on now, and return the delay. * Helper for download_status_increment_failure and @@ -3769,23 +3791,85 @@ find_dl_schedule(download_status_t *dls, const or_options_t *options) STATIC int download_status_schedule_get_delay(download_status_t *dls, const smartlist_t *schedule, + int min_delay, int max_delay, time_t now) { tor_assert(dls); - tor_assert(schedule); + /* We don't need a schedule if we're using random exponential backoff */ + tor_assert(dls->backoff == DL_SCHED_RANDOM_EXPONENTIAL || + schedule != NULL); + /* If we're using random exponential backoff, we do need min/max delay */ + tor_assert(dls->backoff != DL_SCHED_RANDOM_EXPONENTIAL || + (min_delay >= 0 && max_delay >= min_delay && + max_delay <= INT_MAX)); int delay = INT_MAX; + int delay_increment, i; uint8_t dls_schedule_position = (dls->increment_on == DL_SCHED_INCREMENT_ATTEMPT ? dls->n_download_attempts : dls->n_download_failures); + uint8_t entropy; - if (dls_schedule_position < smartlist_len(schedule)) - delay = *(int *)smartlist_get(schedule, dls_schedule_position); - else if (dls_schedule_position == IMPOSSIBLE_TO_DOWNLOAD) - delay = INT_MAX; - else - delay = *(int *)smartlist_get(schedule, smartlist_len(schedule) - 1); + if (dls->backoff == DL_SCHED_DETERMINISTIC) { + if (dls_schedule_position < smartlist_len(schedule)) + delay = *(int *)smartlist_get(schedule, dls_schedule_position); + else if (dls_schedule_position == IMPOSSIBLE_TO_DOWNLOAD) + delay = INT_MAX; + else + delay = *(int *)smartlist_get(schedule, smartlist_len(schedule) - 1); + } else if (dls->backoff == DL_SCHED_RANDOM_EXPONENTIAL) { + /* Check if we missed a reset somehow */ + if (dls->last_backoff_position > dls_schedule_position) { + dls->last_backoff_position = 0; + dls->last_delay_used = 0; + } + + if (dls_schedule_position > 0) { + delay = dls->last_delay_used; + + while (dls->last_backoff_position < dls_schedule_position) { + /* + * Backoff step: we want to multiply by something ~1.5, and then add + * 1 with non-zero probability so we can't get stuck at zero even if + * we start out with zero delay. To do this, pick a uint8_t of + * entropy in the range [0,255], and use it to construct an + * increment. + */ + delay_increment = 0; + /* Get a byte of entropy */ + crypto_rand((char *)(&entropy), sizeof(entropy)); + /* Clamp it just to be sure */ + entropy &= 0xff; + /* If we have non-zero delay; otherwise this is a no-op */ + if (delay > 0) { + /* Use the low 7 bits for the increment */ + for (i = 0; i < 7; ++i) { + if (entropy & (0x1 << i)) delay_increment += (delay >> (i + 1)); + } + } + /* + * Using the remaining bit of entropy, add 1 with probability 50% so + * we can't get stuck at 0 + */ + if (entropy & 0x80) delay_increment += 1; + /* Increment delay, make sure to saturate if we would wrap around */ + if (delay_increment < max_delay - delay) delay += delay_increment; + else delay = max_delay; + ++(dls->last_backoff_position); + } + } else { + delay = min_delay; + } + + /* Clamp it within min/max if we have them */ + if (min_delay >= 0 && delay < min_delay) delay = min_delay; + if (max_delay != INT_MAX && delay > max_delay) delay = max_delay; + + /* Store it for next time */ + dls->last_backoff_position = dls_schedule_position; + dls->last_delay_used = delay; + } /* A negative delay makes no sense. Knowing that delay is * non-negative allows us to safely do the wrapping check below. */ @@ -3846,6 +3930,8 @@ download_status_increment_failure(download_status_t *dls, int status_code, const char *item, int server, time_t now) { int increment = -1; + int min_delay = 0, max_delay = INT_MAX; + tor_assert(dls); /* only count the failure if it's permanent, or we're a server */ @@ -3866,7 +3952,9 @@ download_status_increment_failure(download_status_t *dls, int status_code, /* only return a failure retry time if this schedule increments on failures */ const smartlist_t *schedule = find_dl_schedule(dls, get_options()); - increment = download_status_schedule_get_delay(dls, schedule, now); + find_dl_min_and_max_delay(dls, get_options(), &min_delay, &max_delay); + increment = download_status_schedule_get_delay(dls, schedule, + min_delay, max_delay, now); } download_status_log_helper(item, !dls->increment_on, "failed", @@ -3895,6 +3983,8 @@ download_status_increment_attempt(download_status_t *dls, const char *item, time_t now) { int delay = -1; + int min_delay = 0, max_delay = INT_MAX; + tor_assert(dls); if (dls->increment_on == DL_SCHED_INCREMENT_FAILURE) { @@ -3909,7 +3999,9 @@ download_status_increment_attempt(download_status_t *dls, const char *item, ++dls->n_download_attempts; const smartlist_t *schedule = find_dl_schedule(dls, get_options()); - delay = download_status_schedule_get_delay(dls, schedule, now); + find_dl_min_and_max_delay(dls, get_options(), &min_delay, &max_delay); + delay = download_status_schedule_get_delay(dls, schedule, + min_delay, max_delay, now); download_status_log_helper(item, dls->increment_on, "attempted", "on failure", dls->n_download_attempts, @@ -3941,6 +4033,8 @@ download_status_reset(download_status_t *dls) dls->n_download_failures = 0; dls->n_download_attempts = 0; dls->next_attempt_at = time(NULL) + *(int *)smartlist_get(schedule, 0); + dls->last_backoff_position = 0; + dls->last_delay_used = 0; /* Don't reset dls->want_authority or dls->increment_on */ } diff --git a/src/or/directory.h b/src/or/directory.h index 7646cac..d867aa3 100644 --- a/src/or/directory.h +++ b/src/or/directory.h @@ -146,6 +146,7 @@ STATIC int directory_handle_command_get(dir_connection_t *conn, size_t req_body_len); STATIC int download_status_schedule_get_delay(download_status_t *dls, const smartlist_t *schedule, + int min_delay, int max_delay, time_t now); STATIC char* authdir_type_to_string(dirinfo_type_t auth); @@ -154,6 +155,9 @@ STATIC int should_use_directory_guards(const or_options_t *options); STATIC zlib_compression_level_t choose_compression_level(ssize_t n_bytes); STATIC const smartlist_t *find_dl_schedule(download_status_t *dls, const or_options_t *options); +STATIC void find_dl_min_and_max_delay(download_status_t *dls, + const or_options_t *options, + int *min, int *max); #endif #endif diff --git a/src/or/networkstatus.c b/src/or/networkstatus.c index 5a91dda..db4ed85 100644 --- a/src/or/networkstatus.c +++ b/src/or/networkstatus.c @@ -86,9 +86,9 @@ static time_t time_to_download_next_consensus[N_CONSENSUS_FLAVORS]; static download_status_t consensus_dl_status[N_CONSENSUS_FLAVORS] = { { 0, 0, 0, DL_SCHED_CONSENSUS, DL_WANT_ANY_DIRSERVER, - DL_SCHED_INCREMENT_FAILURE }, + DL_SCHED_INCREMENT_FAILURE, DL_SCHED_DETERMINISTIC, 0, 0 }, { 0, 0, 0, DL_SCHED_CONSENSUS, DL_WANT_ANY_DIRSERVER, - DL_SCHED_INCREMENT_FAILURE }, + DL_SCHED_INCREMENT_FAILURE, DL_SCHED_DETERMINISTIC, 0, 0 }, }; #define N_CONSENSUS_BOOTSTRAP_SCHEDULES 2 @@ -105,10 +105,10 @@ static download_status_t consensus_bootstrap_dl_status[N_CONSENSUS_BOOTSTRAP_SCHEDULES] = { { 0, 0, 0, DL_SCHED_CONSENSUS, DL_WANT_AUTHORITY, - DL_SCHED_INCREMENT_ATTEMPT }, + DL_SCHED_INCREMENT_ATTEMPT, DL_SCHED_DETERMINISTIC, 0, 0 }, /* During bootstrap, DL_WANT_ANY_DIRSERVER means "use fallbacks". */ { 0, 0, 0, DL_SCHED_CONSENSUS, DL_WANT_ANY_DIRSERVER, - DL_SCHED_INCREMENT_ATTEMPT }, + DL_SCHED_INCREMENT_ATTEMPT, DL_SCHED_DETERMINISTIC, 0, 0 }, }; /** True iff we have logged a warning about this OR's version being older than diff --git a/src/or/or.h b/src/or/or.h index 5a19f9a..efe5680 100644 --- a/src/or/or.h +++ b/src/or/or.h @@ -1987,6 +1987,15 @@ typedef enum { #define download_schedule_increment_bitfield_t \ ENUM_BF(download_schedule_increment_t) +/** Enumeration: do we want to use the random exponential backoff + * mechanism? */ +typedef enum { + DL_SCHED_DETERMINISTIC = 0, + DL_SCHED_RANDOM_EXPONENTIAL = 1, +} download_schedule_backoff_t; +#define download_schedule_backoff_bitfield_t \ + ENUM_BF(download_schedule_backoff_t) + /** Information about our plans for retrying downloads for a downloadable * directory object. * Each type of downloadable directory object has a corresponding retry @@ -2033,6 +2042,13 @@ typedef struct download_status_t { download_schedule_increment_bitfield_t increment_on : 1; /**< does this * schedule increment on each attempt, * or after each failure? */ + download_schedule_backoff_bitfield_t backoff : 1; /**< do we use the + * deterministic schedule, or random + * exponential backoffs? */ + uint8_t last_backoff_position; /**< number of attempts/failures, depending + * on increment_on, when we last recalculated + * the delay. */ + int last_delay_used; /**< last delay used for random exponential backoff */ } download_status_t; /** If n_download_failures is this high, the download can never happen. */ diff --git a/src/or/routerlist.c b/src/or/routerlist.c index aaa8fad..6721925 100644 --- a/src/or/routerlist.c +++ b/src/or/routerlist.c @@ -159,6 +159,9 @@ download_status_cert_init(download_status_t *dlstatus) dlstatus->schedule = DL_SCHED_CONSENSUS; dlstatus->want_authority = DL_WANT_ANY_DIRSERVER; dlstatus->increment_on = DL_SCHED_INCREMENT_FAILURE; + dlstatus->backoff = DL_SCHED_DETERMINISTIC; + dlstatus->last_backoff_position = 0; + dlstatus->last_delay_used = 0; /* Use the new schedule to set next_attempt_at */ download_status_reset(dlstatus);

1 0

[tor/master] Unit test for DL_SCHED_RANDOM_EXPONENTIAL
by nickm＠torproject.org 23 Jun '16

23 Jun '16

commit 1553512af40fea10b060db4d236527a4f73e8a6c Author: Andrea Shepard <andrea(a)torproject.org> Date: Sun Jun 12 19:07:38 2016 +0000 Unit test for DL_SCHED_RANDOM_EXPONENTIAL --- src/test/test_dir.c | 69 +++++++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 64 insertions(+), 5 deletions(-) diff --git a/src/test/test_dir.c b/src/test/test_dir.c index 74b753a..7acb4a3 100644 --- a/src/test/test_dir.c +++ b/src/test/test_dir.c @@ -3333,13 +3333,16 @@ test_dir_download_status_schedule(void *arg) (void)arg; download_status_t dls_failure = { 0, 0, 0, DL_SCHED_GENERIC, DL_WANT_AUTHORITY, - DL_SCHED_INCREMENT_FAILURE }; + DL_SCHED_INCREMENT_FAILURE, + DL_SCHED_DETERMINISTIC, 0, 0 }; download_status_t dls_attempt = { 0, 0, 0, DL_SCHED_CONSENSUS, DL_WANT_ANY_DIRSERVER, - DL_SCHED_INCREMENT_ATTEMPT}; + DL_SCHED_INCREMENT_ATTEMPT, + DL_SCHED_DETERMINISTIC, 0, 0 }; download_status_t dls_bridge = { 0, 0, 0, DL_SCHED_BRIDGE, DL_WANT_AUTHORITY, - DL_SCHED_INCREMENT_FAILURE}; + DL_SCHED_INCREMENT_FAILURE, + DL_SCHED_DETERMINISTIC, 0, 0 }; int increment = -1; int expected_increment = -1; time_t current_time = time(NULL); @@ -3355,6 +3358,7 @@ test_dir_download_status_schedule(void *arg) delay1 = 1000; increment = download_status_schedule_get_delay(&dls_failure, schedule, + 0, INT_MAX, TIME_MIN); expected_increment = delay1; tt_assert(increment == expected_increment); @@ -3363,6 +3367,7 @@ test_dir_download_status_schedule(void *arg) delay1 = INT_MAX; increment = download_status_schedule_get_delay(&dls_failure, schedule, + 0, INT_MAX, -1); expected_increment = delay1; tt_assert(increment == expected_increment); @@ -3371,6 +3376,7 @@ test_dir_download_status_schedule(void *arg) delay1 = 0; increment = download_status_schedule_get_delay(&dls_attempt, schedule, + 0, INT_MAX, 0); expected_increment = delay1; tt_assert(increment == expected_increment); @@ -3379,6 +3385,7 @@ test_dir_download_status_schedule(void *arg) delay1 = 1000; increment = download_status_schedule_get_delay(&dls_attempt, schedule, + 0, INT_MAX, 1); expected_increment = delay1; tt_assert(increment == expected_increment); @@ -3387,6 +3394,7 @@ test_dir_download_status_schedule(void *arg) delay1 = INT_MAX; increment = download_status_schedule_get_delay(&dls_bridge, schedule, + 0, INT_MAX, current_time); expected_increment = delay1; tt_assert(increment == expected_increment); @@ -3395,6 +3403,7 @@ test_dir_download_status_schedule(void *arg) delay1 = 1; increment = download_status_schedule_get_delay(&dls_bridge, schedule, + 0, INT_MAX, TIME_MAX); expected_increment = delay1; tt_assert(increment == expected_increment); @@ -3407,6 +3416,7 @@ test_dir_download_status_schedule(void *arg) delay2 = 100; increment = download_status_schedule_get_delay(&dls_attempt, schedule, + 0, INT_MAX, current_time); expected_increment = delay2; tt_assert(increment == expected_increment); @@ -3415,6 +3425,7 @@ test_dir_download_status_schedule(void *arg) delay2 = 1; increment = download_status_schedule_get_delay(&dls_bridge, schedule, + 0, INT_MAX, current_time); expected_increment = delay2; tt_assert(increment == expected_increment); @@ -3427,6 +3438,7 @@ test_dir_download_status_schedule(void *arg) delay2 = 5; increment = download_status_schedule_get_delay(&dls_attempt, schedule, + 0, INT_MAX, current_time); expected_increment = delay2; tt_assert(increment == expected_increment); @@ -3435,6 +3447,7 @@ test_dir_download_status_schedule(void *arg) delay2 = 17; increment = download_status_schedule_get_delay(&dls_bridge, schedule, + 0, INT_MAX, current_time); expected_increment = delay2; tt_assert(increment == expected_increment); @@ -3447,6 +3460,7 @@ test_dir_download_status_schedule(void *arg) delay2 = 35; increment = download_status_schedule_get_delay(&dls_attempt, schedule, + 0, INT_MAX, current_time); expected_increment = INT_MAX; tt_assert(increment == expected_increment); @@ -3455,6 +3469,7 @@ test_dir_download_status_schedule(void *arg) delay2 = 99; increment = download_status_schedule_get_delay(&dls_bridge, schedule, + 0, INT_MAX, current_time); expected_increment = INT_MAX; tt_assert(increment == expected_increment); @@ -3466,15 +3481,58 @@ test_dir_download_status_schedule(void *arg) } static void +test_dir_download_status_random_backoff(void *arg) +{ + download_status_t dls_random = + { 0, 0, 0, DL_SCHED_GENERIC, DL_WANT_AUTHORITY, + DL_SCHED_INCREMENT_FAILURE, DL_SCHED_RANDOM_EXPONENTIAL, 0, 0 }; + int increment = -1; + int old_increment; + time_t current_time = time(NULL); + const int min_delay = 0; + const int max_delay = 1000000; + + (void)arg; + + /* Check the random backoff cases */ + old_increment = 0; + do { + increment = download_status_schedule_get_delay(&dls_random, + NULL, + min_delay, max_delay, + current_time); + /* Test */ + tt_int_op(increment, OP_GE, min_delay); + tt_int_op(increment, OP_LE, max_delay); + tt_int_op(increment, OP_GE, old_increment); + /* We at most double, and maybe add one */ + tt_int_op(increment, OP_LE, 2 * old_increment + 1); + + /* Advance */ + current_time += increment; + ++(dls_random.n_download_attempts); + ++(dls_random.n_download_failures); + + /* Try another maybe */ + old_increment = increment; + } while (increment < max_delay); + + done: + return; +} + +static void test_dir_download_status_increment(void *arg) { (void)arg; download_status_t dls_failure = { 0, 0, 0, DL_SCHED_GENERIC, DL_WANT_AUTHORITY, - DL_SCHED_INCREMENT_FAILURE }; + DL_SCHED_INCREMENT_FAILURE, + DL_SCHED_DETERMINISTIC, 0, 0 }; download_status_t dls_attempt = { 0, 0, 0, DL_SCHED_BRIDGE, DL_WANT_ANY_DIRSERVER, - DL_SCHED_INCREMENT_ATTEMPT}; + DL_SCHED_INCREMENT_ATTEMPT, + DL_SCHED_DETERMINISTIC, 0, 0 }; int delay0 = -1; int delay1 = -1; int delay2 = -1; @@ -4243,6 +4301,7 @@ struct testcase_t dir_tests[] = { DIR(fetch_type, 0), DIR(packages, 0), DIR(download_status_schedule, 0), + DIR(download_status_random_backoff, 0), DIR(download_status_increment, 0), DIR(authdir_type_to_string, 0), DIR(conn_purpose_to_string, 0),

1 0

[tor/master] Use exponential backoffs for consensus downloads
by nickm＠torproject.org 23 Jun '16

23 Jun '16

commit 5104e5645f8bdb2197e57e45eed746470056f0d3 Author: Andrea Shepard <andrea(a)torproject.org> Date: Sun Jun 12 21:07:24 2016 +0000 Use exponential backoffs for consensus downloads --- src/or/networkstatus.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/src/or/networkstatus.c b/src/or/networkstatus.c index db4ed85..d5c8a26 100644 --- a/src/or/networkstatus.c +++ b/src/or/networkstatus.c @@ -86,9 +86,9 @@ static time_t time_to_download_next_consensus[N_CONSENSUS_FLAVORS]; static download_status_t consensus_dl_status[N_CONSENSUS_FLAVORS] = { { 0, 0, 0, DL_SCHED_CONSENSUS, DL_WANT_ANY_DIRSERVER, - DL_SCHED_INCREMENT_FAILURE, DL_SCHED_DETERMINISTIC, 0, 0 }, + DL_SCHED_INCREMENT_FAILURE, DL_SCHED_RANDOM_EXPONENTIAL, 0, 0 }, { 0, 0, 0, DL_SCHED_CONSENSUS, DL_WANT_ANY_DIRSERVER, - DL_SCHED_INCREMENT_FAILURE, DL_SCHED_DETERMINISTIC, 0, 0 }, + DL_SCHED_INCREMENT_FAILURE, DL_SCHED_RANDOM_EXPONENTIAL, 0, 0 }, }; #define N_CONSENSUS_BOOTSTRAP_SCHEDULES 2 @@ -105,10 +105,10 @@ static download_status_t consensus_bootstrap_dl_status[N_CONSENSUS_BOOTSTRAP_SCHEDULES] = { { 0, 0, 0, DL_SCHED_CONSENSUS, DL_WANT_AUTHORITY, - DL_SCHED_INCREMENT_ATTEMPT, DL_SCHED_DETERMINISTIC, 0, 0 }, + DL_SCHED_INCREMENT_ATTEMPT, DL_SCHED_RANDOM_EXPONENTIAL, 0, 0 }, /* During bootstrap, DL_WANT_ANY_DIRSERVER means "use fallbacks". */ { 0, 0, 0, DL_SCHED_CONSENSUS, DL_WANT_ANY_DIRSERVER, - DL_SCHED_INCREMENT_ATTEMPT, DL_SCHED_DETERMINISTIC, 0, 0 }, + DL_SCHED_INCREMENT_ATTEMPT, DL_SCHED_RANDOM_EXPONENTIAL, 0, 0 }, }; /** True iff we have logged a warning about this OR's version being older than

1 0

[tor/master] Use exponential backoff for router descriptor downloads from consensuses
by nickm＠torproject.org 23 Jun '16

23 Jun '16

commit 6370c4ee87204bae264ff9c5a5cf5872958beda9 Author: Andrea Shepard <andrea(a)torproject.org> Date: Sun Jun 12 21:27:22 2016 +0000 Use exponential backoff for router descriptor downloads from consensuses --- src/or/routerparse.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/or/routerparse.c b/src/or/routerparse.c index cefe607..a612a94 100644 --- a/src/or/routerparse.c +++ b/src/or/routerparse.c @@ -3204,6 +3204,8 @@ networkstatus_parse_vote_from_string(const char *s, const char **eos_out, NULL, NULL, ns->consensus_method, flav))) + /* Use exponential-backoff scheduling when downloading microdescs */ + rs->dl_status.backoff = DL_SCHED_RANDOM_EXPONENTIAL; smartlist_add(ns->routerstatus_list, rs); } }

1 0

[tor/master] Changes file for random exponential backoffs
by nickm＠torproject.org 23 Jun '16

23 Jun '16

commit 2905a3484eb02e5cf3bc3474240a42e24c07ab74 Author: Andrea Shepard <andrea(a)torproject.org> Date: Sun Jun 12 21:29:11 2016 +0000 Changes file for random exponential backoffs --- changes/bug15942 | 3 +++ 1 file changed, 3 insertions(+) diff --git a/changes/bug15942 b/changes/bug15942 new file mode 100644 index 0000000..0edc2b7 --- /dev/null +++ b/changes/bug15942 @@ -0,0 +1,3 @@ + o Bugfixes (downloading): + - Use random exponential backoffs when retrying downloads from the dir + servers. Fixes bug 15942; bugfix on ?????.

1 0