May 2018 - tor-commits - lists.torproject.org

[tor/master] man page: FetchUselessDescriptors and DirCache exclude extra infos
by nickm＠torproject.org 03 May '18

03 May '18

commit d96ddf8f1b143934077bc4c1f54fa8a38087e342 Author: teor <teor2345(a)gmail.com> Date: Wed May 2 11:29:48 2018 +1000 man page: FetchUselessDescriptors and DirCache exclude extra infos --- doc/tor.1.txt | 33 ++++++++++++++++++++------------- 1 file changed, 20 insertions(+), 13 deletions(-) diff --git a/doc/tor.1.txt b/doc/tor.1.txt index f003311d9..53a8c8fe5 100644 --- a/doc/tor.1.txt +++ b/doc/tor.1.txt @@ -540,16 +540,21 @@ GENERAL OPTIONS (Default: 1) [[FetchUselessDescriptors]] **FetchUselessDescriptors** **0**|**1**:: - If set to 1, Tor will fetch every consensus flavor, descriptor, and - certificate that it hears about. Otherwise, it will avoid fetching useless - descriptors: flavors that it is not using to build circuits, and authority - certificates it does not trust. Tor stops fetching descriptors if it - hasn't built any circuits lately: this option makes Tor keep fetching - descriptors, even when idle. This option is useful if you're using a + If set to 1, Tor will fetch every consensus flavor, and all server + descriptors and authority certificates referenced by those consensuses, + except for extra info descriptors. When this option is 1, Tor will also + keep fetching descriptors, even when idle. + If set to 0, Tor will avoid fetching useless descriptors: flavors that it + is not using to build circuits, and authority certificates it does not + trust. When Tor hasn't built any application circuits, it will go idle, + and stop fetching descriptors. This option is useful if you're using a tor client with an external parser that uses a full consensus. - This option fetches all documents, **DirCache** fetches and serves - all documents, **UseMicrodescriptors 0** fetches full descriptors and uses - them to build circuits. (Default: 0) + This option fetches all documents except extrainfo descriptors, + **DirCache** fetches and serves all documents except extrainfo + descriptors, **DownloadExtraInfo*** fetches extrainfo documents, and serves + them if **DirCache** is on, and **UseMicrodescriptors** changes the + flavour of consensues and descriptors that is fetched and used for + building circuits. (Default: 0) [[HTTPProxy]] **HTTPProxy** __host__[:__port__]:: Tor will make all its directory requests through this host:port (or host:80 @@ -2392,10 +2397,12 @@ details.) some entry in the policy is accepted. [[DirCache]] **DirCache** **0**|**1**:: - When this option is set, Tor caches all current directory documents and - accepts client requests for them. Setting DirPort is not required for this, - because clients connect via the ORPort by default. Setting either DirPort - or BridgeRelay and setting DirCache to 0 is not supported. (Default: 1) + When this option is set, Tor caches all current directory documents except + extra info documents, and accepts client requests for them. If + **DownloadExtraInfo** is set, cached extra info documents are also cached. + Setting **DirPort** is not required for **DirCache**, because clients + connect via the ORPort by default. Setting either DirPort or BridgeRelay + and setting DirCache to 0 is not supported. (Default: 1) [[MaxConsensusAgeForDiffs]] **MaxConsensusAgeForDiffs** __N__ **minutes**|**hours**|**days**|**weeks**:: When this option is nonzero, Tor caches will not try to generate

1 0

[tor/release-0.3.3] man page: FetchUselessDescriptors and DirCache exclude extra infos
by nickm＠torproject.org 03 May '18

03 May '18

commit d96ddf8f1b143934077bc4c1f54fa8a38087e342 Author: teor <teor2345(a)gmail.com> Date: Wed May 2 11:29:48 2018 +1000 man page: FetchUselessDescriptors and DirCache exclude extra infos --- doc/tor.1.txt | 33 ++++++++++++++++++++------------- 1 file changed, 20 insertions(+), 13 deletions(-) diff --git a/doc/tor.1.txt b/doc/tor.1.txt index f003311d9..53a8c8fe5 100644 --- a/doc/tor.1.txt +++ b/doc/tor.1.txt @@ -540,16 +540,21 @@ GENERAL OPTIONS (Default: 1) [[FetchUselessDescriptors]] **FetchUselessDescriptors** **0**|**1**:: - If set to 1, Tor will fetch every consensus flavor, descriptor, and - certificate that it hears about. Otherwise, it will avoid fetching useless - descriptors: flavors that it is not using to build circuits, and authority - certificates it does not trust. Tor stops fetching descriptors if it - hasn't built any circuits lately: this option makes Tor keep fetching - descriptors, even when idle. This option is useful if you're using a + If set to 1, Tor will fetch every consensus flavor, and all server + descriptors and authority certificates referenced by those consensuses, + except for extra info descriptors. When this option is 1, Tor will also + keep fetching descriptors, even when idle. + If set to 0, Tor will avoid fetching useless descriptors: flavors that it + is not using to build circuits, and authority certificates it does not + trust. When Tor hasn't built any application circuits, it will go idle, + and stop fetching descriptors. This option is useful if you're using a tor client with an external parser that uses a full consensus. - This option fetches all documents, **DirCache** fetches and serves - all documents, **UseMicrodescriptors 0** fetches full descriptors and uses - them to build circuits. (Default: 0) + This option fetches all documents except extrainfo descriptors, + **DirCache** fetches and serves all documents except extrainfo + descriptors, **DownloadExtraInfo*** fetches extrainfo documents, and serves + them if **DirCache** is on, and **UseMicrodescriptors** changes the + flavour of consensues and descriptors that is fetched and used for + building circuits. (Default: 0) [[HTTPProxy]] **HTTPProxy** __host__[:__port__]:: Tor will make all its directory requests through this host:port (or host:80 @@ -2392,10 +2397,12 @@ details.) some entry in the policy is accepted. [[DirCache]] **DirCache** **0**|**1**:: - When this option is set, Tor caches all current directory documents and - accepts client requests for them. Setting DirPort is not required for this, - because clients connect via the ORPort by default. Setting either DirPort - or BridgeRelay and setting DirCache to 0 is not supported. (Default: 1) + When this option is set, Tor caches all current directory documents except + extra info documents, and accepts client requests for them. If + **DownloadExtraInfo** is set, cached extra info documents are also cached. + Setting **DirPort** is not required for **DirCache**, because clients + connect via the ORPort by default. Setting either DirPort or BridgeRelay + and setting DirCache to 0 is not supported. (Default: 1) [[MaxConsensusAgeForDiffs]] **MaxConsensusAgeForDiffs** __N__ **minutes**|**hours**|**days**|**weeks**:: When this option is nonzero, Tor caches will not try to generate

1 0

[tor/master] Merge branch 'maint-0.3.3'
by nickm＠torproject.org 03 May '18

03 May '18

commit 667e66bb091829cb4b5af351af17f854f527d6d7 Merge: 5e0316142 1bbe75da6 Author: Nick Mathewson <nickm(a)torproject.org> Date: Thu May 3 13:37:37 2018 -0400 Merge branch 'maint-0.3.3' doc/tor.1.txt | 35 ++++++++++++++++++++++------------- 1 file changed, 22 insertions(+), 13 deletions(-)

1 0

[tor/release-0.3.3] Merge remote-tracking branch 'teor/bug25998' into maint-0.3.3
by nickm＠torproject.org 03 May '18

03 May '18

commit 1bbe75da6db551e2c5afe32efe68eb8ea7b60048 Merge: c66b51267 d96ddf8f1 Author: Nick Mathewson <nickm(a)torproject.org> Date: Thu May 3 13:37:33 2018 -0400 Merge remote-tracking branch 'teor/bug25998' into maint-0.3.3 doc/tor.1.txt | 35 ++++++++++++++++++++++------------- 1 file changed, 22 insertions(+), 13 deletions(-)

1 0

[tor/release-0.3.3] man page: UseMicrodescriptors auto and 1 are the same now
by nickm＠torproject.org 03 May '18

03 May '18

commit 4a7590c241dbb86571de670fec3f143d27f15cb7 Author: teor <teor2345(a)gmail.com> Date: Wed May 2 10:46:10 2018 +1000 man page: UseMicrodescriptors auto and 1 are the same now --- doc/tor.1.txt | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/doc/tor.1.txt b/doc/tor.1.txt index 2fc5c413c..f003311d9 100644 --- a/doc/tor.1.txt +++ b/doc/tor.1.txt @@ -1636,9 +1636,8 @@ The following options are useful only for clients (that is, if in order to build its circuits. Using microdescriptors makes Tor clients download less directory information, thus saving bandwidth. Directory caches need to fetch regular descriptors and microdescriptors, so this - option doesn't save any bandwidth for them. If this option is set to - "auto" (recommended) then it is on for all clients that do not set - FetchUselessDescriptors. (Default: auto) + option doesn't save any bandwidth for them. For legacy reasons, auto is + accepted, but it has the same effect as 1. (Default: auto) [[PathBiasCircThreshold]] **PathBiasCircThreshold** __NUM__ +

1 0

[tor/release-0.3.3] Merge branch 'maint-0.3.3' into release-0.3.3
by nickm＠torproject.org 03 May '18

03 May '18

commit f6890422aa954ea2dd23bc4bc55f5716e88dcd7e Merge: 8e81d329e 1bbe75da6 Author: Nick Mathewson <nickm(a)torproject.org> Date: Thu May 3 13:37:37 2018 -0400 Merge branch 'maint-0.3.3' into release-0.3.3 doc/tor.1.txt | 35 ++++++++++++++++++++++------------- 1 file changed, 22 insertions(+), 13 deletions(-)

1 0

[tor/release-0.3.3] man page: FetchUselessDescriptors stops tor going idle
by nickm＠torproject.org 03 May '18

03 May '18

commit 9b1d5e8826e93f5d45a9d1ea7b7223fcfee2e484 Author: teor <teor2345(a)gmail.com> Date: Wed May 2 10:46:00 2018 +1000 man page: FetchUselessDescriptors stops tor going idle Also explain how FetchUselessDescriptors and UseMicrodescriptors are different. Closes 25998. --- doc/tor.1.txt | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/doc/tor.1.txt b/doc/tor.1.txt index dd9f9cefa..2fc5c413c 100644 --- a/doc/tor.1.txt +++ b/doc/tor.1.txt @@ -543,10 +543,13 @@ GENERAL OPTIONS If set to 1, Tor will fetch every consensus flavor, descriptor, and certificate that it hears about. Otherwise, it will avoid fetching useless descriptors: flavors that it is not using to build circuits, and authority - certificates it does not trust. This option is useful if you're using a + certificates it does not trust. Tor stops fetching descriptors if it + hasn't built any circuits lately: this option makes Tor keep fetching + descriptors, even when idle. This option is useful if you're using a tor client with an external parser that uses a full consensus. This option fetches all documents, **DirCache** fetches and serves - all documents. (Default: 0) + all documents, **UseMicrodescriptors 0** fetches full descriptors and uses + them to build circuits. (Default: 0) [[HTTPProxy]] **HTTPProxy** __host__[:__port__]:: Tor will make all its directory requests through this host:port (or host:80

1 0

[tor/master] Move unreachable port warnings to a periodic event.
by nickm＠torproject.org 03 May '18

03 May '18

commit b396e4e42906482d387f7c39b90b1d6ef17c70f2 Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue May 1 13:14:18 2018 -0400 Move unreachable port warnings to a periodic event. Arguably, the conditions under which these events happen should be a bit different, but the rules are complex enough here that I've tried to have this commit be pure refactoring. Closes ticket 25952. Finally, before this code goes away, take a moment to look at the amazing way that we used to try to have an event happen every N seconds: get_uptime() / N != (get_uptime()+seconds_elapsed) / N Truly, it is a thing of wonder. I'm glad we didn't start using this pattern everywhere else. --- changes/ticket25952 | 9 ++++++ src/or/main.c | 89 ++++++++++++++++++++++++++++++----------------------- 2 files changed, 60 insertions(+), 38 deletions(-) diff --git a/changes/ticket25952 b/changes/ticket25952 new file mode 100644 index 000000000..e4dd56df0 --- /dev/null +++ b/changes/ticket25952 @@ -0,0 +1,9 @@ + o Minor features (mainloop): + - Move responsibility for + warning relay operators about unreachable ports + from a once-per-second callback to a callback that is only scheduled as + needed. Once enough items are removed from our once-per-second + callback, we can eliminate it entirely to conserve CPU when idle. + Closes ticket + 25952. + diff --git a/src/or/main.c b/src/or/main.c index c03e80dc0..237e45290 100644 --- a/src/or/main.c +++ b/src/or/main.c @@ -1349,6 +1349,7 @@ CALLBACK(heartbeat); CALLBACK(hs_service); CALLBACK(launch_descriptor_fetches); CALLBACK(launch_reachability_tests); +CALLBACK(reachability_warnings); CALLBACK(record_bridge_stats); CALLBACK(rend_cache_failure_clean); CALLBACK(reset_padding_counts); @@ -1392,6 +1393,8 @@ STATIC periodic_event_item_t periodic_events[] = { CALLBACK(check_onion_keys_expiry_time, PERIODIC_EVENT_ROLE_ROUTER, 0), CALLBACK(expire_old_ciruits_serverside, PERIODIC_EVENT_ROLE_ROUTER, PERIODIC_EVENT_FLAG_NEED_NET), + CALLBACK(reachability_warnings, PERIODIC_EVENT_ROLE_ROUTER, + PERIODIC_EVENT_FLAG_NEED_NET), CALLBACK(retry_dns, PERIODIC_EVENT_ROLE_ROUTER, 0), CALLBACK(rotate_onion_key, PERIODIC_EVENT_ROLE_ROUTER, 0), @@ -2329,6 +2332,54 @@ expire_old_ciruits_serverside_callback(time_t now, const or_options_t *options) return 11; } +/** + * Callback: Send warnings if Tor doesn't find its ports reachable. + */ +static int +reachability_warnings_callback(time_t now, const or_options_t *options) +{ + (void) now; + + if (get_uptime() < TIMEOUT_UNTIL_UNREACHABILITY_COMPLAINT) { + return (int)(TIMEOUT_UNTIL_UNREACHABILITY_COMPLAINT - get_uptime()); + } + + if (server_mode(options) && + !net_is_disabled() && + have_completed_a_circuit()) { + /* every 20 minutes, check and complain if necessary */ + const routerinfo_t *me = router_get_my_routerinfo(); + if (me && !check_whether_orport_reachable(options)) { + char *address = tor_dup_ip(me->addr); + log_warn(LD_CONFIG,"Your server (%s:%d) has not managed to confirm that " + "its ORPort is reachable. Relays do not publish descriptors " + "until their ORPort and DirPort are reachable. Please check " + "your firewalls, ports, address, /etc/hosts file, etc.", + address, me->or_port); + control_event_server_status(LOG_WARN, + "REACHABILITY_FAILED ORADDRESS=%s:%d", + address, me->or_port); + tor_free(address); + } + + if (me && !check_whether_dirport_reachable(options)) { + char *address = tor_dup_ip(me->addr); + log_warn(LD_CONFIG, + "Your server (%s:%d) has not managed to confirm that its " + "DirPort is reachable. Relays do not publish descriptors " + "until their ORPort and DirPort are reachable. Please check " + "your firewalls, ports, address, /etc/hosts file, etc.", + address, me->dir_port); + control_event_server_status(LOG_WARN, + "REACHABILITY_FAILED DIRADDRESS=%s:%d", + address, me->dir_port); + tor_free(address); + } + } + + return TIMEOUT_UNTIL_UNREACHABILITY_COMPLAINT; +} + static int dns_honesty_first_time = 1; /** @@ -2456,7 +2507,6 @@ second_elapsed_callback(periodic_timer_t *timer, void *arg) size_t bytes_written; size_t bytes_read; int seconds_elapsed; - const or_options_t *options = get_options(); (void)timer; (void)arg; @@ -2479,43 +2529,6 @@ second_elapsed_callback(periodic_timer_t *timer, void *arg) control_event_circ_bandwidth_used(); control_event_circuit_cell_stats(); - if (server_mode(options) && - !net_is_disabled() && - seconds_elapsed > 0 && - have_completed_a_circuit() && - get_uptime() / TIMEOUT_UNTIL_UNREACHABILITY_COMPLAINT != - (get_uptime()+seconds_elapsed) / - TIMEOUT_UNTIL_UNREACHABILITY_COMPLAINT) { - /* every 20 minutes, check and complain if necessary */ - const routerinfo_t *me = router_get_my_routerinfo(); - if (me && !check_whether_orport_reachable(options)) { - char *address = tor_dup_ip(me->addr); - log_warn(LD_CONFIG,"Your server (%s:%d) has not managed to confirm that " - "its ORPort is reachable. Relays do not publish descriptors " - "until their ORPort and DirPort are reachable. Please check " - "your firewalls, ports, address, /etc/hosts file, etc.", - address, me->or_port); - control_event_server_status(LOG_WARN, - "REACHABILITY_FAILED ORADDRESS=%s:%d", - address, me->or_port); - tor_free(address); - } - - if (me && !check_whether_dirport_reachable(options)) { - char *address = tor_dup_ip(me->addr); - log_warn(LD_CONFIG, - "Your server (%s:%d) has not managed to confirm that its " - "DirPort is reachable. Relays do not publish descriptors " - "until their ORPort and DirPort are reachable. Please check " - "your firewalls, ports, address, /etc/hosts file, etc.", - address, me->dir_port); - control_event_server_status(LOG_WARN, - "REACHABILITY_FAILED DIRADDRESS=%s:%d", - address, me->dir_port); - tor_free(address); - } - } - /** If more than this many seconds have elapsed, probably the clock * jumped: doesn't count. */ #define NUM_JUMPED_SECONDS_BEFORE_WARN 100

1 0

[tor/master] Merge remote-tracking branch 'github/ticket25952'
by nickm＠torproject.org 03 May '18

03 May '18

commit 5e0316142f14df890e34180dba087cd0264fba70 Merge: ed636de4c b396e4e42 Author: Nick Mathewson <nickm(a)torproject.org> Date: Thu May 3 11:59:53 2018 -0400 Merge remote-tracking branch 'github/ticket25952' changes/ticket25952 | 9 ++++++ src/or/main.c | 89 ++++++++++++++++++++++++++++++----------------------- 2 files changed, 60 insertions(+), 38 deletions(-)

1 0

[tor/master] Add a mechanism for the logging system to report queued callbacks
by nickm＠torproject.org 03 May '18

03 May '18

commit b0224bf7282f2ec968a46e35a2a8dab1ddaf0667 Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue May 1 10:05:22 2018 -0400 Add a mechanism for the logging system to report queued callbacks Sometimes the logging system will queue a log message for later. When it does this, the callback will either get flushed at the next safe time, or from the second-elapsed callback. But we're trying to eliminate the second-elapsed callback, so let's make a way for the log system to tell its users about this. --- src/common/log.c | 25 +++++++++++++++++++++++++ src/common/torlog.h | 2 ++ 2 files changed, 27 insertions(+) diff --git a/src/common/log.c b/src/common/log.c index 922e9dd38..ebd50f62d 100644 --- a/src/common/log.c +++ b/src/common/log.c @@ -170,6 +170,9 @@ typedef struct pending_log_message_t { /** Log messages waiting to be replayed onto callback-based logs */ static smartlist_t *pending_cb_messages = NULL; +/** Callback to invoke when pending_cb_messages becomes nonempty. */ +static pending_callback_callback pending_cb_cb = NULL; + /** Log messages waiting to be replayed once the logging system is initialized. */ static smartlist_t *pending_startup_messages = NULL; @@ -538,6 +541,9 @@ logfile_deliver(logfile_t *lf, const char *buf, size_t msg_len, smartlist_add(pending_cb_messages, pending_log_message_new(severity,domain,NULL,msg_after_prefix)); *callbacks_deferred = 1; + if (smartlist_len(pending_cb_messages) == 1 && pending_cb_cb) { + pending_cb_cb(); + } } } else { lf->callback(severity, domain, msg_after_prefix); @@ -825,6 +831,7 @@ logs_free_all(void) logfiles = NULL; messages = pending_cb_messages; pending_cb_messages = NULL; + pending_cb_cb = NULL; messages2 = pending_startup_messages; pending_startup_messages = NULL; UNLOCK_LOGS(); @@ -988,6 +995,24 @@ add_temp_log(int min_severity) } /** + * Register "cb" as the callback to call when there are new pending log + * callbacks to be flushed with flush_pending_log_callbacks(). + * + * Note that this callback, if present, can be invoked from any thread. + * + * This callback must not log. + * + * It is intentional that this function contains the name "callback" twice: it + * sets a "callback" to be called on the condition that there is a "pending + * callback". + **/ +void +logs_set_pending_callback_callback(pending_callback_callback cb) +{ + pending_cb_cb = cb; +} + +/** * Add a log handler to send messages in <b>severity</b> * to the function <b>cb</b>. */ diff --git a/src/common/torlog.h b/src/common/torlog.h index ac632ff52..de389883c 100644 --- a/src/common/torlog.h +++ b/src/common/torlog.h @@ -154,6 +154,8 @@ int add_android_log(const log_severity_list_t *severity, const char *android_identity_tag); #endif // HAVE_ANDROID_LOG_H. int add_callback_log(const log_severity_list_t *severity, log_callback cb); +typedef void (*pending_callback_callback)(void); +void logs_set_pending_callback_callback(pending_callback_callback cb); void logs_set_domain_logging(int enabled); int get_min_log_level(void); void switch_logs_debug(void);

1 0