November 2018 - tor-commits - lists.torproject.org

[torspec/master] Merge remote-tracking branch 'tor-github/pr/43'
by nickm＠torproject.org 27 Nov '18

27 Nov '18

commit bab8a89223eebd28445aa3c4c9e41370b891b8c2 Merge: 4421149 2decb3e Author: Nick Mathewson <nickm(a)torproject.org> Date: Mon Nov 26 16:50:47 2018 -0500 Merge remote-tracking branch 'tor-github/pr/43' dir-spec.txt | 18 ------------------ 1 file changed, 18 deletions(-) diff --cc dir-spec.txt index 3058180,7312dd2..a6547ce --- a/dir-spec.txt +++ b/dir-spec.txt @@@ -2105,44 -2105,8 +2105,26 @@@ If an authority is configured with a V3BandwidthsFile, but parsing fails, this line SHOULD appear in its vote, but without any headers. - Current Keywords: - "timestamp" -- the Unix Epoch time in seconds of the most recent - generator result. - "version" -- the Bandwidth List format document specification version. - "software" -- the name of the software that created the document. - "software_version" -- the version of the software that created the - document. - "file_created" -- the date and time timestamp in ISO 8601 format - and UTC time zone when the file was created. - "generator_started" -- the date and time timestamp in ISO 8601 format - and UTC time zone when the generator started. - "earliest_bandwidth" -- the date and time timestamp in ISO 8601 format - and UTC time zone when the first relay bandwidth was obtained. - "latest_bandwidth" -- the date and time timestamp in ISO 8601 format - and UTC time zone of the most recent generator result. - This time MUST be identical to the initial Timestamp line. - This duplicate value is included to make the format easier for people - to read. First-appeared: Tor 0.3.5.1-alpha. + "bandwidth-file-digest" 1*(SP algorithm "=" digest) NL + + [At most once for votes; does not occur in consensuses.] + + A digest of the bandwidth file used to generate this vote. + "algorithm" is the name of the hash algorithm producing "digest", + which can be "sha256" or another algorithm. "digest" is the + base64 encoding of the hash of the bandwidth file, with trailing =s + omitted. + + If an authority is not configured with a V3BandwidthsFile, this line + SHOULD NOT appear in its vote. + + If an authority is configured with a V3BandwidthsFile, but parsing + fails, this line SHOULD appear in its vote, with the digest(s) of the + unparseable file. + First-appeared: Tor 0.4.0.1-alpha + The authority section of a vote contains the following items, followed in turn by the authority's current key certificate:

1 0

[torspec/master] dir-spec: remove Keywords in bandwidth-file-headers
by nickm＠torproject.org 27 Nov '18

27 Nov '18

commit 2decb3ea271dd316bbe1dab3b3c51667252bea85 Author: juga0 <juga(a)riseup.net> Date: Sun Nov 11 08:33:27 2018 +0000 dir-spec: remove Keywords in bandwidth-file-headers Remove the header's Keywords in the bandwidth-file-headers item, since they are described in bandwidth-file-spec.txt and we should not need to update two specifications every time they change. --- dir-spec.txt | 18 ------------------ 1 file changed, 18 deletions(-) diff --git a/dir-spec.txt b/dir-spec.txt index 8da28c4..7312dd2 100644 --- a/dir-spec.txt +++ b/dir-spec.txt @@ -2105,24 +2105,6 @@ If an authority is configured with a V3BandwidthsFile, but parsing fails, this line SHOULD appear in its vote, but without any headers. - Current Keywords: - "timestamp" -- the Unix Epoch time in seconds of the most recent - generator result. - "version" -- the Bandwidth List format document specification version. - "software" -- the name of the software that created the document. - "software_version" -- the version of the software that created the - document. - "file_created" -- the date and time timestamp in ISO 8601 format - and UTC time zone when the file was created. - "generator_started" -- the date and time timestamp in ISO 8601 format - and UTC time zone when the generator started. - "earliest_bandwidth" -- the date and time timestamp in ISO 8601 format - and UTC time zone when the first relay bandwidth was obtained. - "latest_bandwidth" -- the date and time timestamp in ISO 8601 format - and UTC time zone of the most recent generator result. - This time MUST be identical to the initial Timestamp line. - This duplicate value is included to make the format easier for people - to read. First-appeared: Tor 0.3.5.1-alpha. The authority section of a vote contains the following items, followed

1 0

[torspec/master] Describe SIGNAL ACTIVE and SIGNAL DORMANT.
by nickm＠torproject.org 27 Nov '18

27 Nov '18

commit 4421149986369b4f746fc02a5d78c7337fe5d4ea Author: Nick Mathewson <nickm(a)torproject.org> Date: Mon Nov 19 16:41:19 2018 -0500 Describe SIGNAL ACTIVE and SIGNAL DORMANT. --- control-spec.txt | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/control-spec.txt b/control-spec.txt index 6a04b65..ae4a417 100644 --- a/control-spec.txt +++ b/control-spec.txt @@ -430,7 +430,7 @@ Signal = "RELOAD" / "SHUTDOWN" / "DUMP" / "DEBUG" / "HALT" / "HUP" / "INT" / "USR1" / "USR2" / "TERM" / "NEWNYM" / - "CLEARDNSCACHE" / "HEARTBEAT" + "CLEARDNSCACHE" / "HEARTBEAT" / "ACTIVE" / "DORMANT" The meaning of the signals are: @@ -448,6 +448,12 @@ the client-side DNS cache. (Tor MAY rate-limit its response to this signal.) HEARTBEAT -- Make Tor dump an unscheduled Heartbeat message to log. + DORMANT -- Tell Tor to become "dormant". A dormant Tor will + try to avoid CPU and network usage until it receives + user-initiated network request. (Don't use this + on relays or hidden services yet!) + ACTIVE -- Tell Tor to stop being "dormant", as if it had received + a user-initiated network request. The server responds with "250 OK" if the signal is recognized (or simply closes the socket if it was asked to close immediately), or "552 @@ -463,6 +469,8 @@ DUMP: USR1 DEBUG: USR2 + [SIGNAL DORMANT and SIGNAL ACTIVE were added in 0.4.0.1-alpha.] + 3.8. MAPADDRESS Sent from the client to the server. The syntax is:

1 0

[tor/master] Changes file for "Dormant Mode" (28335, 2149).
by nickm＠torproject.org 27 Nov '18

27 Nov '18

commit e12fdeb18199925070f4e99e60f2c5bda7f6af82 Author: Nick Mathewson <nickm(a)torproject.org> Date: Mon Nov 26 16:39:44 2018 -0500 Changes file for "Dormant Mode" (28335, 2149). --- changes/ticket28335 | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/changes/ticket28335 b/changes/ticket28335 new file mode 100644 index 000000000..eecf7c7fd --- /dev/null +++ b/changes/ticket28335 @@ -0,0 +1,7 @@ + o Major features (client): + - When Tor is running as a client, and it is unused for a long time, it + can now enter a "dormant" state. When Tor is dormant, it avoids + network activity and CPU wakeups until it is reawoken either by a user + request or by a controller command. For more information, see + the configuration options starting with "Dormant". Implements tickets + 2149 and 28335.

1 0

[tor/master] Add a function to schedule a periodic event once, then disable it
by nickm＠torproject.org 27 Nov '18

27 Nov '18

commit 6d84972eb8e27d5e9f1adea36fcc9a9879d718ad Author: Nick Mathewson <nickm(a)torproject.org> Date: Mon Nov 5 16:24:10 2018 -0500 Add a function to schedule a periodic event once, then disable it --- src/core/mainloop/mainloop.c | 6 +++++- src/core/mainloop/periodic.c | 22 +++++++++++++++++----- src/core/mainloop/periodic.h | 7 ++++++- 3 files changed, 28 insertions(+), 7 deletions(-) diff --git a/src/core/mainloop/mainloop.c b/src/core/mainloop/mainloop.c index 12820888f..42f6fb50c 100644 --- a/src/core/mainloop/mainloop.c +++ b/src/core/mainloop/mainloop.c @@ -1600,7 +1600,11 @@ rescan_periodic_events(const or_options_t *options) periodic_event_enable(item); } else { log_debug(LD_GENERAL, "Disabling periodic event %s", item->name); - periodic_event_disable(item); + if (item->flags & PERIODIC_EVENT_FLAG_FLUSH_ON_DISABLE) { + periodic_event_flush_and_disable(item); + } else { + periodic_event_disable(item); + } } } } diff --git a/src/core/mainloop/periodic.c b/src/core/mainloop/periodic.c index c1785eb38..c290c3744 100644 --- a/src/core/mainloop/periodic.c +++ b/src/core/mainloop/periodic.c @@ -45,10 +45,6 @@ periodic_event_dispatch(mainloop_event_t *ev, void *data) periodic_event_item_t *event = data; tor_assert(ev == event->ev); - if (BUG(!periodic_event_is_enabled(event))) { - return; - } - time_t now = time(NULL); update_current_time(now); const or_options_t *options = get_options(); @@ -57,7 +53,7 @@ periodic_event_dispatch(mainloop_event_t *ev, void *data) int next_interval = 0; if (!periodic_event_is_enabled(event)) { - /* The event got disabled from inside its callback; no need to + /* The event got disabled from inside its callback, or before: no need to * reschedule. */ return; } @@ -172,3 +168,19 @@ periodic_event_disable(periodic_event_item_t *event) mainloop_event_cancel(event->ev); event->enabled = 0; } + +/** + * Disable an event, then schedule it to run once. + * Do nothing if the event was already disabled. + */ +void +periodic_event_flush_and_disable(periodic_event_item_t *event) +{ + tor_assert(event); + if (!periodic_event_is_enabled(event)) + return; + + periodic_event_disable(event); + + mainloop_event_activate(event->ev); +} diff --git a/src/core/mainloop/periodic.h b/src/core/mainloop/periodic.h index 4c8c3c96c..7c71be7bc 100644 --- a/src/core/mainloop/periodic.h +++ b/src/core/mainloop/periodic.h @@ -39,6 +39,11 @@ * the net_is_disabled() check. */ #define PERIODIC_EVENT_FLAG_NEED_NET (1U << 0) +/* Indicate that it the event is enabled, it event needs to be run once before + * it becomes disabled. + */ +#define PERIODIC_EVENT_FLAG_FLUSH_ON_DISABLE (1U << 1) + /** Callback function for a periodic event to take action. The return value * influences the next time the function will get called. Return * PERIODIC_EVENT_NO_UPDATE to not update <b>last_action_time</b> and be polled @@ -83,6 +88,6 @@ void periodic_event_destroy(periodic_event_item_t *event); void periodic_event_reschedule(periodic_event_item_t *event); void periodic_event_enable(periodic_event_item_t *event); void periodic_event_disable(periodic_event_item_t *event); +void periodic_event_flush_and_disable(periodic_event_item_t *event); #endif /* !defined(TOR_PERIODIC_H) */ -

1 0

[tor/master] Use macros to make the periodic event table less verbose.
by nickm＠torproject.org 27 Nov '18

27 Nov '18

commit 2070765c7c062c505358d0f1c83f2846181d1667 Author: Nick Mathewson <nickm(a)torproject.org> Date: Mon Nov 5 16:09:13 2018 -0500 Use macros to make the periodic event table less verbose. --- src/core/mainloop/mainloop.c | 78 ++++++++++++++++++++------------------------ 1 file changed, 35 insertions(+), 43 deletions(-) diff --git a/src/core/mainloop/mainloop.c b/src/core/mainloop/mainloop.c index 7eff82fee..12820888f 100644 --- a/src/core/mainloop/mainloop.c +++ b/src/core/mainloop/mainloop.c @@ -1370,74 +1370,66 @@ CALLBACK(write_stats_file); #undef CALLBACK /* Now we declare an array of periodic_event_item_t for each periodic event */ -#define CALLBACK(name, r, f) PERIODIC_EVENT(name, r, f) +#define CALLBACK(name, r, f) \ + PERIODIC_EVENT(name, PERIODIC_EVENT_ROLE_ ## r, f) +#define FL(name) (PERIODIC_EVENT_FLAG_ ## name) STATIC periodic_event_item_t periodic_events[] = { /* Everyone needs to run those. */ - CALLBACK(add_entropy, PERIODIC_EVENT_ROLE_ALL, 0), - CALLBACK(check_expired_networkstatus, PERIODIC_EVENT_ROLE_ALL, 0), - CALLBACK(clean_caches, PERIODIC_EVENT_ROLE_ALL, 0), - CALLBACK(fetch_networkstatus, PERIODIC_EVENT_ROLE_ALL, - PERIODIC_EVENT_FLAG_NEED_NET), - CALLBACK(heartbeat, PERIODIC_EVENT_ROLE_ALL, 0), - CALLBACK(launch_descriptor_fetches, PERIODIC_EVENT_ROLE_ALL, - PERIODIC_EVENT_FLAG_NEED_NET), - CALLBACK(reset_padding_counts, PERIODIC_EVENT_ROLE_ALL, 0), - CALLBACK(retry_listeners, PERIODIC_EVENT_ROLE_ALL, - PERIODIC_EVENT_FLAG_NEED_NET), - CALLBACK(save_state, PERIODIC_EVENT_ROLE_ALL, 0), - CALLBACK(rotate_x509_certificate, PERIODIC_EVENT_ROLE_ALL, 0), - CALLBACK(write_stats_file, PERIODIC_EVENT_ROLE_ALL, 0), + CALLBACK(add_entropy, ALL, 0), + CALLBACK(check_expired_networkstatus, ALL, 0), + CALLBACK(clean_caches, ALL, 0), + CALLBACK(fetch_networkstatus, ALL, 0), + CALLBACK(heartbeat, ALL, 0), + CALLBACK(launch_descriptor_fetches, ALL, FL(NEED_NET)), + CALLBACK(reset_padding_counts, ALL, 0), + CALLBACK(retry_listeners, ALL, FL(NEED_NET)), + CALLBACK(save_state, ALL, 0), + CALLBACK(rotate_x509_certificate, ALL, 0), + CALLBACK(write_stats_file, ALL, 0), /* Routers (bridge and relay) only. */ - CALLBACK(check_descriptor, PERIODIC_EVENT_ROLE_ROUTER, - PERIODIC_EVENT_FLAG_NEED_NET), - CALLBACK(check_ed_keys, PERIODIC_EVENT_ROLE_ROUTER, 0), - CALLBACK(check_for_reachability_bw, PERIODIC_EVENT_ROLE_ROUTER, - PERIODIC_EVENT_FLAG_NEED_NET), - CALLBACK(check_onion_keys_expiry_time, PERIODIC_EVENT_ROLE_ROUTER, 0), - CALLBACK(expire_old_ciruits_serverside, PERIODIC_EVENT_ROLE_ROUTER, - PERIODIC_EVENT_FLAG_NEED_NET), - CALLBACK(reachability_warnings, PERIODIC_EVENT_ROLE_ROUTER, - PERIODIC_EVENT_FLAG_NEED_NET), - CALLBACK(retry_dns, PERIODIC_EVENT_ROLE_ROUTER, 0), - CALLBACK(rotate_onion_key, PERIODIC_EVENT_ROLE_ROUTER, 0), + CALLBACK(check_descriptor, ROUTER, FL(NEED_NET)), + CALLBACK(check_ed_keys, ROUTER, 0), + CALLBACK(check_for_reachability_bw, ROUTER, FL(NEED_NET)), + CALLBACK(check_onion_keys_expiry_time, ROUTER, 0), + CALLBACK(expire_old_ciruits_serverside, ROUTER, FL(NEED_NET)), + CALLBACK(reachability_warnings, ROUTER, FL(NEED_NET)), + CALLBACK(retry_dns, ROUTER, 0), + CALLBACK(rotate_onion_key, ROUTER, 0), /* Authorities (bridge and directory) only. */ - CALLBACK(downrate_stability, PERIODIC_EVENT_ROLE_AUTHORITIES, 0), - CALLBACK(launch_reachability_tests, PERIODIC_EVENT_ROLE_AUTHORITIES, - PERIODIC_EVENT_FLAG_NEED_NET), - CALLBACK(save_stability, PERIODIC_EVENT_ROLE_AUTHORITIES, 0), + CALLBACK(downrate_stability, AUTHORITIES, 0), + CALLBACK(launch_reachability_tests, AUTHORITIES, FL(NEED_NET)), + CALLBACK(save_stability, AUTHORITIES, 0), /* Directory authority only. */ - CALLBACK(check_authority_cert, PERIODIC_EVENT_ROLE_DIRAUTH, 0), - CALLBACK(dirvote, PERIODIC_EVENT_ROLE_DIRAUTH, PERIODIC_EVENT_FLAG_NEED_NET), + CALLBACK(check_authority_cert, DIRAUTH, 0), + CALLBACK(dirvote, DIRAUTH, FL(NEED_NET)), /* Relay only. */ - CALLBACK(check_canonical_channels, PERIODIC_EVENT_ROLE_RELAY, - PERIODIC_EVENT_FLAG_NEED_NET), - CALLBACK(check_dns_honesty, PERIODIC_EVENT_ROLE_RELAY, - PERIODIC_EVENT_FLAG_NEED_NET), + CALLBACK(check_canonical_channels, RELAY, FL(NEED_NET)), + CALLBACK(check_dns_honesty, RELAY, FL(NEED_NET)), /* Hidden Service service only. */ - CALLBACK(hs_service, PERIODIC_EVENT_ROLE_HS_SERVICE, - PERIODIC_EVENT_FLAG_NEED_NET), + CALLBACK(hs_service, HS_SERVICE, FL(NEED_NET)), /* Bridge only. */ - CALLBACK(record_bridge_stats, PERIODIC_EVENT_ROLE_BRIDGE, 0), + CALLBACK(record_bridge_stats, BRIDGE, 0), /* Client only. */ - CALLBACK(rend_cache_failure_clean, PERIODIC_EVENT_ROLE_CLIENT, 0), + CALLBACK(rend_cache_failure_clean, CLIENT, 0), /* Bridge Authority only. */ - CALLBACK(write_bridge_ns, PERIODIC_EVENT_ROLE_BRIDGEAUTH, 0), + CALLBACK(write_bridge_ns, BRIDGEAUTH, 0), /* Directory server only. */ - CALLBACK(clean_consdiffmgr, PERIODIC_EVENT_ROLE_DIRSERVER, 0), + CALLBACK(clean_consdiffmgr, DIRSERVER, 0), END_OF_PERIODIC_EVENTS }; #undef CALLBACK +#undef FL /* These are pointers to members of periodic_events[] that are used to * implement particular callbacks. We keep them separate here so that we

1 0

[tor/master] Move control_per_second_events() into a callback with its own role
by nickm＠torproject.org 27 Nov '18

27 Nov '18

commit a0380b705daceb69c29ccda3a2f1453b9dcbc40d Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Nov 13 08:22:58 2018 -0500 Move control_per_second_events() into a callback with its own role Part of making extra-dormant mode work; closes ticket 28421. --- src/core/mainloop/mainloop.c | 29 +++++++++++++++++++++++------ src/core/mainloop/periodic.h | 5 +++-- src/feature/control/control.c | 2 +- src/test/test_periodic_event.c | 3 ++- 4 files changed, 29 insertions(+), 10 deletions(-) diff --git a/src/core/mainloop/mainloop.c b/src/core/mainloop/mainloop.c index 7e5e5d0ef..a9d5d8155 100644 --- a/src/core/mainloop/mainloop.c +++ b/src/core/mainloop/mainloop.c @@ -1366,6 +1366,7 @@ CALLBACK(save_stability); CALLBACK(save_state); CALLBACK(write_bridge_ns); CALLBACK(write_stats_file); +CALLBACK(control_per_second_events); #undef CALLBACK @@ -1439,6 +1440,9 @@ STATIC periodic_event_item_t periodic_events[] = { /* Directory server only. */ CALLBACK(clean_consdiffmgr, DIRSERVER, 0), + /* Controller with per-second events only. */ + CALLBACK(control_per_second_events, CONTROLEV, 0), + END_OF_PERIODIC_EVENTS }; #undef CALLBACK @@ -1498,6 +1502,8 @@ get_my_roles(const or_options_t *options) int is_hidden_service = !!hs_service_get_num_services() || !!rend_num_services(); int is_dirserver = dir_server_mode(options); + int sending_control_events = control_any_per_second_event_enabled(); + /* We also consider tor to have the role of a client if the ControlPort is * set because a lot of things can be done over the control port which * requires tor to have basic functionnalities. */ @@ -1516,6 +1522,7 @@ get_my_roles(const or_options_t *options) if (is_hidden_service) roles |= PERIODIC_EVENT_ROLE_HS_SERVICE; if (is_dirserver) roles |= PERIODIC_EVENT_ROLE_DIRSERVER; if (is_net_participant) roles |= PERIODIC_EVENT_ROLE_NET_PARTICIPANT; + if (sending_control_events) roles |= PERIODIC_EVENT_ROLE_CONTROLEV; return roles; } @@ -2524,6 +2531,21 @@ hs_service_callback(time_t now, const or_options_t *options) return 1; } +/* + * Periodic callback: Send once-per-second events to the controller(s). + * This is called every second. + */ +static int +control_per_second_events_callback(time_t now, const or_options_t *options) +{ + (void) options; + (void) now; + + control_per_second_events(); + + return 1; +} + /** Timer: used to invoke second_elapsed_callback() once per second. */ static periodic_timer_t *second_timer = NULL; @@ -2546,8 +2568,7 @@ reschedule_per_second_timer(void) tor_assert(second_timer); } - const bool run_per_second_events = - control_any_per_second_event_enabled() || ! net_is_completely_disabled(); + const bool run_per_second_events = ! net_is_completely_disabled(); if (run_per_second_events) { periodic_timer_launch(second_timer, &one_second); @@ -2640,10 +2661,6 @@ second_elapsed_callback(periodic_timer_t *timer, void *arg) */ update_current_time(now); - // TODO XXXX Turn this into a separate event. - /* Maybe some controller events are ready to fire */ - control_per_second_events(); - run_scheduled_events(now); } diff --git a/src/core/mainloop/periodic.h b/src/core/mainloop/periodic.h index 23459ff2b..52d5450ee 100644 --- a/src/core/mainloop/periodic.h +++ b/src/core/mainloop/periodic.h @@ -15,9 +15,10 @@ #define PERIODIC_EVENT_ROLE_BRIDGEAUTH (1U << 4) #define PERIODIC_EVENT_ROLE_HS_SERVICE (1U << 5) #define PERIODIC_EVENT_ROLE_DIRSERVER (1U << 6) +#define PERIODIC_EVENT_ROLE_CONTROLEV (1U << 7) -#define PERIODIC_EVENT_ROLE_NET_PARTICIPANT (1U << 7) -#define PERIODIC_EVENT_ROLE_ALL (1U << 8) +#define PERIODIC_EVENT_ROLE_NET_PARTICIPANT (1U << 8) +#define PERIODIC_EVENT_ROLE_ALL (1U << 9) /* Helper macro to make it a bit less annoying to defined groups of roles that * are often used. */ diff --git a/src/feature/control/control.c b/src/feature/control/control.c index b31b448e9..a5b6ab3bf 100644 --- a/src/feature/control/control.c +++ b/src/feature/control/control.c @@ -368,7 +368,7 @@ control_update_global_event_mask(void) control_get_bytes_rw_last_sec(&r, &w); } if (any_old_per_sec_events != control_any_per_second_event_enabled()) { - reschedule_per_second_timer(); + rescan_periodic_events(get_options()); } #undef NEWLY_ENABLED diff --git a/src/test/test_periodic_event.c b/src/test/test_periodic_event.c index f63adf8e3..6a3e320b2 100644 --- a/src/test/test_periodic_event.c +++ b/src/test/test_periodic_event.c @@ -172,7 +172,8 @@ test_pe_launch(void *arg) for (int i = 0; periodic_events[i].name; ++i) { periodic_event_item_t *item = &periodic_events[i]; - tt_int_op(periodic_event_is_enabled(item), OP_EQ, 1); + tt_int_op(periodic_event_is_enabled(item), OP_EQ, + (item->roles != PERIODIC_EVENT_ROLE_CONTROLEV)); } done:

1 0

[tor/master] Add new "ALL" and "NET_PARTICIPANT" roles for periodic events
by nickm＠torproject.org 27 Nov '18

27 Nov '18

commit b9a88bd53ae79a29c292275381bc7dbaa3804034 Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Nov 6 07:27:31 2018 -0500 Add new "ALL" and "NET_PARTICIPANT" roles for periodic events The previous "ALL" role was the OR of a bunch of other roles, which is a mistake: it's better if "ALL" means "all". The "NET_PARTICIPANT" role refers to the anything that is actively building circuits, downloading directory information, and participating in the Tor network. For now, it is set to !net_is_disabled(), but we're going to use it to implement a new "extra dormant mode". Closes ticket 28336. --- src/core/mainloop/mainloop.c | 36 ++++++++++++++++++++++--------- src/core/mainloop/periodic.h | 7 +++--- src/test/test_periodic_event.c | 49 ++++++++++++++++++++++++++---------------- 3 files changed, 60 insertions(+), 32 deletions(-) diff --git a/src/core/mainloop/mainloop.c b/src/core/mainloop/mainloop.c index 42f6fb50c..3c3f441a9 100644 --- a/src/core/mainloop/mainloop.c +++ b/src/core/mainloop/mainloop.c @@ -1377,16 +1377,27 @@ CALLBACK(write_stats_file); STATIC periodic_event_item_t periodic_events[] = { /* Everyone needs to run those. */ CALLBACK(add_entropy, ALL, 0), - CALLBACK(check_expired_networkstatus, ALL, 0), - CALLBACK(clean_caches, ALL, 0), - CALLBACK(fetch_networkstatus, ALL, 0), CALLBACK(heartbeat, ALL, 0), - CALLBACK(launch_descriptor_fetches, ALL, FL(NEED_NET)), - CALLBACK(reset_padding_counts, ALL, 0), + + /* XXXX Do we have a reason to do this on a callback? */ CALLBACK(retry_listeners, ALL, FL(NEED_NET)), - CALLBACK(save_state, ALL, 0), - CALLBACK(rotate_x509_certificate, ALL, 0), - CALLBACK(write_stats_file, ALL, 0), + + /* We need to do these if we're participating in the Tor network. */ + CALLBACK(check_expired_networkstatus, NET_PARTICIPANT, 0), + CALLBACK(fetch_networkstatus, NET_PARTICIPANT, 0), + CALLBACK(launch_descriptor_fetches, NET_PARTICIPANT, FL(NEED_NET)), + CALLBACK(rotate_x509_certificate, NET_PARTICIPANT, 0), + + /* We need to do these if we're participating in the Tor network, and + * immediately before we stop. */ + CALLBACK(clean_caches, NET_PARTICIPANT, FL(FLUSH_ON_DISABLE)), + CALLBACK(save_state, NET_PARTICIPANT, FL(FLUSH_ON_DISABLE)), + + /* XXXX investigate this. */ + CALLBACK(write_stats_file, NET_PARTICIPANT, FL(FLUSH_ON_DISABLE)), + + /* XXXX investigate this. */ + CALLBACK(reset_padding_counts, NET_PARTICIPANT, FL(FLUSH_ON_DISABLE)), /* Routers (bridge and relay) only. */ CALLBACK(check_descriptor, ROUTER, FL(NEED_NET)), @@ -1418,7 +1429,8 @@ STATIC periodic_event_item_t periodic_events[] = { CALLBACK(record_bridge_stats, BRIDGE, 0), /* Client only. */ - CALLBACK(rend_cache_failure_clean, CLIENT, 0), + /* XXXX this could be restricted to CLIENT even. */ + CALLBACK(rend_cache_failure_clean, NET_PARTICIPANT, FL(FLUSH_ON_DISABLE)), /* Bridge Authority only. */ CALLBACK(write_bridge_ns, BRIDGEAUTH, 0), @@ -1477,7 +1489,7 @@ get_my_roles(const or_options_t *options) { tor_assert(options); - int roles = 0; + int roles = PERIODIC_EVENT_ROLE_ALL; int is_bridge = options->BridgeRelay; int is_relay = server_mode(options); int is_dirauth = authdir_mode_v3(options); @@ -1492,6 +1504,9 @@ get_my_roles(const or_options_t *options) options->ControlPort_set || options->OwningControllerFD != UINT64_MAX; + /* We actually want a better definition here for our work on dormancy. */ + int is_net_participant = ! net_is_disabled(); + if (is_bridge) roles |= PERIODIC_EVENT_ROLE_BRIDGE; if (is_client) roles |= PERIODIC_EVENT_ROLE_CLIENT; if (is_relay) roles |= PERIODIC_EVENT_ROLE_RELAY; @@ -1499,6 +1514,7 @@ get_my_roles(const or_options_t *options) if (is_bridgeauth) roles |= PERIODIC_EVENT_ROLE_BRIDGEAUTH; if (is_hidden_service) roles |= PERIODIC_EVENT_ROLE_HS_SERVICE; if (is_dirserver) roles |= PERIODIC_EVENT_ROLE_DIRSERVER; + if (is_net_participant) roles |= PERIODIC_EVENT_ROLE_NET_PARTICIPANT; return roles; } diff --git a/src/core/mainloop/periodic.h b/src/core/mainloop/periodic.h index 7c71be7bc..23459ff2b 100644 --- a/src/core/mainloop/periodic.h +++ b/src/core/mainloop/periodic.h @@ -16,6 +16,9 @@ #define PERIODIC_EVENT_ROLE_HS_SERVICE (1U << 5) #define PERIODIC_EVENT_ROLE_DIRSERVER (1U << 6) +#define PERIODIC_EVENT_ROLE_NET_PARTICIPANT (1U << 7) +#define PERIODIC_EVENT_ROLE_ALL (1U << 8) + /* Helper macro to make it a bit less annoying to defined groups of roles that * are often used. */ @@ -25,10 +28,6 @@ /* Authorities that is both bridge and directory. */ #define PERIODIC_EVENT_ROLE_AUTHORITIES \ (PERIODIC_EVENT_ROLE_BRIDGEAUTH | PERIODIC_EVENT_ROLE_DIRAUTH) -/* All roles. */ -#define PERIODIC_EVENT_ROLE_ALL \ - (PERIODIC_EVENT_ROLE_AUTHORITIES | PERIODIC_EVENT_ROLE_CLIENT | \ - PERIODIC_EVENT_ROLE_HS_SERVICE | PERIODIC_EVENT_ROLE_ROUTER) /* * Event flags which can change the behavior of an event. diff --git a/src/test/test_periodic_event.c b/src/test/test_periodic_event.c index 86dedd85d..f63adf8e3 100644 --- a/src/test/test_periodic_event.c +++ b/src/test/test_periodic_event.c @@ -19,6 +19,7 @@ #include "feature/hibernate/hibernate.h" #include "feature/hs/hs_service.h" #include "core/mainloop/mainloop.h" +#include "core/mainloop/netstatus.h" #include "core/mainloop/periodic.h" /** Helper function: This is replaced in some tests for the event callbacks so @@ -59,7 +60,9 @@ test_pe_initialize(void *arg) tt_u64_op(item->last_action_time, OP_EQ, 0); /* Every event must have role(s) assign to it. This is done statically. */ tt_u64_op(item->roles, OP_NE, 0); - tt_uint_op(periodic_event_is_enabled(item), OP_EQ, 0); + int should_be_enabled = (item->roles & PERIODIC_EVENT_ROLE_ALL) && + !(item->flags & PERIODIC_EVENT_FLAG_NEED_NET); + tt_uint_op(periodic_event_is_enabled(item), OP_EQ, should_be_enabled); } done: @@ -106,13 +109,12 @@ test_pe_launch(void *arg) /* Now that we've initialized, rescan the list to launch. */ periodic_events_on_new_options(options); + int mask = PERIODIC_EVENT_ROLE_CLIENT|PERIODIC_EVENT_ROLE_ALL| + PERIODIC_EVENT_ROLE_NET_PARTICIPANT; for (int i = 0; periodic_events[i].name; ++i) { periodic_event_item_t *item = &periodic_events[i]; - if (item->roles & PERIODIC_EVENT_ROLE_CLIENT) { - tt_int_op(periodic_event_is_enabled(item), OP_EQ, 1); - } else { - tt_int_op(periodic_event_is_enabled(item), OP_EQ, 0); - } + int should_be_enabled = !!(item->roles & mask); + tt_int_op(periodic_event_is_enabled(item), OP_EQ, should_be_enabled); // enabled or not, the event has not yet been run. tt_u64_op(item->last_action_time, OP_EQ, 0); } @@ -124,7 +126,8 @@ test_pe_launch(void *arg) unsigned roles = get_my_roles(options); tt_uint_op(roles, OP_EQ, - PERIODIC_EVENT_ROLE_RELAY|PERIODIC_EVENT_ROLE_DIRSERVER); + PERIODIC_EVENT_ROLE_RELAY|PERIODIC_EVENT_ROLE_DIRSERVER| + PERIODIC_EVENT_ROLE_ALL|PERIODIC_EVENT_ROLE_NET_PARTICIPANT); for (int i = 0; periodic_events[i].name; ++i) { periodic_event_item_t *item = &periodic_events[i]; @@ -144,17 +147,21 @@ test_pe_launch(void *arg) /* Disable everything and we'll enable them ALL. */ options->SocksPort_set = 0; options->ORPort_set = 0; + options->DisableNetwork = 1; periodic_events_on_new_options(options); for (int i = 0; periodic_events[i].name; ++i) { periodic_event_item_t *item = &periodic_events[i]; - tt_int_op(periodic_event_is_enabled(item), OP_EQ, 0); + int should_be_enabled = (item->roles & PERIODIC_EVENT_ROLE_ALL) && + !(item->flags & PERIODIC_EVENT_FLAG_NEED_NET); + tt_int_op(periodic_event_is_enabled(item), OP_EQ, should_be_enabled); } /* Enable everything. */ options->SocksPort_set = 1; options->ORPort_set = 1; options->BridgeRelay = 1; options->AuthoritativeDir = 1; options->V3AuthoritativeDir = 1; options->BridgeAuthoritativeDir = 1; + options->DisableNetwork = 0; register_dummy_hidden_service(&service); periodic_events_on_new_options(options); /* Note down the reference because we need to remove this service from the @@ -188,41 +195,46 @@ test_pe_get_roles(void *arg) or_options_t *options = get_options_mutable(); tt_assert(options); + const int ALL = PERIODIC_EVENT_ROLE_ALL; + /* Nothing configured, should be no roles. */ + tt_assert(net_is_disabled()); roles = get_my_roles(options); - tt_int_op(roles, OP_EQ, 0); + tt_int_op(roles, OP_EQ, ALL); /* Indicate we have a SocksPort, roles should be come Client. */ options->SocksPort_set = 1; roles = get_my_roles(options); - tt_int_op(roles, OP_EQ, PERIODIC_EVENT_ROLE_CLIENT); + tt_int_op(roles, OP_EQ, PERIODIC_EVENT_ROLE_CLIENT|ALL); /* Now, we'll add a ORPort so should now be a Relay + Client. */ options->ORPort_set = 1; roles = get_my_roles(options); tt_int_op(roles, OP_EQ, (PERIODIC_EVENT_ROLE_CLIENT | PERIODIC_EVENT_ROLE_RELAY | - PERIODIC_EVENT_ROLE_DIRSERVER)); + PERIODIC_EVENT_ROLE_DIRSERVER | ALL)); /* Now add a Bridge. */ options->BridgeRelay = 1; roles = get_my_roles(options); tt_int_op(roles, OP_EQ, (PERIODIC_EVENT_ROLE_CLIENT | PERIODIC_EVENT_ROLE_RELAY | - PERIODIC_EVENT_ROLE_BRIDGE | PERIODIC_EVENT_ROLE_DIRSERVER)); + PERIODIC_EVENT_ROLE_BRIDGE | PERIODIC_EVENT_ROLE_DIRSERVER | + ALL)); tt_assert(roles & PERIODIC_EVENT_ROLE_ROUTER); /* Unset client so we can solely test Router role. */ options->SocksPort_set = 0; roles = get_my_roles(options); tt_int_op(roles, OP_EQ, - PERIODIC_EVENT_ROLE_ROUTER | PERIODIC_EVENT_ROLE_DIRSERVER); + PERIODIC_EVENT_ROLE_ROUTER | PERIODIC_EVENT_ROLE_DIRSERVER | + ALL); /* Reset options so we can test authorities. */ options->SocksPort_set = 0; options->ORPort_set = 0; options->BridgeRelay = 0; roles = get_my_roles(options); - tt_int_op(roles, OP_EQ, 0); + tt_int_op(roles, OP_EQ, ALL); /* Now upgrade to Dirauth. */ options->DirPort_set = 1; @@ -230,7 +242,7 @@ test_pe_get_roles(void *arg) options->V3AuthoritativeDir = 1; roles = get_my_roles(options); tt_int_op(roles, OP_EQ, - PERIODIC_EVENT_ROLE_DIRAUTH|PERIODIC_EVENT_ROLE_DIRSERVER); + PERIODIC_EVENT_ROLE_DIRAUTH|PERIODIC_EVENT_ROLE_DIRSERVER|ALL); tt_assert(roles & PERIODIC_EVENT_ROLE_AUTHORITIES); /* Now Bridge Authority. */ @@ -238,7 +250,7 @@ test_pe_get_roles(void *arg) options->BridgeAuthoritativeDir = 1; roles = get_my_roles(options); tt_int_op(roles, OP_EQ, - PERIODIC_EVENT_ROLE_BRIDGEAUTH|PERIODIC_EVENT_ROLE_DIRSERVER); + PERIODIC_EVENT_ROLE_BRIDGEAUTH|PERIODIC_EVENT_ROLE_DIRSERVER|ALL); tt_assert(roles & PERIODIC_EVENT_ROLE_AUTHORITIES); /* Move that bridge auth to become a relay. */ @@ -246,7 +258,7 @@ test_pe_get_roles(void *arg) roles = get_my_roles(options); tt_int_op(roles, OP_EQ, (PERIODIC_EVENT_ROLE_BRIDGEAUTH | PERIODIC_EVENT_ROLE_RELAY - | PERIODIC_EVENT_ROLE_DIRSERVER)); + | PERIODIC_EVENT_ROLE_DIRSERVER|ALL)); tt_assert(roles & PERIODIC_EVENT_ROLE_AUTHORITIES); /* And now an Hidden service. */ @@ -257,7 +269,8 @@ test_pe_get_roles(void *arg) remove_service(get_hs_service_map(), &service); tt_int_op(roles, OP_EQ, (PERIODIC_EVENT_ROLE_BRIDGEAUTH | PERIODIC_EVENT_ROLE_RELAY | - PERIODIC_EVENT_ROLE_HS_SERVICE | PERIODIC_EVENT_ROLE_DIRSERVER)); + PERIODIC_EVENT_ROLE_HS_SERVICE | PERIODIC_EVENT_ROLE_DIRSERVER | + ALL)); tt_assert(roles & PERIODIC_EVENT_ROLE_AUTHORITIES); done:

1 0

[tor/master] Annotate 1/s callback elements with NET_PARTICIPANT status.
by nickm＠torproject.org 27 Nov '18

27 Nov '18

commit db53bfe8f74dad1b45ba381a5ee3366148a30237 Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Nov 6 11:14:50 2018 -0500 Annotate 1/s callback elements with NET_PARTICIPANT status. --- src/core/mainloop/mainloop.c | 28 ++++++++++++++++++++++------ 1 file changed, 22 insertions(+), 6 deletions(-) diff --git a/src/core/mainloop/mainloop.c b/src/core/mainloop/mainloop.c index 3c3f441a9..7e5e5d0ef 100644 --- a/src/core/mainloop/mainloop.c +++ b/src/core/mainloop/mainloop.c @@ -1379,8 +1379,9 @@ STATIC periodic_event_item_t periodic_events[] = { CALLBACK(add_entropy, ALL, 0), CALLBACK(heartbeat, ALL, 0), - /* XXXX Do we have a reason to do this on a callback? */ - CALLBACK(retry_listeners, ALL, FL(NEED_NET)), + /* XXXX Do we have a reason to do this on a callback? Does it do any good at + * all? For now, if we're dormant, we can let our listeners decay. */ + CALLBACK(retry_listeners, NET_PARTICIPANT, FL(NEED_NET)), /* We need to do these if we're participating in the Tor network. */ CALLBACK(check_expired_networkstatus, NET_PARTICIPANT, 0), @@ -1393,10 +1394,10 @@ STATIC periodic_event_item_t periodic_events[] = { CALLBACK(clean_caches, NET_PARTICIPANT, FL(FLUSH_ON_DISABLE)), CALLBACK(save_state, NET_PARTICIPANT, FL(FLUSH_ON_DISABLE)), - /* XXXX investigate this. */ + /* XXXX investigate this. ??? */ CALLBACK(write_stats_file, NET_PARTICIPANT, FL(FLUSH_ON_DISABLE)), - /* XXXX investigate this. */ + /* XXXX investigate this. ???? */ CALLBACK(reset_padding_counts, NET_PARTICIPANT, FL(FLUSH_ON_DISABLE)), /* Routers (bridge and relay) only. */ @@ -1423,13 +1424,13 @@ STATIC periodic_event_item_t periodic_events[] = { CALLBACK(check_dns_honesty, RELAY, FL(NEED_NET)), /* Hidden Service service only. */ - CALLBACK(hs_service, HS_SERVICE, FL(NEED_NET)), + CALLBACK(hs_service, HS_SERVICE, FL(NEED_NET)), // XXXX break this down more /* Bridge only. */ CALLBACK(record_bridge_stats, BRIDGE, 0), /* Client only. */ - /* XXXX this could be restricted to CLIENT even. */ + /* XXXX this could be restricted to CLIENT+NET_PARTICIPANT */ CALLBACK(rend_cache_failure_clean, NET_PARTICIPANT, FL(FLUSH_ON_DISABLE)), /* Bridge Authority only. */ @@ -1730,18 +1731,24 @@ run_scheduled_events(time_t now) * expired; or if our bandwidth limits are exhausted and we * should hibernate; or if it's time to wake up from hibernation. */ + // TODO: Refactor or rewrite, or NET_PARTICIPANT. Needs separate wakeup + // handling. consider_hibernation(now); /* Maybe enough time elapsed for us to reconsider a circuit. */ + // TODO: NET_PARTICIPANT circuit_upgrade_circuits_from_guard_wait(); if (options->UseBridges && !net_is_disabled()) { /* Note: this check uses net_is_disabled(), not should_delay_dir_fetches() * -- the latter is only for fetching consensus-derived directory info. */ + // TODO: client+NET_PARTICIPANT. + // Also, schedule this rather than probing 1x / sec fetch_bridge_descriptors(options, now); } if (accounting_is_enabled(options)) { + // TODO: refactor or rewrite; NET_PARTICIPANT accounting_run_housekeeping(now); } @@ -1752,6 +1759,7 @@ run_scheduled_events(time_t now) */ /* (If our circuit build timeout can ever become lower than a second (which * it can't, currently), we should do this more often.) */ + // TODO: All expire stuff can become NET_PARTICIPANT, FLUSH_ON_DISABLE circuit_expire_building(); circuit_expire_waiting_for_better_guard(); @@ -1760,10 +1768,12 @@ run_scheduled_events(time_t now) * Do this before step 4, so we can put them back into pending * state to be picked up by the new circuit. */ + // TODO: All expire stuff can become NET_PARTICIPANT, FLUSH_ON_DISABLE connection_ap_expire_beginning(); /* 3c. And expire connections that we've held open for too long. */ + // TODO: All expire stuff can become NET_PARTICIPANT, FLUSH_ON_DISABLE connection_expire_held_open(); /* 4. Every second, we try a new circuit if there are no valid @@ -1773,19 +1783,24 @@ run_scheduled_events(time_t now) */ const int have_dir_info = router_have_minimum_dir_info(); if (have_dir_info && !net_is_disabled()) { + // TODO: NET_PARTICIPANT. circuit_build_needed_circs(now); } else { + // TODO: NET_PARTICIPANT, FLUSH_ON_DISABLE circuit_expire_old_circs_as_needed(now); } /* 5. We do housekeeping for each connection... */ + // TODO: NET_PARTICIPANT channel_update_bad_for_new_circs(NULL, 0); int i; for (i=0;i<smartlist_len(connection_array);i++) { + // TODO: NET_PARTICIPANT, FLUSH_ON_DISABLE run_connection_housekeeping(i, now); } /* 11b. check pending unconfigured managed proxies */ + // NET_PARTICIPANT. if (!net_is_disabled() && pt_proxies_configuration_pending()) pt_configure_remaining_proxies(); } @@ -2625,6 +2640,7 @@ second_elapsed_callback(periodic_timer_t *timer, void *arg) */ update_current_time(now); + // TODO XXXX Turn this into a separate event. /* Maybe some controller events are ready to fire */ control_per_second_events();

1 0

[tor/master] Remove run_scheduled_events() as a separate function.
by nickm＠torproject.org 27 Nov '18

27 Nov '18

commit e535ec8542a1d42243c0b6ae28036aec8262269b Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Nov 13 08:36:38 2018 -0500 Remove run_scheduled_events() as a separate function. (There was nothing else in second_elapsed_callbck() that couldn't go here.) --- src/core/mainloop/mainloop.c | 35 +++++++++++------------------------ 1 file changed, 11 insertions(+), 24 deletions(-) diff --git a/src/core/mainloop/mainloop.c b/src/core/mainloop/mainloop.c index a9d5d8155..ebf9735d4 100644 --- a/src/core/mainloop/mainloop.c +++ b/src/core/mainloop/mainloop.c @@ -1727,13 +1727,22 @@ safe_timer_diff(time_t now, time_t next) } /** Perform regular maintenance tasks. This function gets run once per - * second by second_elapsed_callback(). + * second. */ static void -run_scheduled_events(time_t now) +second_elapsed_callback(periodic_timer_t *timer, void *arg) { + (void) timer; + (void) arg; + const time_t now = time(NULL); const or_options_t *options = get_options(); + /* We don't need to do this once-per-second any more: time-updating is + * only in this callback _because it is a callback_. It should be fine + * to disable this callback, and the time will still get updated. + */ + update_current_time(now); + /* 0. See if we've been asked to shut down and our timeout has * expired; or if our bandwidth limits are exhausted and we * should hibernate; or if it's time to wake up from hibernation. @@ -2642,28 +2651,6 @@ update_current_time(time_t now) current_second = now; } -/** Libevent callback: invoked once every second. */ -static void -second_elapsed_callback(periodic_timer_t *timer, void *arg) -{ - /* XXXX This could be sensibly refactored into multiple callbacks, and we - * could use Libevent's timers for this rather than checking the current - * time against a bunch of timeouts every second. */ - time_t now; - (void)timer; - (void)arg; - - now = time(NULL); - - /* We don't need to do this once-per-second any more: time-updating is - * only in this callback _because it is a callback_. It should be fine - * to disable this callback, and the time will still get updated. - */ - update_current_time(now); - - run_scheduled_events(now); -} - #ifdef HAVE_SYSTEMD_209 static periodic_timer_t *systemd_watchdog_timer = NULL;

1 0