November 2017 - tor-commits - lists.torproject.org

[torspec/master] Add prop286: Controller APIs for hibernation access on mobile
by nickm＠torproject.org 30 Nov '17

30 Nov '17

commit 71dc93753bd6db9d4c9a2d3f1b4b8f7e3682221b Author: Nick Mathewson <nickm(a)torproject.org> Date: Thu Nov 30 07:55:06 2017 -0500 Add prop286: Controller APIs for hibernation access on mobile --- proposals/000-index.txt | 2 + proposals/286-hibernation-api.txt | 194 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 196 insertions(+) diff --git a/proposals/000-index.txt b/proposals/000-index.txt index 3352d02..2bb7363 100644 --- a/proposals/000-index.txt +++ b/proposals/000-index.txt @@ -206,6 +206,7 @@ Proposals by number: 283 Move IPv6 ORPorts from microdescriptors to the microdesc consensus [OPEN] 284 Hidden Service v3 Control Port [OPEN] 285 Directory documents should be standardized as UTF-8 [OPEN] +286 Controller APIs for hibernation access on mobile [OPEN] Proposals by status: @@ -265,6 +266,7 @@ Proposals by status: 283 Move IPv6 ORPorts from microdescriptors to the microdesc consensus [for 0.3.3.x] 284 Hidden Service v3 Control Port 285 Directory documents should be standardized as UTF-8 + 286 Controller APIs for hibernation access on mobile ACCEPTED: 172 GETINFO controller option for circuit information 173 GETINFO Option Expansion diff --git a/proposals/286-hibernation-api.txt b/proposals/286-hibernation-api.txt new file mode 100644 index 0000000..25da5c2 --- /dev/null +++ b/proposals/286-hibernation-api.txt @@ -0,0 +1,194 @@ +Filename: 286-hibernation-api.txt +Title: Controller APIs for hibernation access on mobile +Author: Nick Mathewson +Created: 30-November-2017 +Status: Open + + +1. Introduction + + On mobile platforms, battery life is achieved by reducing + needless network access and CPU access. Tor currently provides + few ways for controllers and operating systems to tune its + behavior. + + This proposal describes controller APIs for better management of + Tor's hibernation mechanisms, and extensions to those mechanisms, + for better power management in mobile environments. + +1.1. Background: hibernation and idling in Tor today + + We have an existing "hibernation" mechanism that we use to + implement "bandwidth accounting" and "slow shutdown" mechanisms: + When a Tor instance is close to its bandwidth limit: it stops + accepting new connections or circuits, and only processes those + it has, until the bandwidth limit is reached. Once the bandwidth + limit is reached, Tor closes all connections and circuits, and + all non-controller listeners, until a new accounting limit + begins. + + Tor handles the INT signal on relays similarly: it stops + accepting new connections or circuits, and gives the existing + ones a short interval in which to shut down. Then Tor closes all + connections and exits the process entirely. + + Tor's "idle" mechanism is related to hibernation, though its + implementation is separate. When a Tor clients has passed a + certain amount of time without any user activity, it declares + itself "idle" and stops performing certain background tasks, such + as fetching directory information, or building circuits in + anticipation of future needs. (This is tied in the codebase to + the "predicted ports" mechanism, but it doesn't have to be.) + + +1.2. Background: power-management signals on mobile platforms + + (I'm not a mobile developer, so I'm about to wildly oversimplify. + Please let me know where I'm wrong.) + + Mobile platforms achieve long battery life by turning off the + parts they don't need. The most important parts to turn off are + the antenna(s) and the screen; the CPU can be run in a slower + mode. + + But it doesn't do much good turning things off when they're + unused, if some background app is going to make sure that they're + always in use! So mobile platforms use signals of various kinds + to tell applications "okay, shut up now". + + Some apps need to do online background activities periodically; + to help this out, mobile platforms give them a signal "Hey, now + is a good time if you want to do that" and "stop now!" + + +1.3. Mostly out-of-scope: limiting CPU wakeups when idle. + + The changes described here will be of limited use if we do not + also alter Tor so that, when it's idle, the CPU is pretty quiet. + That isn't the case right now: we have large numbers of callbacks + that happen periodically (every second, every minute, etc) + whether they need to or not. We're hoping to limit those, but + that's not what this proposal is about. + + +2. Improvements to the hibernation model + + To present a consistent interface that applications and + controllers can use to manage power consumption, we make these + enhancements to our hibernation model. + + First, we add three new hibernation states: "IDLE", + "IDLE_UPDATING", "SLEEP", and "SLEEP_UPDATING". + + "IDLE" is like the current "idle" or "no predicted ports" state: + Tor doesn't launch circuits or start any directory activity, but + its listeners are still open. Tor clients can enter the IDLE + state on their own when they are LIVE, but haven't gotten any + client activity for a while. Existing connections and circuits + are not closed. If the Tor instance receives any new connections, + it becomes LIVE. + + "IDLE_UPDATING" is like IDLE, except that Tor should check for + directory updates as appropriate. If there are any, it should + fetch directory information, and then become IDLE again. + + "SLEEPING" is like the current "dormant state we use for + bandwidth exhaustion, but it is controller-initiated: it begins + when Tor is told to enter it, and ends when Tor is told to leave + it. Existing connections and circuits are closed; listeners are + closed too. + + "SLEEP_UPDATING" is like SLEEP, except that Tor should check for + directory updates as appropriate. If there are any, it should + fetch directory information, and then SLEEP again. + + +2.1. Relay operation + + Relays and bridges should not automatically become IDLE on their + own. + + +2.2. Onion service operation + + When a Tor instance that is running an onion service is IDLE, it + does the minimum to try to remain responsive on the onion + service: It keeps its introduction points open if it can. Once a + day, it fetches new directory information and opens new + introduction points. + + +3. Controller hibernation API + +3.1. Examining the current hibernation state + + We define a new "GETINFO status/hibernation" to inspect the + current hibernation state. Possible values are: + - "live" + - "idle:control" + - "idle:no-activity" + - "sleep:control" + - "sleep:accounting" + - "idle-update:control" + - "sleep-update:control" + - "shutdown:exiting" + - "shutdown:accounting" + - "shutdown:control" + + The first part of each value indicates Tor's current state: + "live" -- completely awake + "idle" -- waiting to see if anything happens + "idle-update" -- waiting to see if anything happens; probing + for directory information + "sleep" -- completely unresponsive + "shutdown" -- unresponsive to new requests; still processing + existing requests. + + The second part of each value indicates the reason that Tor + entered this state: + "control" -- a controller told us to do this. + "no-activity" -- Tor became idle on its own due to not + noticing any requests. + "accounting" -- the bandwidth system told us to enter this + state. + "exiting" -- Tor is in this state because it's getting ready + to exit. + + We add a STATUS_GENERAL hibernation event as follows: + + HIBERNATION + "STATUS=" (one of the status pairs above.) + + Indicates that Tor's hibernation status has changed. + + Note: Controllers MUST accept status values here that they don't + recognize. + + The "GETINFO accounting/hibernating" value and the "STATUS_SERVER + HIBERANATION_STATUS" event keep their old meaning. + +3.2. Changing the hibernation state + + We add the following new possible values to the SIGNAL controller + command: + "SLEEP" -- enter the sleep state, after an appropriate + shutdown interval. + + "IDLE" -- enter the idle state + + "SLEEPWALK" -- If in sleep or idle, start probing for + directory information in the sleep-update or idle-update + state respectively. Remain in that state until we've + probed for directory information, or until we're told to + IDLE or SLEEP again, or (if we're idle) until we get client + activity. Has no effect if not in sleep or idle. + + "WAKEUP" -- If in sleep, sleep-update, idle, idle-update, or + shutdown:sleep state, enter the live state. Has no effect + in any other state. + +3.3. New configuration parameters + + StartIdle -- Boolean. If set to 1, Tor begins in IDLE mode. + +

1 0

[tor/master] Switch -Wnormalized=id to -Wnormalized=nfkc
by nickm＠torproject.org 30 Nov '17

30 Nov '17

commit bef461ef1fac2fbcca5768b2f351f7eeec496759 Author: Fernando Fernandez Mancera <ffernandezmancera(a)gmail.com> Date: Wed Nov 29 17:04:41 2017 +0100 Switch -Wnormalized=id to -Wnormalized=nfkc Modified -Wnormalized flag to nfkc option in configure.ac to avoid source code identifier confusion. Fixes #24467 Signed-off-by: Fernando Fernandez Mancera <ffernandezmancera(a)gmail.com> --- changes/ticket24467 | 3 +++ configure.ac | 2 +- 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/changes/ticket24467 b/changes/ticket24467 new file mode 100644 index 000000000..1b1c223f5 --- /dev/null +++ b/changes/ticket24467 @@ -0,0 +1,3 @@ + o Code simplification and refactoring: + - Switch -Wnormalized=id to -Wnormalized=nfkc in configure.ac to avoid + source code identifier confusion. Closes ticket 24467. diff --git a/configure.ac b/configure.ac index fd098b48d..0b3e1dda2 100644 --- a/configure.ac +++ b/configure.ac @@ -2017,7 +2017,7 @@ if test "x$enable_gcc_warnings_advisory" != "xno"; then -Wnon-literal-null-conversion -Wnon-pod-varargs -Wnonportable-cfstrings - -Wnormalized=id + -Wnormalized=nfkc -Wnull-arithmetic -Wnull-character -Wnull-conversion

1 0

[tor/master] Merge branch 'ticket20020'
by nickm＠torproject.org 30 Nov '17

30 Nov '17

commit 27f017639fa87eb3f235e6d8434f0b7776a38492 Merge: 20fa6a388 93dbce4dd Author: Nick Mathewson <nickm(a)torproject.org> Date: Wed Nov 29 20:24:34 2017 -0500 Merge branch 'ticket20020' changes/ticket20020 | 4 ++++ src/or/routerlist.c | 13 ++++++++++++- 2 files changed, 16 insertions(+), 1 deletion(-)

1 0

[tor/master] Log a better message if a router we're re-adding is expired.
by nickm＠torproject.org 30 Nov '17

30 Nov '17

commit 93dbce4ddd98ca9a4ff2f597b165c87410cabd0e Author: Nick Mathewson <nickm(a)torproject.org> Date: Thu Nov 16 08:35:06 2017 -0500 Log a better message if a router we're re-adding is expired. Addresses ticket 20020. --- changes/ticket20020 | 4 ++++ src/or/routerlist.c | 13 ++++++++++++- 2 files changed, 16 insertions(+), 1 deletion(-) diff --git a/changes/ticket20020 b/changes/ticket20020 new file mode 100644 index 000000000..737fb9598 --- /dev/null +++ b/changes/ticket20020 @@ -0,0 +1,4 @@ + o Minor features (logging): + - Improve a warning message that happens when we fail to re-parse + an old router because of an expired certificate. Closes ticket + 20020. diff --git a/src/or/routerlist.c b/src/or/routerlist.c index fb8225e0d..24c0a4e07 100644 --- a/src/or/routerlist.c +++ b/src/or/routerlist.c @@ -5199,8 +5199,19 @@ update_consensus_router_descriptor_downloads(time_t now, int is_vote, } r = router_add_to_routerlist(ri, &msg, 1, 0); if (WRA_WAS_OUTDATED(r)) { - log_warn(LD_DIR, "Couldn't add re-parsed router: %s", + log_warn(LD_DIR, "Couldn't add re-parsed router: %s. This isn't " + "usually a big deal, but you should make sure that your " + "clock and timezone are set correctly.", msg?msg:"???"); + if (r == ROUTER_CERTS_EXPIRED) { + char time_cons[ISO_TIME_LEN+1]; + char time_cert[ISO_TIME_LEN+1]; + format_iso_time(time_cons, consensus->valid_after); + format_iso_time(time_cert, ri->cert_expiration_time); + log_warn(LD_DIR, " (I'm looking at a consensus from %s; This " + "router's certificates began expiring at %s.)", + time_cons, time_cert); + } } } SMARTLIST_FOREACH_END(sd); routerlist_assert_ok(rl);

1 0

[stem/master] Add an example of using basic hidden service auth
by atagar＠torproject.org 29 Nov '17

29 Nov '17

commit 3f9057b45152ab0de1600c16c4cd11bb5639c14e Author: Damian Johnson <atagar(a)torproject.org> Date: Wed Nov 29 12:15:30 2017 -0800 Add an example of using basic hidden service auth Snippet courtesy of ZeroMux. --- docs/tutorials/over_the_river.rst | 24 ++++++++++++++++++++++++ stem/socket.py | 8 ++++---- stem/util/system.py | 2 ++ 3 files changed, 30 insertions(+), 4 deletions(-) diff --git a/docs/tutorials/over_the_river.rst b/docs/tutorials/over_the_river.rst index 3f8ae0c2..dac78827 100644 --- a/docs/tutorials/over_the_river.rst +++ b/docs/tutorials/over_the_river.rst @@ -70,6 +70,30 @@ Now if we run this... .. image:: /_static/hidden_service.png +.. _hidden-service-authentication: + +Hidden service authentication +----------------------------- + +Hidden services you create can restrict their access, requiring in essence a +password... + +:: + + >>> from stem.control import Controller + >>> controller = Controller.from_port() + >>> controller.authenticate() + >>> response = controller.create_ephemeral_hidden_service({80: 8080}, await_publication=True, basic_auth={'bob': None, 'alice': None}) + >>> response.service_id, response.client_auth + ('l3lnorirzn7hrjnw', {'alice': 'I6AMKiay+UkM5MfrvdnF2A', 'bob': 'VLsbrSGyrb5JYEvZmQ3tMg'}) + +To access this service users simply provide this credential to tor via their +torrc or SETCONF prior to visiting it... + +:: + + >>> controller.set_conf('HidServAuth', 'l3lnorirzn7hrjnw.onion I6AMKiay+UkM5MfrvdnF2A') + .. _ephemeral-hidden-services: Ephemeral hidden services diff --git a/stem/socket.py b/stem/socket.py index ef2d77c7..b05524b5 100644 --- a/stem/socket.py +++ b/stem/socket.py @@ -167,11 +167,11 @@ class ControlSocket(object): # everything down. However, there's a couple cases where this will # cause deadlock... # - # * this socketClosed was *caused by* a close() call, which is joining - # on our thread + # * This SocketClosed was *caused by* a close() call, which is joining + # on our thread. # - # * a send() call that's currently in flight is about to call close(), - # also attempting to join on us + # * A send() call that's currently in flight is about to call close(), + # also attempting to join on us. # # To resolve this we make a non-blocking call to acquire the send lock. # If we get it then great, we can close safely. If not then one of the diff --git a/stem/util/system.py b/stem/util/system.py index 8de186b1..cec095ea 100644 --- a/stem/util/system.py +++ b/stem/util/system.py @@ -1266,6 +1266,8 @@ def call(command, default = UNDEFINED, ignore_exit_status = False, timeout = Non * **CallTimeoutError** if the timeout is reached without a default """ + # TODO: in stem 2.x return a struct with stdout, stderr, and runtime instead + global SYSTEM_CALL_TIME if isinstance(command, str):

1 0

[nyx/master] Resizing could cause nyx to crah
by atagar＠torproject.org 29 Nov '17

29 Nov '17

commit c0276f9efa672dedff6322f04fab4bcb879b2174 Author: Damian Johnson <atagar(a)torproject.org> Date: Wed Nov 29 11:57:24 2017 -0800 Resizing could cause nyx to crah Another great catch from Dbryrtfbcbhgf... https://trac.torproject.org/projects/tor/ticket/24382 Traceback (most recent call last): File "/usr/local/bin/nyx", line 11, in <module> sys.exit(main()) ... File "/usr/local/lib/python3.5/dist-packages/nyx/__init__.py", line 716, in redraw panel.redraw(force = force, top = occupied) File "/usr/local/lib/python3.5/dist-packages/nyx/panel/__init__.py", line 175, in redraw self._last_draw_size = nyx.curses.draw(self._draw, top = self._top, height = self.get_height(), draw_if_resized = draw_dimension) File "/usr/local/lib/python3.5/dist-packages/nyx/curses.py", line 740, in draw curses_subwindow = CURSES_SCREEN.subwin(subwindow_height, subwindow_width, top, left) _curses.error: curses function returned NULL --- nyx/curses.py | 2 ++ web/changelog/index.html | 6 ++++++ 2 files changed, 8 insertions(+) diff --git a/nyx/curses.py b/nyx/curses.py index 976f625..7a5d074 100644 --- a/nyx/curses.py +++ b/nyx/curses.py @@ -747,6 +747,8 @@ def draw(func, left = 0, top = 0, width = None, height = None, background = None curses_subwindow.refresh() return subwindow_dimensions + except curses.error: + return # raw curses access, such as subwin, can raise in edge cases such as resizing finally: CURSES_LOCK.release() diff --git a/web/changelog/index.html b/web/changelog/index.html index f622f04..94d8886 100644 --- a/web/changelog/index.html +++ b/web/changelog/index.html @@ -80,6 +80,12 @@ <li>Errors when saving the configuration could result in a stacktrace (<b><a href="https://trac.torproject.org/projects/tor/ticket/24409">ticket</a></b>)</li> </ul> </li> + + <li><span class="component">Curses</span> + <ul> + <li>Resizing could crash the interface (<b><a href="https://trac.torproject.org/projects/tor/ticket/24382">ticket</a></b>)</li> + </ul> + </li> </ul> <div id="version-2-0" class="section"></div>

1 0

[translation/tails-greeter-2] Update translations for tails-greeter-2
by translation＠torproject.org 29 Nov '17

29 Nov '17

commit f446bf9a8a0b090b2a2062c73e9953bdf2d28775 Author: Translation commit bot <translation(a)torproject.org> Date: Wed Nov 29 19:50:20 2017 +0000 Update translations for tails-greeter-2 --- bn/bn.po | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bn/bn.po b/bn/bn.po index 314105392..f9056ebf0 100644 --- a/bn/bn.po +++ b/bn/bn.po @@ -259,7 +259,7 @@ msgstr "যোগ" #: ../tailsgreeter/gui.py:616 msgid "Back" -msgstr "" +msgstr "পিছনে" #: ../tailsgreeter/gui.py:730 msgid "Shutdown"

1 0

[nyx/master] Add fedora to download page
by atagar＠torproject.org 29 Nov '17

29 Nov '17

commit ac85756eb81da5c6a0a133688efa6781a3b0e45f Author: Damian Johnson <atagar(a)torproject.org> Date: Wed Nov 29 09:16:23 2017 -0800 Add fedora to download page I'm torn on if I should do this or not since a bug's preventing the package from being listed on fedoraproject.org, but according to Juan it's definitely available so guess better to let folks on that platform know. --- web/index.html | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/web/index.html b/web/index.html index 01cde89..c6561bc 100644 --- a/web/index.html +++ b/web/index.html @@ -367,14 +367,14 @@ sudo python setup.py install</pre><br></li> <p>Package derived from Debian for Ubuntu.</p> <pre>% sudo apt-get install tor-arm</pre> </div> + --> <div class="platform"> - <a href="https://apps.fedoraproject.org/packages/tor-arm" id="fedora"><img src="images/download/fedora.png" alt="Fedora"></a> - <a href="https://apps.fedoraproject.org/packages/tor-arm" class="platform-title">Fedora</a> - <p>Packages maintained by Juan for Fedora.</p> - <pre>% sudo yum install tor-arm</pre> + <a href="https://pagure.io/fedora-infrastructure/issue/6526" id="fedora"><img src="images/download/fedora.png" alt="Fedora"></a> + <a href="https://pagure.io/fedora-infrastructure/issue/6526" class="platform-title">Fedora</a> + <p>Packages maintained by Juan for Fedora using <b>dnf</b> or <b>yum</b>.</p> + <pre>% sudo dnf install nyx</pre> </div> - --> <div class="platform"> <a href="https://packages.gentoo.org/packages/net-misc/nyx" id="gentoo"><img src="images/download/gentoo.png" alt="Gentoo"></a>

1 0

[tor/release-0.3.2] Merge branch 'maint-0.3.2' into release-0.3.2
by nickm＠torproject.org 29 Nov '17

29 Nov '17

commit 1e715efb7780c8a697a598ff2c3d03c7daca4340 Merge: 9a9d32fab f0007f2b8 Author: Nick Mathewson <nickm(a)torproject.org> Date: Wed Nov 29 12:06:14 2017 -0500 Merge branch 'maint-0.3.2' into release-0.3.2 src/test/test_config.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)

1 0

[tor/maint-0.3.2] Make the cache_ipv4_answers default become 0 again, for real.
by nickm＠torproject.org 29 Nov '17

29 Nov '17

commit a94c4861027015b4ca4c6a9512fc36512196fea4 Author: Nick Mathewson <nickm(a)torproject.org> Date: Thu Nov 9 16:41:57 2017 -0500 Make the cache_ipv4_answers default become 0 again, for real. Fortunately, use_cached_ipv4_answers was already 0, so we wouldn't actually use this info, but it's best not to have it. Fixes bug 24050; bugfix on 0.2.6.3-alpha --- changes/bug24050 | 5 +++++ src/or/config.c | 3 +-- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/changes/bug24050 b/changes/bug24050 new file mode 100644 index 000000000..d184a77ac --- /dev/null +++ b/changes/bug24050 @@ -0,0 +1,5 @@ + o Minor bugfixes (client): + - By default, do not enable storage of client-side DNS values. + These values were unused by default previously, but they should + not have been cached at all. Fixes bug 24050; bugfix on + 0.2.6.3-alpha. diff --git a/src/or/config.c b/src/or/config.c index 1aeff462e..c48070b58 100644 --- a/src/or/config.c +++ b/src/or/config.c @@ -6166,7 +6166,6 @@ port_cfg_new(size_t namelen) cfg->entry_cfg.ipv4_traffic = 1; cfg->entry_cfg.dns_request = 1; cfg->entry_cfg.onion_traffic = 1; - cfg->entry_cfg.cache_ipv4_answers = 1; cfg->entry_cfg.prefer_ipv6_virtaddr = 1; return cfg; } @@ -6519,7 +6518,7 @@ parse_port_config(smartlist_t *out, bind_ipv4_only = 0, bind_ipv6_only = 0, ipv4_traffic = 1, ipv6_traffic = 0, prefer_ipv6 = 0, dns_request = 1, onion_traffic = 1, - cache_ipv4 = 1, use_cached_ipv4 = 0, + cache_ipv4 = 0, use_cached_ipv4 = 0, cache_ipv6 = 0, use_cached_ipv6 = 0, prefer_ipv6_automap = 1, world_writable = 0, group_writable = 0, relax_dirmode_check = 0,

1 0