November 2017 - tor-dev - lists.torproject.org

UX improvement proposal: Onion auto-redirects using Alt-Svc HTTP header
by George Kadianakis 26 Oct '18

26 Oct '18

UX improvement proposal: Onion auto-redirects using Alt-Svc HTTP header ======================================================================== 1. Motivation: Lots of high-profile websites have onion addresses these days (e.g. Tor , NYT, blockchain, ProPublica). All those websites seem confused on what's the right way to inform their users about their onion addresses. Here are some confusion examples: a) torproject.org does not even advertise their onion address to Tor users (!!!) b) blockchain.info throws an ugly ASCII page to Tor users mentioning their onion address and completely wrecking the UX (loses URL params, etc.) c) ProPublica has a "Browse via Tor" section which redirects to the onion site. Ideally there would be a consistent way for websites to inform their users about their onion counterpart. This would provide the following positives: + Tor users would use onions more often. That's important for user education and user perception, and also to partially dispell the darkweb myth. + Website operators wouldn't have to come up with ad-hoc ways to advertise their onion services, which sometimes results in complete breakage of the user experience (particularly with blockchain) This proposal specifies a simple way forward here that's far from perfect, but can still provide benefits and also improve user-education around onions so that in the future we could employ more advanced techniques. Also see Tor ticket #21952 for more discussion on this: https://trac.torproject.org/projects/tor/ticket/21952 2. Proposal Websites use the Alt-Svc HTTP header to specify their onion counterpart: http://httpwg.org/http-extensions/alt-svc.html The Tor Browser intercepts that header (if any) and auto-redirects the user to the onion site. Tor Browser only does so if the header is served over HTTPS. (That's it.) 3. Improvements 3.1. User education through notifications To minimize the probability of users freaking out about auto-redirects Tor Browser could inform the user that the auto-redirect is happening. This could happen with a small notification bar [*] below the URL bar informing users that "Tor Browser is auto-redirecting you to the secure onion site". The notification bar could also have a question mark button that provides the user with additional information on the merits of onion sites and why they should like them. [*]: like this one: http://www.topdreamweaverextensions.com/UserFiles/Image/firefox-bar.jpg 3.2. Auto-redirects too intrusive? Make them optional. If we think that auto redirects are too intrusive, we should consider making them optional, or letting the website specify the desired behavior. If a website wants to specify an onion address but doesn't like auto-redirects, it could specify that as part of Alt-Svc and we could still inform the user that an onion is available using a notification bar again. 4. Drawbacks This proposal is far from the perfect solution of course. Here are a few reasons why: 4.1. Auto-redirects are scary Novice users or paranoid users might freak out with auto-redirects since it's unexpected. They might think they are under attack or they got hacked. An argument against that is that we are already using HTTPS-everywhere which is basically a redirection layer, and while its redirection technique is simpler, sometimes the derivative URL is not similar to the original URL, and we still don't see people freaking out. I think that's because people trust Tor Browser to do the right thing. Combining this knowledge with section 3.1 might solve this problem. 4.2. No security/performance benefits While we could come up with auto-redirect proposals that provide security and performance benefits, this proposal does not actually provide any of those. As a matter of fact, the security remains the same as connecting to normal websites (since we trust its HTTP headers), and the performance gets worse since we first need to connect to the website, get its headers, and then also connect to the onion. Still _all_ the website approaches mentioned in the "Motivation" section suffer from the above drawbacks, and sysadmins still come up with ad-hoc ways to inform users abou their onions. So this simple proposal will still help those websites and also pave the way forward for future auto-redirect techniques. 4.3. Alt-Svc does not do exactly what we want I read in a blog post [*] that using Alt-Svc header "doesn’t change the URL in the location bar, document.location or any other indication of where the resource is; this is a “layer below” the URL.". IIUC, this is not exactly what we want because users will not notice the onion address, they will not get the user education part of the proposal and their connection will still be slowed down. I think we could perhaps change this in Tor Browser so that it rewrites the onion address to make it clear to people that they are now surfing the onionspace. [*]: https://www.mnot.net/blog/2016/03/09/alt-svc 5. The future As I said this is just a simple proposal to introduce the auto-redirect concept to people, and also to help some sysadmins who are currently coming up with weird ways to inform people about their onions. It's not the best way to do this, but it's definitely one of the simplest ways. In the future we could implement with more advanced auto-redirect proposals like: a) Have a "domains to onions" map into HTTPS-everywhere and have it do the autoredirects for us (performance benefits, and security benefits under many threat models). b) Bake onion addresses into SSL certificates and Let's Encrypt as suggested by comment:42 in #21952. But both of the designs above require non-trivial engineering/policy work and would still confuse people. So I think starting with a simple approach that will educate users and then moving to more advanced designs is a more normative way to go. ====================================================================== Let me know what you think :) (Should this be part of torspec?)

10 25

non-anonymous ephemeral onion services with stem
by Micah Lee 30 Jan '18

30 Jan '18

The stem documentation for create_ephemeral_hidden_service [1] says: "Changed in version 1.5.0: Added support for non-anonymous services." But I can't figure out to actually use this feature. There doesn't seem to be a new argument to say if you want your onion service to be non-anonymous. It also says, "Changed in version 1.5.0: Added the basic_auth argument." But there's a new basic_auth argument you can pass into the function to use that. [1] https://stem.torproject.org/api/control.html#stem.control.Controller.create…

5 12

Proposal 285: Directory documents should be standardized as UTF-8
by Nick Mathewson 10 Jan '18

10 Jan '18

Filename: 285-utf-8.txt Title: Directory documents should be standardized as UTF-8 Author: Nick Mathewson Created: 13 November 2017 Status: Open 1. Summary and motivation People frequently want to include non-ASCII text in their router descriptors. The Contact line is a favorite place to do this, but in principle the platform line would also be pretty logical. Unfortunately, there's no specified way to encode non-ASCII in our directory documents. Fortunately, almost everybody who does it, uses UTF-8 anyway. As we move towards Rust support in Tor, we gain another motivation for standarding on UTF-8, since Rust's native strings strongly prefer UTF-8. So, in this proposal, we describe a migration path to having all directory documents be fully UTF-8. 2. Proposal First, we should have Tor relays reject ContactInfo lines (and any other lines copied directly into router descriptors) that are not UTF-8. At the same time, we should have authorities reject any router descriptors or extrainfo documents that are not valid UTF-8. Simultaneously, we can have all Tor instances reject all non-directory-descriptor directory documents that are not UTF-8, since none should exist today. Finally, once the authorities have updated, we should have all Tor instances reject all directory documents that are not UTF-8. (We should not take this step until the authorities have upgraded, or else the behavior of updated and non-updated clients could be distinguished.) 2.1. Hidden service descriptors' encrypted bodies For the encrypted bodies of hidden service descriptors, we cannot reject them at the authority level, and so we need to take a slightly different approach to prevent client fingerprinting attacks. First, we should make Tor instances start warning about any hidden service descriptors whose bodies, post-decryption, contain non-utf-8 plaintext. At the same time, we add a consensus parameter to indicate that hidden service descriptors with non-utf-8 plantexts should be rejected entirely: "reject-encrypted-non-utf-8". If that parameter is set to 1, then hidden service clients will not only warn, but reject the descriptors. Once the vast majority of clients are running versions that support the "reject-encrypted-non-utf-8" parameter, that parameter can be set to 1.

5 6

Proposal 287: Reduce circuit lifetime without overloading the network.
by Fernando Fernández Mancera 13 Dec '17

13 Dec '17

Hello, I have been working on this proposal for some time and Nick blessed me to send it as Proposal 287. Here is. Filename: 287-reduce-lifetime.txt Title: Reduce circuit lifetime without overloading the network Author: Fernando Fernandez Mancera Created: 30-Nov-2017 Status: Open Motivation: Currently Tor users are reusing a given circuit for ten minutes (by default) after it's first used. This time is too long because a malicious Exit relay can trace a user's pseudonymous profile, especially if connections from multiple protocols are put on the same circuit. This time it is established on MaxCircuitDirtiness parameter and by default its value is ten minutes. I have been thinking in a way to fix this. The first idea that came to my mind was to use StreamIsolationByHost and StreamIsolationByPort on it, but I wasn't able to sort it out. One day, I thought "Why is time so important?" and later on I realized that maybe focusing on the amount of bytes running through the circuit could end up being a better approach on this problem. Design: I propose two options to reduce this problem, both based on taking into account the amount of bytes running through a circuit. MaxCircuitSizeDirtiness (temporal parameter name) will take an integer field that is contained on an interval and represents the maximum amount of bytes that can be written/read (we need to discuss about the use of one for both) by the circuit. If the circuit exceeds that amount, new streams won't use this circuit anymore. MaxCircuitSizeDirtinessByPort (temporal parameter name) will take an array of integers that are contained on an interval and represents the maximum amount of bytes that can be written/read (we need to discuss about the use of one for both) by the circuit per port (StreamIsolationByPort). This array is parallel to the array of ports from StreamIsolationByPort. If the circuit exceeds that amount, new streams won't use this circuit anymore. Regarding default values it would be useful to set up one a bit lower than the average amount of bytes per circuit. On MaxCircuitSizeDirtinessByPort after discuss it we shouldn't set up a default value because someone can identify the port used. About MaxCircuitDirtiness, if the others are set up by default it could be bigger, like thirty minutes, so if the user doesn't send/receive a significant amount of data the circuit will be changed anyway. Security Implications: It is believed that the proposed changes will improve the anonymity for end users. The end user won't reuse a given circuit if they have sent a considerable amount of bytes, thus making more difficult for malicious Exit relays to be able to trace a user's pseudonymous profile. Obviously this is a probability, of course it's possible that sensitive data will leak in a little amount of data but it's more even possible that sensitive data will leak in a large amount. Specification: In order to implement this feature we will need to add some new functionalities. We need to parse MaxCircuitSizeDirtiness and MaxCircuitSizeDirtinessByPort from the torrc config file. We need to create a function or improve one to check the amount of bytes that are running through the circuit and if this amount is higher than the established value, consider the circuit dirty. Compatibility: The proposed changes should not create any compatibility issues. New Tor clients will be able to take advantage of this without any modification to the network. Implementation: It is proposed that MaxCircuitSizeDirtiness will be enabled by default and also increase MaxCircuitDirtiness to thirty minutes. It is proposed that MaxCircuitSizeDirtinessByPort won't be enabled by default for port 22, 53, and port 80 as StreamIsolationByPort. About TorBrowser or any other Tor application that is able to manage circuits by its own because of KeepAliveIsolateSOCKSAuth option being active by default shouldn't be affected by this new feature. As the same form that it currently ignores MaxCircuitDirtiness parameter. Performance and scalability notes: The proposed changes will reduce Tor network stress as users who do not exceed the set amount will reduce circuit generation by three (if default MaxCircuitDirtinesss value is thirty minutes). I want to work on demonstrating that by a research but first it's nice to get the idea accepted. References: Tor project research ideas [https://research.torproject.org/ideas.html] Enhancing Tor's Performance using Real-time Traffic Classification [https://www.cypherpunks.ca/~iang/pubs/difftor-ccs.pdf] (It's not exactly about that, but they talk about circuit lifetime and the ten minutes problem a few times. Also it's an interesting paper.)

3 8

Proposal 286: Controller APIs for hibernation access on mobile
by Nick Mathewson 08 Dec '17

08 Dec '17

Filename: 286-hibernation-api.txt Title: Controller APIs for hibernation access on mobile Author: Nick Mathewson Created: 30-November-2017 Status: Open 1. Introduction On mobile platforms, battery life is achieved by reducing needless network access and CPU access. Tor currently provides few ways for controllers and operating systems to tune its behavior. This proposal describes controller APIs for better management of Tor's hibernation mechanisms, and extensions to those mechanisms, for better power management in mobile environments. 1.1. Background: hibernation and idling in Tor today We have an existing "hibernation" mechanism that we use to implement "bandwidth accounting" and "slow shutdown" mechanisms: When a Tor instance is close to its bandwidth limit: it stops accepting new connections or circuits, and only processes those it has, until the bandwidth limit is reached. Once the bandwidth limit is reached, Tor closes all connections and circuits, and all non-controller listeners, until a new accounting limit begins. Tor handles the INT signal on relays similarly: it stops accepting new connections or circuits, and gives the existing ones a short interval in which to shut down. Then Tor closes all connections and exits the process entirely. Tor's "idle" mechanism is related to hibernation, though its implementation is separate. When a Tor clients has passed a certain amount of time without any user activity, it declares itself "idle" and stops performing certain background tasks, such as fetching directory information, or building circuits in anticipation of future needs. (This is tied in the codebase to the "predicted ports" mechanism, but it doesn't have to be.) 1.2. Background: power-management signals on mobile platforms (I'm not a mobile developer, so I'm about to wildly oversimplify. Please let me know where I'm wrong.) Mobile platforms achieve long battery life by turning off the parts they don't need. The most important parts to turn off are the antenna(s) and the screen; the CPU can be run in a slower mode. But it doesn't do much good turning things off when they're unused, if some background app is going to make sure that they're always in use! So mobile platforms use signals of various kinds to tell applications "okay, shut up now". Some apps need to do online background activities periodically; to help this out, mobile platforms give them a signal "Hey, now is a good time if you want to do that" and "stop now!" 1.3. Mostly out-of-scope: limiting CPU wakeups when idle. The changes described here will be of limited use if we do not also alter Tor so that, when it's idle, the CPU is pretty quiet. That isn't the case right now: we have large numbers of callbacks that happen periodically (every second, every minute, etc) whether they need to or not. We're hoping to limit those, but that's not what this proposal is about. 2. Improvements to the hibernation model To present a consistent interface that applications and controllers can use to manage power consumption, we make these enhancements to our hibernation model. First, we add three new hibernation states: "IDLE", "IDLE_UPDATING", "SLEEP", and "SLEEP_UPDATING". "IDLE" is like the current "idle" or "no predicted ports" state: Tor doesn't launch circuits or start any directory activity, but its listeners are still open. Tor clients can enter the IDLE state on their own when they are LIVE, but haven't gotten any client activity for a while. Existing connections and circuits are not closed. If the Tor instance receives any new connections, it becomes LIVE. "IDLE_UPDATING" is like IDLE, except that Tor should check for directory updates as appropriate. If there are any, it should fetch directory information, and then become IDLE again. "SLEEPING" is like the current "dormant state we use for bandwidth exhaustion, but it is controller-initiated: it begins when Tor is told to enter it, and ends when Tor is told to leave it. Existing connections and circuits are closed; listeners are closed too. "SLEEP_UPDATING" is like SLEEP, except that Tor should check for directory updates as appropriate. If there are any, it should fetch directory information, and then SLEEP again. 2.1. Relay operation Relays and bridges should not automatically become IDLE on their own. 2.2. Onion service operation When a Tor instance that is running an onion service is IDLE, it does the minimum to try to remain responsive on the onion service: It keeps its introduction points open if it can. Once a day, it fetches new directory information and opens new introduction points. 3. Controller hibernation API 3.1. Examining the current hibernation state We define a new "GETINFO status/hibernation" to inspect the current hibernation state. Possible values are: - "live" - "idle:control" - "idle:no-activity" - "sleep:control" - "sleep:accounting" - "idle-update:control" - "sleep-update:control" - "shutdown:exiting" - "shutdown:accounting" - "shutdown:control" The first part of each value indicates Tor's current state: "live" -- completely awake "idle" -- waiting to see if anything happens "idle-update" -- waiting to see if anything happens; probing for directory information "sleep" -- completely unresponsive "shutdown" -- unresponsive to new requests; still processing existing requests. The second part of each value indicates the reason that Tor entered this state: "control" -- a controller told us to do this. "no-activity" -- Tor became idle on its own due to not noticing any requests. "accounting" -- the bandwidth system told us to enter this state. "exiting" -- Tor is in this state because it's getting ready to exit. We add a STATUS_GENERAL hibernation event as follows: HIBERNATION "STATUS=" (one of the status pairs above.) Indicates that Tor's hibernation status has changed. Note: Controllers MUST accept status values here that they don't recognize. The "GETINFO accounting/hibernating" value and the "STATUS_SERVER HIBERANATION_STATUS" event keep their old meaning. 3.2. Changing the hibernation state We add the following new possible values to the SIGNAL controller command: "SLEEP" -- enter the sleep state, after an appropriate shutdown interval. "IDLE" -- enter the idle state "SLEEPWALK" -- If in sleep or idle, start probing for directory information in the sleep-update or idle-update state respectively. Remain in that state until we've probed for directory information, or until we're told to IDLE or SLEEP again, or (if we're idle) until we get client activity. Has no effect if not in sleep or idle. "WAKEUP" -- If in sleep, sleep-update, idle, idle-update, or shutdown:sleep state, enter the live state. Has no effect in any other state. 3.3. New configuration parameters StartIdle -- Boolean. If set to 1, Tor begins in IDLE mode.

5 8

Student project to work on IPv6 support (was: [tor-relays] About relay size)
by Santiago R.R. 06 Dec '17

06 Dec '17

(Moving this thread from tor-relay) El 03/10/17 a las 14:25, teor escribió: > > > On 3 Oct 2017, at 10:57, Roman Mamedov <rm(a)romanrm.net> wrote: > > > > On Tue, 3 Oct 2017 09:53:46 -0400 > > teor <teor2345(a)gmail.com> wrote: > > > >>> For interposing dual-protocoled nodes along the way, how many do there > >>> have to be for it to become "not too limiting"? > >> > >> This is one of the questions we need researchers to answer. > > > > I can't help but feel you are overcomplicating this. > > > > Clients create a circuit by randomly picking 3 nodes out of the all-nodes > > pile, right? If all 3 happen to be IPv6-capable, then the circuit can go over > > IPv6 and all is fine. If some of the 3 happen to be IPv6-only while others are > > IPv4-only, the whole selection can be thrown away and repeated. > > > > That way IPv6-only relays could get some usage on a totally random basis, with > > no compromises and no restraining "of the next hop based on the previous one", > > not hurting anonymity. Clients just need to know which nodes are IPv4-only, > > IPv6-only or dual-stack, to not attempt unworkable combinations, discarding > > them instead. > > Discarding unworkable combinations and restraining node choices seem > equivalent to me, although the relay weights may be different. > > > And as there are more and more dual-stack or IPv6-only relays, the "throw > > away" step will be needed less and less often. > > If you think this will work and is safe for client anonymity, then the next step > is to write a tor proposal. Having a concrete design could help with > analysing the anonymity implications as well. > > I think IPv6-only relays are a lower priority than better IPv6 and dual-stack > client support, and IPv6-only bridge support But we could do both in the > same release. Hello tor-dev, With my colleague JC Bach (in CC), we have proposed a last-year student project to address IPv6-related issues in Tor for the upcoming semester, at IMT Atlantique engineering school. There will be two students working on it. It is hard to say now how far we will arrive, especially because this is our first approach to Tor entrails. So this message is to say we have good chances to come back here looking for help :-) Cheers, -- Santiago

2 4

Rebooting work on proposal #247 (guard discovery)
by George Kadianakis 06 Dec '17

06 Dec '17

Hello Mike, I'm finally getting out of the prop224/microdescriptor bug pile, and getting more time to start working on guard stuff like prop247 again. I'm planning to spend a few days next week to regain knowledge on prop247. I'll check out the notes from the Wilmington hackfest, re-read my old simulator's code, etc. I know you have thought more about prop247 the past months, and it would be great if you could brief me up on any updates that I should know about. Specifically I'm wondering if you have any new insights on how the proposed prop247 changes interact with Tor's guard algorithm (prop271)? Also any other things I should know about from your work on the performance simulator? Perhaps ideas about performance, topology or path restrictions? I'm hoping that such updates might help me boot up faster! Cheers! :-)

2 2

Privacy Pass
by bancfc＠openmailbox.org 29 Nov '17

29 Nov '17

Hi. Are there any plans to include Privacy Pass addon in Tor Browser by default? Privacy Pass is the result of some great work by Ian and his team at University of Waterloo to spare Tor users the torture of solving infinite captchas from Cloudflare.[0][1] [0] https://privacypass.github.io/team/ [1] https://blog.cloudflare.com/privacy-pass-the-math/

4 3

[release] [protocol] Onionoo 4.4-1.8.0
by iwakeh 28 Nov '17

28 Nov '17

Hi there! Onionoo's protocol was extended and has a minor version jump to 4.4. Download available at: https://dist.torproject.org/onionoo/4.4-1.8.0/ Main protocol changes (also summarized in [0]): The version field, which was introduced in 4.1, is now available for bridges, too. The "recommended_version" parameter is provided as an additional search parameter. Details can be found in [1a], [1b], and [2]. The changes are already deployed on all onionoo.torproject.org instances and the update included an upgrade of the geoip data used in Onionoo. Reminder: a major *upcoming version* jump 5.0 lies ahead. Take note of the field "next_major_version_scheduled":"2017-12-17" in documents. Please direct comments and questions to the metrics-team mailing list [3]. Cheers, iwakeh [0] https://metrics.torproject.org/onionoo.html#versions_4_4 [1a] https://metrics.torproject.org/onionoo.html#details_relay_recommended_versi… [1b] https://metrics.torproject.org/onionoo.html#details_bridge_recommended_vers… [2] hhttps://metrics.torproject.org/onionoo.html#parameters_search [3] https://lists.torproject.org/cgi-bin/mailman/listinfo/metrics-team

1 0

Tor and DNS
by N6Ghost 25 Nov '17

25 Nov '17

hi all, saw an open item in the tor projects, about dns and other resource record types. this got me thinking about just trying to understand Tor and DNS. for what I gather so far, is Tor and dns is only about "a" records and quad records "aaaa", thats pretty much it. i think PTR also but just whats need for hostname lookup. I am assuming this means tor clients, and tor browsers but have yet to validate that. any other info on this would be helpful. still trying to dig around and find where the data may be. -N6Ghost

1 0