tor-dev October 2016

tor-dev@lists.torproject.org

52 participants
32 discussions

Proposal 247 (Hidden Service Vanguards) Overhaul
by Mike Perry 09 Jun '17

09 Jun '17

I spent some time trying to clean up proposal 247 based on everyone's comments, as well as based on my own thoughts. Please have a look if you commented on the original proposal, and complain if I've not taken your thoughts into account. (Aaron: In particular, I made several tradeoffs in favor of performance and DoS resistance that may be at odds with some of your suggestions, but I think the end result is still OK after looking into the Sybil rates and thinking about the adversary model in more detail. You may disagree). I've attached my updated version of the proposal inline in this mail, but the canonical updated proposal is in my remote at: https://gitweb.torproject.org/user/mikeperry/torspec.git/tree/proposals/247… Here's a summary of the changes (which are also listed in Git): * Try to make a coherent threat model and specify its assumptions * Fold in my comments about using disjoint sets ("buckets") for the third level guard. * Make the parameter discussion subsection its own section, and include tables with far more detail for the Sybil success rates. * Put the rotation period in a separate subsection from the number of guards * Switch to using min(X,X) and max(X,X) for the distribution for the second and third layer guard lifespans, respectively. Add a subsection describing this distribution (3.2.3) * Changed the default parameters based on these tables, and based on my own intuition about Tor's performance properties. * Move the load balancing, torrc, and other performance considerations to their own section (Section 5). * Move "3.2. Distinguishing new HS circuits from normal HS circuits" to section 4.1. * Fold in some of "3.3. Circuit nodes can now be linked to specific hidden services" into 4.1. Some of it I just removed, though, because I did not find it credible. * Added Roger's concerns about guard linkability to Section 4.2. * Added a denial of service subsection to Section 4.3. ================================ Filename: 247-hs-guard-discovery.txt Title: Defending Against Guard Discovery Attacks using Vanguards Author: George Kadianakis Created: 2015-07-10 Status: Draft 0. Motivation A guard discovery attack allow attackers to determine the guard node of a Tor client. The hidden service rendezvous protocol provides an attack vector for a guard discovery attack since anyone can force an HS to construct a 3-hop circuit to a relay (#9001). Following the guard discovery attack with a compromise and/or coercion of the guard node can lead to the deanonymization of a hidden service. 1. Overview This document tries to make the above guard discovery + compromise attack harder to launch. It introduces an optional configuration option which makes the hidden service also pin the second and third hops of its circuits for a longer duration. With this new path selection, we force the adversary to perform a Sybil attack and two compromise attacks before succeeding. This is an improvement over the current state where the Sybil attack is trivial to pull off, and only a single compromise attack is required. With this new path selection, an attacker is forced to do a one or more node compromise attacks before learning the guard node of a hidden service. This increases the uncertainty of the attacker, since compromise attacks are costly and potentially detectable, so an attacker will have to think twice before beginning a chain of node compromise attacks that he might not be able to complete. 1.1. Visuals Here is how a hidden service rendezvous circuit currently looks like: -> middle_1 -> middle_A -> middle_2 -> middle_B -> middle_3 -> middle_C -> middle_4 -> middle_D HS -> guard -> middle_5 -> middle_E -> Rendezvous Point -> middle_6 -> middle_F -> middle_7 -> middle_G -> middle_8 -> middle_H -> ... -> ... -> middle_n -> middle_n this proposal pins the two middles nodes to a much more restricted set, as follows: -> guard_3A_A -> guard_2_A -> guard_3A_B -> guard_3A_C -> Rendezvous Point HS -> guard_1 -> guard_3B_D -> guard_2_B -> guard_3B_E -> guard_3B_F -> Rendezvous Point Note that the third level guards are partitioned into buckets such that they are only used with one specific second-level guard. In this way, we ensure that even if an adversary is able to execute a Sybil attack against the third layer, they only get to learn one of the second-layer Guards, and not all of them. This prevents the adversary from gaining the ability to take their pick of the weakest of the second-level guards for further attack. 2. Design This feature requires the HiddenServiceGuardDiscovery torrc option to be enabled. When a hidden service picks its guard nodes, it also picks two additional sets of middle nodes `second_guard_set` and `third_guard_set` of size NUM_SECOND_GUARDS and NUM_THIRD_GUARDS respectively for each hidden service. These sets are unique to each hidden service created by a single Tor client, and must be kept separate and distinct. When a hidden service needs to establish a circuit to an HSDir, introduction point or a rendezvous point, it uses nodes from `second_guard_set` as the second hop of the circuit and nodes from that second hop's corresponding `third_guard_set` as third hops of the circuit. A hidden service rotates nodes from the 'second_guard_set' at a random time between MIN_SECOND_GUARD_LIFETIME hours and MAX_SECOND_GUARD_LIFETIME hours. A hidden service rotates nodes from the 'third_guard_set' at a random time between MIN_THIRD_GUARD_LIFETIME and MAX_THIRD_GUARD_LIFETIME hours. These extra guard nodes should be picked with the same path selection procedure that is used for regular middle nodes (though see Section 5.1 for performance reasons to restrict this slightly). Each node's rotation time is tracked independently, to avoid disclosing the rotation times of the primary and second-level guards. XXX how should proposal 241 ("Resisting guard-turnover attacks") be applied here? 2.1. Security parameters We set NUM_SECOND_GUARDS to 4 nodes and NUM_THIRD_GUARDS to 16 nodes (ie four sets of four). XXX: 3 and 12 might be another option here, in which case our rotation period for the second guard position can be reduced to 15 days. We set MIN_SECOND_GUARD_LIFETIME to 1 day, and MAX_SECOND_GUARD_LIFETIME to 33 days, for an average rotation rate of ~11 days, using the min(X,X) distribution specified in Section 3.2.2. We set MIN_THIRD_GUARD_LIFETIME to 1 hour, and MAX_THIRD_GUARD_LIFETIME to 18 hours, for an average rotation rate of ~12 hours, using the max(X,X) distribution specified in Section 3.2.2. XXX make all the above consensus parameters? Yes. Very yes, especially if we decide to change the primary guard lifespan. See Section 3 for more analysis on these constants. 3. Rationale and Security Parameter Selection 3.1. Threat model, Assumptions, and Goals Consider an adversary with the following powers: - Can launch a Sybil guard discovery attack against any node of a rendezvous circuit. The slower the rotation period of the node, the longer the attack takes. Similarly, the higher the percentage of the network is compromised, the faster the attack runs. - Can compromise any node on the network, but this compromise takes time and potentially even coercive action, and also carries risk of discovery. We also make the following assumptions about the types of attacks: 1. A Sybil attack is noisy. It will require either large amounts of traffic, multiple test circuits, or both. 2. A Sybil attack against the second or first layer Guards will be more noisy than a Sybil attack against the third layer guard, since the second and first layer Sybil attack requires a timing side channel in order to determine success, where as the Sybil success is almost immediately obvious to third layer guard, since it will now be returned as a rend point for circuits for the hidden service in question. 3. As soon as the adversary is confident they have won the Sybil attack, an even more aggressive circuit building attack will allow them to determine the next node very fast (an hour or less). 4. The adversary is strongly disincentivized from compromising nodes that may prove useless, as node compromise is even more risky for the adversary than a Sybil attack in terms of being noticed. Given this threat model, our security parameters were selected so that the first two layers of guards should be hard to attack using a Sybil guard discovery attack and hence require a node compromise attack. Ideally, we want the node compromise attacks to carry a non-negligible probability of being useless to the adversary by the time they complete. On the other hand, the outermost layer of guards should rotate fast enough to _require_ a Sybil attack. 3.2. Parameter Tuning 3.2.1. Sybil rotation counts for a given number of Guards The probability of Sybil success for Guard discovery can be modeled as the probability of choosing 1 or more malicious middle nodes for a sensitive circuit over some period of time. P(At least 1 bad middle) = 1 - P(All Good Middles) = 1 - P(One Good middle)^(num_middles) = 1 - (1 - c/n)^(num_middles) c/n is the adversary compromise percentage In the case of Vanguards, num_middles is the number of Guards you rotate through in a given time period. This is a function of the number of vanguards in that position (v), as well as the number of rotations (r). P(At least one bad middle) = 1 - (1 - c/n)^(v*r) Here's detailed tables in terms of the number of rotations required for a given Sybil success rate for certain number of guards. 1.0% Network Compromise: Sybil Success One Two Three Four Five Six Eight Nine Ten Twelve Sixteen 10% 11 6 4 3 3 2 2 2 2 1 1 15% 17 9 6 5 4 3 3 2 2 2 2 25% 29 15 10 8 6 5 4 4 3 3 2 50% 69 35 23 18 14 12 9 8 7 6 5 60% 92 46 31 23 19 16 12 11 10 8 6 75% 138 69 46 35 28 23 18 16 14 12 9 85% 189 95 63 48 38 32 24 21 19 16 12 90% 230 115 77 58 46 39 29 26 23 20 15 95% 299 150 100 75 60 50 38 34 30 25 19 99% 459 230 153 115 92 77 58 51 46 39 29 5.0% Network Compromise: Sybil Success One Two Three Four Five Six Eight Nine Ten Twelve Sixteen 10% 3 2 1 1 1 1 1 1 1 1 1 15% 4 2 2 1 1 1 1 1 1 1 1 25% 6 3 2 2 2 1 1 1 1 1 1 50% 14 7 5 4 3 3 2 2 2 2 1 60% 18 9 6 5 4 3 3 2 2 2 2 75% 28 14 10 7 6 5 4 4 3 3 2 85% 37 19 13 10 8 7 5 5 4 4 3 90% 45 23 15 12 9 8 6 5 5 4 3 95% 59 30 20 15 12 10 8 7 6 5 4 99% 90 45 30 23 18 15 12 10 9 8 6 10.0% Network Compromise: Sybil Success One Two Three Four Five Six Eight Nine Ten Twelve Sixteen 10% 2 1 1 1 1 1 1 1 1 1 1 15% 2 1 1 1 1 1 1 1 1 1 1 25% 3 2 1 1 1 1 1 1 1 1 1 50% 7 4 3 2 2 2 1 1 1 1 1 60% 9 5 3 3 2 2 2 1 1 1 1 75% 14 7 5 4 3 3 2 2 2 2 1 85% 19 10 7 5 4 4 3 3 2 2 2 90% 22 11 8 6 5 4 3 3 3 2 2 95% 29 15 10 8 6 5 4 4 3 3 2 99% 44 22 15 11 9 8 6 5 5 4 3 The rotation counts in these tables were generated with: def count_rotations(c, v, success): r = 0 while 1-math.pow((1-c), v*r) < success: r += 1 return r 3.2.2. Rotation Period As specified in Section 3.1, the primary driving force for the third layer selection was to ensure that these nodes rotate fast enough that it is not worth trying to compromise them, because it is unlikely for compromise to succeed and yield useful information before the nodes stop being used. For this reason we chose 1 to 18 hours, with a weighted distribution (Section 3.2.3) causing the expected average to be 12 hours. From the table in Section 3.2.1, it can be seen that this means that the Sybil attack will complete with near-certainty (99%) in 29*12 hours (14.5 days) for the 1% adversary, 3 days for the 5% adversary, and 1.5 days for the 10% adversary. Since rotation of each node happens independently, the distribution of when the adversary expects to win this Sybil attack in order to discover the next node up is uniform. This means that on average, the adversary should expect that half of the rotation period of the next node is already over by the time that they win the Sybil. With this fact, we choose our range and distribution for the second layer rotation to be short enough to cause the adversary to risk compromising nodes that are useless, yet long enough to require a Sybil attack to be noticeable in terms of client activity. For this reason, we choose a minimum second-layer guard lifetime of 1 day, since this gives the adversary a minimum expected value of 12 hours for during which they can compromise a guard before it might be rotated. If the total expected rotation rate is 11 days, then the adversary can expect overall to have 5.5 days remaining after completing their Sybil attack before a second-layer guard rotates away. 3.2.3. Rotation distribution In order to skew the distribution of the third layer guard towards higher values, we use max(X,X) for the distribution, where X is a random variable that takes on values from the uniform distribution. In order to skew the distribution of the second layer guard towards low values (to increase the risk of compromising useless nodes) we skew the distribution towards lower values, using min(X,X). Here's a table of expectation (arithmetic means) for relevant ranges of X (sampled from 0..N). The current choice for second-layer guards is noted with **, and the current choice for third-layer guards is noted with ***. Range Min(X,X) Max(X,X) 10 2.85 6.15 11 3.18 6.82 12 3.51 7.49 13 3.85 8.15 14 4.18 8.82 15 4.51 9.49 16 4.84 10.16 17 5.18 10.82*** 18 5.51 11.49 19 5.84 12.16 20 6.18 12.82 21 6.51 13.49 22 6.84 14.16 23 7.17 14.83 24 7.51 15.49 25 7.84 16.16 26 8.17 16.83 27 8.51 17.49 28 8.84 18.16 29 9.17 18.83 30 9.51 19.49 31 9.84 20.16 32 10.17** 20.83 33 10.51 21.49 34 10.84 22.16 35 11.17 22.83 36 11.50 23.50 37 11.84 24.16 38 12.17 24.83 39 12.50 25.50 4. Security concerns and mitigations 4.1. Mitigating fingerprinting of new HS circuits By pinning the middle nodes of rendezvous circuits, we make it easier for all hops of the circuit to detect that they are part of a special hidden service circuit with varying degrees of certainty. The Guard node is able to recognize a Vanguard client with a high degree of certainty because it will observe a client IP creating the overwhelming majority of its circuits to just a few middle nodes in any given 10-18 day time period. The middle nodes will be able to tell with a variable certainty that depends on both its traffic volume and upon the popularity of the service, because they will see a large number of circuits that tend to pick the same Guard and Exit. The final nodes will be able to tell with a similar level certainty that depends on their capacity and the service popularity, because they will see a lot of rend handshakes that all tend to have the same second hop. The most serious of these is the Guard fingerprinting issue. When proposal xxx-padding-negotiation is implemented, services that enable this feature should use those padding primitives to create fake circuits to random middle nodes that are not their guards, in an attempt to look more like a client. Additionally, if Tor Browser implements "virtual circuits" based on SOCKS username+password isolation in order to enforce the re-use of paths when SOCKS username+passwords are re-used, then the number of middle nodes in use during a typical user's browsing session will be proportional to the number of sites they are viewing at any one time. This is likely to be much lower than one new middle node every ten minutes, and for some users, may be close to the number of Vanguards we're considering. This same reasoning is also an argument for increasing the number of second-level guards beyond just two, as it will spread the hidden service's traffic over a wider set of middle nodes, making it both easier to cover, and behave closer to a client using SOCKS virtual circuit isolation. 4.2. Hidden service linkability Multiple hidden services on the same Tor instance should use separate second and third level guard sets, otherwise an adversary is trivially able to determine that the two hidden services are co-located by inspecting their current chosen rend point nodes. Unfortunately, if the adversary is still able to determine that two or more hidden services are run on the same Tor instance through some other means, then they are able to take advantage of this fact to execute a Sybil attack more effectively, since there will now be an extra set of guard nodes for each hidden service in use. For this reason, if Vanguards are enabled, and more than one hidden service is configured, the user should be advised to ensure that they do not accidentally leak that the two hidden services are from the same Tor instance. 4.3. Denial of service Since it will be fairly trivial for the adversary to enumerate the current set of rend nodes for a hidden service, denial of service becomes a serious risk for Vanguard users. For this reason, it is important to support a large number of third-level guards, to increase the amount of resources required to bring a hidden service offline by DoSing just a few Tor nodes. 5. Performance considerations The switch to a restricted set of nodes will very likely cause significant performance issues, especially for high-traffic hidden services. If any of the nodes they select happen to be temporarily overloaded, performance will suffer dramatically until the next rotation period. 5.1. Load Balancing Since the second and third level "guards" are chosen from the set of all nodes eligible for use in the "middle" hop (as per hidden services today), this proposal should not significantly affect the long-term load on various classes of the Tor network, and should not require any changes to either the node weight equations, or the bandwidth authorities. Unfortunately, transient load is another matter, as mentioned previously. It is very likely that this scheme will increase instances of transient overload at nodes selected by high-traffic hidden services. One option to reduce the impact of this transient overload is to restrict the set of middle nodes that we chose from to some percentage of the fastest middle-capable relays in the network. This may have some impact on load balancing, but since the total volume of hidden service traffic is low, it may be unlikely to matter. 5.2. Circuit build timeout The adaptive circuit build timeout mechanism in Tor is what corrects for instances of transient node overload right now. The timeout will naturally tend to select the current fastest and least-loaded paths even through this set of restricted routes, but it may fail to behave correctly if there are a very small set of nodes in each guard set, as it is based upon assumptions about the current path selection algorithm, and it may need to be tuned specifically for Vanguards, especially if the set of possible routes is small. 5.3. OnionBalance At first glance, it seems that this scheme makes multi-homed hidden services such as OnionBalance[1] even more important for high-traffic hidden services. Unfortunately, if it is equally damaging to the user for any of their multi-homed hidden service locations to be discovered, then OnionBalance is strictly equivalent to simply increasing the number of second-level guard nodes in use, because an active adversary can perform simultaneous Sybil attacks against all of the rend points offered by the multi-homed OnionBalance introduction points. 5.4. Default vs optional behavior We suggest this torrc option to be optional because it changes path selection in a way that may seriously impact hidden service performance, especially for high traffic services that happen to pick slow guard nodes. However, by having this setting be disabled by default, we make hidden services who use it stand out a lot. For this reason, we should in fact enable this feature globally, but only after we verify its viability for high-traffic hidden services, and ensure that it is free of second-order load balancing effects. Even after that point, until Single Onion Services are implemented, there will likely still be classes of very high traffic hidden services for whom some degree of location anonymity is desired, but for which performance is much more important than the benefit of Vanguards, so there should always remain a way to turn this option off. 6. Future directions Here are some more ideas for improvements that should be done sooner or later: - Maybe we should make the size and rotation period of secondary/third guard sets to be configurable by the user. - To make it harder for an adversary, a hidden service MAY extend the path length of its circuits by an additional static hop. This forces the adversary to use another coercion attack to walk the chain up to the hidden service. 7. Acknowledgments Thanks to Aaron Johnson, John Brooks, Mike Perry and everyone else who helped with this idea. 1. https://onionbalance.readthedocs.org/en/latest/design.html#overview -- Mike Perry

7 14

onionoo.tpo stuck at 2016-05-13 12:00
by nusenu 29 Jan '17

29 Jan '17

Hi Karsten, I was surprised that ornetradar did not send a single email for yesterday's new relays. After looking into it, it turned out it is an onionoo problem. "relays_published":"2016-05-13 12:00:00" https://onionoo.torproject.org/details?limit=4 regards, nusenu

2 11

Proposal 274: A Name System API for Tor Onion Services
by George Kadianakis 26 Jan '17

26 Jan '17

Hello list, we've been busy discussing secure name systems for the past few months, and how we could integrate them with Tor. We had a few people ask us for precise interfaces, so here is an attempt for a modular name system API. It's largely based on the PT API, which has been pretty successful. The proposal was brainstormed on a Sunday morning in Seattle between Yawning, David and me. I wrote the actual proposal contents so any innacuracies, impossibilities and errors are mine :) If people like it and are interested in implementing it, we can draft out an implementation plan for any person who wants to go for it. But for now, please consider this proposal text as a draft that is meant to be improved from this list's feedback. Have fun and looking forward to your comments (and code). --- Filename: 274-naming-layer-api.txt Title: A Name System API for Tor Onion Services Author: George Kadianakis, Yawning Angel, David Goulet Created: 04-Oct-2016 Status: Draft Table Of Contents: 1. Introduction 1.1. Motivation 1.2. Design overview and rationale 2. System Specification 2.1. System overview [SYSTEMOVERVIEW] 2.2. System Illustration 2.3. System configuration [TORRC] 2.3.1. Tor name resolution logic 2.4. Name system initialization [INITPROTOCOL] 2.5. Name resolution using NS API 2.5.1. Message format 2.5.2. RESOLVED status codes 2.5.3. Further name resolution behavior 2.6. Cancelling a name resolution request 2.7. Launching name plugins [INITENVVARS] 2.8. Name plugin workflow [NSBEHAVIOR] 2.8.1. Name plugin shutdown [NSSHUTDOWN] 2.9. Further details of stdin/stdout communication 2.9.1. Message Format 3. Discussion 3.1. Using second-level domains instead of tld 3.2. Name plugins handling all tlds '*' 3.3. Deployment strategy 3.4. Miscellaneous discussion topics 4. Acknowledgements A.1: Example communication Tor <-> name plugin [PROTOEXAMPLE] A.2: Example plugins [PLUGINEXAMPLES] 1. Introduction This proposal specifies a modular API for integrating name systems with Tor. 1.1. Motivation Tor onion service addresses are decentralized and self-authenticated but they are not human-memorable (e.g. 3g2upl4pq6kufc4m.onion). This is a source of poor usability, since Internet users are familiar with the convenient naming of DNS and are not used to addresses being random text. In particular, onion addresses are currently composed of 16 random base32 characters, and they look like this: 3g2upl4pq6kufc4m.onion vwakviie2ienjx7t.onion idnxcnkne4qt76tg.onion vwakviie2ienjx6t.onion When Proposal 224 get deployed, onion addresses will become even bigger: 53 base32 characters. That's: llamanymityx4fi3l6x2gyzmtmgxjyqyorj9qsb5r543izcwymle.onion lfels7g3rbceenuuqmpsz45z3lswakqf56n5i3bvqhc22d5rrsza.onion odmmeotgcfx65l5hn6ejkaruvai222vs7o7tmtllszqk5xbysola.onion qw3yvgovok3dsarhqephpu2pkiwzjdk2fgdfwwf3tb3vgzxr5kba.onion Over the years Tor users have come up with various ad-hoc ways of handling onion addresses. For example people memorize them, or use third-party centralized directories, or just google them everytime. We believe that the UX problem of non-human-memorable problems is not actually solved with the above ad-hoc solutions and remains a critical usability barrier that prevents onion services from being used by a wider audience. 1.2. Design overview and rationale During the past years there has been lots of research on secure naming and various such systems have been proposed (e.g. GNS, Namecoin, etc.). Turns out securely naming things is a very hard research problem, and hence none of the proposed systems is a clear winner: all of them carry various trade-offs. Furthermore, none of the proposed systems has seen widespread use so far, which makes it even harder to pick a single project. Given the plenitude of options, one approach to decide which system is best is to make various decent name systems available and let the Tor community and the sands of time pick the winner. Also, it might be that there is no single winner, and perhaps different specialized name system should be used in different situations. We believe that by getting secure name systems actually get utilized by real users, the whole field will mature and existing systems will get battle-hardened. Hence, the contribution of this proposal is a modular Name System API (NSA) that allows developers to integrate their own name systems in Tor. The interface design is name-system-agnostic, and it's heavily based on the pluggable transports API (proposal 180). It should be flexible enough to accommodate all sorts of name systems (see [PLUGINEXAMPLES]). 2. System Specification A developer that wants to integrate a name system with Tor needs to first write a wrapper that understands the Tor Name System API (NS API). Using the Name System API, Tor asks the name system to perform name queries, and receives the query results. The NS API works using environment variables and stdin/stdout communication. It aims to be portable and easy to implement. 2.1. System overview [SYSTEMOVERVIEW] Here is an overview of the Tor name system: Alice, a Tor user, can activate various name systems by editing her torrc file and specifying which tld each name system is responsible for. For this section, let's consider a simple fictional name system, unicorn, which magically maps domains with the .corn tld to the correct onion address. Here it is: OnionNamePlugin 0 .corn /usr/local/bin/unicorn After Alice enables the unicorn plugin, she attempts connecting to elephantforum.corn. Tor will intercept the SOCKS request, and use the executable at /usr/local/bin/unicorn to query the unicorn name system for elephantforum.corn. Tor communicates with the unicorn plugin using the Tor NS API through which name queries and their results can be transported using stdin/stdout. If elephantforum.corn corresponds to an onion address in the unicorn name system, unicorn should return the onion address to Tor using the Tor NS API. Tor must then internally rewrite the elephantforum.corn address to the actual onion address, and initiate a connection to it. 2.2. System Illustration Here is a diagram illustrating how the Tor Name System API works. The name system used in this example is GNS, but there is nothing GNS-specific here and GNS could be swapped for any other name system (like hosts files, or Namecoin). The example below illustrates how a user who types debian.zkey in their Tor browser gets redirected to sejnfjrq6szgca7v.onion after Tor consults the GNS network. Please revisit this illustration after reading the rest of the proposal. | $~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~$ | 1. $ 4. GNS magic!! $ | User: SOCKS CONNECT to $ debian.zkey -> sejnfjrq6szgca7v.onion$ | http://debian.zkey/ $~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~$~~~~~~~$ | $ +-----|-----------------------------------------+ $ |+----v-----+ 2. +---------+| 3. $ ||Tor | debian.zkey |Tor || debian.zkey +-$-------+ ||Networking------------------------->Naming -------------------------> | ||Submodule | |Submodule|| Tor Name System API | GNS | || <------------------------- <------------------------- wrapper| || | 6. | ||5. | | |+----|-----+ sejnfjrq6szgca7v.onion +---------+|sejnfjrq6szgca7v.onion +---------+ +-----|-----------------------------------------+ | 7. | Tor: Connect to | http://sejnfjrq6szgca7v.onion/ v 2.3. System configuration [TORRC] As demonstrated in [SYSTEMOVERVIEW], a Tor user who wants to use a name system has to edit their configuration file appropriately. Here is the torrc line format: OnionNamePlugin <priority> <tld> <path> where <priority> is a positive integer denoting the priority with which this name plugin should be consulted. <tld> is a string which restricts the scope of this plugin to a particular tld. Finally, <path> is a filesystem path to an executable that speaks the Tor Name System API and can act as an intermediary between Tor and the name system. For example here is a snippet from a torrc file: OnionNamePlugin 0 .hosts /usr/local/bin/local-hosts-file OnionNamePlugin 1 .zkey /usr/local/bin/gns-tor-wrapper OnionNamePlugin 2 .bit /usr/local/bin/namecoin-tor-wrapper OnionNamePlugin 3 .scallion /usr/local/bin/community-hosts-file 2.3.1. Tor name resolution logic When Tor receives a SOCKS request to an address that has a name plugin assigned to it, it needs to perform a query for that address using that name plugin. If there are multiple name plugins that correspond to the requested address, Tor queries all relevant plugins sorted by their priority value, until one of them returns a successful result. If two plugins have the same priority value, Tor MUST abort. If all plugins fail to successfuly perform the name resolution, Tor SHOULD default to using the exit node for name resolution. XXX or not? because of leaks? 2.4. Name system initialization [INITPROTOCOL] When Tor finds OnionNamePlugin lines in torrc, it launches and initializes their respective executables. When launching name plugins, Tor sets various environment variables to pass data to the name plugin (e.g. NS API version, state directory, etc.). More information on the environment variables at [INITENVVARS]. After a name plugin initializes and parses all needed environment variables, it communicates with Tor using its stdin/stdout. The first line that a name plugin sends to stdout signifies that it's ready to receive name queries. This line looks like this: INIT <VERSION> <STATUS_CODE> [<STATUS_MSG>] where VERSION is the Tor NS API protocol version that the plugin supports, STATUS_CODE is an integer status code, and STATUS_MSG is an optional string error message. STATUS_CODE value 0 is reserved for "success", and all other integers are error codes. See [PROTOEXAMPLE] for an example of this protocol. 2.5. Name resolution using NS API Here is how actual name resolution requests are performed in NS API. 2.5.1. Message format When Tor receives a SOCKS request to an address with a tld that has a name plugin assigned to it, Tor performs an NS API name query for that address. Tor does this by printing lines on the name plugin stdout as follows: RESOLVE <QUERY_ID> <NAME_STRING> where QUERY_ID is a unique integer corresponding to this query, and NAME_STRING is the name to be queried. When the name plugin completes the name resolution, it prints the following line in its stdout: RESOLVED <QUERY_ID> <STATUS_CODE> <RESULT> where QUERY_ID is the corresponding query ID and STATUS_CODE is an integer status code. RESULT is the resolution result (an onion address) or an error message if the resolution was not succesful. See [PROTOEXAMPLE] for an example of this protocol. XXX Should <RESULT> be optional in the case of failure? 2.5.2. RESOLVED status codes Name plugins can deliver the following status codes: 0 -- The name resolution was successful. 1 -- Name resolution generic failure. 2 -- Name tld not recognized. 3 -- Name not registered. 4 -- Name resolution timeout exceeded. XXX add more status codes here as needed 2.5.3. Further name resolution behavior Tor and name plugins MAY cache name resolution results in memory as needed. Caching results on disk should be avoided. Tor SHOULD abort (or cancel) an ongoing name resolution request, if it takes more than NAME_RESOLUTION_TIMEOUT seconds. XXX NAME_RESOLUTION_TIMEOUT = ??? Tor MUST validate that the resolution result is a valid .onion name. XXX should we also accept IPs and regular domain results??? XXX perhaps we should make sure that results are not names that need additional name resolution to avoid endless loops. e.g. imagine some sort of loop like this: debian.zkey -> debian-bla.zkey -> debian.zkey -> etc. 2.6. Cancelling a name resolution request Tor might need to cancel an ongoing name resolution request (e.g. because a timeout passed, or the client is not interested in that address anymore). In this case, Tor sends the following line to the plugin stdout as follows: CANCEL <QUERY_ID> to which the name plugin, after performing the cancellation, SHOULD answer with: CANCELED <QUERY_ID> 2.7. Launching name plugins [INITENVVARS] As described in [INITPROTOCOL], when Tor launches a name plugin, it sets certain environment variables. At a minimum, it sets (in addition to the normal environment variables inherited from Tor): "TOR_NS_STATE_LOCATION" -- A filesystem directory path where the plugin should store state if it wants to. This directory is not required to exist, but the plugin SHOULD be able to create it if it doesn't. The plugin MUST NOT store state elsewhere. Example: TOR_NS_STATE_LOCATION=/var/lib/tor/ns_state/ "TOR_NS_PROTO_VERSION" -- To tell the plugin which versions of this configuration protocol Tor supports. Future versions will give a comma-separated list. Plugins MUST accept comma-separated lists containing any version that they recognize, and MUST work correctly even if some of the versions they don't recognize are non-numeric. Valid version characters are non-space, non-comma printing ASCII characters. Example: TOR_NS_PROTO_VERSION=1,1a,2,4B "TOR_NS_PLUGIN_OPTIONS" -- Specifies configuration options for this name plugin as a semicolon-separated list of k=v strings with options that are to be passed to the plugin. Colons, semicolons, equal signs and backslashes MUST be escaped with a backslash. If there are no arguments that need to be passed to any of the plugins, "TOR_NS_PLUGIN_OPTIONS" MAY be omitted. For example consider the following options for the "banana" name plugin: TOR_NS_PLUGIN_OPTIONS=timeout=5;url=https://bananacake.com Will pass to banana the parameters 'timeout=5' and 'url=https://bananacake.com'. XXX Do we like this option-passing interface? Do we have any lessons from our PT experiences? XXX Add ControlPort/SocksPort environment variables. See [PROTOEXAMPLE] for an example of this environment 2.8. Name plugin workflow [NSBEHAVIOR] Name plugins follow the following workflow: 1) Tor sets the required environment values and launches the name plugin as a sub-process (fork()/exec()). See [INITENVVARS]. 2) The name plugin checks its environment, and determines the supported NS API versions using the env variable TOR_NS_PROTO_VERSION. 2.1) If there are no compatible versions, the name plugin writes an INIT message with a failure status code as in [INITPROTOCOL], and then shuts down. 3) The name plugin parses and handles the rest of the environment values. 3.1) If the environment variables are malformed, or otherwise invalid, the name plugin writes an INIT message with a failure status code as in [INITPROTOCOL], and then shuts down. 4) After the name plugin completely initializes, it sends a successful INIT message to stdout as in [INITPROTOCOL]. Then it continues monitoring its stdin for incoming RESOLVE messages. 6) When the name plugin receives a RESOLVE message, it performs the name resolution and replies with the appropriate RESOLVED message. 7) Upon being signaled to terminate by the parent process [NSSHUTDOWN], the name plugin gracefully shuts down. 2.8.1. Name plugin shutdown [NSSHUTDOWN] To ensure clean shutdown of all name plugins when Tor shuts down, the following rules apply for name plugins: Name plugins MUST handle OS specific mechanisms to gracefully terminate (e.g. SIGTERM). Name plugins SHOULD monitor their stdin and exit gracefully when it is closed. 2.9. Further details of stdin/stdout communication 2.9.1. Message Format Tor communicates with its name plugins by writing NL-terminated lines to stdout. The line metaformat is <Line> ::= <Keyword> <OptArgs> <NL> <Keyword> ::= <KeywordChar> | <Keyword> <KeywordChar> <KeyWordChar> ::= <any US-ASCII alphanumeric, dash, and underscore> <OptArgs> ::= <Args>* <Args> ::= <SP> <ArgChar> | <Args> <ArgChar> <ArgChar> ::= <any US-ASCII character but NUL or NL> <SP> ::= <US-ASCII whitespace symbol (32)> <NL> ::= <US-ASCII newline (line feed) character (10)> Tor MUST ignore lines with keywords that it doesn't recognize. 3. Discussion 3.1. Using second-level domains instead of tld People have suggested that users should try to connect to reddit.zkey.onion instead of reddit.zkey. That is, we should always preserve .onion as the tld, and only utilize second-level domains for naming. The argument for this is that this way users cannot accidentally leak addresses to DNS, as the .onion domain is reserved by RFC 7686. The counter-argument here is that this might be confusing to users since they are not used to the second-level domain being special (e.g. co.uk). Also, what happens when someone registers a 16-character name, that matches the length of a vanilla onion address? We should consider the concerns here and take the right decision. 3.2. Name plugins handling all tlds '*' In [TORRC], we assigned a single tld to each name plugin. Should we also accept catch-all tlds using '*'? I'm afraid that this way a name system could try to resolve even normal domains like reddit.com . Perhaps we trust the name plugin itself, but maybe the name system network could exploit this? Also, the catch-all tld will probably cause some engineering complications in this proposal (as it did for PTs). 3.3. Deployment strategy We need to devise a deployment strategy that will allow us to improve the UX of our users as soon as possible, but without taking hasty, sloppy or uneducated decisions. For starters, we should make it easy for developers to write wrappers around their secure name systems. We should develop libraries that speak the NS API protocol and can be used to quickly write wrappers. Similar libraries were quite successful during pluggable transport deployment; see pyptlib and goptlib. In the beginning, name plugins should be third-party applications that can be installed by interested users manually or through package managers. Users will also have to add the appropriate OnionNamePlugin line to their torrc. This will be a testing phase, and also a community-growing phase. After some time, and when we get a better idea of how name plugins work for the community, we can start considering how to make them more easily usable. For example, we can start by including some name plugins into TBB in an optional opt-in fashion. We should be careful here, as people have real incentives for attacking name systems and we should not put our users unwillingly in danger. 3.4. Miscellaneous discussion topics 1. The PT spec tries hard so that a single executable can expose multiple PTs. In this spec, it's clear that each executable is a single name plugin. Is this OK or a bad idea? Should we change interfaces so that each name plugin has an identifier, and then use that identifier for things? 2. Should we make our initialization protocol _identical_ to the PT API initialization protocol? That is, use ENV-ERROR etc. instead of INT? 3. Does it make sense to support reverse queries, from .onion to names? So that people auto-learn the names of the onions they use? 4. Acknowledgements Proposal details discussed during Tor hackfest in Seattle between Yawning, David and me. Thanks to Lunar and indolering for more discussion and feedback. Appendix A.1: Example communication Tor <-> name plugin [PROTOEXAMPLE] Environemnt variables: TOR_NS_STATE_LOCATION=/var/lib/tor/ns_state TOR_NS_PROTO_VERSION=1 TOR_NS_PLUGIN_OPTIONS=timeout=5;cache=/home/user/name_cache Messages between Tor and the banana name plugin: Name plugin (banana) -> Tor: INIT 1 0 Tor -> Name plugin (banana): RESOLVE 1 daewonskate.banana Name plugin (banana) -> Tor: RESOLVED 1 0 jqkscnkne4qt91iq.onion Tor -> Name plugin (banana): RESOLVE 1 architecturedirect.zkey Name plugin (banana) -> Tor (banana): RESOLVE 1 2 "zkey not recognized tld" Tor -> Name plugin (banana): RESOLVE 1 origamihub.banana Name plugin (banana) -> Tor (banana): RESOLVE 1 2 wdxfpaxar4dg12vd.onion Appendix A.2: Example plugins [PLUGINEXAMPLES] Here are a few examples of name plugins for brainstorming: a) Simplest plugin: A local hosts file. Basically a local petname system that maps names to onion addresses. b) A remote hosts file. A centralized community hosts file that people trust. c) Multiple remote hosts files. People can add their own favorite community hosts file. d) Multiple remote hosts files with notaries and reputation trust. Like moxie's convergence tool but for names. e) GNS f) OnioNS g) Namecoin/Blockstart

10 14

Joining Tor Project's software infrastructure
by Jesse V 14 Nov '16

14 Nov '16

Hey everyone, Although it hasn't been obvious, I actually have been making some serious progress on my Onion Name System (OnioNS) project. The paper was accepted into PoPETS, the design is finally stable, and the software is looking really promising, although not yet ready for use. I've focused on improving the maturity of the software and set up the following: * PGP-signed commits using my torproject identity * Continuous integration via Travis CI * Use clang-analyzer and cppcheck static analysis scanners * Automated enforcement of code styling via clang-format * Inclusion of a manpage and command-line help menu * Packaging for Debian (Wheezy+) on my own machine * Automated building for Ubuntu (14.04+) on Launchpad * Protection against memory leaks and buffer overflows * Software is split to minimize installation footprint * Use of CMake, which should help the port to Windows * No RPATH hacks * Support for release and debug builds * Support for Clang and GCC; all warnings enabled for debug * No reliance on OpenSSL (CVE-2016-6309, are you serious?!) * Use of well-maintained libraries for networking and parsing * Minimal dependencies: Botan, JsonCpp, libcurl, libmicrohttpd * Non-native dependencies included as git submodules At this point, I am interested in taking the next step and try to integrate with the Tor Project infrastructure. I am looking for information on the following: 1) How do I upload packages to deb.torproject.org? 2) How do I join Gitian to generate reproducible builds? 3) How do I set up an Onion Name System component in Trac so that I can migrate bug tracking from Github to Trac? 4) How do I set up a repository on gitweb.torproject.org? I'm more comfortable using Github, but there's no need to centralize git. I have a LDAP account, which should help with some of these. I wasn't able to find much useful documentation on these topics, so I would appreciate some help. Also, if anyone has any experience with RPM packaging, I would like to figure that out. I haven't had too much luck with Alien. As I approach a proper release, my overall goal is to improve the maturity, trust, usability of the OnioNS software. Please let me know how I can accomplish the above tasks. Recent commits: https://github.com/Jesse-V/OnioNS-common/tree/json-rpc https://github.com/Jesse-V/OnioNS-server/tree/json-rpc https://github.com/Jesse-V/OnioNS-client/tree/json-rpc I've also updated https://trac.torproject.org/projects/tor/wiki/doc/OnionServiceNamingSystems -- Jesse

3 4

Different trust levels using single client instance
by bancfc＠openmailbox.org 07 Nov '16

07 Nov '16

Summarized question: Do you recommend allowing Workstation VMs of different security levels to communicate with the same Tor instance? Note that they connect via separate internal networks to the Gateway and have different interfaces & controlports so inter-workstation communication should not be possible. Single Tor Gateway, Multiple Workstations Pros: *Same guard node means less chance of picking a malicious one *Single Gateway VM uses less resources Cons: *Some unforeseen way malicious VM "X" can link activities of or influence traffic of VM "Y" **Maybe sending NEWNYM requests in a timed pattern that changes exit IPs of VM Y's traffic, revealing they are behind the same client? **Maybe eavesdropping on HSes running on VM Y's behalf? **Something else we are not aware of? Multi-Tor Gateways mapped 1:1 to Workstation VMs Pros: *Conceptually simple. Uses a different Tor instance so no need to worry about all these questions. Cons: *Uses a different entry guard which can increase chance of running into a malicious relay that can deanonymize some of the traffic. * Uses extra resources (though not much as a Tor Gateway can run with as little as 192MB RAM)

4 6

Revisiting prop224 client authorization
by George Kadianakis 04 Nov '16

04 Nov '16

Hello, we've reached the point in prop224 development where we need to pin down the precise cell formats, so that we can start implementing them. HS client authorization has been one of those areas that are not yet finalized and are still influencing cell format. Here are some topics based on special's old notes, plus some further recent discussion with David and Yawning. a) I think the most important problem here is that the authorization-key logic in the current prop224 is very suboptimal. Specifically, prop224 uses a global authorization-key to ensure that descriptors are only read by authorized clients. However, since that key is global, if we ever want to revoke a single client we need to change the keys for all clients. The current rend-spec.txt does not suffer from this issue, hence I adapted the current technique to prop224. Please see my torspec branch `prop224_client_auth` for the proposed changes: https://gitweb.torproject.org/user/asn/torspec.git/log/?h=prop224_client_au… Some further questions here: i) Should we fake the client-auth-desc-key blob in case client authorization is not enabled? Otherwise, we leak to the HSDir whether client auth is enabled. The drawback here is the desc size increase (by about 330 bytes). Alternatively, we can try to put it in the encrypted part of the descriptor. So that we require subcredential knowledge to access the encrypted part, and then client_auth_cookie knowledge to get the encryption key to decrypt the intro points etc. I feel that this double-encryption design might be too annoying to implement, but perhaps it's worth it? ii) Should we use the descriptor ASCII format to encode all the client-auth-desc-key data? Or is that weird binary format OK? iii) Should we use a fresh IV for each ENCRYPTED_DESC_COOKIE? rend-spec.txt does not do that, and IIUC that's OK because it uses a fresh key for every encryption (even though the plaintext and IV is the same). b) Do we want a NACK from the HS for when client authorization fails? Currently the only way for a client to learn that they are not authorized (or that their keys changed or got revoked) is that they never complete the rendezvous. To achieve that we would need a whole new cell (INTRODUCE_SERVICE_NACK) from the hidden service all the way to the client. Worth it? c) Another suggestion here by special, is to introduce an additional layer of access control at the HSDir request level, such that HSDirs don't even serve descriptors to clients that do not prove knowledge of a pre-shared fetch key. This way unauthorized clients cannot even learn presense information of hidden services. This might be quite useful for applications like Ricochet, who want to hide their presense from revoked clients. Of course this assumes that the HSDir is honest and will honor the fetch key protocol. However, even if the HSDir is dishonest we are just back to the current security level. We started sketching out a solution in the bottom of these notes: https://people.torproject.org/~asn/hs_notes/client_auth.jpg but the solution is not trivial to implement and we are not sure whether it's worth complicating the protocol further (e.g. we need to design a way for apps like Ricochet to get access to the fetch key). d) It might be worthwhile padding the encrypted part of INTRODUCE1 to obscure whether client authorization is in place. As you can see I have mainly worked on point (a) which I consider the most urgent. I welcome feedback on all points, so that we move forward with the design here. Thanks :)

5 14

handling TLS Session Ticket/Identifier for Android
by Hans-Christoph Steiner 31 Oct '16

31 Oct '16

Hey all, Since tor devs have thought about how to handle TLS Session Tickets and Identifiers, I want to capture that approach and stick it into our NetCipher library, which is also used in Orfox. As I understand it, the approach is: * disable TLS Session Tickets entirely * reset TLS Session Identifiers on NEWNYM Any plans to rethink this for TLS v1.3? Any other TLS tracking issues I should be addressing in NetCipher? I'd also appreciate any references on this topic (yes, I know how to find the relevant RFCs ;), like tor trac tickets. My searches have come up with very little. .hc -- PGP fingerprint: EE66 20C7 136B 0D2C 456C 0A4D E9E2 8DEA 00AA 5556 https://pgp.mit.edu/pks/lookup?op=vindex&search=0xE9E28DEA00AA5556

3 3

[Proposal] A simple way to make Tor-Browser-Bundle more portable and secure
by Daniel Simon 31 Oct '16

31 Oct '16

Hello. How it's currently done - The Tor Browser Bundle is dynamically linked against glibc. Security problem - The Tor Browser Bundle has the risk of information about the host system's library ecosystem leaking out onto the network. Portability problem - The Tor Browser Bundle can't be run on systems that don't use glibc, making it unusable due to different syscalls. Solution proposed - Static link the Tor Browser Bundle with musl libc.[1] It is a simple and fast libc implementation that was especially crafted for static linking. This would solve both security and portability issues. What is Tor developers' opinion about this? I personally don't see any drawbacks and would be interested in discussing this further. Sincerely, Daniel [1] https://www.musl-libc.org/

6 8

CollecTor Release 1.1.0
by iwakeh 28 Oct '16

28 Oct '16

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hi there! CollecTor 1.1.0 is available: https://dist.torproject.org/collector/1.1.0/ The release provides many configuration improvements and enables CollecTor to sync descriptors from other instances, i.e. if the sync-run is configured, CollecTor fetches recent descriptors from one or more remote CollecTor instances, verifies these descriptors, and sorts them into the local descriptor store. Synchronization is implemented for relay descriptors (except microdescriptors), sanitized bridge descriptors, and exit lists. To get the most out of the new release it is highly advisable to read INSTALL.md from the release tarball [0]. If you have already a running CollecTor instance, be aware that the configuration is different now and you need to adapt to the new configuration. Please direct comments and questions to the metrics-team mailing list [2]. And, of course, bug reports [3] or feature requests [4] can be filed in trac. Cheers, iwakeh [0] https://dist.torproject.org/collector/1.1.0/ [1] https://collector.torproject.org [2] https://lists.torproject.org/cgi-bin/mailman/listinfo/metrics-team [3] https://trac.torproject.org/projects/tor/newticket?component=Metrics/Collec… [4] https://trac.torproject.org/projects/tor/newticket?component=Metrics/Collec… -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iF4EAREIAAYFAlgTc8oACgkQsANAxTOj/abiQwD/YdmXRYVy7QTGUAEHh6Q8+QJL 3KpzsDGWsx3JoKnGgYEBAIT5T1E6yf+Rqj1c8X3JN1pDiP0bCNH3eHvMU2tGLsyv =JBqs -----END PGP SIGNATURE-----

1 0

Call for help on testing core tor releases
by isabela＠riseup.net 26 Oct '16

26 Oct '16

Hello Tor community! The Core Tor Team would like to improve our release process by getting it more tested so bugs are found earlier, so stable releases can get out faster and without any big bugs. During Tor's Meeting in Seattle a couple of weeks ago, we discussed this topic and decided to organize a QA team (with the help of the community) for core tor. So we decided to make a call for volunteers to help us with testing our 0.2.9 release so we can catch bugs asap. We are planning on freezing our alpha release this Monday, October 17th, the test period is from 17-31 of October. [release calendar: https://trac.torproject.org/projects/tor/wiki/org/teams/NetworkTeam#Releases ] But if you can test it on the week of the 17, please do so because the sooner the better for us to be able to fix the bugs. Also, please understand that giving so many things we have on our plates, we will have to triage and prioritize bugs reported. This means some might be defer to another release. Below you can read more about what we are asking from you ;) and what our goals are. If you think you can help us, please reply to this email letting us know. What we are asking volunteers: * Validate release signature/checksum * Does it build with _no_ Warnings? * Does "make test" pass? * Does "make test-network-all" pass? (you'll need chutney for this - chutney does not work on Windows) * Does it work as a client? As a relay? As a hidden service ? * Does it work if you drop the binary into (eg) torpbrowser? We are hoping to find enough people that we can test the releases on as many OS's as possible. Some OS-specific notes: * on Linux and BSD, Tor requires some dependencies (read INSTALL). * on OS X, you'll need to choose a package manager (HomeBrew, MacPorts, Fink), or install dependencies from source, * either way, you will need to tell configure where the dependencies are located. * on Windows, people sometimes have trouble building Tor. Help us update our build instructions for Windows! Our goals: * We hope to have tor alpha releases tested within a week of it going out * We hope to have people testing it on as many OS's as possible * Reduce the number of bugs found when we launch a release candidate and/or stable release * We want operating systems and configurations other than the ones we use in development to get tested BEFORE we declare the release stable. If you read so far, like the idea but, this is just bad timing for you to join us and help out with the project. No problem! luckily we will have more core tor releases in the future :) so that means there will always be an opportunity to help. We will keep this list updated as we move forward with this project so hopefully you can join us latter! Please let us know if you have any questions about this - and thank you in advance. Cheers, Isabela / on behalf of core tor ps: email from Nick on 0.2.9 freeze dates if you miss it https://lists.torproject.org/pipermail/tor-dev/2016-October/011513.html

8 11

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

tor-dev October 2016