January 2016 - tor-dev - lists.torproject.org

What happens to my hidden service?
by Liste 29 Dec '25

29 Dec '25

Hidden service is unavailable, but tor is running. Tor's log: Dec 12 18:10:27.000 [notice] Your Guard SECxFreeBSD64 ($D7DB8E82604F806766FC3F80213CF719A0481D0B) is failing more circuits than usual. Most likely this means the Tor network is overloaded. Success counts are 199/285. Use counts are 101/101. 253 circuits completed, 0 were unusable, 54 collapsed, and 15 timed out. For reference, your timeout cutoff is 60 seconds. is failing a very large amount of circuits. Most likely this means the Tor network is overloaded, but it could also mean an attack against you or potentially the guard itself. Dec 12 18:15:53.000 [warn] Giving up launching first hop of circuit to rendezvous point [scrubbed] for service xxxxxxxxxxxxxxx. Dec 11 13:08:59.000 [notice] We'd like to launch a circuit to handle a connection, but we already have 32 general-purpose client circuits pending. Waiting until some finish. [268 similar message(s) suppressed in last 600 seconds] Dec 11 13:43:12.000 [warn] onion_skin_client_handshake failed. Dec 11 13:43:12.000 [warn] circuit_finish_handshake failed. Dec 11 13:43:12.000 [warn] connection_edge_process_relay_cell (at origin) failed. Dec 11 00:34:57.000 [warn] Giving up launching first hop of circuit to rendezvous point [scrubbed] for service .... Dec 10 10:09:01.000 [warn] rend_service_introduce(): Bug: Internal error: Got an INTRODUCE2 cell on an intro circ (for service "...") with no corresponding rend_intro_point_t. Dec 10 10:09:03.000 [warn] rend_service_introduce(): Bug: Internal error: Got an INTRODUCE2 cell on an intro circ (for service "...") with no corresponding rend_intro_point_t. Dec 10 10:09:04.000 [warn] rend_service_introduce(): Bug: Internal error: Got an INTRODUCE2 cell on an intro circ (for service "...") with no corresponding rend_intro_point_t. Dec 10 12:42:19.000 [warn] rend_service_introduce(): Bug: Internal error: Got an INTRODUCE2 cell on an intro circ (for service "...") with no corresponding rend_intro_point_t.

3 2

Proposal 247 (Hidden Service Vanguards) Overhaul
by Mike Perry 09 Jun '17

09 Jun '17

I spent some time trying to clean up proposal 247 based on everyone's comments, as well as based on my own thoughts. Please have a look if you commented on the original proposal, and complain if I've not taken your thoughts into account. (Aaron: In particular, I made several tradeoffs in favor of performance and DoS resistance that may be at odds with some of your suggestions, but I think the end result is still OK after looking into the Sybil rates and thinking about the adversary model in more detail. You may disagree). I've attached my updated version of the proposal inline in this mail, but the canonical updated proposal is in my remote at: https://gitweb.torproject.org/user/mikeperry/torspec.git/tree/proposals/247… Here's a summary of the changes (which are also listed in Git): * Try to make a coherent threat model and specify its assumptions * Fold in my comments about using disjoint sets ("buckets") for the third level guard. * Make the parameter discussion subsection its own section, and include tables with far more detail for the Sybil success rates. * Put the rotation period in a separate subsection from the number of guards * Switch to using min(X,X) and max(X,X) for the distribution for the second and third layer guard lifespans, respectively. Add a subsection describing this distribution (3.2.3) * Changed the default parameters based on these tables, and based on my own intuition about Tor's performance properties. * Move the load balancing, torrc, and other performance considerations to their own section (Section 5). * Move "3.2. Distinguishing new HS circuits from normal HS circuits" to section 4.1. * Fold in some of "3.3. Circuit nodes can now be linked to specific hidden services" into 4.1. Some of it I just removed, though, because I did not find it credible. * Added Roger's concerns about guard linkability to Section 4.2. * Added a denial of service subsection to Section 4.3. ================================ Filename: 247-hs-guard-discovery.txt Title: Defending Against Guard Discovery Attacks using Vanguards Author: George Kadianakis Created: 2015-07-10 Status: Draft 0. Motivation A guard discovery attack allow attackers to determine the guard node of a Tor client. The hidden service rendezvous protocol provides an attack vector for a guard discovery attack since anyone can force an HS to construct a 3-hop circuit to a relay (#9001). Following the guard discovery attack with a compromise and/or coercion of the guard node can lead to the deanonymization of a hidden service. 1. Overview This document tries to make the above guard discovery + compromise attack harder to launch. It introduces an optional configuration option which makes the hidden service also pin the second and third hops of its circuits for a longer duration. With this new path selection, we force the adversary to perform a Sybil attack and two compromise attacks before succeeding. This is an improvement over the current state where the Sybil attack is trivial to pull off, and only a single compromise attack is required. With this new path selection, an attacker is forced to do a one or more node compromise attacks before learning the guard node of a hidden service. This increases the uncertainty of the attacker, since compromise attacks are costly and potentially detectable, so an attacker will have to think twice before beginning a chain of node compromise attacks that he might not be able to complete. 1.1. Visuals Here is how a hidden service rendezvous circuit currently looks like: -> middle_1 -> middle_A -> middle_2 -> middle_B -> middle_3 -> middle_C -> middle_4 -> middle_D HS -> guard -> middle_5 -> middle_E -> Rendezvous Point -> middle_6 -> middle_F -> middle_7 -> middle_G -> middle_8 -> middle_H -> ... -> ... -> middle_n -> middle_n this proposal pins the two middles nodes to a much more restricted set, as follows: -> guard_3A_A -> guard_2_A -> guard_3A_B -> guard_3A_C -> Rendezvous Point HS -> guard_1 -> guard_3B_D -> guard_2_B -> guard_3B_E -> guard_3B_F -> Rendezvous Point Note that the third level guards are partitioned into buckets such that they are only used with one specific second-level guard. In this way, we ensure that even if an adversary is able to execute a Sybil attack against the third layer, they only get to learn one of the second-layer Guards, and not all of them. This prevents the adversary from gaining the ability to take their pick of the weakest of the second-level guards for further attack. 2. Design This feature requires the HiddenServiceGuardDiscovery torrc option to be enabled. When a hidden service picks its guard nodes, it also picks two additional sets of middle nodes `second_guard_set` and `third_guard_set` of size NUM_SECOND_GUARDS and NUM_THIRD_GUARDS respectively for each hidden service. These sets are unique to each hidden service created by a single Tor client, and must be kept separate and distinct. When a hidden service needs to establish a circuit to an HSDir, introduction point or a rendezvous point, it uses nodes from `second_guard_set` as the second hop of the circuit and nodes from that second hop's corresponding `third_guard_set` as third hops of the circuit. A hidden service rotates nodes from the 'second_guard_set' at a random time between MIN_SECOND_GUARD_LIFETIME hours and MAX_SECOND_GUARD_LIFETIME hours. A hidden service rotates nodes from the 'third_guard_set' at a random time between MIN_THIRD_GUARD_LIFETIME and MAX_THIRD_GUARD_LIFETIME hours. These extra guard nodes should be picked with the same path selection procedure that is used for regular middle nodes (though see Section 5.1 for performance reasons to restrict this slightly). Each node's rotation time is tracked independently, to avoid disclosing the rotation times of the primary and second-level guards. XXX how should proposal 241 ("Resisting guard-turnover attacks") be applied here? 2.1. Security parameters We set NUM_SECOND_GUARDS to 4 nodes and NUM_THIRD_GUARDS to 16 nodes (ie four sets of four). XXX: 3 and 12 might be another option here, in which case our rotation period for the second guard position can be reduced to 15 days. We set MIN_SECOND_GUARD_LIFETIME to 1 day, and MAX_SECOND_GUARD_LIFETIME to 33 days, for an average rotation rate of ~11 days, using the min(X,X) distribution specified in Section 3.2.2. We set MIN_THIRD_GUARD_LIFETIME to 1 hour, and MAX_THIRD_GUARD_LIFETIME to 18 hours, for an average rotation rate of ~12 hours, using the max(X,X) distribution specified in Section 3.2.2. XXX make all the above consensus parameters? Yes. Very yes, especially if we decide to change the primary guard lifespan. See Section 3 for more analysis on these constants. 3. Rationale and Security Parameter Selection 3.1. Threat model, Assumptions, and Goals Consider an adversary with the following powers: - Can launch a Sybil guard discovery attack against any node of a rendezvous circuit. The slower the rotation period of the node, the longer the attack takes. Similarly, the higher the percentage of the network is compromised, the faster the attack runs. - Can compromise any node on the network, but this compromise takes time and potentially even coercive action, and also carries risk of discovery. We also make the following assumptions about the types of attacks: 1. A Sybil attack is noisy. It will require either large amounts of traffic, multiple test circuits, or both. 2. A Sybil attack against the second or first layer Guards will be more noisy than a Sybil attack against the third layer guard, since the second and first layer Sybil attack requires a timing side channel in order to determine success, where as the Sybil success is almost immediately obvious to third layer guard, since it will now be returned as a rend point for circuits for the hidden service in question. 3. As soon as the adversary is confident they have won the Sybil attack, an even more aggressive circuit building attack will allow them to determine the next node very fast (an hour or less). 4. The adversary is strongly disincentivized from compromising nodes that may prove useless, as node compromise is even more risky for the adversary than a Sybil attack in terms of being noticed. Given this threat model, our security parameters were selected so that the first two layers of guards should be hard to attack using a Sybil guard discovery attack and hence require a node compromise attack. Ideally, we want the node compromise attacks to carry a non-negligible probability of being useless to the adversary by the time they complete. On the other hand, the outermost layer of guards should rotate fast enough to _require_ a Sybil attack. 3.2. Parameter Tuning 3.2.1. Sybil rotation counts for a given number of Guards The probability of Sybil success for Guard discovery can be modeled as the probability of choosing 1 or more malicious middle nodes for a sensitive circuit over some period of time. P(At least 1 bad middle) = 1 - P(All Good Middles) = 1 - P(One Good middle)^(num_middles) = 1 - (1 - c/n)^(num_middles) c/n is the adversary compromise percentage In the case of Vanguards, num_middles is the number of Guards you rotate through in a given time period. This is a function of the number of vanguards in that position (v), as well as the number of rotations (r). P(At least one bad middle) = 1 - (1 - c/n)^(v*r) Here's detailed tables in terms of the number of rotations required for a given Sybil success rate for certain number of guards. 1.0% Network Compromise: Sybil Success One Two Three Four Five Six Eight Nine Ten Twelve Sixteen 10% 11 6 4 3 3 2 2 2 2 1 1 15% 17 9 6 5 4 3 3 2 2 2 2 25% 29 15 10 8 6 5 4 4 3 3 2 50% 69 35 23 18 14 12 9 8 7 6 5 60% 92 46 31 23 19 16 12 11 10 8 6 75% 138 69 46 35 28 23 18 16 14 12 9 85% 189 95 63 48 38 32 24 21 19 16 12 90% 230 115 77 58 46 39 29 26 23 20 15 95% 299 150 100 75 60 50 38 34 30 25 19 99% 459 230 153 115 92 77 58 51 46 39 29 5.0% Network Compromise: Sybil Success One Two Three Four Five Six Eight Nine Ten Twelve Sixteen 10% 3 2 1 1 1 1 1 1 1 1 1 15% 4 2 2 1 1 1 1 1 1 1 1 25% 6 3 2 2 2 1 1 1 1 1 1 50% 14 7 5 4 3 3 2 2 2 2 1 60% 18 9 6 5 4 3 3 2 2 2 2 75% 28 14 10 7 6 5 4 4 3 3 2 85% 37 19 13 10 8 7 5 5 4 4 3 90% 45 23 15 12 9 8 6 5 5 4 3 95% 59 30 20 15 12 10 8 7 6 5 4 99% 90 45 30 23 18 15 12 10 9 8 6 10.0% Network Compromise: Sybil Success One Two Three Four Five Six Eight Nine Ten Twelve Sixteen 10% 2 1 1 1 1 1 1 1 1 1 1 15% 2 1 1 1 1 1 1 1 1 1 1 25% 3 2 1 1 1 1 1 1 1 1 1 50% 7 4 3 2 2 2 1 1 1 1 1 60% 9 5 3 3 2 2 2 1 1 1 1 75% 14 7 5 4 3 3 2 2 2 2 1 85% 19 10 7 5 4 4 3 3 2 2 2 90% 22 11 8 6 5 4 3 3 3 2 2 95% 29 15 10 8 6 5 4 4 3 3 2 99% 44 22 15 11 9 8 6 5 5 4 3 The rotation counts in these tables were generated with: def count_rotations(c, v, success): r = 0 while 1-math.pow((1-c), v*r) < success: r += 1 return r 3.2.2. Rotation Period As specified in Section 3.1, the primary driving force for the third layer selection was to ensure that these nodes rotate fast enough that it is not worth trying to compromise them, because it is unlikely for compromise to succeed and yield useful information before the nodes stop being used. For this reason we chose 1 to 18 hours, with a weighted distribution (Section 3.2.3) causing the expected average to be 12 hours. From the table in Section 3.2.1, it can be seen that this means that the Sybil attack will complete with near-certainty (99%) in 29*12 hours (14.5 days) for the 1% adversary, 3 days for the 5% adversary, and 1.5 days for the 10% adversary. Since rotation of each node happens independently, the distribution of when the adversary expects to win this Sybil attack in order to discover the next node up is uniform. This means that on average, the adversary should expect that half of the rotation period of the next node is already over by the time that they win the Sybil. With this fact, we choose our range and distribution for the second layer rotation to be short enough to cause the adversary to risk compromising nodes that are useless, yet long enough to require a Sybil attack to be noticeable in terms of client activity. For this reason, we choose a minimum second-layer guard lifetime of 1 day, since this gives the adversary a minimum expected value of 12 hours for during which they can compromise a guard before it might be rotated. If the total expected rotation rate is 11 days, then the adversary can expect overall to have 5.5 days remaining after completing their Sybil attack before a second-layer guard rotates away. 3.2.3. Rotation distribution In order to skew the distribution of the third layer guard towards higher values, we use max(X,X) for the distribution, where X is a random variable that takes on values from the uniform distribution. In order to skew the distribution of the second layer guard towards low values (to increase the risk of compromising useless nodes) we skew the distribution towards lower values, using min(X,X). Here's a table of expectation (arithmetic means) for relevant ranges of X (sampled from 0..N). The current choice for second-layer guards is noted with **, and the current choice for third-layer guards is noted with ***. Range Min(X,X) Max(X,X) 10 2.85 6.15 11 3.18 6.82 12 3.51 7.49 13 3.85 8.15 14 4.18 8.82 15 4.51 9.49 16 4.84 10.16 17 5.18 10.82*** 18 5.51 11.49 19 5.84 12.16 20 6.18 12.82 21 6.51 13.49 22 6.84 14.16 23 7.17 14.83 24 7.51 15.49 25 7.84 16.16 26 8.17 16.83 27 8.51 17.49 28 8.84 18.16 29 9.17 18.83 30 9.51 19.49 31 9.84 20.16 32 10.17** 20.83 33 10.51 21.49 34 10.84 22.16 35 11.17 22.83 36 11.50 23.50 37 11.84 24.16 38 12.17 24.83 39 12.50 25.50 4. Security concerns and mitigations 4.1. Mitigating fingerprinting of new HS circuits By pinning the middle nodes of rendezvous circuits, we make it easier for all hops of the circuit to detect that they are part of a special hidden service circuit with varying degrees of certainty. The Guard node is able to recognize a Vanguard client with a high degree of certainty because it will observe a client IP creating the overwhelming majority of its circuits to just a few middle nodes in any given 10-18 day time period. The middle nodes will be able to tell with a variable certainty that depends on both its traffic volume and upon the popularity of the service, because they will see a large number of circuits that tend to pick the same Guard and Exit. The final nodes will be able to tell with a similar level certainty that depends on their capacity and the service popularity, because they will see a lot of rend handshakes that all tend to have the same second hop. The most serious of these is the Guard fingerprinting issue. When proposal xxx-padding-negotiation is implemented, services that enable this feature should use those padding primitives to create fake circuits to random middle nodes that are not their guards, in an attempt to look more like a client. Additionally, if Tor Browser implements "virtual circuits" based on SOCKS username+password isolation in order to enforce the re-use of paths when SOCKS username+passwords are re-used, then the number of middle nodes in use during a typical user's browsing session will be proportional to the number of sites they are viewing at any one time. This is likely to be much lower than one new middle node every ten minutes, and for some users, may be close to the number of Vanguards we're considering. This same reasoning is also an argument for increasing the number of second-level guards beyond just two, as it will spread the hidden service's traffic over a wider set of middle nodes, making it both easier to cover, and behave closer to a client using SOCKS virtual circuit isolation. 4.2. Hidden service linkability Multiple hidden services on the same Tor instance should use separate second and third level guard sets, otherwise an adversary is trivially able to determine that the two hidden services are co-located by inspecting their current chosen rend point nodes. Unfortunately, if the adversary is still able to determine that two or more hidden services are run on the same Tor instance through some other means, then they are able to take advantage of this fact to execute a Sybil attack more effectively, since there will now be an extra set of guard nodes for each hidden service in use. For this reason, if Vanguards are enabled, and more than one hidden service is configured, the user should be advised to ensure that they do not accidentally leak that the two hidden services are from the same Tor instance. 4.3. Denial of service Since it will be fairly trivial for the adversary to enumerate the current set of rend nodes for a hidden service, denial of service becomes a serious risk for Vanguard users. For this reason, it is important to support a large number of third-level guards, to increase the amount of resources required to bring a hidden service offline by DoSing just a few Tor nodes. 5. Performance considerations The switch to a restricted set of nodes will very likely cause significant performance issues, especially for high-traffic hidden services. If any of the nodes they select happen to be temporarily overloaded, performance will suffer dramatically until the next rotation period. 5.1. Load Balancing Since the second and third level "guards" are chosen from the set of all nodes eligible for use in the "middle" hop (as per hidden services today), this proposal should not significantly affect the long-term load on various classes of the Tor network, and should not require any changes to either the node weight equations, or the bandwidth authorities. Unfortunately, transient load is another matter, as mentioned previously. It is very likely that this scheme will increase instances of transient overload at nodes selected by high-traffic hidden services. One option to reduce the impact of this transient overload is to restrict the set of middle nodes that we chose from to some percentage of the fastest middle-capable relays in the network. This may have some impact on load balancing, but since the total volume of hidden service traffic is low, it may be unlikely to matter. 5.2. Circuit build timeout The adaptive circuit build timeout mechanism in Tor is what corrects for instances of transient node overload right now. The timeout will naturally tend to select the current fastest and least-loaded paths even through this set of restricted routes, but it may fail to behave correctly if there are a very small set of nodes in each guard set, as it is based upon assumptions about the current path selection algorithm, and it may need to be tuned specifically for Vanguards, especially if the set of possible routes is small. 5.3. OnionBalance At first glance, it seems that this scheme makes multi-homed hidden services such as OnionBalance[1] even more important for high-traffic hidden services. Unfortunately, if it is equally damaging to the user for any of their multi-homed hidden service locations to be discovered, then OnionBalance is strictly equivalent to simply increasing the number of second-level guard nodes in use, because an active adversary can perform simultaneous Sybil attacks against all of the rend points offered by the multi-homed OnionBalance introduction points. 5.4. Default vs optional behavior We suggest this torrc option to be optional because it changes path selection in a way that may seriously impact hidden service performance, especially for high traffic services that happen to pick slow guard nodes. However, by having this setting be disabled by default, we make hidden services who use it stand out a lot. For this reason, we should in fact enable this feature globally, but only after we verify its viability for high-traffic hidden services, and ensure that it is free of second-order load balancing effects. Even after that point, until Single Onion Services are implemented, there will likely still be classes of very high traffic hidden services for whom some degree of location anonymity is desired, but for which performance is much more important than the benefit of Vanguards, so there should always remain a way to turn this option off. 6. Future directions Here are some more ideas for improvements that should be done sooner or later: - Maybe we should make the size and rotation period of secondary/third guard sets to be configurable by the user. - To make it harder for an adversary, a hidden service MAY extend the path length of its circuits by an additional static hop. This forces the adversary to use another coercion attack to walk the chain up to the hidden service. 7. Acknowledgments Thanks to Aaron Johnson, John Brooks, Mike Perry and everyone else who helped with this idea. 1. https://onionbalance.readthedocs.org/en/latest/design.html#overview -- Mike Perry

7 14

onion moshing
by David Stainton 25 Sep '16

25 Sep '16

I was inspired by onioncat to write a twisted python implementation. Onionvpn doesn't have as many features as onioncat. I've successfully tested that onionvpn and onioncat can talk to each other and play nice. Both onionvpn and onioncat implement a virtual public network. Anyone can send packets to you if they know your onion address or ipv6 address... however injection attacks are unlikely since the attacker cannot know the contents of your traffic without compromising the tor process managing the onion service. I've also tested with mosh; that is, you can use mosh which only works with ipv4 over an ipv4-to-ipv6 tunnel over onionvpn/onioncat. Like this: mosh-client -> udp/ipv4 -> ipv6 -> tun device -> tcp-to-tor -> onion service decodes ipv6 to tun device -> ipv6 -> udp/ipv4 -> mosh-server https://github.com/david415/onionvpn If an onionvpn/onioncat operator were to NAT the onion ipv6 traffic to the Internet then that host essentially becomes a special IPv6 exit node for the tor network. The same can be done for IPv4. Obviously operating such an exit node might be risky due to the potential for abuse... however don't you just love the idea of being about to use low-level network scanners over tor? I wonder if Open Observatory of Network Interference would be interested in this. david

4 3

adding smartcard support to Tor
by Razvan Dragomirescu 03 Jun '16

03 Jun '16

Hello, I am not sure if this has been discussed before or how hard it would be to implement, but I'm looking for a way to integrate a smartcard with Tor - essentially, I want to be able to host hidden service keys on the card. I'm trying to bind the hidden service to a hardware component (the smartcard) so that it can be securely hosted in a hostile environment as well as impossible to clone/move without physical access to the smartcard. I have Tor running on the USBArmory by InversePath ( http://inversepath.com/usbarmory.html ) and have a microSD form factor card made by Swissbit ( www.swissbit.com/products/security-products/overwiev/security-products-over… ) up and running on it. I am a JavaCard developer myself and I have developed embedded Linux firmwares before but I have never touched the Tor source. Is there anyone that is willing to take on a side project doing this? Would it be just a matter of configuring OpenSSL to use the card (I haven't tried that yet)? Thank you, Razvan -- Razvan Dragomirescu Chief Technology Officer Cayenne Graphics SRL

8 34

Quantum-safe Hybrid handshake for Tor
by Zhenfei Zhang 29 Apr '16

29 Apr '16

Hi list, This is a proposal to use quantum-safe hybrid handshake for Tor communications. Given NSA's recent announcement on moving towards quantum-safe cryptography, it would be nice to have a quantum-safe feature for Tor. The idea of the quantum-safe hybrid handshake is to combine both classical key exchange and a key encapsulation mechanism (KEM) instantiated by a quantum safe encryption algorithm, so that the combination gives both (classical) authentication and quantum safety. In a bit more details, the client and the server agrees on a classic pre-master secret, $c$, using the ntor protocol. In parallel, client generates a public/private key pair of the quantum-safe encryption algorithm, and send the public key to the server. The server picks a random string, $q$, encrypts it with the public key and send the ciphertext back to the client. The final secret is the output of KDF(c|q). This proposal defeats the harvest-then-decrypt attack with a minimum impact to the existing ntor protocol. An adversary needs to be able to break the quantum-safe encryption algorithm to learn q. On the other hand, if the quantum-safe encryption algorithm turns out to be not secure, the protocol is still as secure as ntor protocol. In other words, it will at least do no harm to the current security. In addition, this is a modular design that allows us to use any quantum-safe cryptographic primitives. As a proof of concept, we instantiated the protocol with NTRUEncrypt lattice-based crypto. We implemented the the protocol with NTRU parameters that gives 128 bits security. The code is available at https://github.com/NTRUOpenSourceProject/ntru-tor Please find the attachment for the request to change the feature. The proof of the protocol can be found at: https://eprint.iacr.org/2015/287.pdf Some known issue: 1. cell size. As far as we know, all quantum-safe encryption algorithms have large key and/or ciphertext size that exceeds the cell size ~500. So this protocol needs to transmit multiple cells, no matter which quantum-safe encryption algorithm we chose. This is addressed by "Proposal 249: Allow CREATE cells with >505 bytes of handshake data". 2. quantum-safe authentication: there is no quantum-safe authentication in this protocol. We believe that authentication can wait, as future (quantum) adversary cannot come back to present time and break authentication. Hence, we use ntor authentication to keep the proposal compact and simple. It will be a future work after this proposal. Thanks for your time, and happy holidays! Zhenfei Zhang Security Innovation.

9 26

Re: [tor-dev] Should we disable the collection of some stats published in extra-infos?
by Karsten Loesing 28 Mar '16

28 Mar '16

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 15/01/16 23:00, Rob Jansen wrote: > Hello, Hi Rob, I'm moving this discussion from metrics-team@ to tor-dev@, because I think it's relevant for little-t-tor devs who are not subscribed to metrics-team@. Hope you don't mind. > I was recently reviewing the statistics that Tor allows relays to > collect and report to the dir servers [1], which then get published > in extra-info documents [2]. Most of this can be enabled by simply > setting a torrc option. There are quite a few statistics that I > feel should not be collected. I'm wondering if the original purpose > for collecting many of these statistics still exists, and if we > still feel that the privacy compromises that were made when the > collection was implemented are still valid in most cases. > > Here are the stats I am most worried about, and why: > > [unique ips per country code] *-ips (there are many of these, e.g. > "entry-ips") Usually this involves storing individual user IP > addresses in memory (in order to track uniqueness) over some period > of time (usually 24 hours), sometimes for longer than the user > would have otherwise been known to Tor (if a user's session is 1 > hour, Tor could remember the IP for at most 23 additional hours). > This is reported, e.g., per entry; there are many cases in the data > where it is very likely that only one user is connecting to a guard > from a given country (because it is rounded up to 8). Users in > small countries have the greatest risk (intersection attacks become > really easy). I agree that might just lose these statistics. We used them in the past as first approximation to counting users, but obviously that only works as long as clients only connect to a single relay. The only place where we're still using them is in a workaround for estimating bridge users. See #15469 for more details and #8786 for something we'd have to implement before taking these statistics out. > [exit statistics by port number] exit-kibibytes-written > exit-kibibytes-read exit-streams-opened Tor is classifying its > traffic into ports, which could uniquely identify the application > being used by the client. They also track bandwidth usage per port > (and per exit); again, this is bad for those using a random or > unique looking ports (that a given exit does not see very often) > because it could be used to create a fingerprint. Intersection > attacks become easier with this information. Agreed, I can see us dropping these statistics, too. We're currently not using them. But also see my suggestion below. > The less problematic stats: > > [circuit-based cell statistics] cell-processed-cells > cell-queued-cells cell-time-in-queue cell-circuits-per-decile This > provides queue timings and number of cells being processed at a > relay. The number of cells can be used to compute bandwidth of > circuits. It may be possible to launch some attacks that create > several circuits with the intent of moving which decile buckets > some legitimate circuits get placed into, but this is less > worrisome of an attack than the others. I'm less worried about this one. But, suggestion below. > Should Tor still be collecting these things? Should Tor disable the > collection of these statistics until we have a more > privacy-preserving way to collect and aggregate them? > > The good news is that privacy-preserving techniques exist that can > reduce information leakage. I'm developing a tool based on the > secret-sharing variant of PrivEx [3] to collect some of these types > of statistics while providing privacy guarantees. We are currently > using it to collect only those stats that are useful for producing > Tor traffic models. A great advantage of this tool is that the > various counters that we store during the collection phase get > noise added and are randomized during initialization; only the > aggregates are ever known and revealed by the aggregation server, > limiting the information that is lost if a relay is compromised. > This is a large improvement over the current collection method, > which only adds noise before publication and reveals statistics on > a per-relay basis. Suggestion: How about we evaluate these statistics published by relays in the past years to see if there are other benefits or risks we didn't think of, and then we decide whether to leave them in, modify them, or take them out? The reason is that I'd want to avoid removing this code only to realize shortly after that we overlooked a good reason for keeping it. These statistics are being collected for years now, and it might take another year or so for relays to upgrade to stop collecting them. So what's another month. Thanks for (re-)starting this discussion! > All the best, Rob All the best, Karsten > [1] https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt [2] > https://collector.torproject.org/recent/relay-descriptors/extra-infos/ > > [3] www.cypherpunks.ca/~iang/pubs/privex-ccs14.pdf > > > > _______________________________________________ metrics-team > mailing list metrics-team(a)lists.torproject.org > https://lists.torproject.org/cgi-bin/mailman/listinfo/metrics-team > -----BEGIN PGP SIGNATURE----- Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJWnffFAAoJEJD5dJfVqbCrSREH/1G3nZDwC7a8FGFEySgQX3MZ n1dlMcylDU6kypgGMEYs2KmMRwhNYmCVFn6fRJgrFN/KgJ0oZJG1pRcglnIFwNwH 5VUWDZp3+frhY5jBbJ3JXqA4Exhi/0mUYCTRwfcFQ/JOyzjlcFbLRW/rkqFHn62H wLidaVeagiJ+TI/T8zgwtqQjTDSZrZPpmlxvlO57D1bGhW1ZPJVbOeUzuudhNpRS WVu//MH51juCzcML32MeiMV6wWYYOm1irKZ8lZHVCPbsL98qoh3ewtO6fb62THIa sWcQZlwMQdcC0+509PmMeYwkRn+40MUwkk84/glyP57dfpgtOwzZCohL3yQRQ1s= =F6F+ -----END PGP SIGNATURE-----

2 4

Much-revised draft, RFC: removing current obsolete clients from the network
by Nick Mathewson 26 Mar '16

26 Mar '16

[This is a significantly revised version of the last version of this proposal draft, sent here for comment. The last version of this draft was https://lists.torproject.org/pipermail/tor-dev/2015-September/009587.html . This is more or less a complete rewrite, after I convinced myself that the original ideas wouldn't work. Part of the functionality of that old draft has been subsumed in proposal 264.] 1. Introduction Frequently, we find that very old versions of Tor should no longer be supported on the network. To remove relays is easy enough: we simply update the directory authorities to stop listing relays that advertise versions that are too old. But to disable clients is harder. In another proposal I describe a system for letting future clients go gracefully obsolete. This proposal explains how we can safely disable the obsolete clients we have today (and all other client versions of Tor to date, assuming that they will someday become obsolete). 1.1. Why disable clients? * Security. Anybody who hasn't updated their Tor client in 5 years is probably vulnerable to who-knows-what attacks. They aren't likely to get much anonymity either. * Withstand zombie installations. Some Tors out there were once configured to start-on-boot systems that are now unmaintained. (See 1.4 below.) They put needless load on the network, and help nobody. * Be able to remove backward-compatibility code. Currently, Tor supports some truly ancient protocols in order to avoid breaking ancient versions or Tor. This code needs to be maintained and tested. Some of it depends on undocumented or deprecated or non-portable OpenSSL features, and makes it hard to produce a conforming Tor server implementation. * Make it easier to write a conforming Tor relay. If a Tor relay needs to support every Tor client back through the beginning of time, that makes it harder to develop and test compatible implementations. 1.2. Is this dangerous? I don't think so. This proposal describes a way to make older clients gracefully disconnect from the network only when a majority of authorities agree that they should. A majority of authorities already have the ability to inflict arbitrary degrees of sabotage on the consensus document. 1.3. History The earliest versions of Tor checked the recommended-versions field in the directory to see whether they should keep running. If they saw that their version wasn't recommended, they'd shut down. There was an "IgnoreVersion" option that let you keep running anyway. Later, around 2004, the rule changed to "shut down if the version is _obsolete_", where obsolete was defined as "not recommended, and older than a version that is recommended." In 0.1.1.7-alpha, we made obsolete versions only produce a warning, and removed IgnoreVersion. (See 3ac34ae3293ceb0f2b8c49.) We have still disabled old tor versions. With Tor 0.2.0.5-alpha, we disabled Tor versions before 0.1.1.6-alpha by having the v1 authorities begin publishing empty directories only. In version 0.2.5.2-alpha, we completely removed support for the v2 directory protocol used before Tor 0.2.0; there are no longer any v2 authorities on the network. Tor versions before 0.2.1 will currently not progress past fetching an initial directory, because they believe in a number of directory authority identity keys that no longer sign the directory. Tor versions before 0.2.4 are (lightly) throttled in multihop circuit creation, because we prioritize ntor CREATE cells over TAP ones when under load. 1.4. The big problem: slow zombies and fast zombies It would be easy enough to 'disable' old clients by simply removing server support for the obsolete protocols that they use. But there's a problem with that approach: what will the clients do when they fail to make connections, or to extend circuits, or whatever else they are no longer able to do? * Ideally, I'd like such clients to stop functioning _quietly_. If they stop contacting the network, that would be best. * Next best would be if these clients contacted the network only occasionally and at different times. I'll call these clients "slow zombies". * Worse would be if the clients contact the network frequently, over and over. I'll call these clients "fast zombies". They would be at their worst when they focus on authorities, or when they act in synchrony to all strike at once. One goal of this proposal is to ensure that future clients to not become zombies at all; and that ancient clients become slow zombies at worst. 2. Some ideas that don't work. 2.1. Dropping connections based on link protocols. Tor versions before before 0.2.3.6-alpha use a renegotiation-based handshake instead of our current handshake. We could detect these handshakes and close the connection at the relay side if the client attempts to renegotiate. I've tested these changes on versions maint-0.2.0 through maint-0.2.2. They result in zombies with the following behavior: The client contact each authority it knows about, attempting to make a one-hop directory connection. It fails, detects a failure, then reconnects more and more slowly ... but one hour later, it resets its connection schedule and starts again. In the steady state this appears to result in about two connections per client per authority per hour. That is probably too many. (Most authorities would be affected: of the authorities that existed in 0.2.2, gabelmoo has moved and turtles has shut down. The authorities Faravahar and longclaw are new. The authorities moria1, tor26, dizum, dannenberg, urras, maatuska and maatuska would all get hit here.) (We could simply remove the renegotiation-detection code entirely, and reply to all connections with an immediate VERSIONS cell. The behavior would probably be the same, though.) If we throttled connections rather than closing them, we'd only get one connnection per authority per hour, but authorities would have to keep open a potentially huge number of sockets. 2.2. Blocking circuit creation under certain circumstances In tor 0.2.5.1-alpha, we began ignoring the UseNTorHandshake option, and always preferring the ntor handshake where available. Unfortunately, we can't simply drop all TAP handshakes, since clients and relays can still use them in the hidden service protocol. But we could detect these versions by: Looking for use of a TAP handshake from an IP not associated with with any known relay, or on a connection where the client did not authenticate. (This could be from a bridge, but clients don't build circuits that go to an IntroPoint or RendPoint directly after a bridge.) This would still result in clients not having directories, however, and retrying once an hours. 3. Ideas that might work 3.1. Move all authorities to new ports We could have each authority known to older clients start listening for connections at a new port P. We'd forward the old port to the new port. Once sufficiently many clients were using the new ports, we could disable the forwarding. This would result in the old clients turning into zombies as above, but they would only be scrabbling at nonexistent ports, causing less load on the authorities. [This proposal would probably be easiest to implement.] 3.2. Start disabling old link protocols on relays We could have new relays start dropping support for the old link protocols, while maintaining support on the authorities and older relays. The result here would be a degradation of older client performance over time. They'd still behave zombieishly if the authorities dropped support, however. 3.3. Changing the consensus format. We could allow 'f' (short for "flag") as a synonym for 's' in consensus documents. Later, if we want to disable all Tor versions before today, we can change the consensus algorithm so that the consensus (or perhaps only the microdesc consensus) is spelled with 'f' lines instead of 'f' lines. This will create a consensus which older clients and relays parse as having all nodes down, which will make them not connect to the network at all. We could similarly replace "r" with "n", or replace Running with Online, or so on. In doing this, we could also rename fresh-until and valid-until, so that new clients would have the real expiration date, and old clients would see "this consensus never expires". This would prevent them from downloading new consensuses. [This proposal would result in the quietest shutdown.]

7 8

Introducing Snowflake (webrtc pt)
by Serene 18 Mar '16

18 Mar '16

Hi all, Snowflake is a webrtc pluggable transport inspired by flashproxy. (https://gitweb.torproject.org/pluggable-transports/snowflake.git) Arlo, David, and I have made lots of progress on it lately, and it now appears to have reached minimum viability. The following should result in a 100% bootstrap over WebRTC: ``` git clone https://git.torproject.org/pluggable-transports/snowflake.git cd snowflake/client go get go build tor -f torrc ``` In case you have a moment, it'd be cool if you tried it out and let us know whether it works for you. I'd really appreciate hearing about all the thoughts / concerns / issues before we move forward. -- More info -- Q: What are the benefits of this PT compared with other PTs? Snowflake combines the advantages of flashproxy and meek. Primarily: - It has the convenience of Meek, but can support magnitudes more users with negligible CDN costs. (Domain fronting only used for brief signalling / NAT-piercing to setup P2P webrtc which handles the actual traffic) - Arbitrarily high numbers of volunteer proxies are possible like in flashproxy, but NATs are no longer a usability barrier - no need for manual port forwarding! Q: How does it work? 1. Volunteers visit websites which host the "snowflake" proxy. (just like flashproxy) 2. Tor clients automatically find available browser proxies over the domain fronted signaling channel. 3. Tor client and browser proxy establish a WebRTC peer connection. 4. Proxy connects to some relay. 5. Tor occurs. Q: Why is it called Snowflake? There's a bunch of "ICE" negotiation happening for WebRTC, and it also involves a great abundance of ephemeral and short-lived (and special!) volunteer proxies... Anyhow, if Snowflake seems like it would be useful / desired here, it would be awesome if we had more help getting it stable, polished, audited, deployable, etc... Plenty of work to do! <3, ~serene P.S. Make sure you're using a recent version of Go (1.5+) P.P.S. The repo is available in these locations: - https://gitweb.torproject.org/pluggable-transports/snowflake.git - https://github.com/keroserene/snowflake

8 15

Relay Database: Existing Schemas?
by nusenu 27 Feb '16

27 Feb '16

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hi, I'm planing to store relay data in a database for analysis. I assume others have done so as well, so before going ahead and designing a db schema I'd like to make sure I didn't miss pre-existing db schemas one could build on. Data to be stored: - - (most) descriptor fields - - everything that onionoo provides in a details record (geoip, asn, rdns, tordnsel, cw, ...) - - historic records I didn't find something matching so far, so I'll go ahead, but if you know of other existing relay db schemas I'd like to hear about it. thanks, nusenu "GSoC2013: Searchable Tor descriptor archive" (Kostas Jakeliunas) https://www.google-melange.com/gsoc/project/details/google/gsoc2013/wfn/ 5866452879933440 https://lists.torproject.org/pipermail/tor-dev/2013-May/004923.html https://lists.torproject.org/pipermail/tor-dev/2013-September/005357.htm l https://github.com/wfn/torsearch (btw, someone knows the license of this?) > This is true: the summary/details documents (just like in Onionoo > proper) deal with the *last* known info about relays. ernie https://gitweb.torproject.org/metrics-db.git/plain/doc/manual.pdf (didn't find db/tordir.sql mentioned in the pdf) "Instructions for setting up relay descriptor database" https://lists.torproject.org/pipermail/tor-dev/2010-March/001783.html "Set up descriptor database for other researchers" https://trac.torproject.org/projects/tor/ticket/1643 -----BEGIN PGP SIGNATURE----- iQIcBAEBCgAGBQJVLrmVAAoJEFv7XvVCELh0+ycP/0VkRWMGAhx7YwWhixtOPB0q ouitosse3SdLCMd1W7cps7nnQhcGv8E1xeUPYeizNQ+RIK9ldvnwgxhVt9ETOI3e oL6bcx3B/NSbTcGfZhIU6XCCX7+39CXz3/+/gXg4c1a5Cd9rYKuwTn7ZApPUNAK2 9yfckCwHaqh67ZDHB1A7nrHX1BJAjwhri5JRGomHSUXPwNZOyMzpgPkB+I92esqc Z4ajQo2D+Vyk42qF7apqp5ApVBLdt5ycUtW1j4s4KvyJH2BjeP4tCXvvYLKv9IvN TGHMJ111NQXbuk6H1OdaDmMZhhW6utPk8YAbNc7RZHqCdH7No40zFVlikLDrV2Xl uLt8FMlck7hGQv/4udA59Dw3PGHfrSiCbvJXb0S2jarnWOH2O+zSSuMs8jQM/x7k 6tIOlwtRDVGw3VuvNbb4MJnLzdHRxq3qy6ueDYuhmhHslxjD1Cbr3eE1RA7Da9aX kNDgjfXtdah52HWUZbIIxHQYzyCg5G3M5yy51GhUlA2Fe1sbPpNcEBeyEDK/jH3Y +7G+tgR+CdMvSFraCiYKHa1drVPJHeqNvZAqPNHKxpKwOwOoJEYrsW4wb/mJ7ybe o8xNYK691hcaMvHqwI47tw4hZfrI0f0+gQKKzENQQk8/8yUZ/0jgMRlW/Cz0yYIa FZEg+OJ/yEs8vKfYerIN =Rfn4 -----END PGP SIGNATURE-----

3 2

Two major improvements to the Tor Metrics back-end
by Karsten Loesing 15 Feb '16

15 Feb '16

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello list, as some of you may have noticed, Tor Metrics had some trouble updating its aggregate data lately, and the issues there are by no means completely gone now. But I managed to tweak two of the aggregations that previously took many hours. Here's a graph: https://people.torproject.org/~karsten/volatile/metrics-modules-2016-01-07.… The green dots at the end are from the latest update. Changes affecting the 70-run-hidserv-stats.sh module: https://gitweb.torproject.org/metrics-web.git/commit/?id=c0f4c76 https://gitweb.torproject.org/metrics-web.git/commit/?id=6054ddd Changes affecting the 80-run-clients-stats.sh module: https://gitweb.torproject.org/metrics-web.git/commit/?id=44acfe5 Boom! All the best, Karsten -----BEGIN PGP SIGNATURE----- Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJWjprFAAoJEJD5dJfVqbCrPT0IALX6dNrL3PK2gbbwAGmm4OBg 4kuz7qFMshTNFp8iT6HB5H9G9/wldgAPanZn+zLlFM0sOBoY1d16RoL2+ay+VKXQ xTeY85tgqMAnbaS2lQcBRZy5j8RXPui94g/ZcKnw9Bg3TpwLFO2+cMXqh1nhMLtb V+Lj0EE+y7qjdtOQO/IbOQzeUr/S9xtze44dRkGlZ0NmeKQsHXaR3m8w8ReIP32k vZekymZkUgbfzYeRoB+jI+3M3l0CF5agwIelhhHlnWwkJ70jPPVrAGX3rhb8X9SX iSNMUJTG62AC3WLqdRkhyY2YJ5OHSMLtpvQwOaOzqSs1yHWylHDVSLTZx+XCQmM= =tvfK -----END PGP SIGNATURE-----

3 4