- tor-dev - lists.torproject.org

Proposal: Single onion services
by John Brooks 04 Sep '15

04 Sep '15

Here’s a delayed shipment from the hidden services hackfest: Single onion services are a modified form of onion services, which trade service-side location privacy for improved performance, reliability, and scalability. Single onion services have a .onion address identical to any other onion service. The descriptor contains information sufficient to do a relay extend of a circuit to the onion service and to open a stream for the onion address. The introduction point and rendezvous protocols are bypassed for these services. The full proposal by Paul, Roger, and myself is below, and available from: https://gitweb.torproject.org/user/special/torspec.git/plain/proposals/idea… Thanks especially to Paul, who is behind this whole concept, and all of the other participants of the Arlington Accords. - special ----- Filename: xxx-single-onion.txt Title: Single Onion Services Author: John Brooks, Paul Syverson, Roger Dingledine Created: 2015-07-13 Status: Draft 1. Overview Single onion services are a modified form of onion services, which trade service-side location privacy for improved performance, reliability, and scalability. Single onion services have a .onion address identical to any other onion service. The descriptor contains information sufficient to do a relay extend of a circuit to the onion service and to open a stream for the onion address. The introduction point and rendezvous protocols are bypassed for these services. We also specify behavior for a tor instance to publish a single onion service, which requires a reachable OR port, without necessarily acting as a public relay in the network. 2. Motivation Single onion services have a few benefits over double onion services: * Connection latency is much lower by skipping rendezvous * Stream latency is reduced on a 4-hop circuit * Removing rendezvous circuits improves service scalability * A single onion service can use multiple relays for load balancing Single onion services are not location hidden on the service side, but clients retain all of the benefits and privacy of onion services. More details, relation to double onion services, and the rationale for the 'single' and 'double' nomenclature are further described in section 7.4. We believe these improvements, along with the other benefits of onion services, will be a significant incentive for website and other internet service operators to provide these portals to preserve the privacy of their users. 3. Onion descriptors The onion descriptor format is extended to add: "service-extend-locations" NL encrypted-string [At most once] A list of relay extend info, which is used instead of introduction points and rendezvous for single onion services. This field is encoded and optionally encrypted in the same way as the "introduction-points" field. The encoded contents of this field contains no more than 10 entries, each containing the following data: "service-extend-location" SP link-specifiers NL [At start, exactly once] link-specifiers is a base64 encoded link specifier block, in the format described by proposal 224 [BUILDING-BLOCKS] and the EXTEND2 cell. "onion-key" SP key-type NL onion-key [Exactly once] Describes the onion key that must be used when extending to the single onion service relay. The key-type field is one of: "tap" onion-key is a PEM-encoded RSA relay onion key "ntor" onion-key is a base64-encoded NTOR relay onion key [XXX: Should there be some kind of cookie to prove that we have the desc? See also section 7.1. -special] A descriptor may contain either or both of "introduction-points" and "service-extend-locations"; see section 5.2. [XXX: What kind of backwards compatibility issues exist here? Will existing relays accept one of those descriptors? -special] 4. Reaching a single onion service as a client Single onion services use normal onion hostnames, so the client will first request the service's descriptor. If the descriptor contains a "service-extend-locations" field, the client should ignore the introduction points and rendezvous process in favor of the process defined here. The descriptor's "service-extend-locations" information is sufficient for a client to extend a circuit to the onion service, regardless of whether it is also listed as a relay in the network consensus. This extend info must not be used for any other purpose. If multiple extend locations are specified, the client should randomly select one. The client uses a 3-hop circuit to extend to the service location from the descriptor. Once this circuit is built, the client sends a BEGIN cell to the relay, with the onion address as hostname and the desired TCP port. If the circuit or stream fails, the client should retry using another extend location from the descriptor. If all extend locations fail, and the descriptor contains an "introduction-points" field, the client may fall back to a full rendezvous operation. 5. Publishing a single onion service To act as a single onion service, a tor instance (or cooperating group of tor instances) must: * Have a publicly accessible OR port * Publish onion descriptors in the same manner as any onion service * Include a "service-extend-locations" section in the onion descriptor * Accept RELAY_BEGIN cells for the service as defined in section 5.3 5.1. Configuration options The tor server operating a single onion service must accept connections as a tor relay, but is not required to be published in the consensus or to allow extending circuits. To enable this, we propose the following configuration option: RelayAllowExtend 0|1 If set, allow clients to extend circuits from this relay. Otherwise, refuse all extend cells. PublishServerDescriptor must also be disabled if this option is disabled. If ExitRelay is also disabled, this relay will not pass through any traffic. 5.2. Publishing descriptors A single onion service must publish descriptors in the same manner as any onion service, as defined by rend-spec and section 3 of this proposal. Optionally, a set of introduction points may be included in the descriptor to provide backwards compatibility with clients that don't support single onion services, or to provide a fallback when the extend locations fail. 5.3. RELAY_BEGIN When a RELAY_BEGIN cell is received with a configured single onion hostname as the destination, the stream should be connected to the configured backend server in the same manner as a service-side rendezvous stream. All relays must reject any RELAY_BEGIN cell with an address ending in ".onion" that does not match a locally configured single onion service. 6. Other considerations 6.1. Load balancing High capacity services can distribute load by including multiple entries in the "service-extend-locations" section of the descriptor, or by publishing several descriptors to different onion service directories, or by a combination of these methods. 6.2. Benefits of also running a Tor relay If a single onion service also acts as a published tor relay, it will keep connections to many other tor relays. This can significantly reduce the latency of connections to the single onion service, and also helps the tor network. 6.3. Proposal 224 ("Next-Generation Hidden Services") This proposal is compatible with proposal 224, with small changes to the service descriptor format. In particular: The "service-extend-location" sections are included in the encrypted portion of the descriptor, adjacent to any "introduction-point" sections. The "service-extend-locations" field is no longer present. An onion service is also single onion service if any "service-extend-location" field is present. 6.4. Proposal 246 ("Merging Hidden Service Directories and Intro Points") This proposal is compatible with proposal 246. The onion service will publish its descriptor to the introduction points in the same manner as any other onion service. The client may choose to build a circuit to the specified relays, or to continue with the rendezvous protocol. The client should not extend from the introduction point to the single onion service's relay, to avoid overloading the introduction point. The client may truncate the circuit and extend through a new relay. 7. Discussion 7.1. Authorization Client authorization for a single onion service is possible through encryption of the service-extend-locations section in the descriptor, or "stealth" publication under a new onion address, as with traditional onion services. One problem with this is that if you suspect a relay is also serving a single onion service, you can connect to it and send RELAY_BEGIN without any further authorization. To prevent this, we would need to include a cookie from the descriptor in the RELAY_BEGIN information. 7.2. Preventing relays from being unintentionally published Many single onion servers will not want to relay other traffic, and will set 'PublishServerDescriptor 0' to prevent it. Even when they do, they will still generate a relay descriptor, which could be downloaded and published to a directory authority without the relay's consent. To prevent this, we should insert a field in the relay descriptor when PublishServerDescriptor is disabled that instructs relays to never include it as part of a consensus. [XXX: Also see task #16564] 7.3. Ephemeral single onion services (ADD_ONION) The ADD_ONION control port command could be extended to support ephemerally configured single onion services. We encourage this, but specifying its behavior is out of the scope of this proposal. 7.4. Onion service taxonomy and nomenclature Onion services in general provide several benefits. First, by requiring a connection via Tor they provide the client the protections of Tor and make it much more difficult to inadvertently bypass those protections than when connecting to a non .onion site. Second, because .onion addresses are self-authenticating, onion services have look-up, routing, and authentication protections not provided by sites with standard domain addresses. These benefits apply to all onion services. Onion services as originally introduced also provide network location hiding of the service itself: because the client only ever connects through the end of a Tor circuit created by the onion service, the IP address of the onion service also remains protected. Applications and services already exist that use existing onion service protocols for the above described general benefits without the need for network location hiding. This Proposal is accordingly motivated by a desire to provide the general benefits, without the complexity and overhead of also protecting the location of the service. Further, as with what had originally been called 'location hidden services', there may be useful and valid applications of this design that are not reflected in our current intent. Just as 'location hidden service' is a misleading name for many current onion service applications, we prefer a name that is descriptive of the system but flexible with respect to applications of it. We also prefer a nomenclature that consistently works for the different types of onion services. It is also important to have short, simple names lest usage efficiencies evolve easier names for us. For example, 'hidden service' has replaced the original 'location hidden service' in Tor Proposals and other writings. For these reasons, we have chosen 'onion services' to refer to both those as set out in this Proposal and those with the client-side and server-side protections of the original---also for referring indiscriminately to any and all onion services. We use 'double-onion service' to refer to services that join two Tor circuits, one from the server and one from the client. We use 'single-onion' when referring to services that use only a client-side Tor circuit. In speech we sometimes use the even briefer, 'two-nion' and 'one-ion' respectively.

2 1

tor-browser git repository
by tordev123＠Safe-mail.net 04 Sep '15

04 Sep '15

Is there some documentation, how the tor-browser git repository is set up? In particular, how are new Firefox releases imported? How can I get a diff of Tor changes to the underlying Firefox release?

3 2

Donncha's SoP Status Report 7 - Final
by Donncha O'Cearbhaill 03 Sep '15

03 Sep '15

Hello all, This is my seventh, and final biweekly status report for my hidden service scaling / load-balancing SoP project. Since my last status report: * I have been revising my patch for #14846 with the aim of getting it merged before the 0.2.7 feature freeze. * I've been working with Ivan Markin on getting smart cards supported for signing in OnionBalance. Thank you Ivan for your code and enthusiasm. Although the Summer of Privacy program is now over, I plan to continue maintaining and developing this project into the future. My immediate objectives are to: * Package OnionBalance in Debian * Clarifying the documentation (installation, use cases, storing master hidden service keys on an isolated machine). * Implementing smartcard / HSM support master service key storage and signing. Work on the Complex Mode is advancing but I am not yet satisfied with the reliability of that mode. I will also further investigate any potential attacks or risks with that mode. Proposals #224 (next-gen hidden services) and #246 (merging hidden service directories and introduction points) are still developing. I'll continue developing OnionBalance so that if possible, it can facilitate some form of load balancing and redundancy with next-gen hidden services. I'd like to thank my mentors David Goulet and George Kadianakis for their invaluable guidance and feedback during the Summer of Privacy. Other members of the Tor community, in particular s7r, have been exceptional helpful with testing OnionBalance and offering feedback from a hidden service operator perspective. Finally I'd like to thank all in the wider Tor community for the welcome and support you have shown me over the past months. I hope that I can continue making contributions to the Tor community into the future. Regards, Donncha

2 1

Re: [tor-dev] Should cloud-hosted relays be rejected?
by tordev123＠Safe-mail.net 02 Sep '15

02 Sep '15

From: Sharif Olorin <sio(a)tesser.org> To: tor-dev <tor-dev(a)lists.torproject.org> Subject: Re: [tor-dev] Should cloud-hosted relays be rejected? Date: Wed, 02 Sep 2015 08:32:58 +0000 > > I'm not talking about those providers hosting exit nodes. I'm talking about general web sites and resources hosted by them and the traffic that goes there. > > I'm confused. You're worried about people hosting clearnet websites > with cloud hosting providers? Or non-exit relays? Correlation between all kinds of relay traffic and clearnet traffic. Entry traffic would be the worst, allowing end-to-end correlation.

2 1

Re: [tor-dev] Should cloud-hosted relays be rejected?
by tordev123＠Safe-mail.net 02 Sep '15

02 Sep '15

From: nusenu <nusenu(a)openmailbox.org> To: tor-dev(a)lists.torproject.org Subject: Re: [tor-dev] Should cloud-hosted relays be rejected? Date: Tue, 1 Sep 2015 19:23:35 +0200 > > It has other benefits. Those big providers see a huge amount of exit > > traffic and can potentially do correlation against that. > > I disagree on 'huge'. > If you worry about i.e. Amazon hosting to much exit bandwidth you have > to worry about many other* ASes first, and even then, banning them all > completely (exit prob = 0) isn't probably a wise strategy. I'm not talking about those providers hosting exit nodes. I'm talking about general web sites and resources hosted by them and the traffic that goes there.

2 1

Re: [tor-dev] Should cloud-hosted relays be rejected?
by tordev123＠Safe-mail.net 01 Sep '15

01 Sep '15

From: nusenu <nusenu(a)openmailbox.org> To: tor-dev(a)lists.torproject.org Subject: Re: [tor-dev] Should cloud-hosted relays be rejected? Date: Tue, 1 Sep 2015 00:58:05 +0200 > I don't think banning GCE, AWS and MS Azure is an efficient method to > significantly increase the cost of attacks because it is trivial for an > attacker to quickly spin up "a large number of disposable machines" at > other ISPs as well. It has other benefits. Those big providers see a huge amount of exit traffic and can potentially do correlation against that.

3 2

Should cloud-hosted relays be rejected?
by Philipp Winter 01 Sep '15

01 Sep '15

We sometimes see attacks from relays that are hosted on cloud platforms. I have been wondering if the benefit of having cloud-hosted relays outweighs the abuse we see from them. To get an idea of the benefit, I analysed the bandwidth that is contributed by cloud-hosted relays. I first obtained the network blocks owned by three cloud providers (Amazon AWS, Google Cloud, Microsoft Azure), and determined the percent of bandwidth they contributed in July 2015. The results show that there were typically ~200 cloud-hosted relays online: <https://nymity.ch/sybilhunting/png/cloud-hosted_relays_2015-07.png> The spike shortly after hour 200 was caused by a lot of Amazon relays named "DenkoNet". The spike at the very beginning was caused by a number of relays that might very well belong together, too, based on their uptime pattern. What counts, however, is bandwidth. Here's the total bandwidth fraction contributed by cloud-hosted relays over July 2015: <https://nymity.ch/sybilhunting/png/cloud-hosted_bandwidth_2015-07.png> There were no Google Cloud relays to contribute any bandwidth. Amazon AWS-powered relays contributed the majority of bandwidth, followed by Microsoft Azure-powered relays. Here's a summary of the time series in percent: Min. Mean Median Max. 0.2% 0.8% 0.79% 1.5% In an average consensus in July 2015, cloud-hosted relays contributed only around 0.8% of bandwidth. Note, however, that this is just a lower bound. The netblocks I used for the analysis could have changed, and I didn't consider providers other than Google, Amazon, and Microsoft. There are also cloud-hosted bridges. Tor Cloud, however, has shut down, and the number of EC2 bridges is declining: <https://metrics.torproject.org/cloudbridges.html?graph=cloudbridges&start=2…> The harm caused by cloud-hosted relays is more difficult to quantify. Getting rid of them also wouldn't mean getting rid of any attacks. At best, attackers would have to jump through more hoops. If we were to decide to permanently reject cloud-hosted relays, we would have to obtain the netblocks that are periodically published by all three (and perhaps more) cloud providers: <https://docs.aws.amazon.com/general/latest/gr/aws-ip-ranges.html> <https://msdn.microsoft.com/en-us/library/azure/Dn175718.aspx> <https://cloud.google.com/appengine/kb/general?hl=en#static-ip> Note that this should be done periodically because the netblocks are subject to change. Cheers, Philipp

6 7

Where are the GetTor downloads (e.g. GitHub)?
by David Fifield 01 Sep '15

01 Sep '15

Someone IRL asked me about mirrored downloads of Tor Browser on services like GitHub. I know that such services are planned (or already implemented?) for GetTor, and I know about these tickets: Integrate cloud services that are not blocked in mainland China https://bugs.torproject.org/14114 Automate upload of latest Tor Browser to cloud services https://bugs.torproject.org/14744 Script to upload Tor Browser to Github https://bugs.torproject.org/14835 My question is, is there a static URL on GitHub or similar that has the latest downloads? That is, one that people can access even without having used GetTor? Such a URL would be more useful than a typical mirror for many users. I was referred to Lantern's GitHub download page for comparison: https://github.com/getlantern/lantern/releases/tag/latest It has direct links to downloads under githubusercontent.com: https://raw.githubusercontent.com/getlantern/lantern-binaries/master/lanter… https://raw.githubusercontent.com/getlantern/lantern-binaries/master/lanter… https://raw.githubusercontent.com/getlantern/lantern-binaries/master/lanter…

2 2

New Onionoo version 2.6 adds new "alleged_family" and "indirect_family" fields
by Karsten Loesing 31 Aug '15

31 Aug '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello Onionoo users, I just merged and deployed Onionoo protocol version 2.6 which adds two new fields related to relay families and marks the current "family" field as deprecated. The idea of adding three fields for relay families is that we want to support different use cases: - Atlas and Globe want to list all relays that a relay has configured as its family ("effective_family" + "alleged_family"). - Atlas and Globe may further want to mark ineffective family relationships in red ("alleged_family") and effective family relationships in blue ("effective_family"). - Roster needs a family definition that includes family members that are either in a direct family relationship with a relay ("effective_family") or that can be reached by following effective family relationships ("indirect_family"). The current "family" field will stay available until Atlas and Globe are updated. If I should also wait for other clients to be updated, please let me know. We shouldn't wait too long, because all those fingerprints increase the size of Onionoo results for everyone. All the best, Karsten -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJV5JJAAAoJEJD5dJfVqbCrlw0IAJ3NxbTE00M72EN/DLcJw2FD /KMV2vpyQqi6tharTNGtZmZrm7HMZ/RTRhR5qlkmcKE18UDKxF0sX5ezT6+1R0QV dNmUhYspP5cKcF6nqReMdd26dp4am6I5PSWqEmsMiC5vmxZbvp+B3st/9MNfYExE qTQV8rkzsWIGTGSojxXLAjzjQwy9EBXdLrMrchH+jL0igbWmWLeshDSou/N0te1g B6TBnA2LlRLKhLoAtwjjF1TREZN6Tkk9djQ8zHRjppZzu004I8bqPWiJ36qLDEqi UVmuo5zrJUQxQEQ/n+AkRBoVNblVDY7/xeGh35ENhCzs+Ls75jCQzrkzx+7upPw= =yTnY -----END PGP SIGNATURE-----

2 2

[Measurement Team] Next IRC meeting happens on Wednesday, Sep 2, 14:00 UTC in #tor-project
by Karsten Loesing 31 Aug '15

31 Aug '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello list, This is an announcement/reminder that there will be an IRC meeting of the Measurement Team on Wednesday, September 2, 2015, 14:00 UTC in #tor-project https://www.timeanddate.com/worldclock/fixedtime.html?iso=20150902T14 Talk to you on Wednesday! All the best, Karsten -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJV5JhmAAoJEJD5dJfVqbCr69YH/3bpRZnJTfeavkfKKQDog8oq J6uqlFyEKpSl4S2uSc6jT/gwDfUfUuEwWMpXrOUrRzrFSCSQQcMQF8Hiu6Mk5+SD JN398wHGp2gTYp+y6xb6z8TzZfNmEEnJTcSAZtctQrB6kaj3jT29abMqIlsjQW9i F91uCJLJVJWrBhYvqvvFhUbdDhn3S8HYCEx0u6B8Gdd8q1p/Swi5IuSvkpStBHZJ diFGUsrj8HE73RKwOSl1gJcdLIGti3fayDhJZVVuTdV5weS4QF/I96XPJxnPwNqR ek2azn2RteYSMX3vhglkp86z7jKFM9V4sgDXU72S8jp0wWaXXYmbgpY4+v3G7EY= =9+9n -----END PGP SIGNATURE-----

1 0