- tor-dev - lists.torproject.org

Re: [tor-dev] Let's identify which measurement-related tools n eedwork when relays switch from RSA identities to ed25519 identities
by tordev123＠Safe-mail.net 08 Sep '15

08 Sep '15

From: Damian Johnson <atagar(a)torproject.org> To: "tor-dev(a)lists.torproject.org" <tor-dev(a)lists.torproject.org> Subject: Re: [tor-dev] Let's identify which measurement-related tools needwork when relays switch from RSA identities to ed25519 identities Date: Tue, 8 Sep 2015 09:16:15 -0700 > On a side note it would be nice to have a spec patch before changing > things this time. +1 The dir spec is still broken and doesn't cover the new descriptor format with stuff like: extra-info-digest 0589E8C4C8A522814EE8D7DC1A3703920B57208C Qychl+FIG6YEbXgGJaG13mMHW9z9Tci5D9DWCXOwmNg dir-spec: "extra-info-digest" digest NL [At most once] "Digest" is a hex-encoded digest (using upper-case characters) of the router's extra-info document, as signed in the router's extra-info (that is, not including the signature). (If this field is absent, the router is not uploading a corresponding extra-info document.) [Versions before 0.2.0.1-alpha don't recognize this]

1 0

Let's identify which measurement-related tools need work when relays switch from RSA identities to ed25519 identities
by Karsten Loesing 08 Sep '15

08 Sep '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi everyone, Sebastian suggested to me in one of our last measurement team 1-1-1 task exchange rounds [0] to think about the following question: Which measurement-related tools need work when relays switch from RSA identities to ed25519 identities? You'll find below what I came up with. Please send any feedback and corrections to this list. I'll incorporate everything and send an updated document to this list next week. Thanks! All the best, Karsten The switch from RSA to ed25519 will happen in multiple steps (need to confirm these!): 0. Before ed25519 identities were used, each relay had a unique RSA identity. 1. Relays can create an optional ed25519 identity, change it, or stop using it at will. 2. Relays can create an ed25519 identity, but are not allowed to change it or stop using it. 3. Relays can stop using their earlier RSA identity. 4. Relays only use their ed25519 identity. (Add a notion of timeframe here. It seems unlikely that 3 or even 4 will happen before 2017 or even 2018, given how long it takes to deploy a new major Tor version in the network. Sebastian thinks that the timeline may not contain step 1.) ## Directory authority tools: Torflow, Guardfraction There are several tools that provide data to directory authorities for voting on relays. These tools include bandwidth authorities and the yet-to-be-finished Guardfraction tool. They will have to handle new ed25519 identities and provide their data in a format that directory authorities will be able to use for voting. ## Network health scanners: Exitmap, Sybilhunter It's unclear how much work remains to update network scanners like Exitmap and Syilbhunter. ## Descriptor parsing libraries: metrics-lib, Stem, Zoossh Descriptor parsing libraries need to support parsing new fields and later need to support that fields containing RSA fingerprints have become optional. Stem has already implemented ed25519 identities, and there's Java code that metrics-lib could re-use in CollecTor's bridge descriptor sanitizer. Zoossh is probably not updated yet. ## Descriptor archiver: CollecTor The part of CollecTor that archives relay descriptors relies heavily on RSA identities and SHA-1 digests. This includes its implementation of the Tor directory protocol and its structure for storing fetched descriptors in tarballs. Another part of CollecTor that needs to be updated is the bridge descriptor sanitizer. That is already done to the extent of sanitizing ed25519-related descriptor lines. The part that is not updated yet is the tarball structure that still uses SHA-1 digests. ## Network health checkers: DocTor, DepicTor Network health checkers like DocTor and DepicTor use relay fingerprints, but they focus more on network consensus meta data provided by the directory authorities. Updating to ed25519 identities shouldn't be hard. ## Statistics portal: Metrics Most of the data-aggregating modules behind the Metrics website use RSA identities to ensure uniqueness of reported statistics. Switching to another fingerprint format or length should be doable, but will cause some effort. ## Relay address database: ExoneraTor The relay address database ExoneraTor only uses the RSA identity to ensure uniqueness of database entries and for display purposes. Supporting new identities should not be that difficult. ## Onionoo server and clients: Onionoo, Atlas, Globe, Compass, OnionTip, Roster The Onionoo server heavily depends on RSA fingerprints of relays and hashed RSA fingerprints of bridges, both internally and in its API. Supporting ed25519 identities in addition to RSA fingerprints would not be hard, but stopping to support RSA fingerprints will require rewriting major parts of Onionoo. Onionoo clients use the RSA fingerprint provided by the Onionoo server to uniquely identify relays and bridges. They will have to adapt to changes in Onionoo that would first support ed25519 identities and later stop supporting RSA fingerprints. This shouldn't be difficult. ## OONI It's unclear how much effort is needed to update OONI to support new ed25519 identities. ## Exit address scanners: TorDNSEL, TorBEL, Check TorDNSEL and its planned successor TorBEL include RSA identities in their output format. It might be sufficient to simply add ed25519 identities there and leave out RSA identities once they are not used anymore. Check will have to be updated to understand these new formats. [0] 1-1-1 task exchange: you get 1 minute to describe a task that would take somebody else roughly 1 hour and that they will do for you within 1 week (review a document, write some analysis code, fix a small bug, etc.; better come prepared to get the most out of this; give 1, take 1) -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJV7uf1AAoJEJD5dJfVqbCrBJUH/A78UIeNy5UIvctO6EjMp6KU XXeB7jxDoA9RmdhNV7v5MzhSCjblQLfG5tyLmeFV/opX+xG0fMvqFdUB9jupXccV HQXLjeSuNF4Tenayn3W49EUF/O75BHcfwe0fHzmV8KEIhPf66KQn6gNBFFnWr4S0 OfvoSWbIb9Wu7hCya4C7J69FVAhwXPH8JpgpSpKLFYSZWQo1aKbgVqBDKNe/lmrV UO04Ws/O04osG9WWX/HcZgSCfSaN68nBMllnu0yCTJY+fEqFruxYhAaZZV6tXIfG E66vZc2K3S6Rk/1nwHEkNeZKCfjyRG1LthzH28HdCEMzl1mdQhUPkJawNaQXoDM= =rXAd -----END PGP SIGNATURE-----

1 0

Re: [tor-dev] Proposal: Single onion services
by tordev123＠Safe-mail.net 05 Sep '15

05 Sep '15

-------- Original Message -------- From: John Brooks <john.brooks(a)dereferenced.net> To: tor-dev(a)lists.torproject.org Subject: Re: [tor-dev] Proposal: Single onion services Date: Fri, 4 Sep 2015 15:31:15 -0600 > tordev123(a)Safe-mail.net wrote: > > > Doesn't your proposal imply that you are turning all relays into > > exit-nodes lite? The last relay in the path will know what service you are > > connecting to (at least if that service is hosted with a unique relay), > > right? > > A single onion service operates its own server(s). These servers accept OR > connections like a relay does, but they aren’t required to be in the > consensus or to relay traffic. They are the servers listed in the > descriptor. Right, that's how I understood things. > A client connects by extending a circuit to the single onion server. This is > not the same as an exit connection: tor relays will extend circuits to > relays they don't know about, as long as the destination speaks the tor > protocol. It’s possible for any tor relay to be used as the last one before > the single onion server. > > If the single onion server isn’t also a tor relay, it’s possible for the > previous relay to guess the service you’re connecting to. That is a large part of my concern. > This isn’t a risk > to client anonymity, because tor clients will always choose the first three > hops in a circuit before extending to one they didn’t choose. I'm not concerned about clients. > The final circuit looks like: > > Client -> Guard -> Middle -> Middle -> Single Onion > > The client’s traffic is encrypted through to the single onion server as > well. IMO, the second Middle relay can be considered serving as an exit with regards to Single Onion services - that's what I meant with 'exit node lite'. There was the case of an Austrian exit node operator getting prosecuted. It will sometimes be possible to attribute traffic relating to specific transactions to the second Middle node in the path (e.g. when the single onion server keeps detailed logs). So the circumstances of that case could apply to a non-exit relay operator as well. Your proposal is shifting non-exit relays towards performing a role that can be considered exit-like, even if that role is much more limited than normal exits (and there is an additional Tor protocol layer involved).

2 1

Proposal: Single onion services
by John Brooks 04 Sep '15

04 Sep '15

Here’s a delayed shipment from the hidden services hackfest: Single onion services are a modified form of onion services, which trade service-side location privacy for improved performance, reliability, and scalability. Single onion services have a .onion address identical to any other onion service. The descriptor contains information sufficient to do a relay extend of a circuit to the onion service and to open a stream for the onion address. The introduction point and rendezvous protocols are bypassed for these services. The full proposal by Paul, Roger, and myself is below, and available from: https://gitweb.torproject.org/user/special/torspec.git/plain/proposals/idea… Thanks especially to Paul, who is behind this whole concept, and all of the other participants of the Arlington Accords. - special ----- Filename: xxx-single-onion.txt Title: Single Onion Services Author: John Brooks, Paul Syverson, Roger Dingledine Created: 2015-07-13 Status: Draft 1. Overview Single onion services are a modified form of onion services, which trade service-side location privacy for improved performance, reliability, and scalability. Single onion services have a .onion address identical to any other onion service. The descriptor contains information sufficient to do a relay extend of a circuit to the onion service and to open a stream for the onion address. The introduction point and rendezvous protocols are bypassed for these services. We also specify behavior for a tor instance to publish a single onion service, which requires a reachable OR port, without necessarily acting as a public relay in the network. 2. Motivation Single onion services have a few benefits over double onion services: * Connection latency is much lower by skipping rendezvous * Stream latency is reduced on a 4-hop circuit * Removing rendezvous circuits improves service scalability * A single onion service can use multiple relays for load balancing Single onion services are not location hidden on the service side, but clients retain all of the benefits and privacy of onion services. More details, relation to double onion services, and the rationale for the 'single' and 'double' nomenclature are further described in section 7.4. We believe these improvements, along with the other benefits of onion services, will be a significant incentive for website and other internet service operators to provide these portals to preserve the privacy of their users. 3. Onion descriptors The onion descriptor format is extended to add: "service-extend-locations" NL encrypted-string [At most once] A list of relay extend info, which is used instead of introduction points and rendezvous for single onion services. This field is encoded and optionally encrypted in the same way as the "introduction-points" field. The encoded contents of this field contains no more than 10 entries, each containing the following data: "service-extend-location" SP link-specifiers NL [At start, exactly once] link-specifiers is a base64 encoded link specifier block, in the format described by proposal 224 [BUILDING-BLOCKS] and the EXTEND2 cell. "onion-key" SP key-type NL onion-key [Exactly once] Describes the onion key that must be used when extending to the single onion service relay. The key-type field is one of: "tap" onion-key is a PEM-encoded RSA relay onion key "ntor" onion-key is a base64-encoded NTOR relay onion key [XXX: Should there be some kind of cookie to prove that we have the desc? See also section 7.1. -special] A descriptor may contain either or both of "introduction-points" and "service-extend-locations"; see section 5.2. [XXX: What kind of backwards compatibility issues exist here? Will existing relays accept one of those descriptors? -special] 4. Reaching a single onion service as a client Single onion services use normal onion hostnames, so the client will first request the service's descriptor. If the descriptor contains a "service-extend-locations" field, the client should ignore the introduction points and rendezvous process in favor of the process defined here. The descriptor's "service-extend-locations" information is sufficient for a client to extend a circuit to the onion service, regardless of whether it is also listed as a relay in the network consensus. This extend info must not be used for any other purpose. If multiple extend locations are specified, the client should randomly select one. The client uses a 3-hop circuit to extend to the service location from the descriptor. Once this circuit is built, the client sends a BEGIN cell to the relay, with the onion address as hostname and the desired TCP port. If the circuit or stream fails, the client should retry using another extend location from the descriptor. If all extend locations fail, and the descriptor contains an "introduction-points" field, the client may fall back to a full rendezvous operation. 5. Publishing a single onion service To act as a single onion service, a tor instance (or cooperating group of tor instances) must: * Have a publicly accessible OR port * Publish onion descriptors in the same manner as any onion service * Include a "service-extend-locations" section in the onion descriptor * Accept RELAY_BEGIN cells for the service as defined in section 5.3 5.1. Configuration options The tor server operating a single onion service must accept connections as a tor relay, but is not required to be published in the consensus or to allow extending circuits. To enable this, we propose the following configuration option: RelayAllowExtend 0|1 If set, allow clients to extend circuits from this relay. Otherwise, refuse all extend cells. PublishServerDescriptor must also be disabled if this option is disabled. If ExitRelay is also disabled, this relay will not pass through any traffic. 5.2. Publishing descriptors A single onion service must publish descriptors in the same manner as any onion service, as defined by rend-spec and section 3 of this proposal. Optionally, a set of introduction points may be included in the descriptor to provide backwards compatibility with clients that don't support single onion services, or to provide a fallback when the extend locations fail. 5.3. RELAY_BEGIN When a RELAY_BEGIN cell is received with a configured single onion hostname as the destination, the stream should be connected to the configured backend server in the same manner as a service-side rendezvous stream. All relays must reject any RELAY_BEGIN cell with an address ending in ".onion" that does not match a locally configured single onion service. 6. Other considerations 6.1. Load balancing High capacity services can distribute load by including multiple entries in the "service-extend-locations" section of the descriptor, or by publishing several descriptors to different onion service directories, or by a combination of these methods. 6.2. Benefits of also running a Tor relay If a single onion service also acts as a published tor relay, it will keep connections to many other tor relays. This can significantly reduce the latency of connections to the single onion service, and also helps the tor network. 6.3. Proposal 224 ("Next-Generation Hidden Services") This proposal is compatible with proposal 224, with small changes to the service descriptor format. In particular: The "service-extend-location" sections are included in the encrypted portion of the descriptor, adjacent to any "introduction-point" sections. The "service-extend-locations" field is no longer present. An onion service is also single onion service if any "service-extend-location" field is present. 6.4. Proposal 246 ("Merging Hidden Service Directories and Intro Points") This proposal is compatible with proposal 246. The onion service will publish its descriptor to the introduction points in the same manner as any other onion service. The client may choose to build a circuit to the specified relays, or to continue with the rendezvous protocol. The client should not extend from the introduction point to the single onion service's relay, to avoid overloading the introduction point. The client may truncate the circuit and extend through a new relay. 7. Discussion 7.1. Authorization Client authorization for a single onion service is possible through encryption of the service-extend-locations section in the descriptor, or "stealth" publication under a new onion address, as with traditional onion services. One problem with this is that if you suspect a relay is also serving a single onion service, you can connect to it and send RELAY_BEGIN without any further authorization. To prevent this, we would need to include a cookie from the descriptor in the RELAY_BEGIN information. 7.2. Preventing relays from being unintentionally published Many single onion servers will not want to relay other traffic, and will set 'PublishServerDescriptor 0' to prevent it. Even when they do, they will still generate a relay descriptor, which could be downloaded and published to a directory authority without the relay's consent. To prevent this, we should insert a field in the relay descriptor when PublishServerDescriptor is disabled that instructs relays to never include it as part of a consensus. [XXX: Also see task #16564] 7.3. Ephemeral single onion services (ADD_ONION) The ADD_ONION control port command could be extended to support ephemerally configured single onion services. We encourage this, but specifying its behavior is out of the scope of this proposal. 7.4. Onion service taxonomy and nomenclature Onion services in general provide several benefits. First, by requiring a connection via Tor they provide the client the protections of Tor and make it much more difficult to inadvertently bypass those protections than when connecting to a non .onion site. Second, because .onion addresses are self-authenticating, onion services have look-up, routing, and authentication protections not provided by sites with standard domain addresses. These benefits apply to all onion services. Onion services as originally introduced also provide network location hiding of the service itself: because the client only ever connects through the end of a Tor circuit created by the onion service, the IP address of the onion service also remains protected. Applications and services already exist that use existing onion service protocols for the above described general benefits without the need for network location hiding. This Proposal is accordingly motivated by a desire to provide the general benefits, without the complexity and overhead of also protecting the location of the service. Further, as with what had originally been called 'location hidden services', there may be useful and valid applications of this design that are not reflected in our current intent. Just as 'location hidden service' is a misleading name for many current onion service applications, we prefer a name that is descriptive of the system but flexible with respect to applications of it. We also prefer a nomenclature that consistently works for the different types of onion services. It is also important to have short, simple names lest usage efficiencies evolve easier names for us. For example, 'hidden service' has replaced the original 'location hidden service' in Tor Proposals and other writings. For these reasons, we have chosen 'onion services' to refer to both those as set out in this Proposal and those with the client-side and server-side protections of the original---also for referring indiscriminately to any and all onion services. We use 'double-onion service' to refer to services that join two Tor circuits, one from the server and one from the client. We use 'single-onion' when referring to services that use only a client-side Tor circuit. In speech we sometimes use the even briefer, 'two-nion' and 'one-ion' respectively.

2 1

tor-browser git repository
by tordev123＠Safe-mail.net 04 Sep '15

04 Sep '15

Is there some documentation, how the tor-browser git repository is set up? In particular, how are new Firefox releases imported? How can I get a diff of Tor changes to the underlying Firefox release?

3 2

Donncha's SoP Status Report 7 - Final
by Donncha O'Cearbhaill 03 Sep '15

03 Sep '15

Hello all, This is my seventh, and final biweekly status report for my hidden service scaling / load-balancing SoP project. Since my last status report: * I have been revising my patch for #14846 with the aim of getting it merged before the 0.2.7 feature freeze. * I've been working with Ivan Markin on getting smart cards supported for signing in OnionBalance. Thank you Ivan for your code and enthusiasm. Although the Summer of Privacy program is now over, I plan to continue maintaining and developing this project into the future. My immediate objectives are to: * Package OnionBalance in Debian * Clarifying the documentation (installation, use cases, storing master hidden service keys on an isolated machine). * Implementing smartcard / HSM support master service key storage and signing. Work on the Complex Mode is advancing but I am not yet satisfied with the reliability of that mode. I will also further investigate any potential attacks or risks with that mode. Proposals #224 (next-gen hidden services) and #246 (merging hidden service directories and introduction points) are still developing. I'll continue developing OnionBalance so that if possible, it can facilitate some form of load balancing and redundancy with next-gen hidden services. I'd like to thank my mentors David Goulet and George Kadianakis for their invaluable guidance and feedback during the Summer of Privacy. Other members of the Tor community, in particular s7r, have been exceptional helpful with testing OnionBalance and offering feedback from a hidden service operator perspective. Finally I'd like to thank all in the wider Tor community for the welcome and support you have shown me over the past months. I hope that I can continue making contributions to the Tor community into the future. Regards, Donncha

2 1

Re: [tor-dev] Should cloud-hosted relays be rejected?
by tordev123＠Safe-mail.net 02 Sep '15

02 Sep '15

From: Sharif Olorin <sio(a)tesser.org> To: tor-dev <tor-dev(a)lists.torproject.org> Subject: Re: [tor-dev] Should cloud-hosted relays be rejected? Date: Wed, 02 Sep 2015 08:32:58 +0000 > > I'm not talking about those providers hosting exit nodes. I'm talking about general web sites and resources hosted by them and the traffic that goes there. > > I'm confused. You're worried about people hosting clearnet websites > with cloud hosting providers? Or non-exit relays? Correlation between all kinds of relay traffic and clearnet traffic. Entry traffic would be the worst, allowing end-to-end correlation.

2 1

Re: [tor-dev] Should cloud-hosted relays be rejected?
by tordev123＠Safe-mail.net 02 Sep '15

02 Sep '15

From: nusenu <nusenu(a)openmailbox.org> To: tor-dev(a)lists.torproject.org Subject: Re: [tor-dev] Should cloud-hosted relays be rejected? Date: Tue, 1 Sep 2015 19:23:35 +0200 > > It has other benefits. Those big providers see a huge amount of exit > > traffic and can potentially do correlation against that. > > I disagree on 'huge'. > If you worry about i.e. Amazon hosting to much exit bandwidth you have > to worry about many other* ASes first, and even then, banning them all > completely (exit prob = 0) isn't probably a wise strategy. I'm not talking about those providers hosting exit nodes. I'm talking about general web sites and resources hosted by them and the traffic that goes there.

2 1

Re: [tor-dev] Should cloud-hosted relays be rejected?
by tordev123＠Safe-mail.net 01 Sep '15

01 Sep '15

From: nusenu <nusenu(a)openmailbox.org> To: tor-dev(a)lists.torproject.org Subject: Re: [tor-dev] Should cloud-hosted relays be rejected? Date: Tue, 1 Sep 2015 00:58:05 +0200 > I don't think banning GCE, AWS and MS Azure is an efficient method to > significantly increase the cost of attacks because it is trivial for an > attacker to quickly spin up "a large number of disposable machines" at > other ISPs as well. It has other benefits. Those big providers see a huge amount of exit traffic and can potentially do correlation against that.

3 2

Should cloud-hosted relays be rejected?
by Philipp Winter 01 Sep '15

01 Sep '15

We sometimes see attacks from relays that are hosted on cloud platforms. I have been wondering if the benefit of having cloud-hosted relays outweighs the abuse we see from them. To get an idea of the benefit, I analysed the bandwidth that is contributed by cloud-hosted relays. I first obtained the network blocks owned by three cloud providers (Amazon AWS, Google Cloud, Microsoft Azure), and determined the percent of bandwidth they contributed in July 2015. The results show that there were typically ~200 cloud-hosted relays online: <https://nymity.ch/sybilhunting/png/cloud-hosted_relays_2015-07.png> The spike shortly after hour 200 was caused by a lot of Amazon relays named "DenkoNet". The spike at the very beginning was caused by a number of relays that might very well belong together, too, based on their uptime pattern. What counts, however, is bandwidth. Here's the total bandwidth fraction contributed by cloud-hosted relays over July 2015: <https://nymity.ch/sybilhunting/png/cloud-hosted_bandwidth_2015-07.png> There were no Google Cloud relays to contribute any bandwidth. Amazon AWS-powered relays contributed the majority of bandwidth, followed by Microsoft Azure-powered relays. Here's a summary of the time series in percent: Min. Mean Median Max. 0.2% 0.8% 0.79% 1.5% In an average consensus in July 2015, cloud-hosted relays contributed only around 0.8% of bandwidth. Note, however, that this is just a lower bound. The netblocks I used for the analysis could have changed, and I didn't consider providers other than Google, Amazon, and Microsoft. There are also cloud-hosted bridges. Tor Cloud, however, has shut down, and the number of EC2 bridges is declining: <https://metrics.torproject.org/cloudbridges.html?graph=cloudbridges&start=2…> The harm caused by cloud-hosted relays is more difficult to quantify. Getting rid of them also wouldn't mean getting rid of any attacks. At best, attackers would have to jump through more hoops. If we were to decide to permanently reject cloud-hosted relays, we would have to obtain the netblocks that are periodically published by all three (and perhaps more) cloud providers: <https://docs.aws.amazon.com/general/latest/gr/aws-ip-ranges.html> <https://msdn.microsoft.com/en-us/library/azure/Dn175718.aspx> <https://cloud.google.com/appengine/kb/general?hl=en#static-ip> Note that this should be done periodically because the netblocks are subject to change. Cheers, Philipp

6 7