Cloudflare Onions Beta and Network Stability
As some of you may have heard, Cloudflare is beta testing opportunistic onions. This of course is going to create more Tor traffic. Cloudflare has several concerns about running their own relays and says they won’t at this time. That said if every Cloudflare website becomes an Onion Service overnight how would that affect network stability and what can we as relay operators do to prepare for it?
Is there any kind of information about what kind of bandwidth / connections they are expecting to route? Having a sense of scale in a ratio of current numbers I think would let everyone plan for what they're currently seeing multiply by X as a baseline. On 2018-08-20 11:23 AM, Nathaniel Suchy wrote:
As some of you may have heard, Cloudflare is beta testing opportunistic onions. This of course is going to create more Tor traffic. Cloudflare has several concerns about running their own relays and says they won’t at this time. That said if every Cloudflare website becomes an Onion Service overnight how would that affect network stability and what can we as relay operators do to prepare for it?
_______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
In the name of improving security, let's pretend Cloudflare is acting, without a shadow of doubt, in full malice. They want to get every Cloudflare website onion service as fast as possible. That would create a huge amount of circuits and I worry about whether the network can handle it. What information do we have on this? Right now Cloudflare could, in theory, I am not saying that they will, flip a switch and could cause quite the strain on the Tor Network. On Mon, Aug 20, 2018 at 4:44 PM, Robert Keizer <robert@keizer.ca> wrote:
Is there any kind of information about what kind of bandwidth / connections they are expecting to route?
Having a sense of scale in a ratio of current numbers I think would let everyone plan for what they're currently seeing multiply by X as a baseline.
On 2018-08-20 11:23 AM, Nathaniel Suchy wrote:
As some of you may have heard, Cloudflare is beta testing opportunistic onions. This of course is going to create more Tor traffic. Cloudflare has several concerns about running their own relays and says they won’t at this time. That said if every Cloudflare website becomes an Onion Service overnight how would that affect network stability and what can we as relay operators do to prepare for it?
_______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
_______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Cloudflare had a post yesterday[1] on their blog[2] that said they have about 10,000,000 domain names using their service. So that’s a rough number of the maximum number of websites that will be made available over Tor. Now in reality I expect all their large customers to opt-out, unless it’s an opt-in, in which case I don’t expect large websites, only medium to small to join, let’s say <100,000. That said, depending on whether or not this is an opt-in, or opt-out, the number of websites can be from “hundreds of thousands” to “few millions”. In terms of traffic, I’d estimate Cloudflare to be in the “hundreds of Gb/s” to “a few Tb/s”, but their PeeringDB entry[3] doesn’t really say. In general they keep these numbers quite secret (never post chart Y axis, or post relative values). However, they have some info on their IX port capacities, which I believe is their largest bandwidth usage, but not all of it (also rely on private connections / transit): 5,350 Gb/s. This is almost 5.5 Tb/s, and it takes 2-3 minutes to add up all their ports manually from PeeringDB. Of course, this is their *capacity*, not their current bandwidth. All their ports are 10 Gb/s+, so even if they had 100 Mb/s they would go with 10G. Adding up their ports that are 20G+, we get 2,930 Gb/s, which is ~ 3 Tb/s. Even if we only count their 100G ports, we get 2.6 Tb/s. Let’s assume now that Cloudflare has a daily peak of 2.6 Tb/s, so they run at 50% capacity during peak hours. Now let’s also assume that an 80-20 rule is in place. That said, 80% of their traffic is served for the top 20% of their customers. Now let’s make the worst case assumption that this service is opt-in, and their bottom 80% joins, while the top 20% does not.. That means that during peak time, this traffic is 520 Gb/s. So if Cloudflare has a peak of 2.6 Tb/s, and only 20% of its bandwidth-users participate, then that’s 520 Gb/s peak. On top of that, it’s also ~8M hidden services. Now I know that some of these numbers may not seem exactly right, and some are worst case scenarios, but here’s what it means for the Tor network. Unlike Cloudflare, Tor actually does publish their numbers[4]! Currently Tor has ~100,000 Onion Addresses[5]. If Cloudflare adds 8M more, then that’s a pretty huge increase, putting significant load on the directory servers. The current Tor Onion Service traffic is 1.25 Gb/s[6]. Adding 520 Gb/s to it is significant as well. The current Relay Bandwidth is currently ~270 Gb/s[7]. If we divide this by 3, we get ~ 90 Gb/s. So again, 520 Gb/s, is significant. HOWEVER, in terms of traffic, the above assumes that 100% of the visitors will be coming from Tor. This is far from the truth. Now let’s make a really generous estimation and say that 1% of the traffic to these websites (bandwidth-wise) will be from Tor (this is likely orders of magnitude smaller). This means that the total Onion Service traffic will jump from 1.25 Gb/s to ~ 6.5 Gb/s, a significant increase. As far as the total relay bandwidth, the 90 Gb/s that are currently available will be enough, as current utilization is at ~43 Gb/s, so this will become ~75 Gb/s (actual traffic * 6). So as you can see, in terms of traffic, Cloudflare won’t have a huge impact, since the current Tor network is more than 2 times above peak usage. HOWEVER, Cloudflare doesn’t need to hide their location. Everyone knows their servers. So they can use single hop Onion Services, and not the traditional three hop ones. That means that in terms of total traffic, they will use 43 + (3*5) = ~ 60 Gb/s, out of the ~ 90 Gb/s available. As you can see, these are all estimations, and actual traffic will greatly vary. Excuse any mistakes since I’m writing this from my phone, without putting much thought into it. I also didn’t check any beta announcements they may have to see how they will implement this, I just imagined the most straight forward way to do so. From the numbers above I see the main concert is the amount of Onion Services, and not traffic, so to answer your question I guess it will be to work on that front, and not so much on the traffic side of things. In any case, I really hope that Cloudflare moves slowly with this, and in small batches, as it could cause trouble, especially after the 2-3M Onion Service mark (which is untested).. Antonis 1: https://blog.cloudflare.com/african-traffic-growth-and-predictions-for-the-f... 2: https://blog.cloudflare.com/ 3: https://www.peeringdb.com/net/4224 4: https://metrics.torproject.org/ 5: https://metrics.torproject.org/hidserv-dir-onions-seen.html 6: https://metrics.torproject.org/hidserv-rend-relayed-cells.html 7: https://metrics.torproject.org/bandwidth.html
On 20 Aug 2018, at 23:44, Robert Keizer <robert@keizer.ca> wrote:
Is there any kind of information about what kind of bandwidth / connections they are expecting to route?
Having a sense of scale in a ratio of current numbers I think would let everyone plan for what they're currently seeing multiply by X as a baseline.
On 2018-08-20 11:23 AM, Nathaniel Suchy wrote: As some of you may have heard, Cloudflare is beta testing opportunistic onions. This of course is going to create more Tor traffic. Cloudflare has several concerns about running their own relays and says they won’t at this time. That said if every Cloudflare website becomes an Onion Service overnight how would that affect network stability and what can we as relay operators do to prepare for it?
_______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
_______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Hi everyone, tl;dr: stop panicking. This appears to be a per-site opt-in thing. Death by overload is not going to happen, however exciting it might be to predict the apocalypse. Have a cup of tea. Chill. - alec
On 2018-08-20 15:39, DaKnOb wrote:
HOWEVER, Cloudflare doesn’t need to hide their location. Everyone knows their servers. So they can use single hop Onion Services, and not the traditional three hop ones.
That means that in terms of total traffic, they will use 43 + (3*5) = ~ 60 Gb/s, out of the ~ 90 Gb/s available.
One thought that comes to mind: Doesn't this same traffic already likely flow through tor? Right now browsing my personal blog from TBB uses 'x' number of bytes which pass through relays and an exit, won't the primary difference be that we no longer require an exit by routing directly to a Cloudflare hosted single hop onion service? This attention might bring more users to using Tor which is a separate and valid concern in terms of anticipating growth. I'm still waiting for Cloudflare to activate me on this beta program to be able to explore how well it actually works.
Yes, correct. In general in my calculations I took into account worst-case scenarios to see the worst-case result. Realistically I would believe this to be an opt-in, so way way way fewer websites have it, and not see a significant bandwidth increase from Tor to Cloudflare, because of the above. People who used TBB, will keep using TBB. No more people will probably come from this. In general, the only thing I would worry about would be the HSDir size increase, NOT the bandwidth or circuits, or similar.. Also, I don’t think Cloudflare spent so much time in engineering, just to take down Tor.. If they really wanted to do this, they could have done it in “cheaper” ways.. They spent the time so this can work, so they already (probably) took into account the load that will be placed on the network, and determined it to be bearable.. (I hope so :P) Antonis
On 21 Aug 2018, at 08:56, Dave Warren <dw@thedave.ca> wrote:
On 2018-08-20 15:39, DaKnOb wrote: HOWEVER, Cloudflare doesn’t need to hide their location. Everyone knows their servers. So they can use single hop Onion Services, and not the traditional three hop ones. That means that in terms of total traffic, they will use 43 + (3*5) = ~ 60 Gb/s, out of the ~ 90 Gb/s available.
One thought that comes to mind: Doesn't this same traffic already likely flow through tor?
Right now browsing my personal blog from TBB uses 'x' number of bytes which pass through relays and an exit, won't the primary difference be that we no longer require an exit by routing directly to a Cloudflare hosted single hop onion service?
This attention might bring more users to using Tor which is a separate and valid concern in terms of anticipating growth.
I'm still waiting for Cloudflare to activate me on this beta program to be able to explore how well it actually works. _______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
On Tue, 21 Aug 2018 at 07:09, DaKnOb <daknob@daknob.net> wrote:
Also, I don’t think Cloudflare spent so much time in engineering, just to take down Tor..
I've known people at Cloudflare talking about doing something like this for 2+ years, and the goal has been to do something nice for Tor and to make it easier for normal people to adopt Onion networking. I think that it'd be a great idea for more companies to adopt Onion networking, as outlined in my video at: https://www.youtube.com/watch?v=Rd7YKamsliI That goal is not helped by people jumping into the conversation with implausible scenarios that are not even grounded in reasonable common sense. And: if there are arbitrary limits which could be practically exploited to "Br1ng Tor D0wn!!!1!ONE" then I am pretty sure that they will be exploited by the state security apparatuses of the world, well-before actual legitimate usage will reach them. - alec -- http://dropsafe.crypticide.com/aboutalecm
On 21 Aug 2018, at 07:39, DaKnOb <daknob@daknob.net> wrote:
Cloudflare had a post yesterday[1] on their blog[2] that said they have about 10,000,000 domain names using their service. So that’s a rough number of the maximum number of websites that will be made available over Tor. Now in reality I expect all their large customers to opt-out, unless it’s an opt-in, in which case I don’t expect large websites, only medium to small to join, let’s say <100,000. That said, depending on whether or not this is an opt-in, or opt-out, the number of websites can be from “hundreds of thousands” to “few millions”.
Your assumption that Cloudflare will create one onion address per domain is incorrect. They are quite capable of routing traffic from millions of domains through a few onions. They already have multiple domains per TLS certificate, and terminate and route those TLS connections at their edge. My understanding is that having a dedicated onion for your Cloudflare domain will be an opt-in feature, once it becomes available. T
Nathaniel Suchy:
As some of you may have heard, Cloudflare is beta testing opportunistic onions. This of course is going to create more Tor traffic. Cloudflare has several concerns about running their own relays and says they won’t at this time. That said if every Cloudflare website becomes an Onion Service overnight how would that affect network stability and what can we as relay operators do to prepare for it?
thread https://twitter.com/grittygrease/status/1028020391178989568 (unfortunately we never saw the slides if anyone did, please share) They use alt-svc but Tor Browser does not support that currently. I assume opportunistic onions only work for anything that is already on tor. If I understood Nick correctly they also use single onion services, so that means that their traffic portion is not increasing, just shifting away from exits to non-exits (good). -- https://twitter.com/nusenu_ https://mastodon.social/@nusenu
One point that's been completely missed in the hyperbolic fear-mongering so far: Even if Cloudflare onionified a bazillion domain names, there are still only a few million people who use Tor who could generate the load to connect to them. As such: nothing is gonna explode. -a
Except perhaps the directory authorities? On Mon, Aug 20, 2018 at 7:19 PM, Alec Muffett <alec.muffett@gmail.com> wrote:
One point that's been completely missed in the hyperbolic fear-mongering so far:
Even if Cloudflare onionified a bazillion domain names, there are still only a few million people who use Tor who could generate the load to connect to them.
As such: nothing is gonna explode.
-a
_______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
The dirauths don't serve onion service descriptors. https://gitweb.torproject.org/torspec.git/tree/rend-spec-v2.txt (and -v3) Matt On 08/20/2018 07:38 PM, Nathaniel Suchy wrote:
Except perhaps the directory authorities?
On Mon, Aug 20, 2018 at 7:19 PM, Alec Muffett <alec.muffett@gmail.com <mailto:alec.muffett@gmail.com>> wrote:
One point that's been completely missed in the hyperbolic fear-mongering so far:
Even if Cloudflare onionified a bazillion domain names, there are still only a few million people who use Tor who could generate the load to connect to them.
As such: nothing is gonna explode.
-a
On Mon, Aug 20, 2018 at 7:19 PM, Alec Muffett <alec.muffett@gmail.com> wrote:
Even if Cloudflare onionified a bazillion domain names, there are still only a few million people who use Tor who could generate the load to connect to them.
And none of those who could, will, because cloudflare will google recaptcha slave and mine them all to hell as usual, thus would be users will simply leave to patronize more reasonably sane sites [that may use any number of modern approaches to behaviour management discussed elsewhere].
As such: nothing is gonna explode.
Imagine if Cloudflare adds CAPTCHAs to Onion services. Now that’d be something 😂 On Tue, Aug 21, 2018 at 1:18 PM grarpamp <grarpamp@gmail.com> wrote:
On Mon, Aug 20, 2018 at 7:19 PM, Alec Muffett <alec.muffett@gmail.com> wrote:
Even if Cloudflare onionified a bazillion domain names, there are still only a few million people who use Tor who could generate the load to connect to them.
And none of those who could, will, because cloudflare will google recaptcha slave and mine them all to hell as usual, thus would be users will simply leave to patronize more reasonably sane sites [that may use any number of modern approaches to behaviour management discussed elsewhere].
As such: nothing is gonna explode.
tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
nusenu:
Nathaniel Suchy:
As some of you may have heard, Cloudflare is beta testing opportunistic onions. This of course is going to create more Tor traffic. Cloudflare has several concerns about running their own relays and says they won’t at this time. That said if every Cloudflare website becomes an Onion Service overnight how would that affect network stability and what can we as relay operators do to prepare for it?
thread https://twitter.com/grittygrease/status/1028020391178989568
(unfortunately we never saw the slides if anyone did, please share)
They use alt-svc but Tor Browser does not support that currently.
It does in the latest alpha and will in Tor Browser 8. Georg
On Tue, 21 Aug 2018 06:53:00 +0000 Georg Koppen <gk@torproject.org> wrote:
nusenu:
Nathaniel Suchy:
As some of you may have heard, Cloudflare is beta testing opportunistic onions. This of course is going to create more Tor traffic. Cloudflare has several concerns about running their own relays and says they won’t at this time. That said if every Cloudflare website becomes an Onion Service overnight how would that affect network stability and what can we as relay operators do to prepare for it?
thread https://twitter.com/grittygrease/status/1028020391178989568
(unfortunately we never saw the slides if anyone did, please share)
They use alt-svc but Tor Browser does not support that currently.
It does in the latest alpha and will in Tor Browser 8.
That's interesting to know. Is there any document describing how to deploy that on a web server providing the same site on clearnet and on hidden service? For instance, what the protocol name should look like in case of a HTTP 1.1 onion service? RFC[1] only mentions "h2" in its examples. [1] https://tools.ietf.org/html/draft-ietf-httpbis-alt-svc-12 -- With respect, Roman
nusenu:
(unfortunately we never saw the slides if anyone did, please share)
slides: https://perfectoid.space/ -- https://twitter.com/nusenu_ https://mastodon.social/@nusenu
participants (11)
-
Alec Muffett -
DaKnOb -
Dave Warren -
Georg Koppen -
grarpamp -
Matt Traudt -
Nathaniel Suchy -
nusenu -
Robert Keizer -
Roman Mamedov -
teor