Question regarding exit sizing
Hello, I have a question regarding relay sizing to add additional nodes to the network. What scenario is better for the network - adding five 100mbps nodes, or one 500 mbps node? Let’s keep it easy and say all five of those 100 mbps nodes would be in the same datacenter, configured in the same configuration, etc. I’m just curious, because I”m getting ready to add a few more nodes, but I’m wondering if it’ll be better to go big, or just stay small. Thanks, Conrad Rockenhaus
What scenario is better for the network - adding five 100mbps nodes, or one 500 mbps node?
Are we talking bare metal or VPS? A VPS will probably bottleneck on RAM or CPU before hitting 500 Mpbs. Bare metal would stand a chance with the right hardware and tuning, but I wouldn't assume you'll hit 500 Mbps on any given node. Due to the nature of the bandwidth measurements, physical location matters too. You're at the mercy of Tor's bandwidth authorities and in my experience, the further away from Europe, the worse your measurements will be, and so again you may not hit 500 Mbps. Basically, you shouldn't assume that whatever bandwidth you plan for and advertise will come your way. I think you'd have better luck with 5x 100 Mbps nodes, or maybe 3x 200 Mbps nodes. You can also run 2 relays per IP. There are advantages to spreading out the load (like redundancy). I also think Tor's bandwidth measurements and consensus weights are fickle, and some of the variables are out of your control (what else is going on in your rack, datacenter, upstream, etc.). You could use ansible-relayor to turn up a bunch of nodes, wait to see which ones are the most performant, and then keep the best ones. That's what I would do. :)
The 500 Mbps instance would either be one of my private servers in my co-lo, or a dedicated server in one of my private cloud hosting locations. With both contacts, bandwidth costs aren’t an issue, but if one big instance would work I would put it on the same hardware that I am running a server that averages about 700+ mbps consistently. The small servers that I’m hitting 100 mbps on, I’m just getting low cost VPSes for since….they do the job and they do the job well, they’re guaranteed a minimum of 100mbps bandwidth to the first tier 1 hop and Atlas shows them consistently used at that level, so I’m happy. I mainly wanted to give back big to the community because Tor gave me the idea for my latest free for personal/charge for business use idea that I’m going to roll out soon (I’ll gladly send y’all a link, as I think it’s something that would be very useful). I haven’t noticed any bad measurements…the three relays I run now, well, one just started this week so we can throw that one out for now, but the other two are showing 12.55 MiB/s and 12.28 MiB/s, and I’m guaranteed 100 mbps, so I’m doing pretty well on those two. Since the priority is exit nodes, I’ll probably add two more exit nodes in Canada, leaving four exits, and one relay there. But I do get your points, and the more I do think about it, it would be better to just spread it all out, so I guess whenever I start spinning up nodes in Europe I’ll just use VPSes. One other thing I forgot to realize is I’m seeing a steady increase in the amount of DDoS attacks on my exits as of late. My provider tries to mitigate them as much as possible, but it’s annoying for the end users going through the node and it’s annoying for the people who are getting affected by the DDoS. Putting everything on one big box is just screaming “Here, attack me right here plz, kthx."
On Feb 10, 2018, at 1:44 AM, tor <tor@anondroid.com> wrote:
What scenario is better for the network - adding five 100mbps nodes, or one 500 mbps node?
Are we talking bare metal or VPS? A VPS will probably bottleneck on RAM or CPU before hitting 500 Mpbs.
Bare metal would stand a chance with the right hardware and tuning, but I wouldn't assume you'll hit 500 Mbps on any given node.
Due to the nature of the bandwidth measurements, physical location matters too. You're at the mercy of Tor's bandwidth authorities and in my experience, the further away from Europe, the worse your measurements will be, and so again you may not hit 500 Mbps.
Basically, you shouldn't assume that whatever bandwidth you plan for and advertise will come your way.
I think you'd have better luck with 5x 100 Mbps nodes, or maybe 3x 200 Mbps nodes. You can also run 2 relays per IP.
There are advantages to spreading out the load (like redundancy). I also think Tor's bandwidth measurements and consensus weights are fickle, and some of the variables are out of your control (what else is going on in your rack, datacenter, upstream, etc.). You could use ansible-relayor to turn up a bunch of nodes, wait to see which ones are the most performant, and then keep the best ones. That's what I would do. :)
_______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
This is why I chose to run multiple 100 Mhz nodes. I over-advertise the capacity, so that some of them get a bit more traffic than they really should, because the bwauths under-measure NA relays and, worse, keep changing their measurements for no apparent reason. On the same bare metal box one relay can be measured as 3-4x another, easily. -------- Original message --------From: Conrad Rockenhaus <conrad@rockenhaus.com> Date: 2/10/18 12:33 AM (GMT-08:00) To: tor-relays@lists.torproject.org Subject: Re: [tor-relays] Question regarding exit sizing The 500 Mbps instance would either be one of my private servers in my co-lo, or a dedicated server in one of my private cloud hosting locations. With both contacts, bandwidth costs aren’t an issue, but if one big instance would work I would put it on the same hardware that I am running a server that averages about 700+ mbps consistently. The small servers that I’m hitting 100 mbps on, I’m just getting low cost VPSes for since….they do the job and they do the job well, they’re guaranteed a minimum of 100mbps bandwidth to the first tier 1 hop and Atlas shows them consistently used at that level, so I’m happy. I mainly wanted to give back big to the community because Tor gave me the idea for my latest free for personal/charge for business use idea that I’m going to roll out soon (I’ll gladly send y’all a link, as I think it’s something that would be very useful). I haven’t noticed any bad measurements…the three relays I run now, well, one just started this week so we can throw that one out for now, but the other two are showing 12.55 MiB/s and 12.28 MiB/s, and I’m guaranteed 100 mbps, so I’m doing pretty well on those two. Since the priority is exit nodes, I’ll probably add two more exit nodes in Canada, leaving four exits, and one relay there. But I do get your points, and the more I do think about it, it would be better to just spread it all out, so I guess whenever I start spinning up nodes in Europe I’ll just use VPSes. One other thing I forgot to realize is I’m seeing a steady increase in the amount of DDoS attacks on my exits as of late. My provider tries to mitigate them as much as possible, but it’s annoying for the end users going through the node and it’s annoying for the people who are getting affected by the DDoS. Putting everything on one big box is just screaming “Here, attack me right here plz, kthx." On Feb 10, 2018, at 1:44 AM, tor <tor@anondroid.com> wrote: What scenario is better for the network - adding five 100mbps nodes, or one 500 mbps node? Are we talking bare metal or VPS? A VPS will probably bottleneck on RAM or CPU before hitting 500 Mpbs. Bare metal would stand a chance with the right hardware and tuning, but I wouldn't assume you'll hit 500 Mbps on any given node. Due to the nature of the bandwidth measurements, physical location matters too. You're at the mercy of Tor's bandwidth authorities and in my experience, the further away from Europe, the worse your measurements will be, and so again you may not hit 500 Mbps. Basically, you shouldn't assume that whatever bandwidth you plan for and advertise will come your way. I think you'd have better luck with 5x 100 Mbps nodes, or maybe 3x 200 Mbps nodes. You can also run 2 relays per IP. There are advantages to spreading out the load (like redundancy). I also think Tor's bandwidth measurements and consensus weights are fickle, and some of the variables are out of your control (what else is going on in your rack, datacenter, upstream, etc.). You could use ansible-relayor to turn up a bunch of nodes, wait to see which ones are the most performant, and then keep the best ones. That's what I would do. :) _______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
participants (3)
-
Conrad Rockenhaus -
igor.n.mitrofanov -
tor