Also interested in this thread and efforts.
Plan to do your suggestion on 80 core and 128 core Tor server node tests. Will work on posting back here too.
BGP plan to keep separate on router to avoid using Tor server resources.
Does DNS have similar resource concerns as BGP tables? If so, what's best way to handle DNS outside of Tor server resources? On router too?
Can you say more on why you say this, "You can't fully utilize a /24 with 6x 64 core servers on a 100G Router."?
On Tuesday, February 18th, 2025 at 8:43 AM, boldsuck via tor-relays tor-relays@lists.torproject.org wrote:
On Tuesday, 18 February 2025 17:00 usetor.wtf via tor-relays wrote:
Another question - what's the most optimal count of Tor relays per IP when using an IPv4 /24, i.e. roughly 256 IPs? Looking for thoughts / guidance as this can quickly be a costly endeavor with slow turn around times on securing data center capacity.
The number of IPs is unimportant. CPU cores count and network bandwidth, fast cores, the fastest and best cooling! The higher the CPU clock speed, the more MiB/s traffic per tor instance. Slam 60 tor instances onto a 64-core CPU (or 120 instances on 128 core) with 2x10 or 2x25G card and let it run for a few weeks. Then you will see if you can create some more instances. You also have to do DNS. PowerDNS + dnsdist is your friend with 2x10G or more. Where do you do BGP on the server or router? Full table BGP need recources too. You can't fully utilize a /24 with 6x 64 core servers on a 100G Router.
Current hypothesis is around 2 Tor Instances per 256 IPs for 512 relays at 5 MiB/s each needing 21 Gbps port speed. See details below.
Option 1: Is it 8 Tor instances per IP, the current maximum? 2048 total Tor instances across 256 IPs in /24? 1/4 of the current ~8000 running relays (~8200 relays bandwidth measured today)? Seems too many. Example: At 256 IPs, 8 Tor instances per IP, average speed of 10 MiB/s per Tor relay, need roughly 172 Gbps, which is much less common, especially among volunteer Tor relays.
Option 2: Is it 1 Tor instance per IP, the minimum amount per IP? When Tor is blocked, it's done by IP, so have 8 per IP is less efficient when 256 are available to spread out the relays and minimize blockage, unless the full /24 gets blocked? Example: At 256 IPs, 1 Tor instances per IP, average speed of 10 MiB/s per Tor relay, need roughly 21 Gbps, which seems much more reasonable using 2 x 10 Gbps links on one node with ~256 cores or split across 2 nodes of each having 10 Gbps and 128 cores.
If you use a /24 for Tor exit traffic, it is completely blacklisted anyway. Stop doing the math ;-)
Option 3: Seems like the ideal would be however many can be utilized per available bandwidth?
Here's a rough sizing table (attached and inline) of Port Speed in Gbps needed depending on # of available IPs, # of Tor instances per IPv4 and Speed per Tor (MiB/s). Legend: <= 10 Gbps is green, <= 20 Gbps is yellow, and > 20 Gbps is red.
During the Fall of 2021, I saw ~15 MiB/s per Tor Instance and now I see around ~5 MiB/s per Tor Instance (no changes on my servers other than OS and Tor updates).
Current conclusion: I'm looking at the 256, 2, 512, 5, 2560, 21 row as where I'll likely start. 512 is a lot of Tor instances... [image.png]
~8200 relays bandwidth measured today: https://consensus-health.torproject.org/graphs.html
Sent with Proton Mail secure email.
On Monday, February 3rd, 2025 at 8:00 AM, usetor.wtf
usetor.wtf@protonmail.com wrote:
Hi All,
Looking for guidance around running high performance Tor relays on Ubuntu.
Few questions:
- If a full IPv4 /24 Class C was available to host Tor relays, what are
some optimal ways to allocate bandwidth, CPU cores and RAM to maximize utilization of the IPv4 /24 for Tor?
- If a full 10 Gbps connection was available for Tor relays, how many CPU
cores, RAM and IPv4 addresses would be required to saturate the 10 Gbps connection?
- Same for a 20 Gbps connection, how many CPU cores, RAM and IPv4
addresses are required to saturate?
Thanks!
Sent with Proton Mail secure email.
-- ╰_╯ Ciao Marco!
Debian GNU/Linux
It's free software and it gives you freedom!_______________________________________________ tor-relays mailing list -- tor-relays@lists.torproject.org> To unsubscribe send an email to tor-relays-leave@lists.torproject.org
On Wednesday, 19 February 2025 06:35 Tor at 1AEO wrote:
Already replied via PM.
Can you say more on why you say this, "You can't fully utilize a /24 with 6x 64 core servers on a 100G Router."?
6x 64c/128t = 768 DirAuth's allow 8 relays/IP A routed /24 256x8 = 2048 relays, theoretically
On Tuesday, February 18th, 2025 at 8:43 AM, boldsuck via tor-relays <tor-
relays@lists.torproject.org> wrote:
On Tuesday, 18 February 2025 17:00 usetor.wtf via tor-relays wrote:
Another question - what's the most optimal count of Tor relays per IP when using an IPv4 /24, i.e. roughly 256 IPs? Looking for thoughts / guidance as this can quickly be a costly endeavor with slow turn around times on securing data center capacity.
The number of IPs is unimportant. CPU cores count and network bandwidth, fast cores, the fastest and best cooling! The higher the CPU clock speed, the more MiB/s traffic per tor instance. Slam 60 tor instances onto a 64-core CPU (or 120 instances on 128 core) with 2x10 or 2x25G card and let it run for a few weeks. Then you will see if you can create some more instances. You also have to do DNS. PowerDNS + dnsdist is your friend with 2x10G or more. Where do you do BGP on the server or router? Full table BGP need recources too. You can't fully utilize a /24 with 6x 64 core servers on a 100G Router.>
Current hypothesis is around 2 Tor Instances per 256 IPs for 512 relays at 5 MiB/s each needing 21 Gbps port speed. See details below.
Option 1: Is it 8 Tor instances per IP, the current maximum? 2048 total Tor instances across 256 IPs in /24? 1/4 of the current ~8000 running relays (~8200 relays bandwidth measured today)? Seems too many. Example: At 256 IPs, 8 Tor instances per IP, average speed of 10 MiB/s per Tor relay, need roughly 172 Gbps, which is much less common, especially among volunteer Tor relays.
Option 2: Is it 1 Tor instance per IP, the minimum amount per IP? When Tor is blocked, it's done by IP, so have 8 per IP is less efficient when 256 are available to spread out the relays and minimize blockage, unless the full /24 gets blocked? Example: At 256 IPs, 1 Tor instances per IP, average speed of 10 MiB/s per Tor relay, need roughly 21 Gbps, which seems much more reasonable using 2 x 10 Gbps links on one node with ~256 cores or split across 2 nodes of each having 10 Gbps and 128 cores.
If you use a /24 for Tor exit traffic, it is completely blacklisted anyway. Stop doing the math ;-)
Option 3: Seems like the ideal would be however many can be utilized per available bandwidth?
Here's a rough sizing table (attached and inline) of Port Speed in Gbps needed depending on # of available IPs, # of Tor instances per IPv4 and Speed per Tor (MiB/s). Legend: <= 10 Gbps is green, <= 20 Gbps is yellow, and > 20 Gbps is red.
During the Fall of 2021, I saw ~15 MiB/s per Tor Instance and now I see around ~5 MiB/s per Tor Instance (no changes on my servers other than OS and Tor updates).
Current conclusion: I'm looking at the 256, 2, 512, 5, 2560, 21 row as where I'll likely start. 512 is a lot of Tor instances... [image.png]
~8200 relays bandwidth measured today: https://consensus-health.torproject.org/graphs.html
On Monday, February 3rd, 2025 at 8:00 AM, usetor.wtf
usetor.wtf@protonmail.com wrote:
Hi All,
Looking for guidance around running high performance Tor relays on Ubuntu.
Few questions:
- If a full IPv4 /24 Class C was available to host Tor relays, what
are some optimal ways to allocate bandwidth, CPU cores and RAM to maximize utilization of the IPv4 /24 for Tor?
- If a full 10 Gbps connection was available for Tor relays, how many
CPU cores, RAM and IPv4 addresses would be required to saturate the 10 Gbps connection?
- Same for a 20 Gbps connection, how many CPU cores, RAM and IPv4
addresses are required to saturate?
Thanks!
tor-relays@lists.torproject.org
-
boldsuck -
Tor at 1AEO