tor-relays

tor-relays@lists.torproject.org

13 participants
4861 discussions

Re: Guidance on optimal Tor relay server configurations
by Tor at 1AEO 21 Feb '25

21 Feb '25

Also interested in this thread and efforts. Plan to do your suggestion on 80 core and 128 core Tor server node tests. Will work on posting back here too. BGP plan to keep separate on router to avoid using Tor server resources. Does DNS have similar resource concerns as BGP tables? If so, what's best way to handle DNS outside of Tor server resources? On router too? Can you say more on why you say this, "You can't fully utilize a /24 with 6x 64 core servers on a 100G Router."? On Tuesday, February 18th, 2025 at 8:43 AM, boldsuck via tor-relays <tor-relays(a)lists.torproject.org> wrote: > On Tuesday, 18 February 2025 17:00 usetor.wtf via tor-relays wrote: > > > Another question - what's the most optimal count of Tor relays per IP when > > using an IPv4 /24, i.e. roughly 256 IPs? Looking for thoughts / guidance as > > this can quickly be a costly endeavor with slow turn around times on > > securing data center capacity. > > > The number of IPs is unimportant. > CPU cores count and network bandwidth, fast cores, the fastest and best > cooling! The higher the CPU clock speed, the more MiB/s traffic per tor > instance. > Slam 60 tor instances onto a 64-core CPU (or 120 instances on 128 core) with > 2x10 or 2x25G card and let it run for a few weeks. Then you will see if you > can create some more instances. > You also have to do DNS. PowerDNS + dnsdist is your friend with 2x10G or more. > Where do you do BGP on the server or router? Full table BGP need recources > too. You can't fully utilize a /24 with 6x 64 core servers on a 100G Router. > > > Current hypothesis is around 2 Tor Instances per 256 IPs for 512 relays at 5 > > MiB/s each needing 21 Gbps port speed. See details below. > > > > Option 1: Is it 8 Tor instances per IP, the current maximum? 2048 total Tor > > instances across 256 IPs in /24? 1/4 of the current ~8000 running relays > > (~8200 relays bandwidth measured today)? Seems too many. Example: At 256 > > IPs, 8 Tor instances per IP, average speed of 10 MiB/s per Tor relay, need > > roughly 172 Gbps, which is much less common, especially among volunteer Tor > > relays. > > > > Option 2: Is it 1 Tor instance per IP, the minimum amount per IP? When Tor > > is blocked, it's done by IP, so have 8 per IP is less efficient when 256 > > are available to spread out the relays and minimize blockage, unless the > > full /24 gets blocked? Example: At 256 IPs, 1 Tor instances per IP, average > > speed of 10 MiB/s per Tor relay, need roughly 21 Gbps, which seems much > > more reasonable using 2 x 10 Gbps links on one node with ~256 cores or > > split across 2 nodes of each having 10 Gbps and 128 cores. > > > If you use a /24 for Tor exit traffic, it is completely blacklisted anyway. Stop > doing the math ;-) > > > Option 3: Seems like the ideal would be however many can be utilized per > > available bandwidth? > > > > Here's a rough sizing table (attached and inline) of Port Speed in Gbps > > needed depending on # of available IPs, # of Tor instances per IPv4 and > > Speed per Tor (MiB/s). Legend: <= 10 Gbps is green, <= 20 Gbps is yellow, > > and > 20 Gbps is red. > > > > During the Fall of 2021, I saw ~15 MiB/s per Tor Instance and now I see > > around ~5 MiB/s per Tor Instance (no changes on my servers other than OS > > and Tor updates). > > > > Current conclusion: I'm looking at the 256, 2, 512, 5, 2560, 21 row as where > > I'll likely start. 512 is a lot of Tor instances... [image.png] > > > > ~8200 relays bandwidth measured today: > > https://consensus-health.torproject.org/graphs.html > > > > Sent with Proton Mail secure email. > > > > On Monday, February 3rd, 2025 at 8:00 AM, usetor.wtf > > usetor.wtf(a)protonmail.com wrote: > > > > Hi All, > > > > > > Looking for guidance around running high performance Tor relays on Ubuntu. > > > > > > Few questions: > > > 1) If a full IPv4 /24 Class C was available to host Tor relays, what are > > > some optimal ways to allocate bandwidth, CPU cores and RAM to maximize > > > utilization of the IPv4 /24 for Tor? > > > > > > 2) If a full 10 Gbps connection was available for Tor relays, how many CPU > > > cores, RAM and IPv4 addresses would be required to saturate the 10 Gbps > > > connection? > > > > > > 3) Same for a 20 Gbps connection, how many CPU cores, RAM and IPv4 > > > addresses are required to saturate? > > > > > > Thanks! > > > > > > Sent with Proton Mail secure email. > > > > -- > ╰_╯ Ciao Marco! > > Debian GNU/Linux > > It's free software and it gives you freedom!_______________________________________________ > tor-relays mailing list -- tor-relays(a)lists.torproject.org> To unsubscribe send an email to tor-relays-leave(a)lists.torproject.org

2 1

Re: Guidance on optimal Tor relay server configurations
by Tor at 1AEO 21 Feb '25

21 Feb '25

Excellent information, especially the real world scenarios! Exactly what I was looking for! Summary from your email - did I miss anything? To saturate 10 Gbps connection: 1) IPv4 Allocation: Use between 5 and 20. Much lower than 256 in a /24! 2) Tor Relay Count: Run roughly ~40 to ~150, depending on CPU clock speed, i.e. faster clock, fewer relays needed.3) CPU Utilization: 1 Tor relay per physical core preferred but okay to scale to 1 Tor relay per threads/SMT as well, up to 2x Tor relays per core/thread 4) RAM requirements: Maintain a 4:1 RAM-to-core/relay ratio (4GB per core/relay), including extra 32GB per server to cover DoS, OS, networking, etc. overheads In general, some ideals but not required: CPU clock speed: Higher CPU clock speed, better relay performance RAM: Fewer relays, lower RAM requirements RAM: Add ~32GB to overall RAM capacity sizing for OS, DNS, networking, DoS, etc. IPv4: One IPv4 per relay with common traffic ports Scaling: Start with 1 Tor relay per physical core, then add 1 Tor relay per thread/SMT and stop at 2 Tor relays per each core / thread. What are the primary factors that justify running up to two Tor relays per physical core (leveraging SMT) versus a one-to-one mapping? Ex: 37-74 for ~18.5 physical + SMT and 63-126 for ~31.5 physical + SMT. Is one-to-one mapping of Tor relay to core/thread the most compute- and system-efficient approach? Are the clock speeds you listed base or turbo numbers (3.4, 2.25 and 2.0 Ghz)? I'm assuming base. Not sure if anybody has data on how these impact Tor relays? From your real world scenario #2 and advice for the "fastest/best hardware", would this type of server work well for a $20k budget? A single-socket Xeon 6980P (128 physical cores, 256 threads, base clock 2.0 GHz, turbo up to 3.9 GHz) with 1024GB DDR5 (maintaining a 4:1 ratio) and an AIOM Mellanox NIC be optimal? Assuming one relay per core/thread, would this setup be capable of saturating 40 Gbps, given that 10 Gbps typically saturated with ~31.5 physical cores + ~31.5 SMT threads at 2 Ghz and thus 256 relays vs ~63 relays translates roughly to 4×10 Gbps = 40 Gbps? For those curious, according to ChatGPT o3-mini-high deep research: 1) 20% cap on bandwidth contributions for exit relays is roughly 50+ Gbit/s for the largest operators. 2) 10% of Tor's consensus weight in terms of bandwidth for 2025 is roughly 90-95 Gbps of sustained bandwidth. In 2022, it would have been ~68 Gbps. I don't plan to have this issue any time soon, but good to be aware! Screenshots of the lengthy responses below and attached. [image.png] [image.png] [image.png] [image.png] Sent with[Proton Mail](https://proton.me/mail/home)secure email. On Tuesday, February 18th, 2025 at 2:23 PM, mail(a)nothingtohide.nl <mail(a)nothingtohide.nl> wrote: > Hi, > > Many people already replied, but here are my (late) two cents. > >> 1) If a full IPv4 /24 Class C was available to host Tor relays, what are some optimal ways to allocate bandwidth, CPU cores and RAM to maximize utilization of the IPv4 /24 for Tor? > > "Optimal" depends on your preferences and goals. Some examples: > > - IP address efficiency: run 8 relays per IPv4 address. > - Use the best ports: 256 relays (443) or 512 relays (443+80). > - Lowest kernel/system congestion: 1 locked relay per core/SMT thread combination, ideally on high clocked CPUs. > - Easiest to manage: as few relays as possible. > - Memory efficiency: only run middle relays on very high clocked CPUs (4-5 Ghz). > - Cost efficiency: run many relays on 1-2 generations old Epyc CPUs with a high core count (64 or more). > > There are always constraints. The hardware/CPU/memory and bandwidth/routing capability available to you are probably not infinite. Also the Tor Project maximizes bandwidth contributions to 20% and 10% for exit relay and overall consensus weight respectively. > > With 256 IP addresses on modern hardware, it will be very hard to not run in to one of these limitations long before you can make it 'optimal'. Hardware wise, one modern/current gen high performance server only running exit relays will easily push enough Tor traffic to do more than half of the total exit bandwidth of the Tor network. > > My advice would be: > 1) Get the fastest/best hardware with current-ish generation CPU IPC capabilities you can get within your budget. To lower complexity with keeping congestion in control, one socket is easier to deal with than a dual socket system. > > (tip for NIC: if your switch/router has 10 Gb/s or 25 Gb/s ports, get some of the older Mellanox cards. They are very stable (more so than their Intel counterparts in my experience) and extremely affordable nowadays because of all the organizations that throw away their digital sovereignty and privacy of their employees/users to move to the cloud). > > 3) Start with 1 Tor relay per physical core (ignoring SMT). When the Tor relays have ramped up (this takes 2-3 months for guard relays) and there still is considerable headroom on the CPU (Tor runs extremely poorly at scale sadly, so this would be my expectation) then move to 1 Tor relay per thread (SMT included). > > (tip: already run/'train' some Tor relays with a very limited bandwidth (2 MB/s or something) parallel to your primary ones and pin them all to 1-2 cores to let them ramp up in parallel to your primary ones. This makes it *much* less cumbersome to scale up your Tor contribution when you need/want/can do that in the future). > > 4) Assume at least 1 GB of RAM per relay on modern CPUs + 32 GB additionally for OS, DNS, networking and to have some headroom for DoS attacks. This may sound high, especially considering the advice in the Tor documentation. But on modern CPUs (especially with a high clockspeed) guard relays can use a lot more than 512 MB of RAM, especially when they are getting attacked. Middle and exit relays require less RAM. > > Don't skimp out on system memory capacity. DDR4 RDIMMs with decent clockspeeds are so cheap nowadays. For reference: we ran our smaller Tor servers (16C(a)3.4Ghz) with 64 GB of RAM and had to upgrade it to 128 GB because during attacks RAM usage exceeded the amount available and killed processes. > > 5) If you have the IP space available, use one IPv4 address per relay and use all the good ports such as 443. If IP addresses are more scarce, it's also not bad to run 4 or 8 relays per IP address. Especially for middle and exit relays the port doesn't matter (much). Guard relays should ideally always run on a generally used (and generally unblocked) port. > >> 2) If a full 10 Gbps connection was available for Tor relays, how many CPU cores, RAM and IPv4 addresses would be required to saturate the 10 Gbps connection? > > That greatly depends on the CPU and your configuration. I can offer 3 references based on real world examples. They all run a mix of guard/middle/exit relays. > > 1) Typical low core count (16+SMT) with higher clockspeed (3.4 Ghz) saturates a 10 Gb/s connection with ~18.5 physical cores + SMT. > 2) Typical higher core count (64+SMT) with lower clockspeed (2.25 Ghz) saturates a 10 Gb/s connection with ~31.5 physical cores + SMT. > 3) Typical energy efficient/low performance CPU with low core count (16) with very low clockspeed (2.0 Ghz) used often in networking appliances saturates a 10 Gb/s connection with ~75 physical cores (note: no SMT). > > The amount of IP addresses required also depends on multiple factors. But I'd say that you would need between the amount and double the amount of relays of the mentioned core+SMT count in order to saturate 10 Gb/s. This would be 37-74, 63-126 and 75-150 relays respectively. So between 5 and 19 IPv4 addresses would be required at minimum, depending on CPU performance level. > > RAM wise the more relays you run, the more RAM overhead you will have. So in general it's better to run less relays at a higher speed each than run many at a low clock speed. But since Tor scales so badly you need more Relays anyway so optimizing this isn't easy in practice. > >> 3) Same for a 20 Gbps connection, how many CPU cores, RAM and IPv4 addresses are required to saturate? > > Double the amount compared to 10 Gb/s. > > Good luck with your Tor adventure. And let us know your findings with achieving 10 Gb/s when you get there :-). > > Cheers, > > tornth > > Feb 3, 2025, 18:14 by tor-relays(a)lists.torproject.org: > >> Hi All, >> >> Looking for guidance around running high performance Tor relays on Ubuntu. >> >> Few questions: >> 1) If a full IPv4 /24 Class C was available to host Tor relays, what are some optimal ways to allocate bandwidth, CPU cores and RAM to maximize utilization of the IPv4 /24 for Tor? >> >> 2) If a full 10 Gbps connection was available for Tor relays, how many CPU cores, RAM and IPv4 addresses would be required to saturate the 10 Gbps connection? >> >> 3) Same for a 20 Gbps connection, how many CPU cores, RAM and IPv4 addresses are required to saturate? >> >> Thanks! >> >> Sent with[Proton Mail](https://proton.me/mail/home)secure email.

3 2

Guidance on optimal Tor relay server configurations
by usetor.wtf 18 Feb '25

18 Feb '25

Hi All, Looking for guidance around running high performance Tor relays on Ubuntu. Few questions: 1) If a full IPv4 /24 Class C was available to host Tor relays, what are some optimal ways to allocate bandwidth, CPU cores and RAM to maximize utilization of the IPv4 /24 for Tor? 2) If a full 10 Gbps connection was available for Tor relays, how many CPU cores, RAM and IPv4 addresses would be required to saturate the 10 Gbps connection? 3) Same for a 20 Gbps connection, how many CPU cores, RAM and IPv4 addresses are required to saturate? Thanks! Sent with [Proton Mail](https://proton.me/mail/home) secure email.

6 12

Re: Adding falgs to new relays
by Marco Moock 13 Feb '25

13 Feb '25

Am Wed, 12 Feb 2025 04:46:14 +0000 schrieb ZK <zk(a)onionmail.org>: > > >Please give the Nickname for the affected relay. > > family:265E819C80A418EF3690F778B17AF4ED507AC3B1 Check your IPv6 exit policy. It rejects all. Test outgoing connections from the machine. Is there a firewall running? Check the logs for outgoing rejected packets. BadExit indicates that outgoing connections are not possible.

2 2

Re: (No Subject)
by Marco Moock 11 Feb '25

11 Feb '25

Am Mon, 10 Feb 2025 21:03:46 +0000 schrieb Azraxiel <azraxiel(a)proton.me>: > I openened Port 9001 on my router 192.168.178.1 and forwarded it to > my OpenSense 192.168.178.2 In OpenSense(10.0.0.1 Own IP) i created > the following NAT rule and same Firewall rule for the WAN Interface > TCP *AnySource *AnyPort Destination:WAN address Port:9001 IP:10.0.0.1 > Port9001 The External IP is 109.199.164.117 Double NAT is always a bad idea. Why are you doing it in general? Please also provide the IPv6 address, so we can test it too. This is the outcome of my traceroute. m@zbook:~$ sudo traceroute 109.199.164.117 -T -p 9001 traceroute to 109.199.164.117 (109.199.164.117), 30 hops max, 60 byte packets 1 10.254.254.1 (10.254.254.1) 1.787 ms 2.142 ms 2.135 ms 2 172.18.0.1 (172.18.0.1) 5.794 ms 6.896 ms 7.238 ms 3 adslgwffm.tal.de (82.139.222.46) 12.980 ms 13.633 ms * 4 xe-101-0-3-90.fix5-r1.de.syseleven.net (37.49.155.221) 13.947 ms * * 5 po1-1000.r4-fra1-de.as5405.net (45.153.82.4) 14.233 ms * * 6 * * * 7 r3-fra1-de.as5405.net (94.103.180.6) 10.937 ms * * 8 ffm-b5-link.ip.twelve99.net (62.115.148.98) 11.824 ms * * 9 ffm-bb1-link.ip.twelve99.net (62.115.114.88) 11.429 ms * * 10 mcn-b5-link.ip.twelve99.net (62.115.126.111) 18.171 ms * * 11 infrafibre-ic-373861.ip.twelve99-cust.net (213.248.97.162) 21.586 ms * * 12 * * * 13 109.199.164.117 (109.199.164.117) 25.685 ms * * 14 * * * 15 * * * It indicates that the traffic is being dropped. Change you settings to give proper ICMP error messages to make diagnosing more easy.

1 0

(No Subject)
by Azraxiel 10 Feb '25

10 Feb '25

Hello, does anybody use the Opensense Plugin for Tor? I followed every step in the Documention for setting up a Relay but it doesn't work and the opnsense community can't help either. Best regards Azra Sent from Proton Mail Android

2 1

Mass-email sent to relay operators
by Zachary 10 Feb '25

10 Feb '25

Hello my fellow relay operators, I just received an email that was sent to many relay operators' contact emails. The content is as follows: // Start message Hello to all the relays operators. My name is Zakwan Kalb. I'm investigating a possible ongoing end-to-end confirmation attack run by the Torproject. Please reply to me if you have encountered one of the following and provide as much information as possible: 1. Your relay has been removed by the Torproject. (You've seen something like that in your log files: "http status 400 ("Fingerprint and/or ed25519 identity is marked rejected ...") 2. The Torproject added flags it's not supposed to add to your relays (BadExit, MiddleOnly) 3. You have been asked by the Torproject to install unknown software, spy on users or do something else that's suspicious 4. You know something about the ongoing attack I have some evidence of the attack: the Torproject doesn't allow people to run relays by removing them from the network or making them unusable as Guard or Exit for no known reason for years. A random person cannot run a Guard or Exit relay. Thus the Tor network is entirely run by the people chosen by the Torproject by unknown criteria. More evidence and details are needed. I think we need to discuss this issue with each other, contact the media and freedom of speech organizations and let people know what's happening. // End message Looking through his mailing list history it looks like he was asking about this same thing back in 2021. Just wanted to give everyone a heads up. It doesn't seem like there's any malicious intent, maybe a bit of schizophrenia perhaps, but I've reached back out simply asking if he has any proof of anything actually going on just to appease my own curiosity. I have no further comment about this. Zachary

3 2

Adding falgs to new relays
by ZK 10 Feb '25

10 Feb '25

I'm asking the Torpoject to publicly answer the question: why do you add BadExit and MiddleOnly flags to new relays? Please don't lie as you did before and list the criteria here

3 2

Web Tunnel Bridges
by gerard＠bulger.co.uk 29 Jan '25

29 Jan '25

Web tunnel bridges Port 443, https I set up three of these. One in UK, one in Australia and one in USA. Only the USA service is attracting traffic. This might be normal. The others are very quiet. Then I look at it on Tor Relay metric it reports Running <https://metrics.torproject.org/rs.html#search/flag:running> V2Dir <https://metrics.torproject.org/rs.html#search/flag:v2dir> Valid <https://metrics.torproject.org/rs.html#search/flag:valid> Additional Flags none First Seen 2025-01-26 14:22:20 Last Restarted 2025-01-28 20:49:38 Platform Tor 0.4.8.13 on Linux Transport protocols Transport protocols supported by this bridge. webtunnel Bridge distribution mechanism Settings <https://bridges.torproject.org/info#settings> The difference between the quiet ones and the active USA server is Bridge Distribution Mechanism it says "settings" rather than HPPTS for my USA one Does this change automatically or do I need to change something? Gerry

1 0

Re: Having trouble with setting up a relay in a censored country.
by George Hartley 22 Jan '25

22 Jan '25

Hi, sorry for the late reply, I was busy working. The only thing that stands out to me immediately is that you configured two separate ControlPort authentication methods: > HashedControlPassword XX:XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX > CookieAuthentication 1 You might want to comment out the first line. Otherwise, everything else looks correct. P.S: Since you do not run a bridge, there is no need to un-comment the obfs4 related settings: > ServerTransportPlugin obfs4 exec /usr/bin/obfs4proxy > ServerTransportListenAddr obfs4 0.0.0.0:9002 This will do nothing, unless you actually configure the relay to be a bridge. Try to comment both HashedControlPassword and the obfs4 settings, and restart the relay. nyx should now work - additionally, you might want to check the log using journalctl -u tor. There should be a line stating that your ORPort is reachable from the outside, and that your server descriptor has been published. If that is not the case, then I would conclude that your host / ISP is up to some "funny" business, and you might not be able to run a normal relay. Since I can't find any relay with your specified nickname, I assume that it is still not working. However, you could still run an obfs4 bridge, which should work fine in Russia - if you can do this, please do.. any relay or bridge not run by intelligence agencies or government helps! Regards, -GH On Wednesday, January 15th, 2025 at 3:23 PM, nyyymi <nyyymi(a)protonmail.com> wrote: > Hi, here is my torrc file. Enabling cookieauth didn't change anything >

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

tor-relays