Hello,
Just a report-back: https://twitter.com/EmeraldOnion/status/1160694122069422080
"It took us 128 days to go from 1Gbps to 10Gbps advertised bandwidth with the progressive deployment of up to 56 relays (April 2 to August 8, 2019). In terms of actual bandwidth, we have a sustained throughout of roughly 5-6Gbps. We can sustain up to 15Gbps and burst to 20Gbps."
Currently we're the fifth largest exit family at 1.2220% consensus weight, 1247.67 MiB/s advertised bandwidth, and 5.5546% exit probability. It costs us $1,600 per month, in donations, to manage this service. As you may know, we are all volunteers.
There was some additional feedback that we got from Nusenu regarding capping bandwidth of individual Tor processes, and using a second IP for exit traffic. We are still trying to determine the per-process max bandwidth on our two hardware platforms, but we are planning on enabling these caps for quality of the service.
When we originally deployed many new relays using Ansible-Relayor, we used a second IP per process. However, due to routing troubleshooting and relays not showing up on Relay Search, we removed the second IPs. We are still using only one per relay. We also found it challenging to manage this many relay IPs with Ansible-Relayor. Ansible-Relayor uses sequential IPs based on the listing from ifconfig. This presents a challenge because it is difficult to setup forward and reverse DNS in a predictable way.
The second issue we have with running a secondary IP per Tor process is system load. Having more IPs opens more sockets, and we are already putting a lot of load on these multi-core servers. The third issue is that when people block our IPs, they block the scope. Should we use a secondary IP per relay if the IP is in the same scope? Should we then use two /24 scopes, and wouldn't both scopes end up getting blocked? If we were to use a second /24 for relays, how will Ansible-Relayor know to use a second IP scope for exiting?
IPv4 dependency is a real burden. We would like to see Tor Project help Tor network operators more directly, both financially and securing IPv4 scopes for nonprofit organizations to take ownership of. The latter is needed until we can stop using IPv4 completely and operate only with IPv6.
It is discouraging to see so many small and large network operators not using IPv6. Why is this such a problem? Tor Project, please increase your #IPv6 awareness/outreach similar to how ARIN and the other RIRs try very hard to do.
Cheers,
-- Christopher Sheats Executive Director for Emerald Onion Email: yawnbox@emeraldonion.org Office (& Signal): +1-206-739-3390 Website: https://emeraldonion.org/
-----Original Message----- From: tor-relays tor-relays-bounces@lists.torproject.org On Behalf Of grarpamp Sent: Thursday, April 4, 2019 12:45 AM To: tor-relays@lists.torproject.org Subject: Re: [tor-relays] Emerald Onion's new relays
On 4/4/19, Conrad Rockenhaus conrad@rockenhaus.com wrote:
when ISPs are ordered to BGP blackhole some exit IP addresses
I've been assigning a second set of IP addresses to my servers in case I want to run another instance of Tor. Would it be more prudent to use that second set of IP addresses as an OutboundBindAddressExit instead and use different ports as a better practice?
ISP traffic filtering sinks, from the tor browser perspective, affecting traffic exiting a relay out through its exitpolicy to clearnet, can be...
- dst based "sink traffic to there", appears as "cnn.com down", a minor issue, depending on scope of the sink.
- src based "sink traffic from there", appears as "Internet down", a major issue, depending on scope of the sink.
Unlike websites, and unless they're tied playing [geo]politics, ISP's really don't like to keep these sinks in place for a long time.
Relay management such as OS updates, ssh, wget could get blocked if those addresses are in consensus.
Then there is relay-to-relay traffic types that don't "exit", but can still get found and blocked.
And the OR IP must be obviously not be blocked, else depending on scope, the relay won't receive traffic to move out any horizon.
Tor should still allow config of 2 tor instances on one IP.
If IP's are "free", and if operator survey says the exit functions are getting knocked off the tor network more often than entire OR's, try putting out the OutboundBindAddressExit on IP for sacrifice, instead of burning entire OR's which could otherwise be used more quietly as middle relays etc.
An operators own cost, management, and ISP relationships may show running more relays is better IP, or net traffic pushed, wise than enduring a few sinks now and then.
Probably every situation is different. Or try both and see.
Common options from the manpage...
Address address ORPort [address:]PORT|auto [flags] OutboundBindAddress IP OutboundBindAddressOR IP OutboundBindAddressExit IP
First one implemented was OutboundBindAddress, then came OutboundBindAddressOR and OutboundBindAddressExit. All for different matrix of reasons. _______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays