-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Thank you for your reply.
DNS is a complex topic with many considerations, but I think you're on the right track. Latency wise ideally you would find some location that covers a lot of middle ground between the relays, as to prevent some exit relays having a <20 ms DNS latency while others have >200 ms latency. But your relays are spread very far and wide so this may prove challenging.
Unfortunately, some of them would indeed have >200 ms rtt, as measured by a quick ping between them just now. Perhaps in the future, if I find an affordable dedi host in a good AS, I may focus on them and put out a few Gbps, which would make that more feasible. Until then, I'm drifting around looking for smaller hosts that allow Tor. It's been frustrating at times, with one host (vsys.host) saying that they allow Tor exits and happily accepting my payment, then changing their mind and asking me not to run it. Then responding again saying they elevated it to a senior manager and that it was indeed allowed, and then replying again apologizing and saying that no, it's not. At least they recognized the confusion and were willing to issue a refund. Frustratingly, they are still listed as a good ISP on the community wiki, and there are 3 exits hosted successfully on their AS...
* Only allow your Tor exit relay servers to send DNS requests to your DNS load balancer/recursor. This makes it harder for adversaries to check the in-cache TTL values. Or when providing the DNS response, randomize the TTL value then (ideally per RRSet). You could also add latency to DNS responses so they all are similarly slow (as if the record wasn't cached), but this has downsides of its own. * Decouple TTL values from the original DNS record's TTL and the value in your cache. You could even randomize this (again per RRset ideally) to make it even less predictable.
I thought about that after reading your blog post, but I couldn't find out how to do that without patching the Unbound source code. That is something I could do, but at the moment I don't have the time on my hands to maintain such a fork, at least not to the quality standards that would be wanted for Tor.
* Preload the top 1k/10k/100k/1M/10M (depending on your networking and hardware capacity) domain's records and keep them 'hot' by fetching their records with a random offset before they would expire. A previous experiment with a small preload list (1k iirc) increased DNS cache hits from ~74% (without preloading/automatic refreshes) to ~93% (with preloading/automatic refreshes), decreasing DNS cache misses considerably. My assumption is that the DNS cache hit rate will increase significantly with 1) a larger preload list and 2) more queries per second, but this is something I can finally verify when our new infrastructure is in place next year.
I'm unsure how to preload the top domains. I couldn't find anywhere to import an up-to-date cache, and having a script attempt to resolve a large number of domains and keep them hot would probably not go over too well with most hosting providers, I imagine. I suspect that might not be helpful with smaller relays. One of my relays is currently doing ~10 Mbps (it's capable of much more; I'm just assuming it'll take more time before the BW authorities raise its consensus weight), and it has a very low cache hit rate. I suspect this is because TTL expiration starts to be the dominant cause of loss of cache entries, even with prefetch enabled. The hit ratio is not great. Since last reset a few days ago: thread0.num.cachehits=74xxx thread0.num.cachemiss=1637xxx thread0.num.prefetch=28xxx For reference, this is my current Unbound config (cache size varies from system to system depending on available resources): server: outgoing-interface: <outgoing ipv4 goes here> do-ip6: no rrset-cache-size: 64m msg-cache-size: 32m neg-cache-size: 8m prefetch: yes prefetch-key: yes auth-zone: name: "." master: f.root-servers.net master: k.root-servers.net master: l.root-servers.net master: j.root-servers.net fallback-enabled: yes for-downstream: no for-upstream: yes zonefile: "/var/lib/unbound/root.zone"
So "Would it be reasonable to dedicated a single, cheap VPS for DNS queries, and have all my other exits use it as their resolver over DoT?"? I think this is a reasonable approach, even when it adds some DNS latency (within reason) to your exit relays. But I would also try (where possible/feasible) to take some countermeasures against some of the more common/easy attacks. I think in many cases (even without extensive countermeasures) a centralized DNS recursor for your own exit relays, at the very least isn't a significant downgrade from running them locally on your exit servers.
The worst rtt is ~250ms from my test. I know that systemd-resolved will keep its own local cache even if it's using an upstream resolver, but the size is small and fixed. Would a centralized resolver be feasible even in my case? I suppose I could try to find a very well-peered AS and find a cheap VPS on it for only the purposes of DNS resolution (and perhaps as a bridge relay, so bandwidth isn't going to waste).
When our DNS infra is finally upgraded you're free to use it as well (we can whitelist all your IP addresses), but in terms of latency it might be a bad fit since your relays are situated in other parts of the world. Perhaps there are Tor operator that run DNS recursors in South Africa, Moldova, US and Canada for you to use :). You could also ask in the IRC operators channel to see whether other operators can provide you with access to their recursors.
Maybe I can switch to it for the nearest exits. If I had more networking experience, I'd set up my own set of DNS resolvers for the Tor network with anycast, and allow any IP that is a Tor exit to resolve from them. That would at least be better than using 8.8.8.8. But of course, I have neither the time, networking experience, or trust in the community as a new exit operator to do that. Regards, forest -----BEGIN PGP SIGNATURE----- iQIzBAEBCgAdFiEEvLrj6cuOL+I/KdxYBh18rEKN1gsFAmj40hsACgkQBh18rEKN 1gsMbg//Txhgv0mWny5CdsSjDQD1Oba2sdN0QvMorsM3UbtzJFAMi5dk2jwjYmYj GwZd8/pBbqH5gUDq4xFl8whnRUOiNwDezZEHJIS1gTrz9yC9V+tBLVr5sGW5FfvU dGw4MNWLwXubnpDP1q3HlSirvOfQrLKuxP0z7S9FCptzzJpjEG07st8M6Uh3SBAU MjbX/36NbsHDZ1oD44wgbzdpExWlz2ob2BxqMFLudsqxGim3G9IejnxLS5ONFu8y n5nHtm2AKO2Sf74lcfFi/qAqfhMBFWFZ/JKau2jotbR45ZkJffjDtJsUcUCHVt6k sb2suvmSJfDC6/kBd4vgH+8BqwtjYJu5XS9qPp4zNLifr7Gp5yv2nHx9cxiMRAvh 6xf/8z01qDrXZxygVRb1IeLU3kw3xMgZQQHVxAW+1NbJ86F/MMQcNFoIgOGhI2eH dUd0KTdUvC4O+BQEXt2ZCwU5XwskDu+HuO5EAo5dZK4t5vI8pWchAWkdYGVh1x3P nDmemYK3U/ndtSw+8RVTuTlRXfkBBqyavjK4pNkeK9iHuEEYtadU6F3AAKwzBXxx ta7UBWPiMldqd25JTT/9Tj9fboR9IIXZ4TSf9ZuyyV59oz77c0bycsTIhFP4oHs4 7bR2VGZ4yRrouT5u9TlfyUnKK0cO5HZSIRZE+zR8p19oGOM+wPc= =sB2a -----END PGP SIGNATURE-----