On Sat, 16 Apr 2016 20:30:29 +0300 s7r s7r@sky-ip.org wrote:
I agree that teor's O(Kn) is the best approach from performance (no additional memory allocations), simplicity and efficacy point of view. O(Kn) algorithm will clear the entries only based on their expiration time, it won't care to clean the v2 / v3 caches in equal measure which is good, given that we do not know how long HS operators will take / need to upgrade their services to prop 224.
The tap vs ntor situation was a good measure, but the threat model was different (we were trying to ensure new clients using ntor get resources from relays with priority as opposite to non-updated botnet zombies using tap). In the current situation we care about v2 and v3 HS caches exactly the same, for an unknown period of time which might not be short, so we shouldn't penalize v2 in any way.
We do? I can think of numerous reasons as to why v3 is objectively better, and why we should incentive people to move towards 224 style HSes...
What I want is possible with Teor's 2nd algorithm (which is adequate). Make Cache A the v2 cache, and if we've freed enough ram after purging expired entries from A in a given time period, we terminate rather than touching cache B.
In effect this makes it slightly less likely for entries in the B cache to be deallocated (we probably should also halt once we have reached our target deallocation amount when attempting to compact either cache, rather than continuing to purge the rest of the time interval, it's the OOM handler, and an addition, a compare and a branch is cheap enough to be done in a loop).
This needs to be covered regardless how often a HSDir has its OOM triggered. I don't think we should assume it's hard to flood HSDirs with descriptors until the memory is full.
Now that HSDirs will need to handle two caches, is 20% of the total memory allocated for HS descriptors a good value? What harm would increasing it to let's say 25% do?
It would decrease memory available for other things, where other things is "all HSDirs are expected to first and foremost be relays, and do normal relay things like relaying traffic, before being HSDirs".
I don't remember if it's possible to opt out of being a HSDir or not (likewise, with the move to make all relays be dir caches, we already are increasing memory pressure on "things that are comically undersized that shouldn't ever be HSDirs or DirCaches in the first place")....
Regards,