[tor-bugs] #24468 [Core Tor/Tor]: Measure HSDir usage to guide parameter choices

[Tor Bug Tracker & Wiki]

#24468: Measure HSDir usage to guide parameter choices
--------------------------------------------+------------------------------
 Reporter:  teor                            |          Owner:  teor
     Type:  task                            |         Status:  assigned
 Priority:  Medium                          |      Milestone:  Tor:
                                            |  0.3.3.x-final
Component:  Core Tor/Tor                    |        Version:
 Severity:  Normal                          |     Resolution:
 Keywords:  privcount-experimental, tor-hs  |  Actual Points:
Parent ID:  #24425                          |         Points:  3
 Reviewer:                                  |        Sponsor:
--------------------------------------------+------------------------------

Comment (by asn):

 Replying to [ticket:24468 teor]:
 > Split off #24425:
 >
 > Replying to [ticket:24425#comment:5 asn]:
 > > Replying to [ticket:24425#comment:4 teor]:
 > > > If you write down a list of exactly what you want to know, we can
 probably collect some stats on ~18 HSDirs using PrivCount.
 > > ...
 > > here are some basic ideas:
 > >
 > > - How many v2/v3/both descs per HSDir?
 >
 > How is this different to "rate of incoming"?
 >
 > If you mean "cached right now", then I'd need a timeframe so I could
 design an event. I could do this in December or January.
 >

 Yeah I meant "cached right now". How many descs does the HSDir have cached
 at 01:00? At 03:00? At 14:00?

 Perhaps what could be done is a graph with x-axis being time (24 hours),
 and y-axis being boxplots of the number of descriptors through the whole
 observation period. Something like this: https://i.imgur.com/oyryjs4.png

 > > - How much total RAM do all v2/v3/both descs occupy on your hsdirs?
 (max,min,avg,mean over your 18 hsdirs)
 >
 > I think we have some of the data, but I'd need a list of the objects
 that contribute to RAM usage. Do you just want descriptors, or is there a
 replay cache? I could do this in December or January.
 >

 Hm. Don't care much about the replay cache right now, mainly about the
 descs themselves. We could use `rendcache.c:rend_cache_total_allocation`
 (which tracks both v2 and v3), and then `rend_cache_entry_allocation()`
 for v2, and `cache_get_dir_entry_size()` for v3.

 > > - Size variance of v2/v3 descs? (max,min,avg,mean)
 >
 > Already implemented as a histogram, needs defined bin sizes.
 >

 Hm, not sure. Probably depends on the size variance to optimize bin size.

 > > - What's the rate of incoming v2/v3/both descs?
 >
 > Already implemented, needs a time period.
 >

 We could do per hour to get some initial insight?

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/24468#comment:1>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online