[tor-bugs] #7474 [Tor]: Directory authorities don't store 15% or more of uploaded extra-info descriptors

[Tor Bug Tracker & Wiki]

#7474: Directory authorities don't store 15% or more of uploaded extra-info
descriptors
----------------------+-----------------------------------------------------
 Reporter:  karsten   |          Owner:     
     Type:  defect    |         Status:  new
 Priority:  critical  |      Milestone:     
Component:  Tor       |        Version:     
 Keywords:            |         Parent:     
   Points:            |   Actualpoints:     
----------------------+-----------------------------------------------------
 Could it be that the directory authorities reject a number of extra-info
 descriptors for some yet unknown reason?  Is there anything in the logs?

 metrics-db started warning about missing descriptors yesterday:

 {{{
 [java] 2012-11-13 21:05:44 We are missing at least 0.5% of
   server or extra-info descriptors referenced from a consensus
   or vote.
 }}}

 Here are the most recent fractions of descriptors that metrics-db got:

 {{{
 Consensus/Vote, valid-after, votes, server descriptors, extra-infos:
 V, 2012-11-14 14:00:00, NA, 5976/5976 (100.0%), 5081/5976 (85.0%)
 V, 2012-11-14 14:00:00, NA, 5972/5972 (100.0%), 5077/5972 (85.0%)
 V, 2012-11-14 14:00:00, NA, 5976/5976 (100.0%), 5081/5976 (85.0%)
 V, 2012-11-14 14:00:00, NA, 5976/5976 (100.0%), 5081/5976 (85.0%)
 V, 2012-11-14 14:00:00, NA, 5976/5976 (100.0%), 5081/5976 (85.0%)
 V, 2012-11-14 14:00:00, NA, 5976/5976 (100.0%), 5081/5976 (85.0%)
 V, 2012-11-14 14:00:00, NA, 5976/5976 (100.0%), 5081/5976 (85.0%)
 V, 2012-11-14 14:00:00, NA, 5976/5976 (100.0%), 5081/5976 (85.0%)
 V, 2012-11-14 14:00:00, NA, 5956/5956 (100.0%), 5061/5956 (85.0%)
 C, 2012-11-14 14:00:00, 9/9 (100.0%), 3173/3173 (100.0%), 3099/3173
 (97.7%)
 }}}

 So, while metrics-db manages to find 100.0% of server descriptors, it's
 down to 85.0% of referenced extra-info descriptors.

 Here's a quick look at gabelmoo's cached descriptor files that metrics-db
 rsyncs and uses as one relay descriptor source:

 {{{
 $ grep -h "^opt extra-info-digest " cached-descriptors* | \
   cut -d" " -f3- | sort | uniq | wc -l
 19043
 $ grep "^extra-info " cached-extrainfo* | cut -d" " -f3 | \
   sort | uniq | wc -l
 6713
 }}}

 I don't know what typical values are, but having only 6713 of 19043
 referenced extra-info descriptors seems low.

 This ''could'' be a problem in metrics-db.  But it could also be a problem
 with the directory authorities.

-- 
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/7474>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online