[tor-bugs] #13192 [Tor]: Collect aggregate stats of total hidden service usage vs total exit usage in Tor network

[Tor Bug Tracker & Wiki]

#13192: Collect aggregate stats of total hidden service usage vs total exit usage
in Tor network
-----------------------------+---------------------------------
     Reporter:  arma         |      Owner:
         Type:  enhancement  |     Status:  needs_information
     Priority:  normal       |  Milestone:  Tor: 0.2.???
    Component:  Tor          |    Version:
   Resolution:               |   Keywords:  SponsorR, tor-relay
Actual Points:               |  Parent ID:
       Points:               |
-----------------------------+---------------------------------

Comment (by karsten):

 (teor, thanks for the code review and patch.  Please don't think people
 are ignoring you.  Somebody is going to reply to you in the next few
 days.)

 I just finished a very first analysis of reported hidserv-stats.  My
 analysis code has not been reviewed by anyone.  These results might be off
 by orders of magnitude.  Handle with care.

 Please find the attached two graphs:

 The first graph,
 [https://trac.torproject.org/projects/tor/attachment/ticket/13192/hidserv-
 rend-relayed-cells-2014-12-26.png Total number of RELAY cells on
 rendezvous circuits], takes the number of cells that a single relay
 reports, looks up the relay's mean fraction of consensus weight during the
 statistics interval, and plots the quotient of the two numbers.  The graph
 shows that most relays except those near x = 0% show quite similar network
 totals.

 Speaking in absolute numbers (see my warning above about possibly being
 wrong by orders of magnitues), that's roughly 12.5 billion cells on
 rendezvous circuits per day.  At 512 B per cell that's 70.64 MiB/s.  With
 roughly 6,000 MiB/s traffic in the network, that's 1.18% of total traffic.

 The second graph,
 [https://trac.torproject.org/projects/tor/attachment/ticket/13192/hidserv-
 dir-onions-seen-2014-12-26.png Total number of .onion addresses], uses the
 reported number of .onion addresses seen by hidden service directories and
 extrapolates them to the expected number of addresses in the network.

 Calculating the fraction of .onion addresses that a relay would see is
 more complicated though.  In this analysis I'm looking at the mean
 fraction of descriptor space that a directory has handled.  That's the
 difference between the fingerprint of the relay three HSDirs earlier in
 the ring up to the HSDir's own fingerprint, accounting for traversing ring
 end/start, divided by the total ring size (1 << 160).

 Now, each directory has four chances to see a .onion address during 24
 hours: there are two replicas per descriptor, and descriptor identifiers
 change once every 24 hours which most likely doesn't align with the
 statistics interval.

 The graph shows reported .onion numbers divided by calculated descriptor
 space fraction divided by four.

 In absolute numbers (same warning as above) there are roughly 30,000
 .onion addresses in the network.

 So, my main message here is that I'm hopeful that we can extrapolate
 observations by single relays to network totals, at least if these relays
 are not too slow or see a too small part of descriptor space.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/13192#comment:41>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online