[metrics-bugs] #21315 [Circumvention/Snowflake]: publish some realtime stats from the broker?

[Tor Bug Tracker & Wiki]

#21315: publish some realtime stats from the broker?
-------------------------------------+-------------------------------
 Reporter:  arma                     |          Owner:  (none)
     Type:  enhancement              |         Status:  needs_review
 Priority:  Medium                   |      Milestone:
Component:  Circumvention/Snowflake  |        Version:
 Severity:  Normal                   |     Resolution:
 Keywords:                           |  Actual Points:
Parent ID:  #29461                   |         Points:
 Reviewer:  irl                      |        Sponsor:  Sponsor28-can
-------------------------------------+-------------------------------
Changes (by cohosh):

 * status:  needs_revision => needs_review


Comment:

 After looking at #30731, I want to change the proposed collected metrics
 to match the data shown in
 [https://trac.torproject.org/projects/tor/ticket/30693#comment:3 these
 graphs]. I modified the Rscript in those tickets slightly to bin to the
 nearest multiple of 8 and the results are almost identical (due to how
 often proxies are polling). We can get away with even coarser binning but
 I think the multiple of 8 method does enough to disguise individual client
 traffic.

 In addition to this I think it would be interesting to collect geoip data
 on the '''available proxies'''. I don't want to bin snowflake proxies at
 this time because we have too few for this count to be useful at the
 moment, maybe we'd want to add it later but I don't think we need to from
 a client safety perspective. The overall proxy count won't leak client
 information because it only measures whether a proxy has polled at all in
 a 24 hour period and '''not''' whether it was given out (which would leak
 client usage data).

 So I'll propose the following metrics (gathered at a granularity of every
 24 hours):

 {{{
     "snowflake-stats-end" YYYY-MM-DD HH:MM:SS (NSEC s) NL
         [At most once.]

         YYYY-MM-DD HH:MM:SS defines the end of the included measurement
         interval of length NSEC seconds (86400 seconds by default).

     "snowflake-ips" CC=NUM,CC=NUM,... NL
         [At most once.]

         List of mappings from two-letter country codes to the number of
         unique IP addresses of snowflake proxies that have polled.

     "proxy-idle-count" NUM
         [At most once.]

         A count of the number of times a proxy has polled but received
         no client offer, rounded up to the nearest multiple of 8.

     "client-denied-count" NUM
         [At most once.]

         A count of the number of times a client has requested a proxy
         from the broker but no proxies were available, rounded up to
         the nearest multiple of 8.

     "client-proxy-match-count" NUM
         [At most once.]

         A count of the number of times a client successfully received a
         proxy from the broker, rounded up to the nearest multiple of 8.
 }}}

 I'm going to start implementing these metrics and meanwhile put this in
 needs_review for the metrics team to look at.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/21315#comment:19>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online