[tor-bugs] #22983 [Metrics/metrics-lib]: add a descriptor interface and implementation for web-logs

[Tor Bug Tracker & Wiki]

#22983: add a descriptor interface and implementation for web-logs
---------------------------------+-----------------------------------
 Reporter:  iwakeh               |          Owner:  metrics-team
     Type:  enhancement          |         Status:  needs_revision
 Priority:  Medium               |      Milestone:  metrics-lib 2.1.0
Component:  Metrics/metrics-lib  |        Version:
 Severity:  Normal               |     Resolution:
 Keywords:                       |  Actual Points:
Parent ID:                       |         Points:
 Reviewer:                       |        Sponsor:
---------------------------------+-----------------------------------

Comment (by iwakeh):

 Thanks for review!  I'll get to the details later, but first:

 A similar discussion came up during the work on JSON handling parts of
 CollecTor and metrics-lib.  Thinking about this for a while, I'd say we
 should step back a little and tackle the main discussion to find a
 solution for having a clean API without any (cross-product) code
 duplication:

 Goals:
 1) keep metric-lib API a pure client API
 2) avoid all code duplication throughout Metrics' products

 CollecTor is an unusual metrics-lib client, because CollecTor alters
 descriptors and even creates the final descriptors (e.g., adding
 annotations, sanitizing privacy problematic data).  Thus, there is some
 functional overlap (extracting bytes, (de)compression, JSON processing)
 with metrics-lib.

 Adding another Metrics product for these cross-concerns seems too much
 overhead in terms of dealing with the additional dependency in Metrics'
 products, committing to maintain another product and another public API.
 Extending metrics-lib's API (as with the above sanitize method) breaks 1.
 On the other hand, rewriting parsing code (like for bridge-descs-
 sanitation) or de/compression functionality violates 2.

 Suggestion:
 (hope the following doesn't sound crazy)
 What about granting CollecTor the special role (which it has anyway) by
 providing an 'internal' API in metrics-lib?
 This would mainly consist in adding interfaces in sub-packages of
 o.tp.descriptor that allow access to implementational functionality, which
 are explicitly **not** part of the public (as in published javadoc-API)
 API.

 The 'internal' interfaces serve as the special contract between CollecTor
 and metrics-lib, i.e., CollecTor code should not access metrics-lib's
 implementation, but only the 'internal' and public API to avoid code
 duplication.  (For example, I could envision an interface
 o.tp.d.log.InternalLogDescriptor that enables setting the descriptor's
 byte array etc.  And, in case you worry, I would not add a 'sanitize()'
 method to such an interface.)

 This approach seems to have the benefits a new cross-concerns Metrics
 product could offer without the hassle of another code-base and what all
 is attached to it.
 I think, it also would finally pave the way for trimming down and
 streamlining all CollecTor modules.

 If that makes some sense, I can provide branches for the web-log related
 code for both metrics-lib and CollecTor.  So, we can see how the approach
 works in code.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/22983#comment:13>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online