[tor-bugs] #5651 [Metrics Utilities]: Annotation header with descriptor types

[Tor Bug Tracker & Wiki]

#5651: Annotation header with descriptor types
-------------------------------+--------------------------------------------
 Reporter:  atagar             |          Owner:     
     Type:  enhancement        |         Status:  new
 Priority:  normal             |      Milestone:     
Component:  Metrics Utilities  |        Version:     
 Keywords:                     |         Parent:     
   Points:                     |   Actualpoints:     
-------------------------------+--------------------------------------------

Comment(by karsten):

 Replying to [comment:4 atagar]:
 > > For cases 1 to 3 we already have enough context to know what
 descriptor type and version to expect
 >
 > That assumes that these files will always be, say, v3 server
 descriptors? Might they be v2 server descriptors in old versions? Or v4 in
 the future?

 Minor nitpick: they have always been v1 server descriptors, not v3.  The
 current server descriptor format was introduced in version 1 of the
 directory protocol, and it hasn't changed in a backward-incompatible way
 since then.  Keywords have been added, but clients shall ignore keywords
 they don't understand.

 So, if you request a server descriptor via Tor's control or directory port
 using the command or URL that you always used to get a v1 server
 descriptor, and Tor gives you a v2 server descriptor that you can't parse,
 well, then Tor is doing it wrong.  And it would break a lot of things, so
 that's not going to happen, I think.  What I'd expect is that there'd be a
 new command or URL to give you the v2 server descriptor.

 > > for 3), we can derive what descriptors are contained in a cached-*
 file from the filename---it's a hack anyway, because Tor's cached-* files
 are highly implementation-specific and we're messing with them on our own
 risk
 >
 > Yes, it's a hack and reading directly from the data directory is
 completely unsupported by tor. However, I think that this is a bad idea
 and not really necessary.
 >
 > If tor supported, say, "The data directory *may* contain descriptor
 data, identifiable by the first line of its contents." then some use cases
 will no longer require an open control port at all and tor hasn't painted
 itself into a corner in terms of future changes.

 I see the advantage.  The downside, though, is that Tor will create an
 interface that it ''may'' support.  I feel that's not going to happen.
 But I think it's okay to make a best-effort approach for cached-* files.

 > > We also shouldn't mimic Tor's annotation syntax: let's start metrics
 annotations with "# " instead of "@type".
 >
 > Could, though mimicking annotations would make sense if we planned to
 leave the door open for including this with cached descriptors later.

 Right, okay.  We can do `"@type"`.

 > > For example: leaving the original bridge nickname in sanitized
 descriptors instead of "Unnamed" is a minor change (#5684)
 >
 > I've been watching that ticket with some interest since it actually does
 impact stem's parser. On reflection I shouldn't validate bridge scrubbing
 by default - I should move it to an 'is_scrubbed()' method instead...

 Yeah, the scrubbing method isn't set in stone.  Every now and then there
 will be changes like this.  In addition to the nickname thing, there are
 also vague plans to include the country code in the contact line of a
 server descriptor (which doesn't even have a ticket number yet).  And some
 time ago we added the 10.x.x.x thing for IP addresses instead of the
 127.0.0.1 as it was before.  I understand that it's painful to adapt stem
 and metrics-lib to those changes, so I'm trying to avoid them if possible.

 > > The rule would be that we wouldn't parse a descriptor with unknown
 type or higher major version than ours, but that we would parse a
 descriptor with higher minor version, possibly emitting a warning.
 >
 > Sounds good

 Great.

 > > # server-descriptor 1.0
 >
 > Shouldn't this be "server-descriptor-3 1.0"?

 (I don't think so, for the reason stated at the beginning of this
 comment.)

 How do we proceed?

 Should we ask Nick and Roger what they think about including @type
 annotations in descriptors returned by the control port, dir port, and/or
 included in cached-* files?

 And should I specify the annotations for files in metrics tarballs
 somewhere on metrics.tpo and start including them in the existing
 tarballs?  Or is there anything we overlooked?

-- 
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/5651#comment:5>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online