[tor-bugs] #6395 [Metrics Data Processor]: Can we remove GetTor statistics from metrics-*?

[Tor Bug Tracker & Wiki]

#6395: Can we remove GetTor statistics from metrics-*?
------------------------------------+---------------------------------------
 Reporter:  karsten                 |          Owner:     
     Type:  task                    |         Status:  new
 Priority:  normal                  |      Milestone:     
Component:  Metrics Data Processor  |        Version:     
 Keywords:                          |         Parent:     
   Points:                          |   Actualpoints:     
------------------------------------+---------------------------------------

Comment(by karsten):

 Replying to [comment:4 phobos]:
 > I use gettor stats as part of a funding pitch to explain how people
 acquire tor. All I really need however, is to answer the question of "what
 percent of downloads is from https, smtp, xmpp, or bittorrent?" If we can
 answer this question, I'm find with dropping the gettor graphs themselves.

 We can't answer that question.

 > I also don't understand how this is too much to maintain. It sounds like
 gettor was broken, not the metrics part.

 Metrics was affected, because GetTor provided ill-formatted data that made
 metrics choke and send me hourly error emails, so I had to disable parsing
 temporarily.  kaner and I exchanged a few (Trac) emails to track down the
 problem.  When GetTor worked again I had to manually merge its old and new
 data and re-enable parsing on metrics.  This incident cost me a few hours
 overall.

 Apart from that, every piece of code of course needs maintenance.  The
 quickly written GetTor parsing code in metrics-* is no exception there.
 If it breaks, I need to fix it.  Also, this code makes it harder for me to
 refactor metrics-* which is desperately needed to make it easier to
 "automate graphing xy" as Roger asks me to do every few months.  And if I
 want to provide a library to parse all metrics code or document all
 metrics documents on a website, that always includes GetTor statistics,
 too.  If I can throw out parts from metrics-* that are mostly unneeded,
 I'd really prefer to do that.

 > The current gettor installation is a mess and needs to be moved to a new
 server. Gettor probably needs to be rewritten and to provide stats via
 some other method than http.

 How exactly is a GetTor rewrite not going to generate work on my side to
 adapt the metrics side of things?

 What makes me so sad about this thread is that maintenance is taken for
 granted, and that we don't have any process for removing less used
 features.  "Oh, but it's working, why remove it" is not helpful as a
 reply.  No, we cannot keep every feature we ever built.

 So, how do we proceed here?  I'd really, really prefer if GetTor
 statistics on metrics went away.  If I write a small Python script that
 graphs GetTor statistics and we add that to GetTor's repository to be
 maintained by the GetTor maintainer, can we then keep it out of metrics?

-- 
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/6395#comment:7>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online