[tor-bugs] #21303 [Internal Services/Tor Sysadmin Team]: monitor our fastly usage for early warning of overage charges

[Tor Bug Tracker & Wiki]

#21303: monitor our fastly usage for early warning of overage charges
-----------------------------------------------------+------------------
     Reporter:  arma                                 |      Owner:  hiro
         Type:  task                                 |     Status:  new
     Priority:  Medium                               |  Milestone:
    Component:  Internal Services/Tor Sysadmin Team  |    Version:
     Severity:  Normal                               |   Keywords:
Actual Points:                                       |  Parent ID:
       Points:                                       |   Reviewer:
      Sponsor:                                       |
-----------------------------------------------------+------------------
 https://docs.fastly.com/api/stats says that we can fetch our fastly stats
 in an automated way:
 {{{
 curl -H "Fastly-Key: api-key" https://api.fastly.com/stats/usage
 }}}
 gives me
 {{{
 {"data":{"africa":{"bandwidth":0,"requests":0},"anzac":{"bandwidth":21746725539,"requests":36556},"asia":{"bandwidth":236910848733,"requests":371271},"europe":{"bandwidth":62146174145799,"requests":90885278},"latam":{"bandwidth":14281347382,"requests":25199},"usa":{"bandwidth":12092179356871,"requests":18200808}},"status":"success","msg":null,"meta":{"from":"2016-12-24
 19:08:47 UTC","to":"2017-01-24 19:08:47 UTC","by":"day","region":"all"}}
 }}}

 where api-key is a secret number that we can get from our fastly account
 (let me know and I'll tell you our current number).

 We should run this fetch periodically, e.g. daily or several times a day,
 and use it to notice if our numbers are way bigger than we expect them to
 be.

 Specifically, fastly has given us $20k of free money each month, and we're
 using around $5k-6k of it each month, and I don't know what happens if we
 suddenly use a lot more than $20k, but it could be ugly.

 weasel suggests that if we write a script we can put in cron that writes a
 file, with the first line being (OK|WARNING|CRITICAL|UNKNOWN) and then
 more info on the second line, that would be easy to glue into the current
 monitoring and reporting infrastructure.

 It's probably 20 lines of python for the person who knows how to import
 json and add up the bandwidth numbers and compare them to a set of
 thresholds.

 Alternatively, it's possible that the Internet has this script already
 written and maintained.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/21303>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online