[metrics-bugs] #21050 [Metrics/Onionoo]: Onionoo serves invalid cache headers

[Tor Bug Tracker & Wiki]

#21050: Onionoo serves invalid cache headers
---------------------------------+--------------------------
     Reporter:  lukechilds       |      Owner:  metrics-team
         Type:  defect           |     Status:  new
     Priority:  Medium           |  Milestone:
    Component:  Metrics/Onionoo  |    Version:
     Severity:  Normal           |   Keywords:
Actual Points:                   |  Parent ID:
       Points:                   |   Reviewer:
      Sponsor:                   |
---------------------------------+--------------------------
 Onionoo serves invalid cache headers the first time you receive a 304. All
 304 responses after that are ok until the cache max-age is reached. Then
 the next 304 response will have invalid headers and after that it's ok
 again until the new max-age is reached etc.

 They are invalid because the Date header isn't reset once the max-age is
 reached and the Age header carries on incrementing. This results in an age
 that is higher than the max-age, which means the headers are saying the
 content has already expired before it's been sent.

 Example:

 ----

 Make a request at 09:10:32

 {{{
 Response code: 200
 Date: Wed, 21 Dec 2016 09:10:32 GMT
 Age: 0
 Cache-Control: public, max-age=300
 Last-Modified: Wed, 21 Dec 2016 08:23:36 GMT
 }}}

 AKA This content is from 09:10:32 (now) and you can cache it until
 09:15:32 (5 minutes)

 ----

 Now lets make a request at 09:15:41, it's been more than 5 minutes so
 we'll set the If-Modified-Since header to see if our cached version is
 still up to date

 {{{
 Response code: 304
 Date: Wed, 21 Dec 2016 09:10:32 GMT
 Age: 309
 Cache-Control: public, max-age=300
 Last-Modified: Wed, 21 Dec 2016 08:23:36 GMT
 }}}

 AKA This content is from 09:15:41 (now) and you can cache it until
 09:15:32 (9 seconds ago).

 So the headers are saying it was out of date 9 seconds before the response
 was sent. It wasn't, it's still fresh, it's just the Date header wasn't
 updated so the max-age was calculated from an old date.

 ----

 If we make another request with If-Modified-Since set at 09:15:54 (13
 seconds later) everything catches up:

 {{{
 Response code: 304
 Date: Wed, 21 Dec 2016 09:15:41 GMT
 Age: 13
 Cache-Control: public, max-age=300
 Last-Modified: Wed, 21 Dec 2016 08:23:36 GMT
 }}}

 AKA This content is from 09:15:54 (now) and you can cache it until
 09:20:41 (4:47 minutes)

 Notice how the Date header is now correctly set as the date of our
 previous request with the age header calculated from that.

 ----

 This is technically not abiding by the HTTP protocol:

 https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html
 > 14.9 Cache-Control
 >
 > The Cache-Control general-header field is used to specify directives
 that MUST be obeyed by all caching mechanisms along the request/response
 chain.

 In the second request in the example above the max-age in the Cache-
 Control headers is 300 and the age is 309 which means that Onionoos
 caching mechanism isn't obeying the Cache-Control header.

 The end result of this is that the first time a 304 response is sent it
 will never be cached by a client even though the content is still fresh.
 In the above example 3 HTTP requests needed to be made, if the headers
 were set correctly only 2 requests would have been required. Fixing this
 should reduce the amount of traffic that hits the Onionoo server.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/21050>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online