[tor-bugs] #16907 [Onionoo]: Stop returning a 500 Internal Server Error when the hourly updater has not run for 6+ hours

[Tor Bug Tracker & Wiki]

#16907: Stop returning a 500 Internal Server Error when the hourly updater has not
run for 6+ hours
-------------------------+------------------------------
 Reporter:  karsten      |          Owner:
     Type:  enhancement  |         Status:  needs_review
 Priority:  Medium       |      Milestone:
Component:  Onionoo      |        Version:
 Severity:  Normal       |     Resolution:
 Keywords:               |  Actual Points:
Parent ID:               |         Points:
  Sponsor:               |
-------------------------+------------------------------

Comment (by karsten):

 I'm still investigating the December 31 issue.  Apparently, there was a
 host issue with fetching data from CollecTor at 01:19 UTC (`"No route to
 host"`), possibly followed by a termination of the back-end process (which
 I don't see in the logs, though), followed by a (manual?) host restart at
 15:30 UTC.  I'm still looking into all that, but I think it's unrelated to
 this ticket, so I'll open another ticket once I know whether it's an
 actual issue in Onionoo.

 So, after some more thinking about the patch above I'm inclined to reject
 it in favor of just returning a 200 status code regardless of how old the
 data is.  Some reasons:
  - Returning a 5xx status code ''and'' including (stale) data seems very
 non-standard to me.  I'd want to see an example of another web application
 using HTTP that way before implementing such a thing.
  - I fear that a non-standard use of HTTP status codes might make it more
 difficult to replicate Onionoo server parts in the future.  For example,
 if we had a separate layer of web caches that get their data from two or
 three Onionoo front-ends, we shouldn't confuse those web caches by
 returning unusual HTTP status codes.
  - It should be up to the application developer of the Onionoo client to
 decide whether data received from the Onionoo server is still recent
 enough.  I can imagine that different applications have different
 requirements, and it shouldn't be decided for them by the server, not even
 indicated by using different status codes.  The maximum age of six hours
 for content was picked rather arbitrary in the early stages of developing
 Onionoo, and it hasn't become less arbitrary since then.  (Onionoo clients
 can look at the `Last-Modified` header to learn how old the contained data
 is, and they can look at the `relays_published` or `bridges_published`
 fields of the content to see how old the data is that was used to build
 the response.)

 Regarding possible complaints about stale data, I think it's up to the
 applications to inform their users about that.  I commented on #15178 for
 changing this in Atlas and created #16908 for Globe four months ago, but
 I'm not aware of any changes to those two applications.  Usually, I'd say
 let's make this a major protocol change and give a one-month heads-up, but
 this behavior of sending a 500 status code after six hours is not even
 specified.  That's why I'm inclined not to change the protocol version at
 all and instead just give a one week heads-up on tor-dev@ before removing
 the six-hour check.

 Please let me know if I overlooked something or whether my reasoning
 doesn't make sense, and I'll reconsider.  Otherwise I'll move forward in a
 few days.  Thanks, everyone!

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/16907#comment:13>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online