[tor-bugs] #9529 [Tor Check]: Replace check.tp.o webservers with Arlo's Go version

[Tor Bug Tracker & Wiki]

#9529: Replace check.tp.o webservers with Arlo's Go version
-----------------------+----------------------------------------------------
 Reporter:  arma       |          Owner:     
     Type:  task       |         Status:  new
 Priority:  normal     |      Milestone:     
Component:  Tor Check  |        Version:     
 Keywords:             |         Parent:     
   Points:             |   Actualpoints:     
-----------------------+----------------------------------------------------
Changes (by mccajm):

 * cc: adam@… (added)


Comment:

 I would be interested in contributing to this if Arlo would like a hand.
 I'm also able to benchmark if needed.

 500 req/s seems like an _easily_ achievable number.  Although all software
 is different, we've benchmarked similar go services up to 65k req/s.  That
 used 15% of a core and a few hundred MB of RAM.  I need to read
 TorCheck.py further to understand what I'm missing, but from speaking to
 arma the current problems seems to be the wait time: to spawn a cgi
 process and iowait for things like dns.  Is the current Apache compiled
 with a modified MAX_CLIENTS?  If the connections are taking longer than a
 second, Apache will only be able to handle 255 simultaneous connections.

 A few thoughts on the go code:

 - Check seems to serve pages which either occasionally change (like
 RecommendedTBBVersions and exit-addresses) or a segment changes based on a
 dns lookup, such as the index page.  The total amount of data here is
 really small. I would try to eliminate disk reads where possible by
 loading these into a buffer and serving them directly from there.  These
 buffers could be reloaded in response to a SIGHUP.
 - Responses should be gzipped to close connections more quickly.
 - I don't think the mutexes around ExitMap are necessary.  The variable is
 only written to in LoadLists.

-- 
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/9529#comment:5>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online