TCP: request_sock_TCP: Possible SYN flooding on port 80. Sending cookies. Check SNMP counters. what it can mean? also my 2 relays go offline for a few hours once a day, then are restored.
The TCP protocol builds a new connection by the client sending a SYN packet to the server, the server responding with a SYN+ACK packet and then the client responding with an ACK packet:
Client Server | | | -----SYN------> | | | | <---SYN+ACK---- | | | | -----ACK------> | {connection established}
The server must somehow keep track of the connections during that handshake and the naïve way of doing that is having a queue that hold information on them. The client must do the same, by the way.
Now imagine that someone sends a ton of SYN packets to the server, without ever having intention of making a connection and therefore not keeping track of them. The poor server responds with SYN+ACKs and keeps half-open connections in the queue in a hope that the client will send the final ACK packet: but that one will never arrive. That may fill up the queue quickly and if a normal client tries to connect, it can’t because the queue is full. That kind of an DoS attack is based on assymetry between the resources needed by the attacker and the victim: the victim will dedicate its resources to half-open connections, while the attacker just sends SYN packets without ever storing anything.
Attacker Server queue | | [ ][ ][ ][ ][ ][ ] | -----SYN------> | [x][ ][ ][ ][ ][ ] | <---SYN+ACK---- | [x][ ][ ][ ][ ][ ] | -----SYN------> | [x][x][ ][ ][ ][ ] | <---SYN+ACK---- | [x][x][ ][ ][ ][ ] | -----SYN------> | [x][x][x][ ][ ][ ] | <---SYN+ACK---- | [x][x][x][ ][ ][ ] | -----SYN------> | [x][x][x][x][ ][ ] | <---SYN+ACK---- | [x][x][x][x][ ][ ] | -----SYN------> | [x][x][x][x][x][ ] | <---SYN+ACK---- | [x][x][x][x][x][ ] | -----SYN------> | [x][x][x][x][x][x] full | <---SYN+ACK---- | [x][x][x][x][x][x] full ' | . | . Honest client | . | | . | -----SYN------> | [x][x][x][x][x][x] full :( :(
Here the story of “Possible SYN flooding on port 80” ends. That is what a SYN flood is.
A way to prevent that type of an attack is to not store information on the server. Instead, the server uses the protocol in a creative way to store the information in the SYN+ACK packet itself (in the sequence number, as SYN cookies). A honest client must then respond with an ACK packet that contains a predictably modified information, which can be used to reconstruct the original one. Only then actual resources are assigned to the connection. Of course an attacker may till simply open a ton of connections, but this way there is a symmetry between resources required by both sides. Yes, the server may have their resources exhausted by the attacker, but the attacker must allocate as much resources to the attack. That method, however, has its drawbacks and therefore is applied only if an actual attack is suspected. That’s what “Sending cookies” means.
What causes your nodes to go down is harder to tell. If both events are clearly correlated, something may be actually making a large number of connections, overloarding the nodes and making them unavailable to honest clients. That has happened in the past, with hundreds and hundreds of simulaneaous connections suddenly appearing from a limited range of addresses. SYN cookies will not prevent that in any way (it’s not their job), but the SYN flood prevention mechanism firing may be a sign that a ton of TCP connections are being established fast.
You may try to debug that by collecting information on connections to you node. E.g. by executing `ss` (possibly `netstat` on some systems) periodically (for example each 15s) to count connections to the node (and only them), counting them and logging that information over many days, including some unwanted events you have described, and then see if you see any correlation.
BTW: using Latin script and English for the title will increase chances of getting attention. Otherwise people may even simply filter out your messages “visually”, assuming it’s spam. :)