Hey Nick,
Some general advice for securing a VPS;
- disable root ssh login - disable ssh password login and only allow key based logins (https://www.digitalocean.com/community/tutorials/how-to-set-up-ssh-keys--2) - install fail2ban - disable / remove any services you aren't using - use full disk encryption - use a decent iptables config (https://www.linode.com/docs/security/securing-your-server#creating-a-firewal...) - keep the box up to date using unattended upgrades (https://wiki.debian.org/UnattendedUpgrades)
Generally fail2ban, key-only login, iptables, and software updates should be enough to keep the box secure. You probably got hacked either through pwd bruteforcing (which would be prevented by fail2ban or disabling pwd login) or by a vulnerability (which would be prevented by software upgrades).
I've generally found Edis to be very good (e.g. one of my VPS has an uptime > 1 year).
Cheers,
Dan
On 25/10/14 16:36, Nick Sheppard wrote:
For the last month I've been running a middle relay (no guard flag yet) on a 512 MB VPS provided by Edis.at in Switzerland (4.99 euro per month). The software is Tor 0.2.5.8-rc with obfsproxy 2 and 3 running on up-to-date Debian Wheezy 7.6. I left the iptables at their defaults.
Last night Edis suspended my VPS because of a suspected outgoing DDoS attack, and emailed me. I rang their (very helpful) phone support in London and they de-suspended the VPS so I could ssh into it and investigate. ( I told them it was running a tor non-exit relay - that wasn't a problem.)
Sure enough the Solus control panel traffic graphs show the tor relay traffic stopping abruptly last night, followed about 40 minutes later by an exponentially increasing spike of outgoing traffic, soon cut off when the VPS was suspended.
I ssh'd into the VPS ("last login" showed my own last login, so no problem there) and looked at logs and top. notices.log simply cut off when the tor traffic stopped, but showed nothing unusual before that.
The Solus control panel traffic graph started showing (a very small amount of) outgoing traffic as soon as the VPS was de-suspended, so I assumed the malware was still active, and used top.
This is typical of what I found.
1 root 20 0 10604 832 700 S ... 0:00.10 init 2 root 20 0 0 0 0 S ... 0:00.00 kthreadd/3277 3 root 20 0 0 0 0 S ... 0:00.00 khelper/3277 1370 root 20 0 36976 660 492 S ... 0:00.38 hxyqbutesc 1400 root 20 0 109m 1616 1188 S ... 0:00.00 rsyslogd 1446 root 20 0 18836 884 680 S ... 0:00.00 cron 1473 root 20 0 49888 1212 608 S ... 0:00.00 sshd 3187 root 20 0 27556 744 504 S ... 0:00.00 vzctl 3188 root 20 0 17808 1968 1500 S ... 0:00.00 bash 5374 root 20 0 21584 1416 1072 R ... 0:00.00 top 5440 root 20 0 6244 536 296 S ... 0:00.00 akcviaxtbl 5443 root 20 0 6244 532 296 S ... 0:00.00 akcviaxtbl 5446 root 20 0 6244 532 296 S ... 0:00.00 akcviaxtbl 5448 root 20 0 6244 536 300 S ... 0:00.00 akcviaxtbl 5449 root 20 0 6244 532 296 S ... 0:00.00 akcviaxtbl
There is a strange line near the top for process "hxyqbutesc", which didn't change; and a strange block of lines at the bottom, which changed every second or two. Sometimes it was five similar lines, as above; sometimes it was several block of lines, eg this:
5276 root 20 0 6244 528 292 S 0.0 0.1 0:00.00 qscntoweqb 5277 root 20 0 6244 536 300 S 0.0 0.1 0:00.00 qscntoweqb 5280 root 20 0 6244 532 296 S 0.0 0.1 0:00.00 qscntoweqb 5282 root 20 0 6244 536 300 S 0.0 0.1 0:00.00 qscntoweqb 5283 root 20 0 6244 528 292 S 0.0 0.1 0:00.00 qscntoweqb 5290 root 20 0 6244 532 296 S 0.0 0.1 0:00.00 saqizxaihz 5294 root 20 0 6244 528 300 S 0.0 0.1 0:00.00 saqizxaihz 5295 root 20 0 6244 532 296 S 0.0 0.1 0:00.00 saqizxaihz 5296 root 20 0 6244 524 296 S 0.0 0.1 0:00.00 saqizxaihz 5298 root 20 0 6244 528 296 S 0.0 0.1 0:00.00 saqizxaihz
Each block is always 5 lines, and the names (always 10 lower-case letters) seem to be different every time. The blocks change fairly regularly every second or two.
I shut the VPS down to stop it doing any more harm, but I didn't delete anything; I can restart it and ssh in again for further investigation if necessary.
Eventually I'll have to reinstall everything from scratch, straightforward enough, but what can I do to make sure it doesn't happen again? Would hardening my iptables work? Has anyone else seen this?
Nick Sheppard
tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays