[tor-relays] My VPS relay has just been hacked

Sat Oct 25 15:56:55 UTC 2014

Hey Nick,

Some general advice for securing a VPS;

- disable root ssh login
- disable ssh password login and only allow key based logins 
(https://www.digitalocean.com/community/tutorials/how-to-set-up-ssh-keys--2)
- install fail2ban
- disable / remove any services you aren't using
- use full disk encryption
- use a decent iptables config 
(https://www.linode.com/docs/security/securing-your-server#creating-a-firewall)
- keep the box up to date using unattended upgrades 
(https://wiki.debian.org/UnattendedUpgrades)

Generally fail2ban, key-only login, iptables, and software updates 
should be enough to keep the box secure. You probably got hacked either 
through pwd bruteforcing (which would be prevented by fail2ban or 
disabling pwd login) or by a vulnerability (which would be prevented by 
software upgrades).

I've generally found Edis to be very good (e.g. one of my VPS has an 
uptime > 1 year).

Cheers,

Dan

On 25/10/14 16:36, Nick Sheppard wrote:
> For the last month I've been running a middle relay (no guard flag 
> yet) on a 512 MB VPS provided by Edis.at in Switzerland (4.99 euro per 
> month).  The software is Tor 0.2.5.8-rc with obfsproxy 2 and 3 running 
> on up-to-date Debian Wheezy 7.6.  I left the iptables at their defaults.
>
> Last night Edis suspended my VPS because of a suspected outgoing DDoS 
> attack, and emailed me.  I rang their (very helpful) phone support in 
> London and they de-suspended the VPS so I could ssh into it and 
> investigate. ( I told them it was running a tor non-exit relay - that 
> wasn't a problem.)
>
> Sure enough the Solus control panel traffic graphs show the tor relay 
> traffic stopping abruptly last night, followed about 40 minutes later 
> by an exponentially increasing spike of outgoing traffic, soon cut off 
> when the VPS was suspended.
>
> I ssh'd into the VPS ("last login" showed my own last login, so no 
> problem there) and looked at logs and top.  notices.log simply cut off 
> when the tor traffic stopped, but showed nothing unusual before that.
>
> The Solus control panel traffic graph started showing (a very small 
> amount of) outgoing traffic as soon as the VPS was de-suspended, so I 
> assumed the malware was still active, and used top.
>
> This is typical of what I found.
>
> 1    root 20 0 10604  832  700 S ... 0:00.10 init
> 2    root 20 0     0    0    0 S ... 0:00.00 kthreadd/3277
> 3    root 20 0     0    0    0 S ... 0:00.00 khelper/3277
> 1370 root 20 0 36976  660  492 S ... 0:00.38 hxyqbutesc
> 1400 root 20 0  109m 1616 1188 S ... 0:00.00 rsyslogd
> 1446 root 20 0 18836  884  680 S ... 0:00.00 cron
> 1473 root 20 0 49888 1212  608 S ... 0:00.00 sshd
> 3187 root 20 0 27556  744  504 S ... 0:00.00 vzctl
> 3188 root 20 0 17808 1968 1500 S ... 0:00.00 bash
> 5374 root 20 0 21584 1416 1072 R ... 0:00.00 top
> 5440 root 20 0  6244  536  296 S ... 0:00.00 akcviaxtbl
> 5443 root 20 0  6244  532  296 S ... 0:00.00 akcviaxtbl
> 5446 root 20 0  6244  532  296 S ... 0:00.00 akcviaxtbl
> 5448 root 20 0  6244  536  300 S ... 0:00.00 akcviaxtbl
> 5449 root 20 0  6244  532  296 S ... 0:00.00 akcviaxtbl
>
> There is a strange line near the top for process "hxyqbutesc", which 
> didn't change; and a strange block of lines at the bottom, which 
> changed every second or two. Sometimes it was five similar lines, as 
> above; sometimes it was several block of lines, eg this:
>
> 5276 root 20 0 6244 528 292 S  0.0  0.1 0:00.00 qscntoweqb
> 5277 root 20 0 6244 536 300 S  0.0  0.1 0:00.00 qscntoweqb
> 5280 root 20 0 6244 532 296 S  0.0  0.1 0:00.00 qscntoweqb
> 5282 root 20 0 6244 536 300 S  0.0  0.1 0:00.00 qscntoweqb
> 5283 root 20 0 6244 528 292 S  0.0  0.1 0:00.00 qscntoweqb
> 5290 root 20 0 6244 532 296 S  0.0  0.1 0:00.00 saqizxaihz
> 5294 root 20 0 6244 528 300 S  0.0  0.1 0:00.00 saqizxaihz
> 5295 root 20 0 6244 532 296 S  0.0  0.1 0:00.00 saqizxaihz
> 5296 root 20 0 6244 524 296 S  0.0  0.1 0:00.00 saqizxaihz
> 5298 root 20 0 6244 528 296 S  0.0  0.1 0:00.00 saqizxaihz
>
> Each block is always 5 lines, and the names (always 10 lower-case 
> letters) seem to be different every time.  The blocks change fairly 
> regularly every second or two.
>
> I shut the VPS down to stop it doing any more harm, but I didn't 
> delete anything; I can restart it and ssh in again for further 
> investigation if necessary.
>
> Eventually I'll have to reinstall everything from scratch, 
> straightforward enough, but what can I do to make sure it doesn't 
> happen again?  Would hardening my iptables work?  Has anyone else seen 
> this?
>
> Nick Sheppard
>
>
>
>
> _______________________________________________
> tor-relays mailing list
> tor-relays at lists.torproject.org
> https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays

-- 
Dan Rogers
+44 7539 552349
skype: dan.j.rogers
gpg key <https://secure.techwang.com/gpg/public_key.txt>
linkedin <http://www.linkedin.com/in/danrogerslondon> | twitter 
<http://twitter.com/danjrog> | spotify 
<http://open.spotify.com/user/bonkbonkonk> | music 
<http://holdingitwrong.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.torproject.org/pipermail/tor-relays/attachments/20141025/32ab6c96/attachment.html>