I spent the last 4 evenings learning about Linux security and putting it to practice right away.
After the attack was discovered and the server was shutdown, I took some time to analyze the attack in detail. You can read about my analysis in a separate blog post here “Debian Linux home server compromised – Discovery and Analysis”.
Shortly after I shutdown the server I took some time to come up with a contingency plan:
- Take down compromised server
- Alert users that system is offline
- Disconnect/disable any network interfaces
- Connect monitor and keyboard to compromised server
- Analyze (disconnected!) compromised server
- Learn from attack
- Re-install server (start from scratch!) with hardened security
- Bring server back online
- Notify users
- Important: pro-actively monitor server and keep installed software updated
The contingency plan differs from situation to situation, from organization to organization (policies), and from person to person (skill) etc. You could for example make an image of the hard disk (or system partition) and mount it read only somewhere, so you can thoroughly inspect it without the risk of making any changes whatsoever. But this is simply a home server with a pretty vanilla Debian Linux installation. So there’s no need to take it to such levels. And there’s plenty of reason to assume that the attacker is only a script kiddie.
After the analysis was done I proceeded by wiping the hard disk (of course, if you’re really paranoid you could consider trashing the hard disk and buy a new one) and installed Debian Linux on the box.
This time I did my homework and the new setup is quiet different as you can tell from the comparison matrix below:
Comparison of old setup versus new setup
| Old setup | New setup | |
|---|---|---|
| Firewall | Not present (behind NAT, couple of ports forwarded) | Rules to allow certain services only from LAN. Rules to block hammering. |
| Intrusion detection | Not present | Installed portsentry and tripwire. Report anomalies to me via email. |
| SSH | Password auth | Disabled password auth. User white list (no root). Limit authentication retries. Keys with pass phrases. Installed fail2ban. |
| Bash | No precaution | Extended default history size. Added time stamping. Protected variables. Append only. Show logged on users whenever I logon. |
| Temp directories | No precaution | Prevent executing commands in /dev/shm, /tmp and /var/tmp. |
| Logfiles | Default logrotate configuration | Installed logcheck. Extended defaults for logrotate to store logfiles longer. |
| Anti-virus and rootkit checkers | Not present | Installed clamav. Installed chkrootkit. Installed rkhunter. Report anomalies to me via email. |
| Password policy | Not present | Configured using PAM. |
I have tested the installation with some basic penetration testing and everything seems to work perfectly!
There’s still some stuff left for me to do such as:
- Automatically poll for new updates and install them.
- Transfer backup archives to a remote server (over SSH+rsync).
- Setup disk/user quota.
I considered reporting anomalies/alerts via SMS, but to be honest I can’t be bothered with it when I’m at work or at school. There are more important things in life then your home server. 
If you’re interested in securing/hardening your Debian Linux installation I suggest you take a look at my bookmarks on Delicious: My bookmarks on Delicious.com tagged security. Look for links in the period from 10th of January to the 14th of January.
Damn Romanians! Always trouble ha ha. Two really excellent entries on security by the way.