Preventing Server Attacks

First of all, I don’t claim to be an expert in linux, server management, or security.  But I’ve been doing this a long time and have dealt with many MANY problems and always seem to find a solution.  However, I’ve mostly been working on other people’s servers or working on VPS (Virtual Private Server) or a managed server, and so often didn’t have to deal with “hacking” (for lack of a better term).  However, I’ve purchased a few of my very own servers over the past few years, and just this last month I started running into some of these problems.  My goal here is just to outline the problems I face(d) and how I solve(d) them.

Symptoms

Well, basically my server was unreachable (could not ping or log in via SSH or even remote console).  I restarted the server, and everything was OK.  It went down again within minutes, so I restarted from the remote console, disabled the networks and had a look at the log files.

Shell Login Attempts

I was getting about 30 shell login requests per minute.  Ok, so I looked on the net for a solution, and it was clear: IP tables.  But how to automate login attempts and ban the IP addresses?  fail2ban.  After installing and configuring, my server was back on track for the next 2 days or so.

SMTP Login Attempts

Ok, so now we got the same thing going on: multiple failed attempts at SMTP login.  My servers uses dovecot to handle mail, so I did some research as how to do something similar as fail2ban for these types of requests.  My answer was found at the dovecot wiki.  So, I got that all set up and working, and again, my server was ok for a day or two.

Reverse DNS-Lookup

So lat night my server was again unreachable, so I again logged in via remote console, closed the networks and had a look at the log files.  It seemed that I was getting quite a few reverse DNS lookup requests.  Here it gets a bit tricky, and I’m not 100% sure of my solution, or even if it is the biggest of problems.

In this case, I’m referring to asking an IP address which domain name/server name is the default.  This is actually an important feature for mass-email programs (newsletters and such).  When a relatively large e-mail provider (say, gmail), sees that 100,000 of its users are receiving similar mails, then it’s probably going to check out which IPs are sending them out, and maybe even make sure that the IPs are linked with the domain in the From: MIME header.  That’s why I always try to use trusted mailing-list providers when I or a customer wants to send out mass e-mails.  But that’s another story for another time.

In any event, based on my research, I concluded that indeed this was a form or hack on my system.  I concluded that because my error messages were exactly like the ones described at Boston Conman.  He describes it there in quite good detail, and concludes in part 2 that a good firewall should take care of this.

Firewall

So I went into aptitude and searched for “firewall”.  The first one that came up was apf-firewall.  It seemed pretty decent, modern, easy, and powerful, so I gave it a shot.  I went through the config file and set it up to meet my needs.  This is an IMPORTANT step, as it explains every feature in good detail, and the default settings are such that it won’t work.  So you really have to read the fine print, and most often a value will be suggested to you.

Firewall + fail2ban

I now knew that I had 2 programs communicating with IP tables, and this could be problematic.  So after a quick search, I discovered how to use apf-firewall and fail2ban together.

Apache mod_security

I mention this first here perhaps only for hierarchical reasons, but actually the first thing I did in this whole process was install mod_security for my Apache server.  However, after installing apf-firewall and fail2ban, I decided to see if mod_security would also play nice.  It turns out the fail2ban wiki has all the answers.

Conclusion

I just had to write this article now, because I noticed I’ve already put a bunch of work into this, and of course it could be that in 2 days my server will again come crashing down.  But I see this at the moment as a “good thing”.  I’m finally getting my feet wet, so to speak, in real-life server security.  It’s only very strange that none of my other servers has this problem, although I went with a pre-install with plesk (ugh! don’t get me started), and they have their own firewalls built in.

If you’re inclined to bullet-points:

• mod_security for Apache web server
• firewall (I chose apf-firewall)
• fail2ban (configured to work with mod_security, dovecot and firewall)

So, I hope this to be the end of my “hacker woes”, but if not, I’ll keep you updated!

Update

So, everything worked fine again for a few days, and again the server became unreachable, even through the remote serial console.  I ordered a new server and moved everything over.  That was 3 days ago, and since then, I’ve had no problems.  Therefore, I would conclude that I had a bad modem, despite all the successful hardware tests.  Oh well, at least I got to whet my whistle in the wonderful world of server protection.  I hope you had as much fun as I did!