How to auto-detect and possibly to stop or prevent a DoS attack?

G'day,

I woke this morning to find my linode server functioning poorly. After a bit of poking around to see what was going on I rebooted the system but the effect was minimal. I logged into the dash-board and among other things I noticed some oddities with a few stats on the graphic display. Two noteworthies were it looked like CPU usage and available disk space were both maxed out. I poked around a bit more and found a 16gb apache log file. Once deleted, and after a bit of tweaking to restore minor damage caused by a full disk, everything seems to be running okay again. This afternoon I received an alert from linode, which read…

Your Linode has exceeded the notification threshold (90) for CPU Usage by averaging 99.8% for the last 2 hours.

The message is timestamped for 2:30 this morning. So the email 'arrived' about 12 hours after the problem took place. (I don't have a problem with the email or the delay because I would have been asleep at the time anyway; and, having the notification is a helpful tool that helps figure out what's going on - so by all means don't stop them from coming in situations like this).

One last note. Somewhere between 11 P.M. and midnight my connection to the server dropped from slow to timeouts. I live in a very remote location and this sort of thing often happens. I didn't think anything of it at the time but it 'may' have signalled the beginning of the problem.

My conclusion from all this is that somewhere between midnight and the time of the linode alert my server was hit with a DoS attack. I host a few domains and as far as I can tell only one of them was hit. Unfortunately, I deleted what was probably the most informative log. I had no choice if I wanted the server back up and running.

Question 1. What methods and strategies are available to auto detect this sort of attack?

I can monitor log file sizes with a cron job and delete anything over a certain limit. But for it to be effective I'd have to run it every half hour or so. That would address the full disk but not glutting the CPU.

Question 2. Is there any point to trying to figure out where the attack came from or who was behind it?

All logs are still available; just not the bloated one I had to remove. But even if I query the logs for activity around 1 AM, for example, wouldn't a tech savvy attacker spoof their point of origin or run the attack through proxies? Is there value in this approach?

At this point in time, I can only surmise the obvious. It was either a random or deliberate attack. If random, the only defense would be improved detection. If deliberate, I/we'd have to prove beyond all doubt where it came from or we're no further ahead.

My first step will be to pass-word protect the area I believe the attacker exploited. It's only a couple small files I use for personal use but we can't have somebody pounding them to malicious effect. After that, I would be grateful if anybody can recommend any ready-made scripts for monitoring logs (I'll make my own if I have to), and of course any measure to auto add attacking sources to firewall or the like.

Thank you. I look forward to hearing about your experiences and recommendations.

PS. It might be important to ask whether anybody else felt the effects of an attack yesterday. And can linode use their tools to shed light on what took place? Thanks again.