DDoS Mitigation

You can monitor your bandwidth burn rate using http://www.linode.com/forums/viewtopic.php?t=1558, and then have your monitoring stuff (e.g. munin) yell if you're going through too much stuff so you can take manual action. Or, depending on your web server, you might be able to limit a site's bandwidth usage per month/day/hour, if it was HTTP traffic causing the issue. There are also ways to enforce limits using iptables, but I haven't used those in production.

A lot of it boils down to what, exactly, the DDoS was – the trick to mitigating one is figuring out how it is impacting you and then figure out what is common to all the bad traffic (and makes it identifiably different from "good" traffic). It's a somewhat complex field, and there's no stick-this-in-your-rc.local instant fix, unfortunately.

Thanks for the advice, I'll have to a deeper look into the subject I think.

For the record, it was not HTTP traffic, the first thing I did was turn off apache, it seems to have been a UDP flood or some other form of junk packets being sent to the server in high volume.

For the moment though the linode staff have helped control the situation, they've nullrouted the servers IP address until this attack subsides. Hopefully that should keep things in one piece.

Cool. Yup, that's the sort that's a wicked pisser to deal with locally and probably just needs to be handled upstream. Hmm, I should whip up a Munin plugin to monitor bandwidth burn rate…

At least you didn't get hit like the .gov websites did. Burning 45.5GB per second of bandwidth for several hours.

I've had some experience with DDoS attacks. Back about 4 years ago (when I was with an old host) I was the target of a HUGE DDoS attack that brought down part of the RackSpace (at the time) datacenter. Before the Linode owners get nervous, nothing like that will ever happen again.

Anyway, one of the early attacks (and not major) was on an IRCd that I ran. Basically they would have thousands of bots connecting every minute. Luckily I found that none of them replied to CTCP VERSION requests, so I developed a mIRC script that Versioned every connection and glined it if there was no response.

It wasnt a flat out DDoS, but I had this issue where I was almost hitting my bandwidth cap every month, only to check apache logs and find there were all these .br users downloading files from the files directory on my server. I eventually just blocked all .br IPs with IPtables… I removed the entries a few months ago and bandwidth usage hasn't changed so i guess they moved on.

Hoopycat was right, its all about finding what makes them different from the average user. I have heard of various bandwidth-usage limiting techniques, most involve IPTables.