Firewall method (fail2ban, DenyHosts, iptables-only, other?)
At the moment, I'm building a LAMP server with email (Postfix/Dovecot) in order to host my own projects, starting with a blog but moving towards indie game development down the track, so I'm trying to be up to small business levels of functionality. Whatever I host there it's going to be just me who needs access, so I can limit things down pretty severely. Once I get this Linode running smoothy, I also don't want to have to spend too much of my time doing basic maintenance. But on the flipside I do want to at least have a basic understanding of what's going on so I can actively avoid serious issues.
Currently I think I've locked down SSH access pretty tight. You can only log in as a single specified non-root user with a public access key (no password access) on a non-standard port. I think the only further thing I could do is lock down the IPs, but given my home ISP access doesn't have a static IP and there's a not insignificant chance I might need to change ISPs in the future, I think that might be overkill. I don't think it's likely someone will break in via SSH, but I don't know enough about problems with other ports or whether I need to worry about attacks being an issue simply because of their sheer numbers.
I'm thinking from what I've read that for firewalls, I probably just need a simple method of stopping brute force attacks after a certain number of failed attempts.
My current firewall is a very simple iptables bash script that I wrote from the info in this tutorial and linked to from /etc/rc.local: http://www.howtoforge.com/linuxiptablessarge
Here's the code, modified a bit in case anyone uses it with their SSH port and IP address (note though that I'm brand new at this, so use at your own risk
#!/bin/sh
iptables=/sbin/iptables
myip= # insert your server IP address in here!
mysshport=22 #or whatever you changed it to!
# clear the tables just in case anything was already set...
$iptables --flush
# make sure I can log in...
# SSH
$iptables -A INPUT -p tcp --dport $mysshport -j ACCEPT
# the basics
$iptables -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
$iptables -A FORWARD -i eth0 -m state --state RELATED,ESTABLISHED -j ACCEPT
$iptables -A OUTPUT -m state --state NEW,RELATED,ESTABLISHED -j ACCEPT
# anything I want to reject (make sure it's not me!)
# bad guys, spammers, add them here...
# MySQL (remote access)
# do I need access to MySQL? Might need to fix this line if I do...
# $iptables -A -d $myip -p tcp --dport 3306 -j ACCEPT
# Email/Postfix
$iptables -A INPUT -d $myip -p tcp --dport 25 -j ACCEPT
# HTTP/Apache
$iptables -A INPUT -d $myip -p tcp --dport 80 -j ACCEPT
# SSL/Apache
$iptables -A INPUT -d $myip -p tcp --dport 443 -j ACCEPT
# IMAP
$iptables -A INPUT -d $myip -p tcp --dport 587 -j ACCEPT
# IMAPS
$iptables -A INPUT -d $myip -p tcp --dport 993 -j ACCEPT
# Localhost traffic
$iptables -A INPUT -d $myip -s 127.0.0.1 -j ACCEPT
# ICMP/Ping
$iptables -A INPUT -d $myip -p icmp -j ACCEPT
# Reject everything else
$iptables -A INPUT -j REJECT
$iptables -A OUTPUT -j REJECT
$iptables -A FORWARD -j REJECT
If I understand what I've done correctly, I've allowed access to the ports I need while blocking everything else, which is a pretty straightforward place to start.
Now I want to put in a temporary block for anyone trying to do repeat requests, but not in such a way that it's likely that it will block me out.
I've had a look at fail2ban, but I was finding it hard to understand how to configure it, or more accurately how to configure it for what I wanted to do. I found some tutorials on their website nothing that went through and told me explicitly what each option did, which made me a bit nervous given I wanted to set it to a non-standard SSH port.
I've also heard about DenyHosts, which I'm reading about now but am unsure if it's really what I need. It's not iptables based either, which may or may not be a good thing - I'm not sure.
The other option which I've seen mentioned is I could just write a few custom iptables lines myself, which I guess would go under the "bad guys" comment I've made in my simple iptables script. I'm not entirely sure what those lines should be, though
I don't think I need anything too fancy, but then again I'm new at this and I'm not really sure what level of action I should be taking here. I'd like to hear what recommenations more experienced server administrators would make to a beginner in my situation.
Thanks!
26 Replies
At this point, if I were in your position I would probably put off learning/installing fail2ban or denyhosts, unless logs showed high numbers of failed login attempts. With password login disabled, you don't have to be concerned about the password-guessing robots, and changing ports probably deflects most attempts off the bat. You can always save this for later and learn about it* when you have less on your plate. Another option along these lines might be to create a rate-limiting rule in iptables to limit the number of new ssh connections per minute. The downside to all these is the possibility of locking yourself out, so it'd be wise to take the time to understand before deploying.
(If this server held sensitive personal information or similar high-value data I might be more paranoid and change my tune, but it sounds like you're just setting up an ordinary web/e-mail server.)
*And there's plenty to learn - for example, fail2ban works by inserting iptables rules to block traffic from bad hosts, while denyhosts uses /etc/hosts.deny which blocks attempts via TCPwrappers.
The only thing I'm a bit unsure about is a Denial of Service attack, given I don't yet have any iptables rules to throtte down traffic. I was hoping it would be relatively straightforward to find the right iptables rules to do that, but it doesn't seem to be the case. I've found similar rules, but it requires a fair amount of reading to understand what they actually throttle. And I think I've overloaded my brain this week learning all this new stuff.
My gut feeling is to stick with iptables based approaches, as the more modules I need for a firewall the greater the chance one of them will go wrong. I've been looking at iptables generators to help me, but most of them seem to get complicated very quickly, often assuming you're building a multi-zone system.
Currently I've found FireHOL which has given me a good first impression. It looks the closest thing to approachable as these firewalls seem to get to a newbie.
#!/bin/sh
iptables -P FORWARD DROP
iptables -P INPUT ACCEPT
iptables -A INPUT -i eth0 -p tcp --syn --destination-port 0:79 -j DROP
iptables -A INPUT -i eth0 -p tcp --syn --destination-port 81:442 -j DROP
iptables -A INPUT -i eth0 -p tcp --syn --destination-port 444: -j DROP
@zunzun:
My current iptables script for
http://zunzun.com :#!/bin/sh iptables -P FORWARD DROP iptables -P INPUT ACCEPT iptables -A INPUT -i eth0 -p tcp --syn --destination-port 0:79 -j DROP iptables -A INPUT -i eth0 -p tcp --syn --destination-port 81:442 -j DROP iptables -A INPUT -i eth0 -p tcp --syn --destination-port 444: -j DROP
Perhaps this is good for you, but this is no good for LAMP server, it needs at least to allow incoming port 25 and maybe port 110/143/465/993 for roaming users to send/receive e-mails on a move.
Coming back to the topic. Personally, I have changed the port number from 22 to ….and immidiately, I have much less brute force attacks on my server.
I know that the firewall I've currently got works, in the sense that I can do everything I need to do (send and receive emails, view web pages). I'm just moderately paranoid that I haven't yet limited enough of what it shouldn't do.
But I haven't had enough experience of administering a server let loose in the wild. If locking down SSH and switching the port is enough for a normal small site, maybe I'm being overconcerned and should be moving on to other areas.
@zunzun:
Coming back to the topic. Personally, I have changed the port number from 22 to ….and immidiately, I have much less brute force attacks on my server.
Same here. i had thousands and thousands of brute-force attempts consuming bandwidth and CPU but as soon as i moved the port to … it was all quiet.
@trazoi:
# Reject everything else $iptables -A INPUT -j REJECT $iptables -A OUTPUT -j REJECT $iptables -A FORWARD -j REJECT
As a security admin, I usually prefer to use a special REJECT instead of a standard REJECT or DROP.
# Reject everything else
$iptables -A INPUT -j REJECT --reject-with icmp-host-unreachable
$iptables -A OUTPUT -j REJECT --reject-with icmp-host-unreachable
$iptables -A FORWARD -j REJECT --reject-with icmp-host-unreachable
My reason for doing this is more obscurity, but it does help with scanners like nmap. If you use a standard REJECT then you are probably going to return a REJECT packet for TCP and an admin-prohibited for everything else. This lets an attacker know you are using an ACL and there could be an open port, so they will keep scanning. If you use a DROP, the only time this should happen is where there is a firewall in place between the two hosts. They also help aid DOS attacks when combined with spoofed source addresses.
Sending an icmp-host-unreachable packet is slightly different. It fools most people and tools. Nmap, for example, will stop scanning if it receives a host unreachable packet. It doesn't care that it came from the IP address that it is scanning. When a person will ping an IP address they receive a reply, such as:
H:\>ping host.domain.tld
Pinging host.domain.tld [172.16.0.1] with 32 bytes of data:
Reply from 172.16.0.1: Destination host unreachable.
Reply from 172.16.0.1: Destination host unreachable.
Reply from 172.16.0.1: Destination host unreachable.
Reply from 172.16.0.1: Destination host unreachable.
Ping statistics for 172.16.0.1:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
That reply will usually fool somebody who isn't expecting it and even some that are.
Using icmp-host-unreachable will also reduce the ability to use your system as a DOS contributor. When an attacker sends a packet to DOS an attackee, they can forge your IP address, causing the attackee to wait for your response to complete the connection. A DROP will cause the attackee to leave memory open waiting for the response that you will never send. icmp-host-unreachable will immediately free that memory.
> We will show three 0-day denial-of-service attacks caused by remote log injection on BlockHosts, DenyHosts and fail2ban.
*This paper talks about remote log injection, where an external attacker can modify a log, based on the input it provides to an application (in our case OpenSSH and vsftpd). By modifying the way the application logs, we are able to attack these log analysis tools. We are not talking about local log modification or "syslog injection".
ConfigServer Firewall
csf is an iptables front-end and works with a Login Failure Daemon (ldf).
This is the default cpanel firewall that we can run as a generic firewall.
@jbglenn:
http://www.ossec.net/en/attacking-loganalysis.html
DoS attacks are hard to defend against. One more reason to appreciate Linode's out-of-band console access. :-)
@Vance:
@jbglenn:
http://www.ossec.net/en/attacking-loganalysis.html
DoS attacks are hard to defend against. One more reason to appreciate Linode's out-of-band console access.:-)
So Lish works even under a complete DoS hammering? That's good to know. Given I'm positive I can't predict all the possible ways something could go wrong it's nice to know I've got a way in to correct things in any condition.
@trazoi:
$iptables -A FORWARD -i eth0 -m state --state RELATED,ESTABLISHED -j ACCEPT
If you're not acting as a router, then you should change this to:
iptables -P DROP
@trazoi:
# MySQL (remote access) # do I need access to MySQL? Might need to fix this line if I do... # $iptables -A -d $myip -p tcp --dport 3306 -j ACCEPT
:shock: You're allowing anyone on the internet access to login to MySQL? I hope you've got a secure password, and there's no 0-day vulnerabilities…
@fukawi2:
:shock: You're allowing anyone on the internet access to login to MySQL? I hope you've got a secure password, and there's no 0-day vulnerabilities…
I hope not:) . That line was commented out, so if I understand the table rules the general catch-all "drop everything" line at the end should come into effect. Please correct me if I'm wrong, because I don't want MySQL to be open to the internet.
@trazoi:
@Vance:
@jbglenn:
http://www.ossec.net/en/attacking-loganalysis.html
DoS attacks are hard to defend against. One more reason to appreciate Linode's out-of-band console access. :-)
So Lish works even under a complete DoS hammering? That's good to know. Given I'm positive I can't predict all the possible ways something could go wrong it's nice to know I've got a way in to correct things in any condition.
While flooding a host with traffic is one type of DoS attack, there are others, like the ones described in the article to lock you out by tricking your installation of fail2ban or denyhosts. Lish and the AJAX console let you get into your Linode even if the firewall configuration or hosts.deny has been subverted to keep you out.
I don't think there's much an individual host owner can do to deal with a large traffic flood; someone upstream will have to (try to) block the traffic before it enters their network.
* Edit the sshdconfig file so that root login is not allowed, and only allow users from a specific group to get access.
````
PermitRootLogin no
AllowGroups specialfriends
````
You will need to create the 'special_friends' group (name it what you like), and make sure that your regular user is added to that group. Then you can restart the ssh server:
````
groupadd -g 9000 special_friends
usermod -aG ssh_allow <user_name></user_name>
````
I learned this method right <url url="http://www.linode.com/forums/viewtopic.php?t=4288">~~[](http://www.linode.com/forums/viewtopic.php?t=4288)~~[here on this board](http://www.linode.com/forums/viewtopic.php?t=4288)</url>.
This set of rules in iptables:
iptables -A INPUT -p tcp --dport 22 -m state --state NEW -m recent --set \ --name SSH iptables -A INPUT -p tcp --dport 22 -m state --state NEW -m recent --update \ --seconds 60 --hitcount 4 --rttl --name SSH -m limit --limit 2/min -j LOG \ --log-prefix "SSH_brute_force: " iptables -A INPUT -p tcp --dport 22 -m state --state NEW -m recent --update \ --seconds 60 --hitcount 4 --rttl --name SSH -j DROP
This method is adapted from
andhere . It limits a single ip address to 3 logins within 60 seconds (no distinction for good or bad guys, just 3 in that time limit). The logging is limited to two log notices per minute. I picked up the logging limit trick (which will cut down on 24 hour brute force attacks filling your /var/log/syslog with useless crap) from thishere .nice tutorial
You can do even more (limit ssh login to people who have public keys on your system, only allow specific ip ranges, change the ssh port, use portknocking), but I find that for the moment, this is a good balance for me of security and manageability. Check your logs (/var/log/auth and /var/log/syslog) regularly, and if you're messing with iptables rules, be patient. You may lock yourself out, but usually a manual reboot will fix you there. (Unless you've already set the machine to load iptables rules automatically whenever the network starts. Don't do that until you're pretty sure of your rule-set.)
@Telemachus:
You may lock yourself out, but usually a manual reboot will fix you there. (Unless you've already set the machine to load iptables rules automatically whenever the network starts. Don't do that until you're pretty sure of your rule-set.)
No need to do that, you can just usewhich doesn't use a network interface. Lish
@zengei:
@Telemachus:You may lock yourself out, but usually a manual reboot will fix you there. (Unless you've already set the machine to load iptables rules automatically whenever the network starts. Don't do that until you're pretty sure of your rule-set.)
No need to do that, you can just usewhich doesn't use a network interface. Lish
Yup, I still haven't gotten out of my old habits. Thanks for the reminder about Lish.
@Telemachus:
It limits a single ip address to 3 logins within 60 seconds (no distinction for good or bad guys, just 3 in that time limit).
Word of warning: This has bitten me in the ass more then it has helped. bash-completion on scp and this do not mix well.
@mwalling:
@Telemachus:It limits a single ip address to 3 logins within 60 seconds (no distinction for good or bad guys, just 3 in that time limit).
Word of warning: This has bitten me in the ass more then it has helped. bash-completion on scp and this do not mix well.
I hear you, but for me - so far - it's working well. I'm not doing the kind of work into the box via scp that you may be. I tend to log in once or twice, work in that terminal while checking, eg, css changes in a web browser. So far, I'm good. But I will keep my eye on it.
@Zengei: I've used public key auth, but that has bitten me. Moving to a new machine to work and not having a key-stick. That's a pain.
No perfect solution for every situation, probably.
@zengei:
As for myself, I only allow public key authentication so brute forcing is basically useless but I was annoyed by all the failed authentication messages in my auth.log so I use iptables to restrict access to port 22 from IP addresses in my current /16 range. If my dynamic IP address changes to something outside the specified ranges I just login via Lish and add the new range.
I use iptables to restrict port 22 to two of my ISP's netblocks, but I also have sshd running unrestricted on other ports.
@mnordhoff:
@zengei:As for myself, I only allow public key authentication so brute forcing is basically useless but I was annoyed by all the failed authentication messages in my auth.log so I use iptables to restrict access to port 22 from IP addresses in my current /16 range. If my dynamic IP address changes to something outside the specified ranges I just login via Lish and add the new range.
I use iptables to restrict port 22 to two of my ISP's netblocks, but I also have sshd running unrestricted on other ports.
Wow, that's a really good idea, thanks. And it'll work on a physical server as well where you don't have the benefit of Lish.