My sshd was bruteforced!
Looks like it has been hacked.
Here's a snipplet of auth.log (notice, that log time is gmt+10):
Feb 6 14:18:04 samolet sshd[10763]: Invalid user students from 69.164.208.111
Feb 6 14:18:04 samolet sshd[10763]: pam_unix(sshd:auth): check pass; user unknown
Feb 6 14:18:04 samolet sshd[10763]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=li123-111.members.linode.com
Feb 6 14:18:06 samolet sshd[10763]: Failed password for invalid user students from 69.164.208.111 port 47634 ssh2
Feb 6 14:18:08 samolet sshd[10765]: Invalid user students from 69.164.208.111
Feb 6 14:18:08 samolet sshd[10765]: pam_unix(sshd:auth): check pass; user unknown
Feb 6 14:18:08 samolet sshd[10765]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=li123-111.members.linode.com
Feb 6 14:18:10 samolet sshd[10765]: Failed password for invalid user students from 69.164.208.111 port 49052 ssh2
Feb 6 14:18:12 samolet sshd[10767]: Invalid user students from 69.164.208.111
Feb 6 14:18:12 samolet sshd[10767]: pam_unix(sshd:auth): check pass; user unknown
Feb 6 14:18:12 samolet sshd[10767]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=li123-111.members.linode.com
Feb 6 14:18:14 samolet sshd[10767]: Failed password for invalid user students from 69.164.208.111 port 50710 ssh2
Feb 6 14:18:17 samolet sshd[10769]: Invalid user students from 69.164.208.111
Feb 6 14:18:17 samolet sshd[10769]: pam_unix(sshd:auth): check pass; user unknown
Feb 6 14:18:17 samolet sshd[10769]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=li123-111.members.linode.com
Feb 6 14:18:19 samolet sshd[10769]: Failed password for invalid user students from 69.164.208.111 port 52243 ssh2
Feb 6 14:18:21 samolet sshd[10771]: Invalid user squid from 69.164.208.111
Feb 6 14:18:21 samolet sshd[10771]: pam_unix(sshd:auth): check pass; user unknown
Feb 6 14:18:21 samolet sshd[10771]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=li123-111.members.linode.com
Feb 6 14:18:23 samolet sshd[10771]: Failed password for invalid user squid from 69.164.208.111 port 53818 ssh2
Feb 6 14:18:25 samolet sshd[10773]: Invalid user squid from 69.164.208.111
Feb 6 14:18:25 samolet sshd[10773]: pam_unix(sshd:auth): check pass; user unknown
Feb 6 14:18:25 samolet sshd[10773]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=li123-111.members.linode.com
Feb 6 14:18:27 samolet sshd[10773]: Failed password for invalid user squid from 69.164.208.111 port 55409 ssh2
Feb 6 14:18:29 samolet sshd[10775]: Invalid user support from 69.164.208.111
Feb 6 14:18:29 samolet sshd[10775]: pam_unix(sshd:auth): check pass; user unknown
Feb 6 14:18:29 samolet sshd[10775]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=li123-111.members.linode.com
Feb 6 14:18:31 samolet sshd[10775]: Failed password for invalid user support from 69.164.208.111 port 56912 ssh2
Feb 6 14:18:33 samolet sshd[10777]: Invalid user support from 69.164.208.111
Feb 6 14:18:33 samolet sshd[10777]: pam_unix(sshd:auth): check pass; user unknown
Feb 6 14:18:33 samolet sshd[10777]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=li123-111.members.linode.com
Feb 6 14:18:35 samolet sshd[10777]: Failed password for invalid user support from 69.164.208.111 port 58437 ssh2
Feb 6 14:18:37 samolet sshd[10779]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=li123-111.members.linode.com user=sys
Feb 6 14:18:39 samolet sshd[10779]: Failed password for sys from 69.164.208.111 port 60046 ssh2
Feb 6 14:18:41 samolet sshd[10781]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=li123-111.members.linode.com user=sys
Feb 6 14:18:43 samolet sshd[10781]: Failed password for sys from 69.164.208.111 port 33356 ssh2
Feb 6 14:18:45 samolet sshd[10783]: Invalid user sysadmin from 69.164.208.111
Feb 6 14:18:45 samolet sshd[10783]: pam_unix(sshd:auth): check pass; user unknown
Feb 6 14:18:45 samolet sshd[10783]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=li123-111.members.linode.com
Feb 6 14:18:47 samolet sshd[10783]: Failed password for invalid user sysadmin from 69.164.208.111 port 34917 ssh2
Feb 6 14:18:49 samolet sshd[10787]: Invalid user sysadmin from 69.164.208.111
Feb 6 14:18:49 samolet sshd[10787]: pam_unix(sshd:auth): check pass; user unknown
Feb 6 14:18:49 samolet sshd[10787]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=li123-111.members.linode.com
Feb 6 14:18:51 samolet sshd[10787]: Failed password for invalid user sysadmin from 69.164.208.111 port 36397 ssh2
Feb 6 14:18:53 samolet sshd[10789]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=li123-111.members.linode.com user=sync
Feb 6 14:18:56 samolet sshd[10789]: Failed password for sync from 69.164.208.111 port 37983 ssh2
Feb 6 14:18:58 samolet sshd[10791]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=li123-111.members.linode.com user=sync
Feb 6 14:18:59 samolet sshd[10791]: Failed password for sync from 69.164.208.111 port 39625 ssh2
Feb 6 14:19:02 samolet sshd[10793]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=li123-111.members.linode.com user=sync
Feb 6 14:19:03 samolet sshd[10793]: Failed password for sync from 69.164.208.111 port 41021 ssh2
Feb 6 14:19:06 samolet sshd[10795]: Invalid user tech from 69.164.208.111
Feb 6 14:19:06 samolet sshd[10795]: pam_unix(sshd:auth): check pass; user unknown
Feb 6 14:19:06 samolet sshd[10795]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=li123-111.members.linode.com
Feb 6 14:19:08 samolet sshd[10795]: Failed password for invalid user tech from 69.164.208.111 port 42470 ssh2
Feb 6 14:19:10 samolet sshd[10797]: Invalid user tech from 69.164.208.111
Feb 6 14:19:10 samolet sshd[10797]: pam_unix(sshd:auth): check pass; user unknown
Feb 6 14:19:10 samolet sshd[10797]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=li123-111.members.linode.com
Feb 6 14:19:12 samolet sshd[10797]: Failed password for invalid user tech from 69.164.208.111 port 44090 ssh2
Feb 6 14:19:14 samolet sshd[10799]: Invalid user telnetd from 69.164.208.111
Feb 6 14:19:14 samolet sshd[10799]: pam_unix(sshd:auth): check pass; user unknown
Feb 6 14:19:14 samolet sshd[10799]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=li123-111.members.linode.com
Feb 6 14:19:15 samolet sshd[10799]: Failed password for invalid user telnetd from 69.164.208.111 port 45447 ssh2
Feb 6 14:19:18 samolet sshd[10804]: Invalid user telnetd from 69.164.208.111
Feb 6 14:19:18 samolet sshd[10804]: pam_unix(sshd:auth): check pass; user unknown
Feb 6 14:19:18 samolet sshd[10804]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=li123-111.members.linode.com
Feb 6 14:19:18 samolet sshd[1228]: Received signal 15; terminating.
30 Replies
mwalling@youtoo:~$ whois 69.164.208.111 | grep -i abuse
RAbuseHandle: LAS12-ARIN
RAbuseName: Linode Abuse Support
RAbusePhone: +1-609-593-7103
RAbuseEmail: abuse@linode.com
OrgAbuseHandle: LAS12-ARIN
OrgAbuseName: Linode Abuse Support
OrgAbusePhone: +1-609-593-7103
OrgAbuseEmail: abuse@linode.com
[Edit] Sorry, misread that post as from a linode customer, not other way aroud … my gaff
@mooseday:
Don't know how many other people do this, but on any new install the first thing is change the SSH port from 22 to something else ( 22222 for example ).
[Edit] Sorry, misread that post as from a linode customer, not other way aroud … my gaff
:P
Thats exactly what i do but they still find a way to figure out the port hence why i use DenyHost
Ive installed DenyHost yesterday as i was reading on the linode forum that people were doing "back yard" attacks where they bruted machines on the same network!
I dont actually see the point with brute forcing.. Two of our old server were bruted into before we looked into DenyHost.. Why do people actually brute force do they actually get anything out of it?
Password1
Football1
Dolphin1
Zzzzzzzz
Abcd1234
Abcdefg1
While there are lots of reasons that these shouldn't even be allowed as passwords, it illustrates that users will generally choose simplicity over complexity.
@Key:
Ive installed DenyHost yesterday as i was reading on the linode forum that people were doing "back yard" attacks where they bruted machines on the same network!
This is why I firewall off my private interface. :)
That said, when computers are compromised, it is quite common that the attacker will take a look at the interfaces and then go for any other devices they can see – with an emphasis on machines on the same LAN as the compromised host. That way, if the admin cleans one machine, they still have another… and it is likely that the admin will leave the same hole as they did previously.
@Key:
I dont actually see the point with brute forcing.. Two of our old server were bruted into before we looked into DenyHost.. Why do people actually brute force do they actually get anything out of it?
Yes. Because people are lazy and configure accounts with dumb names and weak passwords. They don't need root to DDoS a site, just basic connectivity. When you think about it, you can do quite a lot with a regular account.
Brute forcers make me angry we now have to install extra software like DenyHost or completely disable the service to stop them attempting to break in!
There should be a way to report and take them down for good!
@Key:
Brute forcers make me angry we now have to install extra software like DenyHost or completely disable the service to stop them attempting to break in!
There should be a way to report and take them down for good!
The business I run off of Linode is a computer security business. I am slowly working on scripts that will watch for brute force attempts and centralize the source IP addresses the attempts are coming from. This data will be available (not sure if it will be free or a small subscription fee) so that customers can block hosts using Netfilter/iptables based on the information gathered. Essentially a Spamhaus for SSH brute force attacks.
@vonskippy:
People still setup SSH to use passwords???
This.
No reason not to have SSH configured to allow only PK authentication.
@kmweber:
@vonskippy:People still setup SSH to use passwords???
This.
No reason not to have SSH configured to allow only PK authentication.
+1
Even helps you out if you have to manage X boxes, you can use single key, some ssh-agent magick and ssh in and out of them without ever needed to invent, remember or type any passwords.
@kmweber:
No reason not to have SSH configured to allow only PK authentication.
I don't carry a storage device around with me at all times, so having credentials that can be memorized is useful to me.
@Vance:
@kmweber:No reason not to have SSH configured to allow only PK authentication.
I don't carry a storage device around with me at all times, so having credentials that can be memorized is useful to me.
Pssh, you dont have your 4096 bit key memorized?
@Vance:
I don't carry a storage device around with me at all times, so having credentials that can be memorized is useful to me.
You'd rather use a untrusted system with a password then carry around a teeny tiny thumbdrive?
If I'm not at my desktop (home or work) or I'm on the road and I don't have my smartphone or netbook or notebook, then I don't log in.
@vonskippy:
@Vance:I don't carry a storage device around with me at all times, so having credentials that can be memorized is useful to me.
You'd rather use a untrusted system with a password then carry around a teeny tiny thumbdrive?If I'm not at my desktop (home or work) or I'm on the road and I don't have my smartphone or netbook or notebook, then I don't log in.
+1 * 1e400
Totally agree.
@vonskippy:
You'd rather use a untrusted system with a password then carry around a teeny tiny thumbdrive?
Exactly what benefit do you think a private key will give you if you plug it into an untrusted system? All I need to do is copy off the data on your thumbdrive (eh; just copy files that are used) and perform keyboard logging, and I now have your key and passphrase.
(And that assumes you have your own ssh client on the thumbdrive and aren't using an insecure one on the machine).
Untrusted systems are, by definition, untrusted.
I've been using it with its PAM module for a couple months now, and it works very well. It shows up as a HID keyboard, and when plugged into a virgin port on a Windows machine, is ready to go in ~ 5 seconds. It does mean that you still have to accept password logins over SSH, but you can remove passwd from PAM for the ssh service (or something like that. I come from Slackware where PAM is the root of all evil, so it is a little bit voodoo magic).
@vonskippy:
If I'm not at my desktop (home or work) or I'm on the road and I don't have my smartphone or netbook or notebook, then I don't log in.
Okay. That's the decision you've made.
The assertion was that there was "no reason" to use password authentication. You may disagree with my reason, but it's still a reason.
As Mr. Schneier is fond of saying, security is about making tradeoffs. In some cases I have looked at the value of what is being protected, the convenience of being able to log in via password, and the countermeasures taken to prevent illegitimate logins, and decided password authentication is worthwhile.
So just don't use untrusted computers to login to your server…
That being said, it's not a good argument against key based login at all.
The way to secure the SSHD is:
1. move the port from default 22 to something random
2. create an everyday account that you will use to login
3. disable "sudo su" if you are using distro that is set up that way by default (like Ubuntu).
4. disable remote root login
5. setup key based login
5. setup private/public key (with password) and make sure you can login as non-root user to the server
5. disable login by just password - hence leaving PK as only option
That means from that point on you'll be logging in to your server as non-root, password protected key file will mean that you've got both the private key file and password, and you'll need to become root (and have root password) before you can do any damage.
So it's :
one root password
- vs -
1. find sshd port
2. somehow steal the private key file (which is the hardest part)
3. brute force the password used on the private key file
4. brute force the root password
Yeah, bad news is that you'll need to remember 2 passwords, 1. for key, and 2. for root. Though if you really really want to remember just one password - then it's better to have a passwordless private key file for non-root account + really strong root password.
@mwalling:
SelfishMan told me about
http://www.yubico.com/products/yubikey/
Interesting device, but inadequate by itself for security; what happens if it's stolen? It's good as part of a two-factor authentication system (eg password plus yubikey OTP).
In 2+ years of running sshd on port 22 of my linodes, there has not been a single attempt that even supplied a valid username.
Paranoia is great, but well-chosen passwords are still plenty secure for my purposes.
> In 2+ years of running sshd on port 22 of my linodes, there has not been a single attempt that even supplied a valid username.
Do you change root to something else then?
In ~5 minutes of setting up a fresh Linode and having sshd on port 22, I had 100s of hits trying to guess the root password.
AllowUsers someusername
PermitRootLogin no
@vonskippy:
People still setup SSH to use passwords???
It amazes me too. There is no excuse for falling to simple username/password dictionary attacks.
Here is a clue people:
Use SSH public key authentication
Limit keys to certain source ips if practical ( yes it does that )
Limit keys to certain commands if practical ( yes it does that too )
Disable root authentication or disable root password authentication
Don't use predictable usernames
Implement a decent password policy
Implement SSH connection rate limiting in iptables
Changing the SSH port isn't security. At best it reduces the rate of attacks
If you know enough to argue with the above ( like the people who say denyhosts is better than iptables rate limiting ) then do what you know to be right.