server compromised?

Do you have FTP access setup?

Yes, but with only one user that does not have access to /var/www obviously.

You don't mention what version(s) of WP you were running? Who owned the files?

I wouldn't be surprised if you got hit by some WP security hole that hasn't been patched. Not only would I start backing up :) But I'd also start reading up on locking down and securing WP.

And disable FTP, period, there is no reason for it. You have SSH and SFTP. Create a "limited" user (or users, one for each domain) that doesn't have sudo privileges and can only create files under your site(s) webroots.

Lock all the files down so only those user accounts can write to those files. Ensure that the web server and PHP cannot change the files or directories. That should really limit how much damage someone can do if they compromise some insecure software running on your site.

And review your Apache access logs, looks like you've only looked at your error logs.

To be more specific, I do not have regular FTP, but it is sftp and that one user is locked down. It was WP version 2.7, and owned by www-data, so that could be the reason. But if the WP is owned by a user, how do I get it to be writable by the server?

I've looked at the access logs but I'm not quite sure what to look for. One minute users were able to access the site, then all of a sudden it is 200

> But if the WP is owned by a user, how do I get it to be writable by the server?

Either lock it down so only a few directories that you want to have updated (content, themes, etc) are 777 or 770 (depending on setup) so the web server can write to it, or setup something like php-cgiwrap (if you're running PHP for everything dynamic), where the web server runs as the user. But if that software has a hole in it that can be exploited then the web server will have access to everything that user has access to. So best to make sure each site on the server is run under a different account. This is how my site(s) are setup at my current host. Well all my sites are run under one account, but still… Someone else's site running under the web server account can't modify my files if they get hacked and if my sites ever got hacked, they wouldn't be able to do squat to any other sites on the server.

> One minute users were able to access the site, then all of a sudden it is 200

That's where I'd look. Either someone hacked your account logged in via ssh and cleaned you out, cleaning up after themselves or it was done via something that got compromised on one of your sites.

@xtine:

I go in to bash history of root and the only other system account (me) to see if someone logged in and deleted the files… nope
If someone attacks your server, the first thing he does is to redirect the HISTFILE to /dev/null so you won't find anything in the history file…

@xtine:

To be more specific, I do not have regular FTP, but it is sftp and that one user is locked down. It was WP version 2.7,
It is an old version, after that there was 2.7.1 and now 2.8.

If you try to google for "wordpress 2.7 vulnerability" you can see that it is really recommended to upgrade!

@xtine:

and owned by www-data, so that could be the reason. But if the WP is owned by a user, how do I get it to be writable by the server?
You can use suphp: it is "a tool for executing PHP scripts with the permissions of their owners".

Well inspecting more of my server… it's pretty clear that most likely a malicious user got ahold of www-data and deleted what they could. All of my webfiles that were owned by root or someone else were untouched.

I am lucky that I can restore what I can with the mysql backups and copies of templates I have on my own computer. I've really learned a lesson, and will never have anything important (or not backed up) owned by www-data ever again. I will look into suexec and suphp. And I was running the version before the recent 2.8, so it was 2.7.1. But Wordpress is so inherently insecure, I am actually not that surprised my sites were deleted but I am still annoyed.

I've been inspecting my access log for the one site that had the access and error logs owned by root (so it wasn't deleted!) and there is one user that keeps on accessing the site as 200 "/". But I don't see any fishy behavior besides that. It's literally one minute users were accessing the theme files like normally, and then all of a sudden they get a blank index file. Grrr.

I've learned my share of basic apache security and kept more aware of locking Wordpress down.

Thanks for the help everyone.

How did you lock your server down? Just curious.

I'd recommend you looking into something like mpm-itk as well.

Here is my lock down (I'm a little lazy but it works)
* All users have /bin/false as shell unless they are an active user (only 2 actives)

• Key auth only via ssh, group allowed access, and key is 2048 bits long, no root login allowed via ssh

I'd also recommend re-installing your OS, you don't know how far they actually got into your system.

I didn't lock it down that much besides firehol. Only allowed port 80 and 22 to go through. As I was looking at users I realized I should have had everything set to /bin/false besides my user.

I'm really considering reinstalling the OS when I have a free weekend so I can re-do everything right.

are you running mod_security (http://www.modsecurity.org/) ? i found that that helps out a bunch with securing stuff.