linode compromised. need assistance recovery
All of my sites are showing Database error message. Upon trying to access phpMyAdmin it opens a survey page (redirect) & not able to access server via SSH/putty. I shut down the server as to absorb what has been done & what to do now.
The linode support dont offer anything much apart from throwing some links of library.
I think I dont have time nor have confidence to recover and deploy the hacked system.
I'm ready to shell extra money for Linode managed but they do not offer a recovery either.
How would you recover your linode system if it's unaccessible via ssh. (I could login via lish? but I shut the server)
14 Replies
shouuld I download the linux via SFTP?
@johnycage:
But how to recover the data?
You don't - you get the last known good (and non-compromised) backup and use that.
![](
- Les
I think it would 've been easier if I had static html sites but I've around 10 wordpress sites, restoring these would be a problem since I dont have backup system in place.
How to check which part is corrupted? how to see the log files?
all my sites are showing database error. Upon trying to visit phpmyadmin url it redirects to some survey.
@johnycage:
Should I dare to boot my linode now?
Absolutely not.
As others have said, your machine is compromised, and you have no way of knowing which bits have been altered. Re-install from the ground up and restore from backup.
For more info on how to appropriately respond, check out this
@johnycage:
but I've around 10 wordpress sites, restoring these would be a problem since I dont have backup system in place.
Then you my friend are totally f**ked.
Why in the world would you run 10 wordpress sites and NOT have a backup (both files and databases)?
And why are you downloading the hacked image? Unless you plan on running forensic diagnostics to determine how you got screwed, it's completely useless.
@anderiv:
For more info on how to appropriately respond, check out
Q/A over on Serverfault on how to deal with a compromised server. this
Good article except for the part that says "Don't Panic".
Compromised with no backups seems to be the perfect time to panic.
About 5 different people, including Linode staff, have suggested deleting this image and restoring from a backup. We are not suggesting this because we are assholes and want you to lose your work. We are suggesting this because that machine has been cracked and it's full of malware, botkits, and all other kind of nasties. Getting anything off it is a very risky job. You can try if you want but you should know that the most experienced amongst us would exercise extreme care if we did it at all, which in all likelihood we would not. Whatever you do -DO NOT BOOT THE IMAGE-.
In future you need backups. Even if it's just shutting things down periodically and using the Linode manager to copy the disk images it's a lot better than nothing. Really backups should be off-site. There are lots of ways to do it. I like BackupPC personally.
You need to do security updates for all your software.
You need some kind of monitoring, in the crudest case that could be nosing though the system periodically to look for changes you didn't make or processes that should not be running. Tripwire is good but it's a PITA to setup.
If they don't have backups then things get a whole lot more complicated, in the case of wordpress it's best to create a new node, reinstall wordpress, reinstall any plugins/themes, then if you have any custom code go through it line by line looking for anything weird. You also can't trust any images uploaded you'll need to reprocess those to make sure they don't contain anything weird. Lastly you restore your database after checking each and every entry for anything weird. It's a pain staking process.
In the future have backups, at a minimum the Linode backup service will do it's not expensive. I personally use duplicity.
~~![](<URL url=)
I'm totally serious though. Do not boot the disk image. It's very easy for a skilled hacker to hide malware. No one is going to leave backdoor.py in your home directory.
Also, always make backups, daily if possible. Run a script every day to back up databases and put them in some database folder. Use rsync on a local computer to back up all the files and databases. Use a local backup system on your own computer that stores the versions of the files for each day so that you get a history of backups. Getting a daily backup of the server can be made really simple; it's very important to do. At least pay for Linode backups.
I'm planning to write about security advice here on the website for my GitHub profile:
Here's the source for it if anyone wants to contribute; I can add push access
And for backup advice and GitHub, upload any custom software that you use to GitHub so that you don't have to recreate it if you lose the linode. It will be open source, but unless you're actually planning to sell it, you might as well make it free for other people to use.
EDIT
Do not use PHPmyAdmin. It is a total backdoor.~~