Wordpress Websites Being Redirected To Spam Websites

That probably means your site(s) have gotten hacked. You might need to rebuild the sites.

Dear @LouWestin, rebuilding can't solve this if the hacker has backdoor entry. I want a solution so that I can seal and secure the Linode and kick the spammer out of the server. Even if we rebuild the sites the files will again reappear. We did rebuild a website and the problem started just in a day. Please suggest if you have something else.

What I mean is you need to rebuild the sites by actually starting fresh. Change passwords, redo the databases, etc. the breach could be due a plugin, guessable passwords, no SSL, websites not updated, etc.

You also should look at your log files and find out who’s accessing what and where if possible.

Greetings @TechTwigs,

As LouWestin noted, what is likely the most effective solution is to redeploy. You can find our guide on doing so here:

https://www.linode.com/docs/security/recovering-from-a-system-compromise/

If you'd like to investigate more, we recommend checking the following locations:

• /var/log/auth.log : Check this log file for signs of unauthorized access and brute-force attempts. Use the last command to cross reference recent account logins with this file.
• /tmp : This directory is often used by malicious parties to store files?
• Web server logs: There may be a vulnerable script or web application. The location of these log files depends on your web server (apache, nginx, etc.) configuration.
• ps aux : Use this command to audit running processes for foreign processes

It also may be worth running a ClamAV scan as well as checking for rootkits. Here are some suggestions:

https://www.clamav.net/
https://www.linode.com/docs/security/vulnerabilities/scanning-your-linode-for-malware/
http://askubuntu.com/questions/250290/how-do-i-scan-for-viruses-with-clamav

And some rootkit scanner suggestions:

https://www.rfxn.com/projects/linux-malware-detect/
http://rkhunter.sourceforge.net
http://www.chkrootkit.org/

Best,
Preston
Linode Support Team

Hi Preston,

Thank you for the detailed comment. I tried to investigate the issue so that I do not face the same in future ever again if I find out the root cause of it. I followed your instructions to investigate. Here is what I found.

1) /var/log/auth.log : Check this log file for signs of unauthorized access and brute-force attempts. Use the last command to cross reference recent account logins with this file.
Result: Found out cron jobs and sshd access. There are failed attempts of sshd access with different names. I have enabled fail2ban and hope that will take care of banning such user ips. But, I also found out few entries like : sshd[25960]: fatal: Unable to negotiate with 195.34.238.67 port 54125: no matching cipher found. Their offer: aes128-cbc,3des-cbc,aes256-cbc,aes192-cbc [preauth]. And sshd[17738]: error: Received disconnect from 69.30.243.248 port 60043:3: com.jcraft.jsch.JSchException: Auth fail [preauth]. sshd[11524]: fatal: Unable to negotiate with 155.64.38.121 port 60311: no matching key exchange method found. Their offer: diffie-hellman-group1-sha1,diffie-hellman-group-exchange-sha1 [preauth] etc

2) /tmp : This directory is often used by malicious parties to store files?
Result: /tmp has 777 set and t is appended (drwxrwsrwxt). The directory contains the following empty directories with the same permission: .font-unix, .ICE-unix, .Test-unix,.X11-unix, .XIM-unix. There is one more directory named systemd-private-{some digits which I have removed for security reasons}-systemd-timesyncd.service-6D9zGT and has 700 as permission. This directory contains an empty tmp directory with 777 permission.
/var/tmp laso has drwxrwxrwxt as permission. Contains three directories like systemd-private-{some digits which I have removed for security reasons}-systemd-timesyncd.service-6D9zGT with 700 as permission. These directories contain tmp directory with 777 as permission.

3) Web server logs: There may be a vulnerable script or web application. The location of these log files depends on your web server (apache, nginx, etc.) configuration.
Result:
I am running Apache server. Here is the Apache server log from /var/log/Apache2 directory.
error.log:
[Fri Oct 19 06:25:03.710720 2018] [mpmprefork:notice] [pid 885] AH00163: Apache/2.4.18 (Ubuntu) configured -- resuming normal operations [Fri Oct 19 06:25:03.710799 2018] [core:notice] [pid 885] AH00094: Command line: '/usr/sbin/apache2' sh: 1: /usr/sbin/sendmail: not found sh: 1: /usr/sbin/sendmail: not found sh: 1: /usr/sbin/sendmail: not found sh: 1: /usr/sbin/sendmail: not found [Sat Oct 20 00:27:00.595903 2018] [mpmprefork:error] [pid 885] AH00161: server reached MaxRequestWorkers setting, consider raising the MaxRequestWorkers setting
[Sat Oct 20 00:28:39.767026 2018] [core:notice] [pid 885] AH00052: child pid 25893 exit signal Segmentation fault (11)
[Sat Oct 20 00:28:46.003792 2018] [core:notice] [pid 885] AH00052: child pid 26834 exit signal Segmentation fault (11)
[crit] Memory allocation failed, aborting process.
[Sat Oct 20 00:39:09.638074 2018] [core:notice] [pid 885] AH00052: child pid 26877 exit signal Aborted (6)

mmap() failed: [12] Cannot allocate memory

mmap() failed: [12] Cannot allocate memory

mmap() failed: [12] Cannot allocate memory

mmap() failed: [12] Cannot allocate memory

mmap() failed: [12] Cannot allocate memory

mmap() failed: [12] Cannot allocate memory
[Sat Oct 20 00:49:49.180337 2018] [core:notice] [pid 885] AH00052: child pid 26820 exit signal Segmentation fault (11)

access.log files are old. nad there are no recent entries. If you want me to send something specific, please tell me so that I can find it.

4) ps aux : Use this command to audit running processes for foreign processes.
Result: Output is-

USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 0.0 0.0 37844 572 ? Ss Oct20 0:04 /sbin/init
root 2 0.0 0.0 0 0 ? S Oct20 0:00 [kthreadd]
root 3 0.0 0.0 0 0 ? I< Oct20 0:00 [rcugp] root 4 0.0 0.0 0 0 ? I< Oct20 0:00 [rcupargp] root 6 0.0 0.0 0 0 ? I< Oct20 0:00 [kworker/0:0H-k root 8 0.0 0.0 0 0 ? I< Oct20 0:00 [mmpercpuwq] root 9 0.0 0.0 0 0 ? S Oct20 0:09 [ksoftirqd/0] root 10 0.0 0.0 0 0 ? I Oct20 0:24 [rcupreempt]
root 11 0.0 0.0 0 0 ? I Oct20 0:01 [rcusched] root 12 0.0 0.0 0 0 ? I Oct20 0:00 [rcubh]
root 13 0.0 0.0 0 0 ? S Oct20 0:32 [rcuc/0]
root 14 0.0 0.0 0 0 ? S Oct20 0:00 [rcub/0]
root 15 0.0 0.0 0 0 ? S Oct20 0:00 [migration/0]
root 16 0.0 0.0 0 0 ? S Oct20 0:00 [watchdog/0]
root 17 0.0 0.0 0 0 ? S Oct20 0:00 [cpuhp/0]
root 18 0.0 0.0 0 0 ? S Oct20 0:00 [kdevtmpfs]
root 19 0.0 0.0 0 0 ? I< Oct20 0:00 [netns]
root 20 0.0 0.0 0 0 ? S Oct20 0:00 [rcutaskskthr
root 21 0.0 0.0 0 0 ? S Oct20 0:00 [kauditd]
root 24 0.0 0.0 0 0 ? S Oct20 0:00 [khungtaskd]
root 25 0.0 0.0 0 0 ? S Oct20 0:00 [oomreaper] root 26 0.0 0.0 0 0 ? I< Oct20 0:00 [writeback] root 27 0.0 0.0 0 0 ? S Oct20 0:00 [kcompactd0] root 28 0.0 0.0 0 0 ? SN Oct20 0:00 [ksmd] root 29 0.0 0.0 0 0 ? SN Oct20 0:20 [khugepaged] root 30 0.0 0.0 0 0 ? I< Oct20 0:00 [crypto] root 31 0.0 0.0 0 0 ? I< Oct20 0:00 [kintegrityd] root 32 0.0 0.0 0 0 ? I< Oct20 0:00 [kblockd] root 33 0.0 0.0 0 0 ? I< Oct20 0:00 [atasff]
root 34 0.0 0.0 0 0 ? I< Oct20 0:00 [md]
root 35 0.0 0.0 0 0 ? I< Oct20 0:00 [edac-poller]
root 36 0.0 0.0 0 0 ? I< Oct20 0:00 [ib-comp-wq]
root 37 0.0 0.0 0 0 ? I< Oct20 0:00 [ibmcast] root 38 0.0 0.0 0 0 ? I< Oct20 0:00 [ibnlsawq]
root 39 0.0 0.0 0 0 ? I< Oct20 0:00 [devfreqwq] root 40 0.0 0.0 0 0 ? S Oct20 0:00 [watchdogd] root 42 0.0 0.0 0 0 ? I< Oct20 0:00 [rpciod] root 43 0.0 0.0 0 0 ? I< Oct20 0:00 [kworker/u3:0] root 44 0.0 0.0 0 0 ? I< Oct20 0:00 [xprtiod] root 123 0.1 0.0 0 0 ? S Oct20 4:39 [kswapd0] root 124 0.0 0.0 0 0 ? S Oct20 0:00 [ecryptfs-kthre root 125 0.0 0.0 0 0 ? I< Oct20 0:00 [nfsiod] root 126 0.0 0.0 0 0 ? I< Oct20 0:00 [cifsiod] root 127 0.0 0.0 0 0 ? I< Oct20 0:00 [cifsoplockd] root 128 0.0 0.0 0 0 ? S Oct20 0:00 [jfsIO] root 129 0.0 0.0 0 0 ? S Oct20 0:00 [jfsCommit] root 130 0.0 0.0 0 0 ? S Oct20 0:00 [jfsSync] root 131 0.0 0.0 0 0 ? I< Oct20 0:00 [xfsalloc] root 132 0.0 0.0 0 0 ? I< Oct20 0:00 [xfsmrucache] root 133 0.0 0.0 0 0 ? I< Oct20 0:00 [glockworkqueu
root 134 0.0 0.0 0 0 ? I< Oct20 0:00 [deleteworkque root 135 0.0 0.0 0 0 ? I< Oct20 0:00 [gfsrecovery]
root 226 0.0 0.0 0 0 ? I< Oct20 0:00 [kthrotld]
root 227 0.0 0.0 0 0 ? I< Oct20 0:00 [acpithermalp
root 229 0.0 0.0 0 0 ? I< Oct20 0:00 [ttmswap] root 230 0.0 0.0 0 0 ? I< Oct20 0:00 [knbd-recv] root 231 0.0 0.0 0 0 ? I< Oct20 0:00 [drbd-reissue] root 232 0.0 0.0 0 0 ? I< Oct20 0:00 [rbd] root 233 0.0 0.0 0 0 ? I< Oct20 0:00 [iscsieh]
root 234 0.0 0.0 0 0 ? S Oct20 0:00 [scsieh0]
root 235 0.0 0.0 0 0 ? I< Oct20 0:00 [scsitmf0]
root 236 0.0 0.0 0 0 ? I< Oct20 0:00 [nvme-wq]
root 237 0.0 0.0 0 0 ? I< Oct20 0:00 [nvme-reset-wq]
root 238 0.0 0.0 0 0 ? I< Oct20 0:00 [nvme-delete-wq
root 240 0.0 0.0 0 0 ? S Oct20 0:00 [scsieh1]
root 241 0.0 0.0 0 0 ? I< Oct20 0:00 [scsitmf1]
root 242 0.0 0.0 0 0 ? S Oct20 0:00 [scsieh2]
root 243 0.0 0.0 0 0 ? I< Oct20 0:00 [scsitmf2]
root 244 0.0 0.0 0 0 ? S Oct20 0:00 [scsieh3]
root 245 0.0 0.0 0 0 ? I< Oct20 0:00 [scsitmf3]
root 246 0.0 0.0 0 0 ? S Oct20 0:00 [scsieh4]
root 247 0.0 0.0 0 0 ? I< Oct20 0:00 [scsitmf4]
root 248 0.0 0.0 0 0 ? S Oct20 0:00 [scsieh5]
root 249 0.0 0.0 0 0 ? I< Oct20 0:00 [scsitmf5]
root 250 0.0 0.0 0 0 ? S Oct20 0:00 [scsieh6]
root 251 0.0 0.0 0 0 ? I< Oct20 0:00 [scsitmf6]
root 258 0.1 0.0 0 0 ? I< Oct20 3:10 [kworker/0:1H-k
root 260 0.0 0.0 0 0 ? I< Oct20 0:00 [raid5wq]
root 262 0.0 0.0 0 0 ? I< Oct20 0:00 [dmbufiocache
root 263 0.0 0.0 0 0 ? I< Oct20 0:00 [rdmacm] root 267 0.0 0.0 0 0 ? I< Oct20 0:00 [ipv6addrconf]
root 276 0.0 0.0 0 0 ? I< Oct20 0:00 [kstrp]
root 277 0.0 0.0 0 0 ? I< Oct20 0:00 [ceph-msgr]
root 313 0.0 0.0 0 0 ? I< Oct20 0:00 [charger_manage
root 315 0.0 0.0 0 0 ? S Oct20 0:17 [jbd2/sda-8]
root 316 0.0 0.0 0 0 ? I< Oct20 0:00 [ext4-rsv-conve
root 330 0.0 0.2 35272 4180 ? Ss Oct20 0:08 /lib/systemd/sy
root 409 0.0 0.0 42256 36 ? Ss Oct20 0:00 /lib/systemd/sy
systemd+ 497 0.0 0.0 100324 316 ? Ssl Oct20 0:00 /lib/systemd/sy
syslog 729 0.0 0.0 256392 260 ? Ssl Oct20 0:04 /usr/sbin/rsysl
message+ 735 0.0 0.0 42900 576 ? Ss Oct20 0:00 /usr/bin/dbus-d
root 748 0.0 0.0 275868 228 ? Ssl Oct20 0:06 /usr/lib/accoun
root 749 0.0 0.0 29008 452 ? Ss Oct20 0:02 /usr/sbin/cron
root 758 0.0 0.0 28624 1052 ? Ss Oct20 0:00 /lib/systemd/sy
root 797 0.0 0.0 65508 464 ? Ss Oct20 0:01 /usr/sbin/sshd
root 822 0.0 0.0 277108 460 ? Ssl Oct20 0:00 /usr/lib/policy
root 845 0.0 0.0 15936 0 tty1 Ss+ Oct20 0:00 /sbin/agetty --
root 846 0.0 0.0 15752 0 ttyS0 Ss+ Oct20 0:00 /sbin/agetty --
mysql 879 0.8 6.3 1424180 128472 ? Sl Oct20 21:42 /usr/sbin/mysql
root 885 0.0 0.1 412008 4024 ? Ss Oct20 0:10 /usr/sbin/apach
root 892 0.1 0.2 440916 4892 ? Sl Oct20 2:58 /usr/bin/python
root 893 0.0 0.8 114300 16256 ? Ss Oct20 1:18 linode-longview
www-data 19646 0.2 4.2 533812 86724 ? S 06:25 0:46 /usr/sbin/apach
www-data 19647 0.3 4.6 538920 93688 ? S 06:25 1:03 /usr/sbin/apach
www-data 19648 0.2 4.6 536556 93532 ? S 06:25 0:59 /usr/sbin/apach
www-data 19649 0.2 3.4 519580 69440 ? S 06:25 1:00 /usr/sbin/apach
www-data 19650 0.2 3.9 522604 79364 ? S 06:25 1:00 /usr/sbin/apach
www-data 19651 0.2 3.7 523644 75984 ? S 06:25 0:53 /usr/sbin/apach
www-data 19652 0.2 3.8 607588 78468 ? S 06:25 0:59 /usr/sbin/apach
www-data 19653 0.2 2.8 512792 58116 ? S 06:25 0:54 /usr/sbin/apach
www-data 19654 0.2 3.7 520500 75108 ? S 06:25 0:49 /usr/sbin/apach
www-data 19655 0.2 5.1 550468 103368 ? S 06:25 0:53 /usr/sbin/apach
www-data 19656 0.2 4.3 534636 88840 ? S 06:25 1:00 /usr/sbin/apach
www-data 19657 0.2 2.7 506128 56592 ? S 06:25 0:51 /usr/sbin/apach
www-data 19658 0.2 4.0 615488 81320 ? S 06:25 0:59 /usr/sbin/apach
www-data 19659 0.2 4.6 539808 94884 ? S 06:25 0:54 /usr/sbin/apach
www-data 19660 0.2 3.0 509304 62736 ? S 06:25 1:00 /usr/sbin/apach
www-data 19661 0.2 3.8 604288 78852 ? S 06:25 0:59 /usr/sbin/apach
www-data 19662 0.2 4.0 536720 82684 ? S 06:25 0:53 /usr/sbin/apach
www-data 19663 0.2 4.8 546620 98400 ? S 06:25 0:53 /usr/sbin/apach
www-data 19664 0.2 5.5 553780 112296 ? S 06:25 0:58 /usr/sbin/apach
www-data 19666 0.2 4.5 546176 92128 ? S 06:25 0:58 /usr/sbin/apach
www-data 19667 0.3 4.8 540868 97840 ? S 06:25 1:07 /usr/sbin/apach
www-data 19668 0.2 5.5 631616 113008 ? S 06:25 0:56 /usr/sbin/apach
www-data 19699 0.2 3.0 505576 62060 ? S 06:31 0:56 /usr/sbin/apach
root 19767 0.0 0.0 0 0 ? I 06:35 0:00 [kworker/0:2-cg
www-data 19930 0.3 3.9 597212 80052 ? S 06:56 1:04 /usr/sbin/apach
www-data 19931 0.3 5.0 630276 103332 ? S 06:56 1:08 /usr/sbin/apach
www-data 20054 0.2 3.5 517368 71080 ? S 07:09 0:44 /usr/sbin/apach
www-data 20136 0.2 4.1 528672 84472 ? S 07:23 0:48 /usr/sbin/apach
www-data 20138 0.2 4.4 535848 90508 ? S 07:23 0:43 /usr/sbin/apach
www-data 21365 0.2 4.9 553160 100420 ? S 10:11 0:16 /usr/sbin/apach
www-data 21366 0.1 2.8 508656 58252 ? S 10:11 0:14 /usr/sbin/apach

Found out that there are number of processes by web user(www-data) which are accessing /usr/sbin folder. I went ahead and found something . /usr/bin, /usr/sbin contains symlinks with permissions set as 777. And the files contained in these folders have permission set as 755. I do not remember doing any of these. In fact this is the first time I am navigating to this directory. Isn't this suspicious? Can you please guide me to fix this if this is not correct.

Can you help me pinpoint what's exactly is causing the spammer to get access to our www folder.

@LouWestin Thanks for your response.

I’d recommend starting a new VM and going though the securing your linode guide since you mentioned turning on fail2ban recently. Since the server is probably already compromised it’s not going to help at this point.

Edit: You’re going to have to copy over the web data too and start with fresh installations. The WP databases could be tainted as well, so I wouldn’t recommend copying them over.

The multiple failed ssh attempts is kind of normal if the user name root is the that’s mainly failing since it’s commonly known (obviously) and why it’s recommended to disable remote login for it.

If the other user names failing are ones you created and use to login, then that would raise some concerns with me as to how they know that.

The multiple www-data processes would be the Apache handlers running. However it’s noted In the logs that you’re reaching your max limit. This means that the amount of traffic is exceeding what you have set in your Apache configuration. This could be from the spam traffic.

You mentioned too about a Cron jobs in the /tmp directory and permissions set to 777 that you don’t recall changing. It could be either from getting compromised or something you installed.

Dear @LouWestin,

Thanks for the suggestion. I could have redone everything from scratch. But, we have around 16 live websites with data. It would be great if you can tell me how to rectify this issue rather than taking altogether a new route.

If the server has been breached and the hacker has gotten root access to your server then there’s really no other recourse than to rebuild from scratch.

My question is, is every website on the server being redirected or displaying some form of spamming or malware?

@LouWestin, Yes. Every website gets redirected to a spam website if the user accesses it after finding the search result in Google. Once the website is browsed, it does not redirect to spam website. I found out that this is some kind of Pharma attack where search results in google takes the user to a spam website first time. This saves the session in cookie and does not happen again for next 20-30 days as set by the spammer. I have read similar experiences all over the internet.

Ok. We were talking about it yesterday in the Linode chat. One person mentioned that it could’ve been something to do with privilege escalation giving the hacker root access.

If every website is getting redirected then its possible that the hacker/spammer was able to modify the files on the server. If that’s the case, then it goes back to what I originally said; you’re going to have to transfer over all the website’s content and rebuild the server.

@LouWestin, Thank you for your views and guidance. Really appreciate it.

@pambrosky, Do you have anything to add? I do not want to redo all of these websites and setting up a new Linode is again a pain. Is there any other way?

Quite correction. Actually what you found in the /tmp directory might actually be fine. Source: https://serverfault.com/questions/784681/are-directories-with-mode-drwxrwxrwt-safe-in-tmp-on-a-nix-system

Thanks @LouWestin.