nginx+php-fpm+wp = cpu spike 504 gateway error
The server was performing well (for a month without downtime) until yesterday where the CPU suddenly spiked, memory maxed out (1024MB) and load average of 20, 19.6, 18.4 (quad core). I have to restart the php-fpm every 5 minutes to keep it running.
After some meddling and troubleshooting, I have managed to identify the issue with the wordpress database. It is probably due to a corrputed database. The frontend was working fine without any issue, but everytime i save post in the backend, the CPU spike, php5-fpm process took over and crash the server. The problem is, whatever i do, it keeps returning a 504 gateway error.
When I save post in wp: 504 gateway error
when i import database in phpmyadmin: 504 gateway error
I have made plenty of changes to the nginx.conf, php.ini,
The situation now is that the site is still live, but I can't touch anything in the WordPress backend as it will cause the cpu to spike and server crash.
What actually went wrong? Can anyone help?
Below is my settings:
nginx.conf
# Generic startup file.
user damien;
worker_processes 4;
error_log /var/log/nginx/error.log;
pid /var/run/nginx.pid;
# Keeps the logs free of messages about not being able to bind().
#daemon off;
events {
worker_connections 1024;
}
http {
# rewrite_log on;
include mime.types;
default_type application/octet-stream;
access_log /var/log/nginx/access.log;
sendfile on;
# tcp_nopush on;
keepalive_timeout 0;
# tcp_nodelay on;
# gzip on;
client_max_body_size 200M;
client_body_buffer_size 128k;
client_header_buffer_size 256k;
client_header_timeout 300;
client_body_timeout 300;
send_timeout 300;
index index.php index.html index.htm;
# Upstream to abstract backend connection(s) for PHP.
upstream php {
# server unix:/tmp/php-fpm.sock;
server 127.0.0.1:9000;
}
server {
location ~* wp-config.php {
deny all;
}
# pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
location ~ \.php$ {
try_files $uri =404;
fastcgi_split_path_info ^(.+\.php)(/.+)$;
fastcgi_pass 127.0.0.1:9000;
# With php5-fpm:
#fastcgi_pass unix:/var/run/php5-fpm.sock;
fastcgi_index index.php;
include fastcgi_params;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_param SCRIPT_NAME $fastcgi_script_name;
fastcgi_read_timeout 1800;
fastcgi_send_timeout 1800;
fastcgi_connect_timeout 1800;
fastcgi_buffer_size 128k;
fastcgi_buffers 4 256k;
fastcgi_busy_buffers_size 256k;
fastcgi_temp_file_write_size 256k;
fastcgi_intercept_errors on;
}
location ~ /\. {
access_log off;
log_not_found off;
deny all;
}
if ($host ~* ^www\.(.*)) {
set $remove_www $1;
rewrite ^(.*)$ http://$remove_www$1 permanent;
}
}
include sites-enabled/*;
}
pm = dynamic
pm.max_children = 35
pm.start_servers = 10
pm.min_spare_servers = 2
pm.max_spare_servers = 32
;pm.process_idle_timeout = 10s;
pm.max_requests = 400
my.conf
[client]
port = 3306
socket = /var/run/mysqld/mysqld.sock
[mysqld_safe]
socket = /var/run/mysqld/mysqld.sock
nice = 0
[mysqld]
user = mysql
pid-file = /var/run/mysqld/mysqld.pid
socket = /var/run/mysqld/mysqld.sock
port = 3306
basedir = /usr
datadir = /var/lib/mysql
tmpdir = /tmp
lc-messages-dir = /usr/share/mysql
skip-external-locking
log-error = /var/log/mysql.err
bind-address = 127.0.0.1
key_buffer = 256M
max_allowed_packet = 16M
thread_stack = 192K
thread_cache_size = 20
myisam-recover = BACKUP
max_connections = 80
table_cache = 512
thread_concurrency = 10
query_cache_limit = 8M
query_cache_size = 64M
myisam_sort_buffer_size = 32M
sort_buffer_size = 1M
read_buffer_size = 1M
join_buffer_size = 5M
log_slow_queries = /var/log/mysql/mysql-slow.log
long_query_time = 2
log-queries-not-using-indexes
# * InnoDB
#
#skip-innodb
# InnoDB is enabled by default with a 10MB datafile in /var/lib/mysql/.
# Read the manual for more InnoDB related options. There are many!
#
# * Security Features
#
# Read the manual, too, if you want chroot!
# chroot = /var/lib/mysql/
#
# For generating SSL certificates I recommend the OpenSSL GUI "tinyca".
#
# ssl-ca=/etc/mysql/cacert.pem
# ssl-cert=/etc/mysql/server-cert.pem
# ssl-key=/etc/mysql/server-key.pem
max_heap_table_size = 80M
tmp_table_size = 80M
[mysqldump]
quick
quote-names
max_allowed_packet = 16M
[mysql]
#no-auto-rehash # faster start of mysql but no tab completition
[isamchk]
key_buffer = 16M
[myisamchk]
key_buffer = 64M
sort_buffer_size = 64M
read_buffer = 16M
write_buffer = 16M
4 Replies
-Tim
Both the mysql and php are causing issue. Even after I fix the leap second issue, the problem still persists.
What I did is to remove the php-apc and reinstall it. Additionally I also installed memcached and everything is back to normal and is running even faster. Seems that the php-apc module is the one causing trouble.