My linode is suddenly using 100% cpu for klogd
My linode instance: Linode 2GB: 1 CPU, 50GB Storage, 2GB RAM
is suddenly using 100% CPU for the last couple hours out of nowhere. My instance runs nginx, nodejs, laravel and a postgresql database.
Using htop I see:
USER: postgres, command: /sbin/klogd -c 1 -x -x and cpu 52%+
I tried stopping postgres but the task seems to start up again.
When I do a trace -p [pid] I see this over and over again:
select(8, [3], NULL, NULL, {tv_sec=0, tv_usec=0}) = 0 (Timeout)
select(8, [3], NULL, NULL, {tv_sec=0, tv_usec=0}) = 0 (Timeout)
Anyone know what other logs I can look at or know why this is happening?
Killing that task with "kill" does fix it, but not sure why this happened and if it will occur again.
1 Reply
tbaka
Linode Staff
Hi there,
This can be a tricky situation because there are a few things going on. If I were in your shoes, I would start by checking for any zombie processes by running:
$ ps aux | grep 'Z'
Anything with a status of Z is a zombie process, take note of its PID. You can then kill the parent process by running:
$ kill -s SIGCHLD $PID
This would likely resolve your issue in the short term. For a more longterm solution you can consider the possibility that klogd is simply writing extremely verbose logs. There is information you may not need. You can change the log level of this daemon by running:
$ klogd -c $level
In this case $level indicates the desired level of verbosity. This page provides a good description of each level.
In researching this I have also found that klogd is a deprecated process. This essentially means that it has been superseded by newer, more efficient logging daemons. For this as well as security reasons it may be worthwhile to consider upgrading your system to a more current distro.
Lastly, there are two good posts about identifying and trouble shooting high CPU usage. Here and here. I would recommend taking a look at them to see if they can give you any additional insight.
I hope this helps get you off to a good start!