High CPU
When I less /var/log/messages I see the majority of the messages are *Firewall: *TCP_IN Blocked** messages. Is this the source of my problem, or should I be looking deeper?
9 Replies
@akerl:
Have you considered checking what is using your CPU?
Actually, I did and I should have mentioned that. However, while I understand some of the things going on I don't understand everything. I'm not a sysadmin by training and only in practice when I have to be.
As I look at top I don't see anything that jumps out at me as a problem. The sites being hosted are low traffic sites, so I don't expect to see and am not seeing a lot of activity from MySQL and PHP.
mysql 20 0 1192m 416m 6276 S 2 27.6 1423:56 mysqld
7 root 20 0 0 0 0 S 0 0.0 1762:16 rcu_sched
16486 root 20 0 4796 2540 2364 S 0 0.2 0:00.09 bash
17345 root 20 0 2532 1980 1656 R 0 0.1 0:00.09 top
1 root 20 0 2080 76 0 S 0 0.0 9:26.50 init
2 root 20 0 0 0 0 S 0 0.0 0:00.17 kthreadd
3 root 20 0 0 0 0 S 0 0.0 90:08.32 ksoftirqd/0
4 root 20 0 0 0 0 S 0 0.0 0:00.06 kworker/0:0
5 root 0 -20 0 0 0 S 0 0.0 0:00.00 kworker/0:0H
8 root 20 0 0 0 0 S 0 0.0 0:00.00 rcu_bh
9 root RT 0 0 0 0 S 0 0.0 13:16.63 migration/0
10 root RT 0 0 0 0 S 0 0.0 11:08.58 migration/1
11 root 20 0 0 0 0 S 0 0.0 93:30.14 ksoftirqd/1
12 root 20 0 0 0 0 S 0 0.0 0:00.00 kworker/1:0
13 root 0 -20 0 0 0 S 0 0.0 0:00.00 kworker/1:0H
14 root RT 0 0 0 0 S 0 0.0 10:58.86 migration/2
15 root 20 0 0 0 0 S 0 0.0 93:46.01 ksoftirqd/2
16 root 20 0 0 0 0 S 0 0.0 0:00.00 kworker/2:0
17 root 0 -20 0 0 0 S 0 0.0 0:00.00 kworker/2:0H
18 root RT 0 0 0 0 S 0 0.0 11:12.97 migration/3
19 root 20 0 0 0 0 S 0 0.0 93:25.68 ksoftirqd/3
20 root 20 0 0 0 0 S 0 0.0 0:00.00 kworker/3:0
21 root 0 -20 0 0 0 S 0 0.0 0:00.00 kworker/3:0H
22 root RT 0 0 0 0 S 0 0.0 11:56.06 migration/4
23 root 20 0 0 0 0 S 0 0.0 89:39.02 ksoftirqd/4
24 root 20 0 0 0 0 S 0 0.0 0:00.00 kworker/4:0
25 root 0 -20 0 0 0 S 0 0.0 0:00.00 kworker/4:0H
26 root RT 0 0 0 0 S 0 0.0 12:16.67 migration/5
27 root 20 0 0 0 0 S 0 0.0 93:40.16 ksoftirqd/5
28 root 20 0 0 0 0 S 0 0.0 0:00.00 kworker/5:0
29 root 0 -20 0 0 0 S 0 0.0 0:00.00 kworker/5:0H
@sneakyfitch:
…while I understand some of the things going on I don't understand everything. I'm not a sysadmin by training and only in practice when I have to be.
I read where elsewhere that something to look at was to run echo l > /proc/sysrq-trigger and then see what it reports by using dmesg.
Doing that I get a lot of the same things I saw before. There are lots of *Firewall: *TCPIN Blocked** and an occasional *Firewall: *UDPIN Blocked** and *Firewall: *Port Flood**.
At the end of that report I see this, but I don't know what it means of it it has any significance at all:
> SysRq : Show backtrace of all active CPUs
sending NMI to all CPUs:
xen: vector 0x2 is not implemented
SysRq : Show backtrace of all active CPUs
sending NMI to all CPUs:
xen: vector 0x2 is not implemented
Any utility like top or htop will show you what is using CPU. Based on what you pasted above, it's MySQL.
Figure out what's happening when your usage spikes, and then figure out how to make that not use as much CPU.
- Les
@akerl:
That would probably be saner to read if you'd left the headers, but it looks like MySQL is what's using the most CPU.
Sorry. You are correct. I don't know why it didn't copy.
aegir:~# top
top - 15:24:42 up 101 days, 16:29, 1 user, load average: 1.14, 1.19, 1.34
Tasks: 168 total, 3 running, 165 sleeping, 0 stopped, 0 zombie
Cpu(s): 16.1%us, 2.8%sy, 0.0%ni, 80.6%id, 0.0%wa, 0.0%hi, 0.0%si, 0.5%st
Mem: 1546224k total, 1435604k used, 110620k free, 35932k buffers
Swap: 262140k total, 0k used, 262140k free, 698112k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
32435 smithbas 20 0 68176 42m 18m R 35 2.8 0:01.06 php
32460 smithbas 20 0 62420 36m 18m R 15 2.4 0:00.46 php
7 root 20 0 0 0 0 S 0 0.0 1762:34 rcu_sched
28148 root 20 0 2532 1976 1656 R 0 0.1 0:00.41 top
31290 root 20 0 4824 2544 2348 S 0 0.2 0:00.03 bash
1 root 20 0 2080 76 0 S 0 0.0 9:26.59 init
2 root 20 0 0 0 0 S 0 0.0 0:00.17 kthreadd
3 root 20 0 0 0 0 S 0 0.0 90:09.04 ksoftirqd/0
4 root 20 0 0 0 0 S 0 0.0 0:00.06 kworker/0:0
5 root 0 -20 0 0 0 S 0 0.0 0:00.00 kworker/0:0H
8 root 20 0 0 0 0 S 0 0.0 0:00.00 rcu_bh
9 root RT 0 0 0 0 S 0 0.0 13:16.71 migration/0
10 root RT 0 0 0 0 S 0 0.0 11:26.89 migration/1
11 root 20 0 0 0 0 S 0 0.0 93:30.78 ksoftirqd/1
12 root 20 0 0 0 0 S 0 0.0 0:00.00 kworker/1:0
13 root 0 -20 0 0 0 S 0 0.0 0:00.00 kworker/1:0H
14 root RT 0 0 0 0 S 0 0.0 11:12.72 migration/2
15 root 20 0 0 0 0 S 0 0.0 93:46.97 ksoftirqd/2
16 root 20 0 0 0 0 S 0 0.0 0:00.00 kworker/2:0
17 root 0 -20 0 0 0 S 0 0.0 0:00.00 kworker/2:0H
18 root RT 0 0 0 0 S 0 0.0 11:13.05 migration/3
19 root 20 0 0 0 0 S 0 0.0 93:26.51 ksoftirqd/3
20 root 20 0 0 0 0 S 0 0.0 0:00.00 kworker/3:0
21 root 0 -20 0 0 0 S 0 0.0 0:00.00 kworker/3:0H
22 root RT 0 0 0 0 S 0 0.0 12:02.22 migration/4
23 root 20 0 0 0 0 S 0 0.0 89:39.61 ksoftirqd/4
24 root 20 0 0 0 0 S 0 0.0 0:00.00 kworker/4:0
25 root 0 -20 0 0 0 S 0 0.0 0:00.00 kworker/4:0H
26 root RT 0 0 0 0 S 0 0.0 12:19.94 migration/5
27 root 20 0 0 0 0 S 0 0.0 93:40.97 ksoftirqd/5
28 root 20 0 0 0 0 S 0 0.0 0:00.00 kworker/5:0
29 root 0 -20 0 0 0 S 0 0.0 0:00.00 kworker/5:0H
30 root RT 0 0 0 0 S 0 0.0 12:06.26 migration/6
That may or may not be helpful, but this is where I start to run short on understanding what's going on.
@akerl:
Go look at the processes section to see which processes that refers to
If you mean look at the Process Explorer section of Longview, nothing jumps out at me there. The highest CPU consumer is mysqld at just 2%.