Kernel page allocation failure

Question

Kernel page allocation failure

general

This morning I had some trouble with network connectivity to my server. I noticed the the following kernel stack trace in /var/log/messages. I will assume it's nothing and reboot for good measure but I thought I'd post it for posterity.

Mike

…

Jul 22 09:15:06 www ntpd[2880]: kernel time sync enabled 0001

Jul 22 09:49:15 www ntpd[2880]: kernel time sync enabled 4001

Jul 22 10:23:22 www ntpd[2880]: kernel time sync enabled 0001

Jul 22 11:44:31 www kernel: swapper: page allocation failure. order:5, mode:0x20

Jul 22 11:44:31 www kernel: Pid: 0, comm: swapper Not tainted 2.6.38.3-linode32 #1

Jul 22 11:44:31 www kernel: Call Trace:

Jul 22 11:44:31 www kernel: [] ? _allocpages_nodemask+0x4f0/0x670

Jul 22 11:44:31 www kernel: [] ? cacheallocrefill+0x2f9/0x520

Jul 22 11:44:31 www kernel: [] ? __kmalloc+0xb4/0xe0

Jul 22 11:44:31 www kernel: [] ? pskbexpandhead+0x12e/0x200

Jul 22 11:44:31 www kernel: [] ? xenforceevtchn_callback+0x17/0x30

Jul 22 11:44:31 www kernel: [] ? _pskbpull_tail+0x4d/0x2b0

Jul 22 11:44:31 www kernel: [] ? xenrestorefldirectend+0x0/0x1

Jul 22 11:44:31 www kernel: [] ? devhardstart_xmit+0x26a/0x520

Jul 22 11:44:31 www kernel: [] ? schdirectxmit+0xb2/0x170

Jul 22 11:44:31 www kernel: [] ? nf_iterate+0x74/0xa0

Jul 22 11:44:31 www kernel: [] ? ipfinishoutput+0x0/0x300

Jul 22 11:44:31 www kernel: [] ? devqueuexmit+0x121/0x550

Jul 22 11:44:31 www kernel: [] ? ipfinishoutput+0x0/0x300

Jul 22 11:44:31 www kernel: [] ? ipfinishoutput+0x134/0x300

Jul 22 11:44:31 www kernel: [] ? ip_output+0xaa/0xe0

Jul 22 11:44:31 www kernel: [] ? ipfinishoutput+0x0/0x300

Jul 22 11:44:31 www kernel: [] ? iplocalout+0x18/0x20

Jul 22 11:44:31 www kernel: [] ? ipqueuexmit+0x137/0x3a0

Jul 22 11:44:31 www kernel: [] ? xenforceevtchn_callback+0x17/0x30

Jul 22 11:44:31 www kernel: [] ? tcptransmitskb+0x372/0x7e0

Jul 22 11:44:31 www kernel: [] ? tcpwritexmit+0x198/0x980

Jul 22 11:44:31 www kernel: [] ? _tcppushpendingframes+0x24/0x90

Jul 22 11:44:31 www kernel: [] ? tcprcvestablished+0x146/0x840

Jul 22 11:44:31 www kernel: [] ? tcpv4do_rcv+0xd6/0x230

Jul 22 11:44:31 www kernel: [] ? localbhenable+0x16/0x80

Jul 22 11:44:31 www kernel: [] ? tcpv4rcv+0x6cc/0x7b0

Jul 22 11:44:31 www kernel: [] ? iplocaldeliver_finish+0x97/0x220

Jul 22 11:44:31 www kernel: [] ? iplocaldeliver_finish+0x0/0x220

Jul 22 11:44:31 www kernel: [] ? iprcvfinish+0xf6/0x3c0

Jul 22 11:44:31 www kernel: [] ? _netifreceive_skb+0x32d/0x510

Jul 22 11:44:31 www kernel: [] ? xenforceevtchn_callback+0x17/0x30

Jul 22 11:44:31 www kernel: [] ? netifreceiveskb+0x67/0x70

Jul 22 11:44:31 www kernel: [] ? xennet_poll+0x7f5/0xc20

Jul 22 11:44:31 www kernel: [] ? netrxaction+0x9a/0x130

Jul 22 11:44:31 www kernel: [] ? _dosoftirq+0x7c/0x130

Jul 22 11:44:31 www kernel: [] ? _dosoftirq+0x0/0x130

Jul 22 11:44:31 www kernel: [] ? irq_exit+0x65/0x70

Jul 22 11:44:31 www kernel: [] ? xenevtchndo_upcall+0x1d/0x30

Jul 22 11:44:31 www kernel: [] ? xendoupcall+0x7/0xc

Jul 22 11:44:31 www kernel: [] ? hypercall_page+0x3a7/0x1010

Jul 22 11:44:31 www kernel: [] ? xensafehalt+0xf/0x20

Jul 22 11:44:31 www kernel: [] ? default_idle+0x2f/0x60

Jul 22 11:44:31 www kernel: [] ? cpu_idle+0x42/0x70

Jul 22 11:44:31 www kernel: [] ? start_kernel+0x2da/0x2df

Jul 22 11:44:31 www kernel: [] ? unknown_bootoption+0x0/0x190

Jul 22 11:44:31 www kernel: [] ? xenstartkernel+0x530/0x538

Jul 22 11:44:31 www kernel: Mem-Info:

Jul 22 11:44:31 www kernel: DMA per-cpu:

Jul 22 11:44:31 www kernel: CPU 0: hi: 0, btch: 1 usd: 0

Jul 22 11:44:31 www kernel: CPU 1: hi: 0, btch: 1 usd: 0

Jul 22 11:44:31 www kernel: CPU 2: hi: 0, btch: 1 usd: 0

Jul 22 11:44:31 www kernel: CPU 3: hi: 0, btch: 1 usd: 0

Jul 22 11:44:31 www kernel: Normal per-cpu:

Jul 22 11:44:31 www kernel: CPU 0: hi: 186, btch: 31 usd: 150

Jul 22 11:44:31 www kernel: CPU 1: hi: 186, btch: 31 usd: 135

Jul 22 11:44:31 www kernel: CPU 2: hi: 186, btch: 31 usd: 108

Jul 22 11:44:31 www kernel: CPU 3: hi: 186, btch: 31 usd: 180

Jul 22 11:44:31 www kernel: activeanon:15618 inactiveanon:15212 isolated_anon:0

Jul 22 11:44:31 www kernel: activefile:49699 inactivefile:33152 isolated_file:0

Jul 22 11:44:31 www kernel: unevictable:1136 dirty:1 writeback:0 unstable:0

Jul 22 11:44:31 www kernel: free:3443 slabreclaimable:3771 slabunreclaimable:1830

Jul 22 11:44:31 www kernel: mapped:3869 shmem:105 pagetables:507 bounce:0

Jul 22 11:44:31 www kernel: DMA free:3044kB min:84kB low:104kB high:124kB activeanon:0kB inactiveanon:140kB activefile:668kB inactivefile:2728kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15808kB mlocked:0kB dirty:0kB writeback:0kB mapped:72kB shmem:0kB slabreclaimable:128kB slabunreclaimable:392kB kernelstack:32kB pagetables:0kB unstable:0kB bounce:0kB writebacktmp:0kB pagesscanned:0 allunreclaimable? no

Jul 22 11:44:31 www kernel: lowmem_reserve[]: 0 500 500 500

Jul 22 11:44:31 www kernel: Normal free:10728kB min:2816kB low:3520kB high:4224kB activeanon:62472kB inactiveanon:60708kB activefile:198128kB inactivefile:129880kB unevictable:4544kB isolated(anon):0kB isolated(file):0kB present:512064kB mlocked:4544kB dirty:4kB writeback:0kB mapped:15404kB shmem:420kB slabreclaimable:14956kB slabunreclaimable:6928kB kernelstack:824kB pagetables:2028kB unstable:0kB bounce:0kB writebacktmp:0kB pagesscanned:0 allunreclaimable? no

Jul 22 11:44:31 www kernel: lowmem_reserve[]: 0 0 0 0

Jul 22 11:44:31 www kernel: DMA: 4094kB 628kB 2516kB 432kB 464kB 1128kB 0256kB 0512kB 01024kB 02048kB 0*4096kB = 3044kB

Jul 22 11:44:31 www kernel: Normal: 25404kB 658kB 316kB 032kB 064kB 0128kB 0256kB 0512kB 01024kB 02048kB 0*4096kB = 10728kB

Jul 22 11:44:31 www kernel: 84794 total pagecache pages

Jul 22 11:44:31 www kernel: 966 pages in swap cache

Jul 22 11:44:31 www kernel: Swap cache stats: add 14067, delete 13101, find 200063/200381

Jul 22 11:44:31 www kernel: Free swap = 255592kB

Jul 22 11:44:31 www kernel: Total swap = 262140kB

Jul 22 11:44:31 www kernel: 133104 pages RAM

Jul 22 11:44:31 www kernel: 0 pages HighMem

Jul 22 11:44:31 www kernel: 5637 pages reserved

Jul 22 11:44:31 www kernel: 90512 pages shared

Jul 22 11:44:31 www kernel: 68737 pages non-shared

Jul 22 12:13:15 www kernel: device eth0 entered promiscuous mode

Jul 22 12:13:21 www kernel: device eth0 left promiscuous mode

Jul 22 12:13:23 www kernel: device eth0 entered promiscuous mode

Jul 22 12:13:29 www kernel: device eth0 left promiscuous mode

Jul 22 12:14:24 www kernel: device eth0 entered promiscuous mode

Jul 22 12:14:29 www kernel: device eth0 left promiscuous mode

Jul 22 12:25:09 www dhclient: DHCPREQUEST on eth0 to 64.22.84.11 port 67

Jul 22 12:25:09 www dhclient: DHCPACK from 64.22.84.11

Jul 22 12:25:09 www dhclient: bound to 173.230.137.181 – renewal in 42564 seconds.

Jul 22 13:14:00 www ntpd[2880]: kernel time sync enabled 4001

4 Replies

forum:Nibbler · Answer 1 · July 22, 2011, 1:59 p.m.

forum:Nibbler 13 years, 5 months ago

I think that's been fixed in more recent kernels.

caker · Answer 2 · July 22, 2011, 2:27 p.m.

caker 13 years, 5 months ago Linode Staff

Thanks for the report. Yes, please always try to reproduce using the latest kernel.

-Chris

forum:ioplex · Answer 3 · July 22, 2011, 2:35 p.m.

forum:ioplex 13 years, 5 months ago

~~@caker:~~

Thanks for the report. Yes, please always try to reproduce using the latest kernel.
How does one use the latest kernel? My profile reads:

My CentOS 5.5 Profile (Latest 2.6 Paravirt (2.6.39.1-linode34))

even though the log fragement mentions 2.6.38.3-linode32.

Do I simply reboot?

I have not actually rebooted yet. I was going to wait until later tonight.

Mike

forum:glg · Answer 4 · July 22, 2011, 3:06 p.m.

forum:glg 13 years, 5 months ago

If you haven't changed the profile, then you probably haven't rebooted since 2.6.38.3-linode32 was latest. Answer is indeed to simply reboot.

Compute

Storage

Networking

Databases

Services

Developer Tools

Industries

Pricing

Community

Engage With Us

Kernel page allocation failure

4 Replies

Reply

Tips: