Network issues with latest Kernel
I have noticed that the network interfaces for my Linode are having issues after using a kernel after 4.15.13-x86_64-linode106. Latest is now -linode110 and when I reboot, I see repeated messages in /var/log/message like this:
Jul 31 10:01:03 optimus dhclient[1700]: DHCPDISCOVER on lan1 to 255.255.255.255 port 67 interval 15 (xid=0x1bc37b30)
Jul 31 10:01:18 optimus dhclient[1700]: DHCPDISCOVER on lan1 to 255.255.255.255 port 67 interval 18 (xid=0x1bc37b30)
Jul 31 10:01:36 optimus dhclient[1700]: DHCPDISCOVER on lan1 to 255.255.255.255 port 67 interval 15 (xid=0x1bc37b30)
Jul 31 10:01:40 optimus NetworkManager[638]: <warn> [1533056500.5511] dhcp4 (lan1): request timed out
Jul 31 10:01:40 optimus NetworkManager[638]: <info> [1533056500.5512] dhcp4 (lan1): state changed unknown -> timeout
Jul 31 10:01:40 optimus NetworkManager[638]: <info> [1533056500.6160] dhcp4 (lan1): canceled DHCP transaction, DHCP client pid 1700
Jul 31 10:01:40 optimus NetworkManager[638]: <info> [1533056500.6161] dhcp4 (lan1): state changed timeout -> done
Jul 31 10:01:40 optimus NetworkManager[638]: <info> [1533056500.6168] device (lan1): state change: ip-config -> failed (reason 'ip-config-unavailable', sys-iface-state: 'managed')
Jul 31 10:01:40 optimus NetworkManager[638]: <warn> [1533056500.6172] device (lan1): Activation: failed for connection 'Wired connection 1'
Jul 31 10:01:40 optimus NetworkManager[638]: <info> [1533056500.6176] device (lan1): state change: failed -> disconnected (reason 'none', sys-iface-state: 'managed')
Jul 31 10:01:40 optimus NetworkManager[638]: <info> [1533056500.6352] policy: auto-activating connection 'Wired connection 1'
Jul 31 10:01:40 optimus NetworkManager[638]: <info> [1533056500.6364] device (lan1): Activation: starting connection 'Wired connection 1' (23ff739f-79bf-3d65-9552-3391be50aa00)
Jul 31 10:01:40 optimus NetworkManager[638]: <info> [1533056500.6366] device (lan1): state change: disconnected -> prepare (reason 'none', sys-iface-state: 'managed')
Jul 31 10:01:40 optimus NetworkManager[638]: <info> [1533056500.6369] device (lan1): state change: prepare -> config (reason 'none', sys-iface-state: 'managed')
Jul 31 10:01:40 optimus NetworkManager[638]: <info> [1533056500.6374] device (lan1): state change: config -> ip-config (reason 'none', sys-iface-state: 'managed')
Jul 31 10:01:40 optimus NetworkManager[638]: <info> [1533056500.6376] dhcp4 (lan1): activation: beginning transaction (timeout in 45 seconds)
Jul 31 10:01:40 optimus NetworkManager[638]: <info> [1533056500.6398] dhcp4 (lan1): dhclient started with pid 1738
Jul 31 10:01:40 optimus dhclient[1738]: DHCPDISCOVER on lan1 to 255.255.255.255 port 67 interval 6 (xid=0x54f4509d)
Jul 31 10:01:46 optimus dhclient[1738]: DHCPDISCOVER on lan1 to 255.255.255.255 port 67 interval 9 (xid=0x54f4509d)</info></info></info></info></info></info></info></info></warn></info></info></info></info></warn>
Whatever is the underlying issue appears to be affecting web services on my Linode.
I am on CentOS Linux release 7.5.1804 (Core).
Any ideas on what I need to look into?
Thanks.
11 Replies
I misspoke when I said it was affected web services.
But I did isolate it to the fact that for some reason, NetworkManager was thinking there were more interfaces to manage: lan1 through lan4, wlan0, and wlan0.
I was able to make some changes to the NetworkManager.conf file so it was able to recognize that lan1 through lan4 was not connected so dhclient isn't trying to poll for an IP, but for some reason, wlan0 and wlan1 appear as interfaces on the system. I don't have network scripts declaring these two interfaces. Can't locate how they are defined.
sjacobs
Linode Staff
We have been seeing some sporadic issues with our latest kernel. Can you please try selecting the following kernel in your Configuration Profile, saving, and rebooting?
4.17.11-x86_64-linode111
We made some changes and hope it resolves your issues. If you give it a try, please let us know how it goes.
Thanks in advance!
Yes, this one seems to eliminate all the errant network interfaces.
An nmcli device status only shows eth0 and lo, with nothing else. /var/log/messages show no dhclient errors for interface dhcp polling.
Though build 111 has its own issues; seems to be missing the universal tun/tap driver now:
vpn openvpn: Tue Jul 31 20:25:49 2018 ERROR: Cannot open TUN/TAP dev /dev/net/tun: No such file or directory (errno=2)
ouch thanks @hkrtrsh for heads up. Any Linode staff members can confirm if the build 1111 kernel has TUN/TAP disabled ?
looks like 4.17.12-x86_64-linode112 was just released https://www.linode.com/kernels/?ver=1533646944#312
4.17.14-x86_64-linode113 is out https://www.linode.com/kernels/?ver=1533933336#138
https://cdn.kernel.org/pub/linux/kernel/v4.x/ChangeLog-4.17.14
@sjacobs looks like 4.17 kernels all result in no boot up for me with CentOS 7.5 64bit linode vpses ??
4.17 fails to boot up with Glish hanging at message
Welcome to emergency mode!
right after
Started Initialize the iWARP/InfiniBand/RDMA stack in the kernel line
tried following kernels
- 4.17.14-x86_64-linode113
- 4.17.12-x86_64-linode112
- 4.17.11-x86_64-linode111
4.17.14-x86_64-linode113 breaks docker for me.
docker daemon returns this whenever I try to run a container:
failed to create endpoint (…) on network bridge: failed to add the host (…) <=> sandbox (…) pair interfaces: operation not supported.
I reverted to 4.14.19-x86_64-linode100 and the issue resolved.