K8S - PersistentVolume and node failure
I would like to know if the attached storage to a given worker node in K8s cluster will be re-attached to another node should that node fail or brought down for maintenance?
considering that I'm using Linode Block Storage CSI Driver.
Example:
MERN app requires access to MongoDB which has attached block storage attached to worker node 1 in 3 node clusters.
Node 1 fails, will k8S CSI driver will disconnect attached storage and connect it to node 2 or 3? (Since its a fully managed k8s) I'm guessing there is no action I'm required to take when this issue occurs.
1 Reply
jdutton
Linode Staff
Hey @deank - I reached out to our LKE team about this one, and the short answer here is yes!
The CSI driver will attach the Volume to the node on which the pod is running. That means that if a node goes down, the pod will be evicted and rescheduled to an available node. The only downside is that the process can take up to 30 minutes. According to our LKE team, this timeframe is something they're working on reducing.
If you need to avoid downtime altogether, we actually have another Community site post that outlines steps you can take to migrate your workload to new nodes that don't need to undergo maintenance. Here's a link to the post:
How can I prepare an LKE Cluster for Scheduled Maintenance?
Otherwise, there shouldn't be any action you would need to take in terms of detaching/reattaching the Block Storage Volume.
Hope that answers your question!