Is there a best practice for (bash) scripting LKE cluster to deployment?
I'm (bash) scripting LKE cluster creation and the Deployment of a container that requires imageSecrets
and a PVC.
I'm challenged by:
- Best signal for successful (ready) cluster creation.
- After Namespace creation, I occasionally get errors creating a Pod
- After PVC deletion, the Linode Volume does not appear to auto-delete
lke cluster-create
and lke cluster-list
do not appear to include a return value that can be used as a signal for the cluster's state:
[
{
"id": 12345,
"created": "2023-02-14T00:00:00",
"updated": "2023-02-14T00:00:00",
"label": "foo",
"region": "us-west",
"k8s_version": "1.25",
"control_plane": {
"high_availability": false
},
"tags": []
}
]
It would be useful if getting clusters included some value as to the server's state.
Instead, I'm using lke kubeconfig-view ${ID} --json
and until
the returned JSON includes kubeconfig
rather than [{"field":"","reason":"Cluster kubeconfig is not yet available"}]
Is this the best way to do this?
NOTE Too small gaps between
lke cluster-create
andlke cluster-delete
can delete the cluster without deleting underlying Linodes.
When I'm done, I delete everything (including the PVC). It's defined with:
storageClassName: linode-block-storage
But, the Linode Volume does not get deleted even after the Linode that it's bound to is deleted with the cluster's deletion. I'm having to enumerate the volumes as a fail-safe and delete any that remain.
NOTE Linode
NodeBalancers
are correctly programmed (creation|deletion) as Kubernetes Servicestype: LoadBalancer
areapply
|delete
.
Scripts
Constants
IMAGE_LINODE="docker.io/linode/cli:5.31.0'
Create Cluster & KUBECONFIG
function create_linode_cluster () {
local IMAGE="${IMAGE_LINODE}"
local LABEL="foo"
# Get latest Kubernetes version
local VERSION=$(\
podman run \
--interactive --tty --rm \
--volume=${HOME}/.config/linode-cli:/home/cli/.config/linode-cli \
${IMAGE} \
lke versions-list --json \
| jq -r '.[0].id') && echo ${VERSION}
if [ -z "${VERSION}" ]
then
echo "Unable to determine Kubernetes version: ${VERSION}"
return 1
fi
local ID=$(\
podman run \
--interactive --rm \
--volume=${HOME}/.config/linode-cli:/home/cli/.config/linode-cli \
${IMAGE} \
lke cluster-create \
--label=${LABEL} \
--region=us-west \
--k8s_version=${VERSION} \
--node_pools.type=g6-standard-1 \
--node_pools.count=1 \
--no-defaults \
--json \
| jq -r '.[0].id') && echo ${ID}
# Export LKE KUBECONFIG to temp file
local CONFIG="${PWD}/tmp/linode.$(date +%y%m%d).yaml"
local KUBECONFIG=""
until [ -n "${KUBECONFIG}" ]
do
sleep 15s
KUBECONFIG=$(\
podman run \
--interactive --rm \
--volume=${HOME}/.config/linode-cli:/home/cli/.config/linode-cli \
${IMAGE} \
lke kubeconfig-view ${ID} --json \
| jq -r .[0].kubeconfig)
done
echo ${KUBECONFIG} | base64 --decode > ${CONFIG}
}
Create PVC, Secret, Deployment
function create_linode_foo_server () {
local NAME="foo"
local HOST="foo.example.com"
local NAMESPACE="bar"
local IMAGE=${IMAGE_FOO}
local CONFIG="${PWD}/tmp/linode.$(date +%y%m%d).yaml"
kubectl \
create namespace ${NAMESPACE} \
--kubeconfig=${CONFIG}
# Create PVC
kubectl apply \
--filename=${PWD}/linode/kubernetes/pvc.yaml \
--kubeconfig=${CONFIG} \
--namespace=${NAMESPACE}
# Create temporary Pod w/ PVC mount to X509 cert
kubectl apply \
--filename=${PWD}/linode/kubernetes/shell.yaml \
--kubeconfig=${CONFIG} \
--namespace=${NAMESPACE}
# Wait for it to come Ready
kubectl wait pod/shell \
--for=condition=Ready \
--namespace=${NAMESPACE} \
--kubeconfig=${CONFIG}
# Copy X509 cert and account into PVC
kubectl cp \
${PWD}/linode/certs/. \
${NAMESPACE}/shell:/certs \
--kubeconfig=${CONFIG}
# Delete temporary Pod only
kubectl delete \
--filename=${PWD}/linode/kubernetes/shell.yaml \
--kubeconfig=${CONFIG} \
--namespace=${NAMESPACE}
# Required because IMAGE_FOO is private
kubectl create secret docker-registry ghcr \
--kubeconfig=${CONFIG} \
--namespace=${NAMESPACE} \
--docker-server=reg.io \
--docker-username=${USER} \
--docker-password=${PASS} \
--docker-email=${EMAIL}
local DEPLOYMENT="${PWD}/tmp/linode.${NAME}.$(date +%y%m%d).yaml"
sed \
--expression="s|image: IMAGE|image: ${IMAGE}|g" \
--expression="s|host=HOST|host=${HOST}|g" \
${PWD}/linode/kubernetes/autocert.tmpl > ${DEPLOYMENT}
# Deploy autocert server w/ PVC mount to access ${HOST} X509 cert
kubectl apply --filename=${DEPLOYMENT} \
--kubeconfig=${CONFIG} \
--namespace=${NAMESPACE}
}
1 Reply
You may find that Terraform and the Linode Terraform Provider offer some of the functionality you're looking for with a bit more ease of use. Looking over the Deploy a Linode Kubernetes Engine Cluster Using Terraform documentation, a success message is printed to your output after the cluster is created with Terraform.
You also mention that your PVC is not being deleted even though it is defined as storageClassName: linode-block-storage
. Again, according to the documentation, this should allow for the PVC to be deleted when the cluster is deleted. If this is not occurring during your deletion process, I suggest doublechecking what you have named above as pvc.yaml
file to ensure the configs are correct. You may also want to try to delete the namespace where your PVC resides first, which will delete the PVC and the pod it is attached to, then delete the cluster. If this doesn't resolve your issue, I suggest bringing this to the attention of the Support Team in a ticket.