Let's Encrypt on Kubernetes with cert-manager: webhook failure when creating ClusterIssuer

Hello,

While following this Linode HOW-TO on setting up Let's Encrypt TLS on Linode Kubernetes Engine, I get the following error when I try to create a ClusterIssuer CRD:

$ kubectl create -f acme-issuer-staging.yaml
Error from server (InternalError): error when creating "acme-issuer-staging.yaml": Internal error occurred: failed calling webhook "webhook.cert-manager.io": Post https://cert-manager-webhook.cert-manager.svc:443/mutate?timeout=10s: dial tcp 10.128.124.39:443: i/o timeout

Has anybody else encountered this?

5 Replies

In trying to recreate this issue, I was able to successfully create a ClusterIssuer CRD following the linked instructions - I didn't run into the same error as above.

First, if you haven't already, I would check that your cert-manager pods are running:

kubectl get pods --namespace cert-manager

The output should look similar to the below:

NAME                                      READY   STATUS    RESTARTS   AGE
cert-manager-7747db9d88-htjnk             1/1     Running   0          30m
cert-manager-cainjector-87c85c6ff-wkg68   1/1     Running   0          30m
cert-manager-webhook-64dc9fff44-ggjhj     1/1     Running   0          30m

If that checks out, you may want to consider checking the logs for that cert-manager (replace "64dc9fff44-ggjhj" with your correct input), as this may help pinpoint the issue:

kubectl logs cert-manager-webhook-64dc9fff44-ggjhj -n cert-manager

It seems as though others have run into this issue, too - some were able to find resolutions, some not - it would largely depend on your configuration. Many have pinpointed firewall rules as the main problem, others have cited outdated versions of various applications, which could be worth looking into:

I'm having the same issue.
The webhook pod is never ready

NAME                                       READY   STATUS    RESTARTS   AGE
cert-manager-75dbbd5d6-g5bnd               1/1     Running   0          2m15s
cert-manager-cainjector-85c559fd6c-qs2ms   1/1     Running   3          2m15s
cert-manager-webhook-6c77dfbdb8-546tq      0/1     Running   0          2m15s

and the logs of the pod are like so:

E1119 11:30:45.504379       1 dynamic_source.go:88] cert-manager/webhook "msg"="Failed to generate initial serving certificate, retrying..." "error"="failed verifying CA keypair: tls: failed to find any PEM data in certificate input"  "interval"=1000000000

Also noticing that these other pods have issues (unsure if it's related):

kube-system    pod/calico-kube-controllers-6c5ccf657c-g2dfn         0/1     CrashLoopBackOff        444        38h
kube-system    pod/csi-linode-controller-0                          0/4     Init:0/1                330        38h
kube-system    pod/csi-linode-node-xkdxv                            0/2     Init:CrashLoopBackOff   330        38h

If you also want to support DNS01 (e.g. to have wildcard certificates), you might want to have a look at our webhook adapter: https://github.com/monostream/cert-manager-linode

there is a similar project here: https://github.com/slicen/cert-manager-webhook-linode but has not worked in my tests using sub-domains.

Hi @jtoscani
I'm with this same problem now, my certs are coming to expire and I cannot reissue them.

I tried to uninstall and reinstall and already read a ton of pages.

Could you help?

https://www.linode.com/community/questions/24953/cert-manager-problem-with-webhook-io-timeout

I'm also going through the same issues, I'm glad it's not just me. I've used similar setup on Digital Ocean and it was fine, but Linode seems to have issues.

Reply

Please enter an answer
Tips:

You can mention users to notify them: @username

You can use Markdown to format your question. For more examples see the Markdown Cheatsheet.

> I’m a blockquote.

I’m a blockquote.

[I'm a link] (https://www.google.com)

I'm a link

**I am bold** I am bold

*I am italicized* I am italicized

Community Code of Conduct