ProxyProtocol, NodeBalancer, Traefik and LKE
Hello!
I'm trying to get NodeBalancer + ProxyProtocol + Traefik working with my LKE cluster and I am confused about which IPs I should whitelist for ProxyProtocol in Traefik.
I have a single node cluster, and if I whitelist the IP of this node then ProxyProtocol works however this confuses me. Should it not suffice to whitelist the IP of the load balancer?
LoadBalancer Service:
apiVersion: v1
kind: Service
metadata:
name: linode-nb-xxyyzz
annotations:
service.beta.kubernetes.io/linode-loadbalancer-preserve: true
service.beta.kubernetes.io/linode-loadbalancer-nodebalancer-id: xxyyzz
service.beta.kubernetes.io/linode-loadbalancer-default-proxy-protocol: v2
spec:
type: LoadBalancer
selector:
app: traefik
ports:
- protocol: TCP
port: 80
name: web
targetPort: 80
- protocol: TCP
port: 443
name: websecure
targetPort: 443
Traefik Deployment:
apiVersion: apps/v1
kind: Deployment
metadata:
name: traefik
labels:
app: traefik
spec:
replicas: 1
selector:
matchLabels:
app: traefik
template:
metadata:
labels:
app: traefik
spec:
serviceAccountName: traefik-ingress-controller
containers:
- name: traefik
image: traefik:v2.5
args:
- --log.level=debug
- --api.dashboard=true
- --api.insecure=true
- --entryPoints.web.address=:80
- --entryPoints.web.proxyProtocol.trustedIPs=192.168.255.0/24,172.xxx.yyy.zzz
- --entryPoints.websecure.address=:443
- --providers.kubernetesingress
ports:
- name: web
containerPort: 80
- name: websecure
containerPort: 443
imagePullSecrets:
- name: regcred
(172.xxx.yyy.zzz is the IP of my NodeBalancer and 192.168.255.0/24 is the range of private IPs that the NodeBalancer can have)
This does not work, curling whoami:
$ curl 172.xxx.yyy.zzz/all
Hostname: whoami-5c47f9fb66-99fnc
IP: 127.0.0.1
IP: 10.2.0.48
RemoteAddr: 10.2.0.51:33100
GET /all HTTP/1.1
Host: 172.xxx.yyy.zzz
User-Agent: curl/7.74.0
Accept: */*
Accept-Encoding: gzip
X-Forwarded-For: 139.xxx.yyy.zzz
X-Forwarded-Host: 172.xxx.yyy.zzz
X-Forwarded-Port: 80
X-Forwarded-Proto: http
X-Forwarded-Server: traefik-6966d8c548-vgzf8
X-Real-Ip: 139.xxx.yyy.zzz
X-Real-Ip and X-Forwarded-For are set to the ip of the node. Also the Traefik logs say:
time="2021-10-10T15:08:47Z" level=debug msg="IP 139.xxx.yyy.zzz is not in trusted IPs list, ignoring ProxyProtocol Headers and bypass connection" entryPointName=web
If I instead use
--entryPoints.web.proxyProtocol.trustedIPs=139.xxx.yyy.zzz
In Traefik Deployment then it works as expected.
Am I missing something or will I need to explicitly whitelist each node by IP? Is this safe?
2 Replies
I have exactly the same questions.
Furthermore, I don't know from where the 139.xxx.yyy.zzz (mine are like 95.xxx.yyy.zzz and 192.168.255.144) comes from.
Even after almost 3 years of posting this question, the problem still exists. nodebalancer-id
annotation does not work. Removed the annotation and started using external-dns
to automatically setup DNS for the allocated external-ip
.