How to whitelist certain IPs on an ingress on k8s?

I have a k8s deployment and ingress on my Linode K8s cluster that I would like to only be available to specific IPs on the Internet.

In my ingress yaml I have put this annotation:

ingress.kubernetes.io/whitelist-source-range: "x.x.x.x, y.y.y.y"

After putting that in, I am now getting 403 forbidden from every IP I try from, eg x.x.x.x, y.y.y.y and z.z.z.z is forbidden.

The ingress-nginx pod logs show this:

2023/03/23 22:29:40 [error] 718#718: *94091058 access forbidden by rule, client: 10.2.0.1, server: myapp.mydomain.com, request: "GET / HTTP/2.0", host: "myapp.mydomain.com"

According to this post: https://stackoverflow.com/questions/47990836/kubernetes-whitelist-source-range-blocks-instead-of-whitelist-ip

I also needed to set the ingress-nginx controller service to externalTrafficPolicy: Local

I change this setting, but it still says 403 forbidden from all public IPs.

Looking at the ingress-nginx-controller pod logs, I see:

2023/03/23 22:36:51 [error] 718#718: *94102418 access forbidden by rule, client: 192.168.255.41, server: myapp.mydomain.com, request: "GET / HTTP/2.0", host: "myapp.mydomain.com"

Changing the externalTrafficPolicy looks like the request now comes from 192.168.255.41 which is still not a public IP but it's not a 10.x.x.x cluster IP address (the Linode NodeBalancer private IP connecting to my cluster?)

I only want to use the whitelist on one of my deployed apps, not all of them.

Any help configuring this whitelist would be greatly appreciated.