/v1/ssh-keys is empty when deploying a private image
Hello,
I noticed that /v1/ssh-keys returns an empty value when booting a Linode image from a private image with a custom config.
core@172-234-24-26 ~ $ curl -H "Metadata-Token: $TOKEN" http://169.254.169.254/v1/ssh-keys
core@172-234-24-26 ~ $ curl -H "Metadata-Token: $TOKEN" http://169.254.169.254/v1/network
ipv4.public: 172.234.24.26/32
ipv6.link_local: fe80::f03c:94ff:fe1a:6760/128
ipv6.slaac: 2600:3c06::f03c:94ff:fe1a:6760/128
I created and booted the Linode by following those steps:
$ export REGION="us-ord"
$ linode-cli linodes create \
--region "${REGION}" \
--booted false \
--type g6-nanode-1 \
--metadata.user_data "$(base64 -w0 ignition.json)" \
--label flatcar \
--no-defaults
$ export LINODE_ID=$(linode-cli linodes list --label flatcar --json | jq -r '.[0].id')
$ linode-cli linodes config-create \
--kernel linode/direct-disk \
--helpers.updatedb_disabled true \
--helpers.distro false \
--helpers.modules_dep false \
--helpers.network false \
--helpers.devtmpfs_automount false \
--label default \
--devices.sda.disk_id "${DISK_ID}" \
--root_device sda \
"${LINODE_ID}"
$ linode-cli linodes boot "${LINODE_ID}"
I also tried to create the instance with --authorized_users $USER but same result. I have two public SSH keys on my account.
Thanks and let me know if you need more information.
Mathieu
5 Replies
CassandraD
Linode Staff
I think first making sure that the SSH-keys are actually being added to the instance is the first step. I believe there are two places you can pass the ssh-keys to your server when creating it with the CLI.
Either it needs to be included in your cloud-config file discussed in this guide or by adding the param in the CLI call, as discussed in this guide.
Those linked resources provide examples of how to format the ssh-key in those specific situations. I'd recommend giving both of those methods a try and seeing if that helps.
Also, I confirmed I get the same result from the meta-data call you shared as I do when I run cat ~/.ssh/authorized_keys so checking that file to make sure the keys are there might be helpful.
Otherwise, it might be helpful to try to run the command from inside a server made from a cloud-init compatible public image just to see if only the private image is having problems. You can run linode-cli images list to see which images work with cloud-init.
Feel free to follow up if you're still struggling and share anything you learned when trying any of these steps.
Hi,
I uploaded a new-image with the cloud-init compatible flag and it worked. Thanks!
I think it's a bit counter-intuitive as the purpose of this kind of endpoint (/v1/ssh-keys) is to provide keys without initial provisioning (e.g cloud-init).
Hello, I think I was mistaken. The issue is still there, I did some Cluster API tests and the instance does not get SSH keys from the instance metadata service (while my profile has SSH keys). I just have an empty result when querying the ssh-keys endpoint.
CassandraD
Linode Staff
If you think that something on our side isn't working as intended, feel free to open a Support Ticket.
At the moment, I don't think I have enough information to add much here. Can you elaborate on what steps you're taking and what you're trying to accomplish? I know you added some information before, but I think it would be helpful to make sure we're on the same page.
Can you provide the command you're using the query the endpoint?
Can you try to log into the server over ssh using the -v flag to see if that provides any information.
Again, if something on our side isn't working, feel free to open a ticket. Otherwise, if you can let us know the most recent steps you took, hopefully someone in our Community can provide more precise help.
Hi, I am sorry for the long delay - I missed your answer. Flatcar OS relies on two separate mechanisms to provision the instance:
- Ignition will provision the instance from the initramfs
- Afterburn will configure the instance after the booting the instance by setting some values known after boot (e.g ssh keys from metadata service or IP addresses)
Afterburn is supposed to fetch SSH public keys from the Akamai metadata service (https://techdocs.akamai.com/cloud-computing/docs/metadata-service-api#ssh-keys-v1ssh-keys) but it fails because Akamai metadata service returns an empty result:
core@172-236-119-109 ~ $ journalctl --boot -u coreos-metadata-sshkeys@core.service
Sep 09 14:18:27 localhost systemd[1]: Starting coreos-metadata-sshkeys@core.service - Flatcar Metadata Agent (SSH Keys)...
Sep 09 14:18:27 localhost coreos-metadata[1431]: Sep 09 14:18:27.504 INFO Putting http://169.254.169.254/v1/token: Attempt #1
Sep 09 14:18:27 localhost coreos-metadata[1431]: Sep 09 14:18:27.662 INFO Fetching http://169.254.169.254/v1/ssh-keys: Attempt #1
Sep 09 14:18:27 172-236-119-109 coreos-metadata[1431]: Sep 09 14:18:27.809 INFO Fetch successful
Sep 09 14:18:27 172-236-119-109 update-ssh-keys[1469]: Error: failed to update authorized keys
Sep 09 14:18:27 172-236-119-109 update-ssh-keys[1469]: Caused by: update-ssh-keys: no keys found in "/home/core/.ssh/authorized_keys.d"
Sep 09 14:18:27 172-236-119-109 systemd[1]: coreos-metadata-sshkeys@core.service: Control process exited, code=exited, status=1/FAILURE
Sep 09 14:18:27 172-236-119-109 systemd[1]: coreos-metadata-sshkeys@core.service: Failed with result 'exit-code'.
Sep 09 14:18:27 172-236-119-109 systemd[1]: Failed to start coreos-metadata-sshkeys@core.service - Flatcar Metadata Agent (SSH Keys).
Manually querying the endpoint returns a similar result:
$ curl -H "Metadata-Token: $TOKEN" --location --head http://169.254.169.254/v1/ssh-keys
HTTP/1.1 200 OK
Connection: keep-alive
Content-Length: 1
Content-Type: text/plain
Date: Mon, 09 Sep 2024 14:23:53 GMT
Retry-After: 1
Server: nginx/1.18.0
X-Ratelimit-Limit: 10
X-Ratelimit-Remaining: 9
X-Ratelimit-Reset: 1725891835
But I have a SSH key configured on my profile, example here with a Debian based Linode:
root@localhost:~# curl -H "Metadata-Token: $TOKEN" http://169.254.169.254/v1/ssh-keys
users.root: ssh-rsa AAAAB3N...