Terraform: Orchestrating creation of A record and rDNS
Using Terraform, I'm trying to create an A record (pointing from test.{domain}
to a Linode instance test_server
, and then a reverse DNS record pointing from the server back to test.{domain}
:
resource "linode_domain_record" "a_record" {
domain_id = linode_domain.example.id
record_type = "A"
name = "test"
target = linode_instance.test_server.ip_address
}
resource "linode_rdns" "test" {
address = linode_instance.test_server.ip_address
rdns = "${linode_domain_record.a_record.name}.${linode_domain.example.domain}"
}
The problem is that, while Terraform seems to wait until the A record is confirmed created before creating the rDNS record, the A record is still not completely ready and the operation fails:
linode_domain_record.a_record: Creating...
linode_domain_record.a_record: Creation complete after 1s [id=xxxxxxx]
linode_rdns.test: Creating...
: Error creating a Linode RDNS: [400] [rdns] No match was found for 'test.{domain}'. Reverse DNS must have a matching forward entry. If your entry was added to your domain service recently, it make take some time for the changes to propagate.
What would be a good way to orchestrate this?
5 Replies
Terraform also waits until any provisioner is complete, so at first I thought it might help to add a small sleep:
resource "linode_domain_record" "a_record" {
domain_id = linode_domain.example.id
record_type = "A"
name = "test"
target = linode_instance.test_server.ip_address
provisioner "local-exec" {
command = "sleep 30"
}
}
But unfortunately it doesn't help with just a small sleep, it seems like it takes much longer (~ 10 minutes) for the A record to propagate even though it's managed by Linode.
This is something that I've been wrestling with for a while, and the only ways I've found to do it are ones that I consider to be a bit messy. Terraform uses a declarative approach, and there doesn't seem to be a way to do things conditionally in Terraform, or to tell it "wait for this to happen, and then do that".
The workaround that I've found is to upload a script that runs on the server using the file
provisioner and then use the remote-exec
provisioner to pass any needed data to the script, make it executable on the Linode, and then run it. On execution, the script can determine if the DNS record has propagated, and then use an API call to add the rDNS record once it has. For example, you could add code to your Terraform plan to upload a script to your Linode and execute it like this:
resource "linode_instance" "DB-instance"{
label = "db1"
group = "databases"
...
provisioner "file" {
source = "remote_exec_db.sh"
destination = "/root/remote_exec_db.sh"
}
provisioner "remote-exec" {
inline = [
"chmod +x /root/remote_exec_db.sh",
"/root/remote_exec_db.sh ${var.db_configscript_data} ${linode_instance.exampleWeb-instance-1.ipv4[0]} ${linode_instance.exampleWeb-instance-2.ipv4[0]}",
]
}
Then in the remote_exec_db.sh
script:
#!/usr/bin/env bash
...
function set_rdns {
curl -H "Content-Type: application/json" \
-H "Authorization: Bearer ${1}" \
-X PUT -d '{
"rdns": "'"${2}"'"
}' https://api.linode.com/v4/networking/ips/"${3}"
}
...
# Wait for DNS to propagate before setting the rDNS
printf "Waiting for DNS propagation before setting rDNS..."
declare a=1
while [ $a -ne 0 ]; do
result="$(dig +short "${DOMAIN}")"
if [ "${result}" == "${ip}" ]; then
a=0
else
printf "."
sleep 60
fi
done
printf "\nPropagation complete! Setting rDNS...\n"
# Add the rDNS record
set_rdns "${LINODE_API_TOKEN}" "${DOMAIN}" "${ip}"
The second option, which I feel isn't really all that different, is to do the same thing using a StackScript. At minimum, this method will clean up your code a little bit by eliminating all of the extra steps involved in using Terraform's file and remote-exec provisioners.
If I ever come across a better answer, I'll be sure to update this thread, but if anyone out there knows of a better/cleaner way to do this with Terraform in the meantime, I would love to know about it.
I use Terraform pretty infrequently, and that code was thrown together from another plan I have that waited for a StackScript to complete before running a follow-up script, so the remote-exec
approach is a bit over-complicated here. Terraform also has a local-exec
provisioner which might be a cleaner approach. I haven't tried this myself, but hypothetically you might be able to use local-exec
to run the same script locally on the machine that runs your Terraform plan:
resource "linode_instance" "DB" {
...
provisioner "local-exec" {
command = "./script.sh <arguments>"
}
}
Noticed that "Instantaneous DNS Manager updates" was mentioned in https://www.linode.com/2019/12/30/2019-a-year-in-review/ , not sure exactly what is meant by that, but it sounds like it might perhaps improve the situation?
Thanks for sharing your approach! Glad to see I'm not the only one struggling with this. Was also thinking about polling dig
.. While a bit messy, perhaps that's the best approach for now(?)
Agree it's appealing to use local-exec
since you don't have to copy the script to the instance. Only issue that it seems like you cannot query the Linode nameservers (nsX.linode.com) from outside, so you have to use some other ones like 1.1.1.1. When I tested now, it only took 6 minutes from creation until propagation to 1.1.1.1 though, so it seems like the propagation from Linode to 1.1.1.1 in itself is fast enough.
./helpers/wait_for_dns.sh
:
#!/bin/bash
DOMAIN="$1"
IP="$2"
echo "Waiting for $DOMAIN : $IP"
until [ "$(dig +short @1.1.1.1 ${DOMAIN})" == "${IP}" ]; do
sleep 10
done
main.tf
:
resource "linode_domain_record" "a_record" {
domain_id = linode_domain.example.id
record_type = "A"
name = "test"
target = linode_instance.test_server.ip_address
provisioner "local-exec" {
command = "./helpers/wait_for_dns.sh $DOMAIN $IP"
environment = {
DOMAIN = "test.{domain}"
IP = linode_instance.test_server.ip_address
}
}
}
resource "linode_rdns" "test" {
address = linode_instance.test_server.ip_address
rdns = "${linode_domain_record.a_record.name}.${linode_domain.example.domain}"
}
Thanks very much for this solution; it worked for me.
Only issue that it seems like you cannot query the Linode nameservers (nsX.linode.com) from outside,
This seems incorrect. If you set nsX.linode.com as the nameserver authority for your domain (managed by the registrar), it must provide public listings for other DNS servers to access. I successfully used ns1.linode.com on a local-exec
provisioner, and saved about 15 min over using 1.1.1.1.
However, apparently PTR records take a lot longer to propagate through Linode's systems. I cannot find any clear answer from Linode about how long it takes to register. Even ns1.linode.com reports the wrong/old PTR hours after updating the rDNS.
I also cannot find any cloud providers that provide better PTR support. For example, AWS requires a support request process.
Noticed that "Instantaneous DNS Manager updates" was mentioned in https://www.linode.com/2019/12/30/2019-a-year-in-review/ , not sure exactly what is meant by that, but it sounds like it might perhaps improve the situation?
Seems likely that COVID-19 will affect the 2020 roadmap. Even if they do get the insta-DNS feature out the door, I wouldn't expect it to speed up PTR record changes. Apparently those happen through a very different flow.