DNS traffic query limits?
Linode
Linode Staff
So I am gathering information on a large number of domains for a project I'm working. I need to run DNS queries to grab the A record for each domain (millions of them).
Is there a recommended timing or practice in order to prevent throttling or disabling of my network traffic? Something like, "X amount of queries per period of time?"
I would like to do this without breaking any rules and follow best practices. I know this kind of stuff might look malicious if I do it in the wrong way, so was hoping to avoid that if possible.
Thanks!
1 Reply
mtcotton846
Linode Staff
My understanding of the limits on outgoing requests is that they are arbitrarily set by the registrar. Doing some googling I found this post at serverfault that talks about how Google Public DNS handles this sort of stuff:
To protect other DNS nameservers against DoS attacks that could be launched from our resolver servers, Google Public DNS enforces per-nameserver QPS limits on outgoing requests from each serving cluster.
As far as Linode is concerned, I am not aware of any specific limits on the number of DNS queries you can perform, but I would probably recommend you to start low (maybe something closer to 10 queries a minute) and then work your way up slowly from there in order to ensure you are not hitting any limits. A more conservative rate of querying should help mitigate any negative repercussions on your IP (like getting throttled).