Failed to open stream (HTTP Request)
I have a script (as a part of a WordPress plugin) that scrapes data once per page load. This script works perfectly on my development computer (running WordPress on a Xampp stack and windows) yet fails to open a stream and produces HTTP/1.0 429 (which corresponds to "Too Many Requests") on my Linode running Ubuntu. I have checked the firewall and it shouldn't be a problem.
The script uses simplehtmldom.php and the exact warning is thrown at file_get_contents and I'm not sure why. I am fairly certain that only one http request is requested, and shouldn't cause code 429.
Any insight would be greatly appreciated.
1 Reply
UPDATE:
I figured file_get_contents might be disabled, so I tried using curl:
$useragent = "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:61.0) Gecko/20100101 Firefox/61.0";
$ch = curl_init ("");
curl_setopt ($ch, CURLOPT_URL, $url);
curl_setopt ($ch, CURLOPT_USERAGENT, $useragent); // set user agent
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
$exec = curl_exec ($ch);
curl_close($ch);
However, curl_exec gives me an empty response, so not really sure what's up.