Unknown storage alerts pagerduty / Icinga2
We are using Icinga2 for monitoring this server. We are randomly getting UNKNOWN storage alerts. We have Icinga2 tied into PagerDuty and this server is alerting us all times of the day about an UNKNOWN storage. As far as errors, we are not seeing anything other than disk space usage alerts. Not seeing any errors related to storage in /var/log/messages.
When we login to server we run "df -h" we seeing the filesystems mounted.
1 Reply
It will be important to know where the alert is coming from, as Icinga2 and pagerduty can both produce alerts.
You can learn more about what service is alerting you by checking the Icinga2 logs and the pagerduty logs. Here are detailed guys with step-by-step instructions on troubleshooting and debugging with error logs here:
https://www.icinga.com/docs/icinga2/latest/doc/15-troubleshooting/
https://www.icinga.com/docs/icinga2/latest/doc/21-development/#development
https://v1.developer.pagerduty.com/documentation/rest/log_entries/list
https://v1.developer.pagerduty.com/documentation/rest/log_entries/show
https://v1.developer.pagerduty.com/documentation/rest/log_entries
You can check your systems storage availability and inode availability with the following:
$ df -h
$ df -i
Hope that helps get you some more information to help you troubleshoot.