rsyslogd crashing on Debian 6.0 and freezing Apache
I've been on Linode for some years now with no major issue. Since a few days, I've started noticing something troublesome on one of my Debian 6.0 boxes:
rsyslogd: [origin software="rsyslogd" swVersion="4.6.4" x-pid="1248" x-info="http://www.rsyslog.com"] rsyslogd was HUPed, type 'lightweight'.
When I get this line in /var/log/messages, Apache just freezes (presumably because Apache logs a lot) and the only way for me to get things going again is to restart both the rsyslogd and apache2 services manually.
Anyone noticing something similar? Any clue?
Thanks a lot.
4 Replies
Typically a HUP signal is sent to a daemon to tell it to reload its configuration files, or to close and re-open any log files it is writing to. If this happens at a predictable time, I would guess that a log rotation script is sending the HUP signal to rsyslogd, but I don't know why that would cause it to freeze.
This would make the most sense if it's happening around the time system logs are being rotated out.
-Tim
# time logrotate -f /etc/logrotate.conf
real 0m7.945s
user 0m3.007s
sys 0m0.529s
# tail /var/log/messages
Jul 10 14:37:27 server01 rsyslogd: [origin software="rsyslogd" swVersion="4.6.4" x-pid="5785" x-info="http://www.rsyslog.com"] rsyslogd was HUPed, type 'lightweight'.
And, interestingly, Apache was not affected by the log rotation. So it seems that I've been erroneously thinking that the HUP signal being sent to rsyslogd and Apache's behavior were somewhat related.
I guess I'll need to wait for Apache to freeze again to know what's really happening…
Thanks for your help anyway.
In fact, Apache was freezing because of APC. When the APC cache becomes full, PHP apps, if wrongly written (and a lot of them are) tend to freeze.
My solution was to increase the APC cache from the default 32Mb to 128Mb and Apache has been working fantastically well for the past week now. It hasn't frozen once.