Some Linode software is sending SIGWINCH signal to Apache server
Hello - We run an Apache server on Linode. For some reason, that we do not understand, every few months the Apache server receives a SIGWINCH signal. Apache2 uses this signal as a indication that it should shutdown.
The SIGWINCH signal is supposed to be issued when the window size that an application is running in changes. Since the Apache2 runs in the background (there is no window associated with a background task), it should never get this signal.
Here are the log entries from the most recent occurrence:
[Tue Jul 30 13:00:01.907344 2024] [mpm_prefork:notice] [pid 1661892] AH00170: caught SIGWINCH, shutting down gracefully
[Tue Jul 30 13:00:02.247342 2024] [mpm_prefork:notice] [pid 1662632] AH00163: Apache/2.4.52 (Ubuntu) mod_jk/1.2.48 OpenSSL/3.0.2 configured -- resuming normal operations
[Tue Jul 30 13:00:02.247392 2024] [core:notice] [pid 1662632] AH00094: Command line: '/usr/sbin/apache2'
[Tue Jul 30 13:08:04.358431 2024] [php:notice] [pid 1662634] [client 66.249.79.134:39387] populate: adding as, referer: https://www.hear2read.com/
Any help or suggestions on tracing as to which process is sending the SIGWINCH signal is greatly appreciated.
Thanks,
Ganesh3
1 Reply
CassandraD
Linode Staff
While you're correct about the original use of SIGWINCH, which came from SIGnal WINdow CHange, everything I'm reading has suggested that for Apache, that's not necessarily what it means. For example, I found this in a ServerFault discussion.
SIGWINCH and "graceful kill" implies that something was asking httpd to shutdown (this is not what SIGWINCH was originally intended for - Apache re-purposed it for this).
So, it sounds like you're right that something is sending this signal, I've researched this and I haven't found a definitive or even most likely culprit. That said, here are some suggestions for possibly getting that information.
One thing you can do is change your Apache configurations to verbose logging to get more information the next time it happens. Hopefully this says what's killing your service. You may also want to check other system logs, though I'm reading that it's difficult to find what is sending signals like this. I did find a post with suggestions for doing so, though it sounds pretty complicated.
Someone there did mention a utility called killsnoop. Researching that SIGWINCH is listed in some documentation for that utility, so that may be a way to find what's sending the signal.
In the meantime, some things that can shut down processes can be looked at more generally for problems.
First, running out of memory can cause your server to shut down non-essential processes. You can use these resources to help you investigate memory problems:
Next, this StackOverflow discussion recommends checking cron jobs and unattended-upgrades or other automatic updates to packages. These guides may help you check for these things.
Cron Jobs
Configuring Automated Security Updates on Ubuntu
Seeing how those are both configured should help you figure out how to check if they are already configured if you're not sure.
Also, I've noticed a lot of the resources I'm finding are from users who are running Apache through Docker. If you're using Docker, it may change how you investigate, as with the instructions in that link for getting logs.
Finally, we do have a few general guides for troubleshooting Apache that may provide some additional places to look for what's going on. Likely that are more basic than you need right now, but I wanted to share them just in case.
Troubleshooting Web Servers, Databases, and Other Services
Troubleshooting Common Apache Issues