How to figure out why Apache stops responding

On three different occasions last night, my apache server stopped responding to requests. Twice just for three minutes, the third time for a couple hours until I restarted it. I am using LogLevel debug but the error logs don't show anything at all over that time period. The access logs show requests being served normally until they just stop. There wasn't an unusual level of load, although I had just resized the linode earlier in the evening.

I'm proxying to apache with nginx, and checking the nginx access logs on each of those occasions, nginx just shows 499 errors for a while, then 504 errors for the rest of the time. That seems like what I would expect if Apache stops responding for some reason.

So where can I go from here to figure out what happened? I'm using django+postgres but I don't know how to investigate with no useful apache logs.

4 Replies

Can you tell how many children apache has at that time? I.e. are they deadlocked and just stacking up to the max?

Did you resize to a larger linode or to a smaller one? If you shrinked your linode, the usual suspect would be OOM (out-of-memory). You can use Munin to monitor memory usage over time. Even if memory turns out not to be the culprit, at least you can eliminate one possibility this way, and Munin also gives you other useful statistics.

I resized to a larger linode. But I probably should be using Munin anyways, that's a good point.

Looking at the logs, it didn't appear that there was a deadlock. Using worker mpm there were three wsgi daemon processes and each were serving requests normally up to the time when the server went dead. Would you expect to see something in error.log if apache ran out of children?

Normally when Apache runs out of children itll tell you in the error_log that it hit the MaxClients setting.

Are your Apache logs being rotated properly? An oversized log can cause strange issues. Do you have any RLimits configured?

Reply

Please enter an answer
Tips:

You can mention users to notify them: @username

You can use Markdown to format your question. For more examples see the Markdown Cheatsheet.

> I’m a blockquote.

I’m a blockquote.

[I'm a link] (https://www.google.com)

I'm a link

**I am bold** I am bold

*I am italicized* I am italicized

Community Code of Conduct