Apache memory issues
This is what I get from a ps aux | grep apache:
root 370 0.0 0.2 142652 148 ? S Nov25 0:01 /usr/sbin/apache
www-data 7102 0.0 3.8 146944 2284 ? S 12:25 0:01 /usr/sbin/apache
www-data 7103 0.0 11.4 146732 6776 ? S 12:25 0:04 /usr/sbin/apache
www-data 7120 0.0 12.8 147100 7648 ? S 12:28 0:01 /usr/sbin/apache
www-data 7130 0.0 12.0 146708 7140 ? S 12:30 0:02 /usr/sbin/apache
www-data 7137 0.0 12.9 147292 7668 ? S 12:31 0:02 /usr/sbin/apache
www-data 7138 0.0 12.3 146980 7356 ? S 12:31 0:01 /usr/sbin/apache
www-data 7151 0.0 8.3 147316 4960 ? S 12:33 0:01 /usr/sbin/apache
www-data 7159 0.0 10.0 146416 5980 ? S 12:33 0:02 /usr/sbin/apache
www-data 7179 0.0 13.8 147480 8212 ? S 12:35 0:02 /usr/sbin/apache
www-data 7216 0.0 7.2 147768 4320 ? S 12:44 0:02 /usr/sbin/apache
root 8826 0.0 0.7 1336 472 pts/0 S 17:38 0:00 grep apache
These are the modules that I am loading:
LoadModule config_log_module /usr/lib/apache/1.3/mod_log_config.so
LoadModule mime_magic_module /usr/lib/apache/1.3/mod_mime_magic.so
LoadModule mime_module /usr/lib/apache/1.3/mod_mime.so
LoadModule negotiation_module /usr/lib/apache/1.3/mod_negotiation.so
LoadModule status_module /usr/lib/apache/1.3/mod_status.so
LoadModule includes_module /usr/lib/apache/1.3/mod_include.so
LoadModule autoindex_module /usr/lib/apache/1.3/mod_autoindex.so
LoadModule dir_module /usr/lib/apache/1.3/mod_dir.so
LoadModule cgi_module /usr/lib/apache/1.3/mod_cgi.so
LoadModule userdir_module /usr/lib/apache/1.3/mod_userdir.so
LoadModule alias_module /usr/lib/apache/1.3/mod_alias.so
LoadModule rewrite_module /usr/lib/apache/1.3/mod_rewrite.so
LoadModule access_module /usr/lib/apache/1.3/mod_access.so
LoadModule auth_module /usr/lib/apache/1.3/mod_auth.so
LoadModule expires_module /usr/lib/apache/1.3/mod_expires.so
LoadModule unique_id_module /usr/lib/apache/1.3/mod_unique_id.so
LoadModule setenvif_module /usr/lib/apache/1.3/mod_setenvif.so
LoadModule perl_module /usr/lib/apache/1.3/mod_perl.so
LoadModule php4_module /usr/lib/apache/1.3/libphp4.so
and this is from free at low usage:
total used free shared buffers cached
Mem: 59356 56988 2368 0 3300 6456
-/+ buffers/cache: 47232 12124
Swap: 394232 59440 334792
13 Replies
I imagine you already tried to disable each of these modules one by one to make sure none are superfluous.
LoadModule access_module modules/mod_access.so
LoadModule auth_module modules/mod_auth.so
LoadModule auth_anon_module modules/mod_auth_anon.so
LoadModule auth_dbm_module modules/mod_auth_dbm.so
LoadModule auth_digest_module modules/mod_auth_digest.so
LoadModule include_module modules/mod_include.so
LoadModule log_config_module modules/mod_log_config.so
LoadModule env_module modules/mod_env.so
LoadModule mime_magic_module modules/mod_mime_magic.so
LoadModule cern_meta_module modules/mod_cern_meta.so
LoadModule expires_module modules/mod_expires.so
LoadModule headers_module modules/mod_headers.so
LoadModule usertrack_module modules/mod_usertrack.so
LoadModule unique_id_module modules/mod_unique_id.so
LoadModule setenvif_module modules/mod_setenvif.so
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_connect_module modules/mod_proxy_connect.so
LoadModule proxy_ftp_module modules/mod_proxy_ftp.so
LoadModule proxy_http_module modules/mod_proxy_http.so
LoadModule mime_module modules/mod_mime.so
LoadModule status_module modules/mod_status.so
LoadModule autoindex_module modules/mod_autoindex.so
LoadModule asis_module modules/mod_asis.so
LoadModule info_module modules/mod_info.so
LoadModule cgi_module modules/mod_cgi.so
LoadModule cgid_module modules/mod_cgid.so
LoadModule vhost_alias_module modules/mod_vhost_alias.so
LoadModule negotiation_module modules/mod_negotiation.so
LoadModule dir_module modules/mod_dir.so
LoadModule imap_module modules/mod_imap.so
LoadModule actions_module modules/mod_actions.so
LoadModule speling_module modules/mod_speling.so
LoadModule userdir_module modules/mod_userdir.so
LoadModule alias_module modules/mod_alias.so
LoadModule rewrite_module modules/mod_rewrite.so
LoadModule file_cache_module modules/mod_file_cache.so
LoadModule echo_module modules/mod_echo.so
LoadModule charset_lite_module modules/mod_charset_lite.so
LoadModule cache_module modules/mod_cache.so
LoadModule disk_cache_module modules/mod_disk_cache.so
LoadModule mem_cache_module modules/mod_mem_cache.so
LoadModule ext_filter_module modules/mod_ext_filter.so
LoadModule case_filter_module modules/mod_case_filter.so
LoadModule case_filter_in_module modules/mod_case_filter_in.so
LoadModule deflate_module modules/mod_deflate.so
LoadModule logio_module modules/mod_logio.so
plus modPHP and modSSL are loaded from the startup command line.
My ps aux gives:````
root 23494 0.0 0.4 24576 248 ? Ss Nov05 0:01 /usr/sbin/apache2 -k start -D SSL -D PHP4
apache 23500 0.0 0.0 23584 4 ? S Nov05 0:00 /usr/sbin/apache2 -k start -D SSL -D PHP4
apache 23502 0.0 13.2 33416 7904 ? S Nov05 0:16 /usr/sbin/apache2 -k start -D SSL -D PHP4
apache 23855 0.0 13.6 33856 8104 ? S Nov05 0:15 /usr/sbin/apache2 -k start -D SSL -D PHP4
apache 18797 0.0 19.3 32880 11516 ? S Nov09 0:01 /usr/sbin/apache2 -k start -D SSL -D PHP4
apache 21083 0.0 13.8 32700 8236 ? S Nov10 0:02 /usr/sbin/apache2 -k start -D SSL -D PHP4
apache 1167 0.0 14.1 27968 8424 ? S 14:09 0:00 /usr/sbin/apache2 -k start -D SSL -D PHP4
````
so my Apache has a maximum resident set size just over 11MiB and maximum virtual memory size less than 34 MiB. Your Apache instances have virtual memory sizes approaching 150MiB - that's where your swapping problem is originating - Apache needs to swap to do just about anything.
Try restarting Apache and see what the memory usage is immediately after startup. If it is much lower, then something is leaking memory. If the VM size is high from the start, there is something strange about your config.
It may be that apache1 is more memory hungry than apache2 (I use apache2 myself with only a handful of modules and it nicely fits into my Linode64), but I didn't expect that.
In other words, if you use PHP with Apache 2, you still have to use the one-child-per-connection model, so you won't save much memory at all.
@pclissold:
Try restarting Apache and see what the memory usage is immediately after startup. If it is much lower, then something is leaking memory. If the VM size is high from the start, there is something strange about your config.
If it is a memory-leaking app, a workaround is to set the MaxRequestsPerChild directive in your httpd.conf to a non-zero value.
See:
Try starting with a value like 50 – though you might need to adjust it depending on how much memory is being leaked with each request and how busy your site is. Nevertheless, it will stop the child processes from getting too large.
HTH,
Cliff
First of all, thank you so much for your help guys! I'm still a newbie, so thanks for bearing with me.
I decided against upgrading to Apache2 after reading your posts and realising it wasn't the answer. What I did instead was run 'free' and watched what happened to memory as I surfed my sites, started FTP'ing etc.
When I went to my usual websites, memory usage went up a little, but nothing too alarming (however, I was wondering at what point Apache releases memory as it never seemed to go back up).
Then when I used Squirrelmail webmail, free memory took a nosedive. I then tested to see if it was my Qmail installation by using Outlook Express to connect through IMAP but it had little effect on memory. I tried Squirrelmail again and it ate up memory again. So I guess I've found the culprit.
I'm not sure how to fix this memory leak in Squirrelmail. I'll probably have a look on their forums. If anyone has any ideas, please share! Thanks!
Any ideas what could be causing this?
Apache, PHP, qmail, courier-imap (I'm guessing your using Courier), ftpd, and you haven't mentioned it, but I'm guessing your also running MySQL.
For every page you load up in Squirrelmail, your using one of the apache processes, loading up an instance of PHP, and starting usually at least one instance of courier-imap to do whatever job is on the page (which I've found leaves an average footprint of greater than 1MB). That's just for one page, so that's why you see spikes when using Squirrelmail.
If your not running MySQL, you should still be able to pull this off without a hitch. Are you also running spam checkers? That's another service I've found you can't really pull off too easily on a Linode 64.
Yes, I tried to connect manually using Outlook Express and it does not have much effect on memory.
Hi Tierra,
Thanks for the explanation about Squirrelmail. It makes sense and I guess it is a lot to be running on a 64. It all basically works as long as its only me connecting to it and not doing anything major. BTW, I am running mysql as well, but I stopped using spamd.
I guess it may be time to upgrade to a 96 and I've logged a ticket with Caker. Do you think a Linode 96 will be sufficient to run everything smoothly? Would it be better to get another 64 and seperate the mail server?
@stylehk:
Thanks for the explanation about Squirrelmail. It makes sense and I guess it is a lot to be running on a 64. It all basically works as long as its only me connecting to it and not doing anything major. BTW, I am running mysql as well, but I stopped using spamd.
I guess it may be time to upgrade to a 96 and I've logged a ticket with Caker. Do you think a Linode 96 will be sufficient to run everything smoothly? Would it be better to get another 64 and seperate the mail server?
With a Linode 64, it is possible to pull off Apache, MySQL, PHP, qmail, courier-imap, squirrelmail, and a couple other small things given you know your way around and have cleaned out Apache modules, optimized Apache server settings, MySQL server and buffer settings, as well as giving PHP a smaller footprint by compiling binaries with stuff stripped out that it builds in by default that you don't need, and I'm sure you can find other places to cut back. It just comes down to the question of whether or not you have a choice of upgrading, have the time to do optimizations, whether or not you really need all those services and their included features, etc. Kind of a money vs. time issue.
If you want spam filtering, definately get another Linode to handle mail separate though. There's not a lot you can optimize with that, and in general it's a big job.