What are you using to display your website?

Static HTML, dynamic PHP/Python/Perl or some custom solution?

It would be also interesting to know what backend services like MySQL or some other memory-expensive applications that may eat up your memory/cpu (time) you use to find out what the problem is.

You may need to bring your PassengerMaxPoolSize down a bit. I don't run any Passenger setups, but a quick Googling suggests that the default value for PassengerMaxPoolSize is a bit high for memory constrained environments.

What database backend are you using? If you haven't tuned your database at all, it can also be easting up large amounts of memory.

Have you implemented any sort of caching? A little caching can get you a long way in terms of reducing database usage.

@paulengstler:

It would be also interesting to know what backend services like MySQL or some other memory-expensive applications that may eat up your memory/cpu (time) you use to find out what the problem is.

Try to run munin, that gives you a good idea about what's going on, for example, if you start swapping.

Your CPU usage isn't high at all. If upgrading your linode helped stop the crashes, the problem almost certainly has to do with using too much RAM.

Type top at the terminal and press Shift+M to sort your processes by memory usage. Post a screenshot here and we'll be able to figure out what's using all your RAM.

paulengstler …

I'm developed the application using Ruby on Rails, the os is ubuntu 10.04, the db is MySQL

JshWright …

Thank you very much I'll try to play with PassengerMaxPoolSize and see if it fix the crashing, heres my mysql conf

==============================

key_buffer = 16K

maxallowedpacket = 1M

thread_stack = 64K

threadcachesize = 8

This replaces the startup script and checks MyISAM tables if needed

the first time they are touched

myisam-recover = BACKUP

max_connections = 100

table_cache = 4

thread_concurrency = 10

#

* Query Cache Configuration

#

querycachelimit = 1M

querycachesize = 16M

============================

I don't have any caching , but a friend suggested varnish-cache.org and I tried it at a seperate vps and didn't notice a differrence in loading the pages (didn't use anything for that I was just counting)

any reccommendationس for a good caching solution please?

hybinet…

I think its because of the RAM as well, now I went down again from 2GB to 1GB, I came back to my office to find out that the server been crashed for the 3 houres 'very imperessing', anyway here is 'top' results as you suggested, would be grateful if u have a look

~~http://212.110.169.40/linode-03.jpg" />

by theway I also get this auto email from linode saying that my linode has exceeded the notification threshold (1000) for disk io rate by averaging 1775.45 for the last 2 hours

Thanx guys~~

The disk IO isn't that high however it does suggest mysql may not be as efficient as it could be check out mysqltuner.pl and mysql workbench both can tell you how efficient the sql cache and key indexes are.

The high disk I/O warning is because when you ran out of memory, your system started using the swap disk very heavily in a desperate (but futile) attempt to keep its head above water. Don't worry about that, since it'll go away when you fix your memory problem.

Your Ruby processes are eating a ton of ram. The Ruby processes listed in htop are combining to use half the RAM available to the box. I'm not a Ruby dev, but it seems to me like your app is probably using more memory than it should.

As far as caching goes… there are a lot of different ways to implement caching. You can do whole-page caching with something like Varnish (nginx can also do caching). Alternatively, you could do object or query level caching in your app, using something like memcached.

Optimizing MySQL is never a bad idea, but it's probably not going to help you out all that much here.

Step one would be reducing PassengerMaxPoolSize. I'd set it to 2 and see if that helps keep your server afloat. Step 2 would be figuring out why your app is eating so much memory… Once you fix that, you can start bringing PassengerMaxPoolSize back up.