RAM, Swap, Virtual Memory

@caker:

I want to encourage everyone with a Linode to have a swap file of at least 256M (you can resize your disk images to make room). I've recently changed the suggested swap file size in the Distro wizard to 256M. I believe a lot of the problems we've encountered with disk contention is caused by Linodes with a very small swap size (64M) constantly thrashing their swap file.

Stay tuned for more performance posts!

-Chris

If you don't have disk space available for the raw disk partition by resizing the partition, you can always create a swap file over the filesystem:

• dd if=/dev/zero of=/swap1 count=256k

(count is in the block count, not kBytes, ie 256k creates swap file of ~128MBytes)

• mkswap /swap1

• /sbin/swapon /swap1 (add this to somewhere in your rc boot files)

This probably takes a slight performance hit vs. raw partition but I haven't noticed problems with it in my linode.

Risto

Hi,

Wouldnt it be easier to just resize the filesystem and then add the space to the swap partition.

Adam

@adamgent:

Hi,

Wouldnt it be easier to just resize the filesystem and then add the space to the swap partition.

Adam
Its very easy to add a swap file if you don't have the partition space. I created my Linode 64 with a 64 Mb swap and 1984 Mb main partitions. I used the defaults because I didn't know better. When I realised that I needed more swap I added a 64 Mb swap file on the main partition. That never gets close to running low. This was before we could downsize partitions. It sounds like I may be paying a small performance cost so I have two questions:

If I use the resizing utility, about how long would it take to downsize my 1984 partition to 1920 Mb? Its about 50% full.

How safe is it really? I backup the live data frequently and I'm going to try to do a total rsync backup overnight tonight but it would still be a major pita to lose everything.

Ross

@caker:

On the other hand, not having a large enough swap can be much worse for performance. This is because Linux has a lot less total VM to work with. It has to swap harder and more frequently to get its jobs done.
This is really a general Linux question. I'm no expert so I'm not trying to contradict you but I would have thought if it doesn't have enough VM then the consequences would be more serious than just swapping harder and more frequently. If your physical memory plus swap is not enough to run what you're asking it to do then wouldn't things crash or simply not work? Are you talking about the effect of applications being smart enough to adjust their behaviour when they are running in limited memory?

Ross

@tetranz:

This is really a general Linux question. I'm no expert so I'm not trying to contradict you but I would have thought if it doesn't have enough VM then the consequences would be more serious than just swapping harder and more frequently. If your physical memory plus swap is not enough to run what you're asking it to do then wouldn't things crash or simply not work? Are you talking about the effect of applications being smart enough to adjust their behaviour when they are running in limited memory?
You're right – when Linux exceeds its VM, the Out-Of-Memory (OOM) Killer kicks in and decides which processes to kill to try and free up some memory. But, the OOM killer is a last resort.

What does Linux do before calling on the OOM Killer? It swaps like crazy. Linux will start to swap out more aggressively, processes have no where to go (RAM or SWAP), and you end up in a nobody-wins situation.

Having more swap means more things can be placed into the swap, without having to thrash things out (thereby consuming RAM and making matters worse) to put something else in.

-Chris

@rko:

• dd if=/dev/zero of=/swap1 count=256k

(count is in the block count, not kBytes, ie 256k creates swap file of ~128MBytes)
dd if=/dev/zero of=/swap1 bs=1M count=128

Will make a 128 MB empty file… a little easier to read.

@adamgent:

Wouldnt it be easier to just resize the filesystem and then add the space to the swap partition.
It depends – perhaps they don't want to reboot a production system, or need to execute a large process.

-Chris

@tetranz:

If I use the resizing utility, about how long would it take to downsize my 1984 partition to 1920 Mb? Its about 50% full.
I'd guess less than 2 minutes.

@tetranz:

How safe is it really? I backup the live data frequently and I'm going to try to do a total rsync backup overnight tonight but it would still be a major pita to lose everything.
It's worked every time. But, never a bad idea to make backups.

-Chris

Recommend a beginner template for us boots:

Would it be possible to make a capurted picture example under basic settup recommendations and put them on your webpage so we can actually see. Reason being i just tried to get my linode up as fast as possible when i got it and have no clue wether i did it with the right intent or purpose. I would like to know what it looks like for a basic example setup would be for maybe just a webserv or one that has many ips and many web pages and showing how the memory was setup in different cases. I read everything you guys post a couple times at least and then try to figure out how to apply it. So pictures and actual commands how they were used for basic setups would help me a great deal.

It is just an idea for you(I know how to capture pics in windows if that would help)

Linode is on the leading edge and is what i have been looking for for years and your skills here is above what I have ever seen or thought possible.

@You_Wish:

Would it be possible to make a capurted picture example under basic settup recommendations and put them on your webpage so we can actually see.

Can I assume you are speaking about values for the Control Panel ?

The following is what I have for my linode64 account

http://www.opencurve.org/~sunny/misc/linode.png

http://www.opencurve.org/~sunny/misc/linode2.png

Bill Clinton

Yup those pictures talk 100 times more to me then 3 pages of stuff that is over my head.

Thanks for that give me more pictures.

I love you man

Just a little tip, never make a swap file on a file system. At least, not in a production system.

The kernel has a few more layers to go to access the swap (make calls to the filesystem) which is not what you want especially for something as time critical as swap.

If you need more swap, do it right. Resize the filesystems.

@Cool_blade:

Just a little tip, never make a swap file on a file system. At least, not in a production system.

The kernel has a few more layers to go to access the swap (make calls to the filesystem) which is not what you want especially for something as time critical as swap.

If you need more swap, do it right. Resize the filesystems.

I'm sure it's more effective to use raw file as swap vs. file over a filesystem but can anyone quantify the performance difference? Is it 0.5, 5% or 50% overhead or what kind of difference we are talking about? Any studies or tests done about this?

Risto

@caker:

I want to encourage everyone with a Linode to have a swap file of at least 256M….
If possible, I'd prefer to avoid swapping entirely. Is it okay to operate without any swap space at all, assuming that all the software I run can fit into the RAM allocated to my Linode? I'm running only Linux itself, SSH, Stunnel, and a tiny web server that preallocates all its RAM. Here's my memory usage now:

  $ free -m
               total       used       free     shared    buffers     cached
  Mem:            92         51         41          0          2         21
  -/+ buffers/cache:         27         65
  Swap:          100          0        100

Thanks for you advice.

caker wanted to encourage 256 MiB swap as opposed to the then-standard 64 MiB. These small swap spaces were thought to cause Linodes to thrash as they tried to make the best use of insufficient swap.

No swap is fine as long as you are confident that you will never come close to using all of your physical memory.

@pclissold:

No swap is fine as long as you are confident that you will never come close to using all of your physical memory.

True, but I'd rather have some available in case it's needed. Better than running afoul of nasty thrashing or worse the OOM killer (I've seen it in action, it's not pretty).

If your running applications will consistently fit into RAM (with some room to spare), generally the box will rarely, if ever, swap anything out to disk. You can also control this to some degree by adjusting the value of '/proc/sys/vm/swappiness'… I just looked and my linode is set to '60'. Higher numbers will make you swap more. Lower will swap less. Unfortunately, this sort of tuning can be the source of great holy wars and flame fests on various boards. Preferable values will vary depending on the sort of system you're running, performance testing, etc.

hi there ! after 17 years, most of community in market are against using Swap. Their argument is ''swap is slow'' . But in my opinion swap is getting faster with new disk technologies just like RAM. Maybe there wont be any RAM in future and we will just use swap, why not since its fast enough ?
i m an amateur who want to do more jobs with less sources. Being faster cant be the only fact for me. Using much more sources for being just a little bit faster does not seem to me very logical. Since the speed is not only related with Ram , everything can affect it, and if the users has slow machines and connections it wont change a lot that the source machine is very fast.
so i would like to learn your advice and idea about swap size for today ?