Different CPU usage on identical servers
I have 2 Linode 32 GB servers that are set up identically as web servers. I have NodeBalancers set up to distribute traffic to these web servers evenly. The NodeBalancers are set to use "Least Connections" and "Table" session stickiness.
I observe that one of these servers consistently has higher CPU usage than the other. Over the past 12 hours, I can see that generally, it's using 100%-200% higher CPU than the other.
The size of the nginx access logs is comparable, so it doesn't appear one is receiving more traffic than the other. Looking at the process stats in Longview, the main web server process for handling requests has a higher CPU %, but so does nginx. There are no additional processes using significant CPU.
I can think of a few potential reasons for all of this:
- They are not in fact identical; there is some configuration difference between them.
- The nature of how the NodeBalancers work cause one to receive "harder" requests than the other (e.g. more likely to get the first request for a resource, making it miss the cache).
- One is on hardware that is busier than the other.
I'm wondering if there's any info (especially regarding 2 and 3) that would be helpful in solving this problem.
1 Reply
There are a number of things you can do to check CPU usage on your Linodes. While they might be identical in structure, different things can be going on in each of them that could account for what you're seeing.
What I want to do is point you to some documentation that will show you how to look into this. Take a look at this other post which goes into detail about CPU usage and why it can be higher in some Linodes than others:
Why is my Linode suddenly using 100% of its CPU?
One of recommended troubleshooting tips in the above post is the use of the top
command. This will allow you to see the server's processor activity in real time, and show you which processes are consuming the highest amounts of CPU:
Using top to Monitor Server Performance
I also want to point you to the iostat
command that is listed in the first linked post. This will help you diagnose potential CPU steal. This post goes into more detail about that: