nginx/load balancing I don't know what's the best solution..

I'm not an expert here, but the way I see it is that if you do a load balancer, your main concern will be network traffic – if you need each server to keep it's data synchronized with each other, it will need to update all the other servers each time the data is updated.

One possibility is to rely on one server for a short amount of time, then have it synchronize it's data to another server and redirect everybody to that server, and have this run in a sort of server cycle, where after a set amount of time, the current server will synchronize the next server then redirect to the new one, until you cycle around to the first server, then start the process over again.

Since I'm not too familiar with load balancing or with your specific needs, you should wait for others to provide their suggestions.

You're going to need to look at the performance data you're collecting to figure out what's going on. There are three possible bottlenecks: network, disk, and CPU. There are also a number of limits (e.g. the "too many open files" error, which is ominous), some of which are adjustable, some of which aren't.

Are you running munin or another similar tool to gather ongoing system data? If not, you need to seriously consider it. You might also consider something similar to Etsy's StatsD for timing various parts of your application, although I will note that if you aren't having I/O bottlenecks now, you will be if you start running munin and graphite and statsd on the same box as your application :-)

nginx, generally speaking, is not resource-intensive. What you're probably going to want to scale is your application itself, which gets into big questions of synchronization and session management and all that. Moving from one server to two servers is going to be hard (especially if it wasn't planned for from the start), but it is probably a good idea. Or, go for three… they tell me that going from two to three is harder than going from one to two, so I go straight for three.

Also, beware the distributed single points of failure.

Thanks for your response, I will try monitoring tools for sure.

In the meantime, I checked the Linode monitoring graphs, and I noted that in most loaded event the CPU went up to 120% (not bad, i had almost 200 concurrent users and I have 400% to use) and the I/O went up to about 150 kbits/sec, what is not much, I think… BW is not high either… so I guess that my bottleneck is CPU.

There is a workaround for doing load balancing in case of CPU bottleneck and not I/O bottleneck? what if I have one load balancer on front, two nginx servers in the middle (reducing CPU load) and a single server for disk I/O in the back?

Oh, and BTW, yes, I feel stupid for not checking the graphs before. xD