Expanding Laterally
I run a website running XenForo, 2 different Wordpress installations, a Wikimedia installation, and the forum alone starting to approach 400 active users every day during peak hours. This runs on a single 4GB package, and now, we're starting to see almost 400% CPU and 3.25GiB RAM being used up. I think it's time for an upgrade.
My question is relatively complicated.
I am interested, for personal experimentation's sake, in expanding laterally. While an 8GB server would definitely resolve the issue, I'm more interested in buying a second 4GB server. This is because, at 8GB, the # of cores lower than the GB size of the package, meaning the cost effectiveness of the CPU is less than multiple boxes. I would like to put the MySQL database that runs all applications on a single box, and then buy a second 4GB linode. If that one becomes peaked too even without running MySQL, I'd then buy a 3rd 4GB linode and utilize load balancer.
My questions are:
1) Is this the way to go, or am I being silly?
2) If I buy my 2nd server at the same datacenter, is this a latency issue? Can I use internal IPs?
6 Replies
You want to go from one box to Admin to several boxes to Admin.
You want to go from your current set of attack vectors to multiples of that set of attack vectors.
All because …..?
I'd vote (at least for now) "you're being silly".
Of course if you continue to grow, the single box option might no longer be on the table.
If you want to experiment/learn about running distributed web services - setup a local box with VirtualBox and learn the easy way in your own free secure virtual sandbox. Then you'll have the know how to move it to the real world if you outgrow the single box solution.
YMMV
@vonskippy:
So you want to go from one point of failure to several points of failure.
Isn't that the point? You can take one box down and run maintenance on that without a hitch.
@vonskippy:
You want to go from one box to Admin to several boxes to Admin.
That doesn't seem like that big of a deal.
@vonskippy:
You want to go from your current set of attack vectors to multiples of that set of attack vectors.
How can my MySQL DB be attacked if the firewall is set to ignore non-local connections? The entire point is to isolate the DB and run scripts on boxes that can be broken down, cloned, and updated en masse repeatedly.
Each box you add to your Admin list is just one more hassle, one more to keep updated, one more to test, one more to document, one more …
As long as you never remove that isolation to test, or via a typo, or via just plain carelessness, or… or… or…
But whatever - try out the multi-box scenario and see how it works out for you - you can always change back.
Personally, simplicity works best for me. Less hassles, less work, less worries. I only grow more complex when there is no other simple solution.
Ignore vonskippy, he's the resident curmudgeon. Splitting your webservice and database are the natural first step when increasing capacity. It's the logical first step in expanding horizontally, and it's the easiest one too. Once you've got your database on its own 4GB box, don't forget to tweak MySQL's settings to take advantage of the RAM. You can start with the biggest sample file that MySQL ships with (my-huge.cnf, meant for servers with 1-2GB of RAM that mostly just run MySQL), and then you can use mysqltuner to help tweak it from there.
Note that you need to let the MySQL server gather statistics before running mysqltuner. The idea is that you let your database server run with real load on it for 24 hours or more, and then you run mysqltuner, which will look at the statistics the server has gathered and suggest changes to your config file based on your workload.
To give an overview:
I have 3 servers.
Alpha is designated by the MySQL server and is a 4GB Linode.
Beta is a "vulnerable process" server and is a 1GB Linode.
Charlie is the new Script image server and is a 4GB Linode.
Alpha was the original box and had literally everything tucked into it. The only way to continue improving performance with more load is to buy a bigger Linode. Alpha has been re-designated to dynamic storage, and has memcached and MySQL loaded onto it.
When my website was DDoS attacked, I read up on network security to protect my website from an identification attack. Already behind cloudflare, I made sure that the attackers were forced to go through their service so that I could manage it more easily. Beta was born out of this necessity, as I offloaded IP-vulnerable services like email and teamspeak to it. If the attackers chose to take down Beta, I don't have to care because it's not something uber critical to services. Also, since Beta is not a web server, application level attacks are much harder to pull off by novices.
This month, we experienced a huge spike in traffic that burdened an already growing number of daily users. The website slowed to the crawl, and my incredibly generous MySQL configuration ended up crashing the database multiple times at peak hours as it overconsumed RAM. I decided that I wanted to free up memory by moving Apache to its own box, which is why I now have Charlie. Charlie interacts with Alpha and Beta, and is set up so that no service is dependent on it.
Having Charlie be the dependent server with nothing requiring it to exist is super important. As long as Charlie remains a middle-man that Alpha and Beta don't need, I can then go on to set up Delta and Epsilon, also on cheap 4GB Linodes, using mirror images of Charlie. If Alpha or Beta need an upgrade, I'll probably just bump up the Linode to 8GB/2GB respectively, since RAM is the most critical aspect to each, not CPU like it is with Apache services. Since CPUs become less cost-effective on Linode after 4GB, this makes the most sense to me.
Just wanted to spout off on web stuff. Thanks for the input.