Load balancing servers in different data centers

Gigenet Cloud has a datacenter in Chicago and a datacenter in Los Angeles and they do support private IP traffic between the two for free. It should be possible to setup such a config, but I warn you that their reliability (in my personal experience) is FAR lower than that of Linode and their support isn't quite as good, either.

Softlayer also allows private IP traffic between datacenters, but their cloud seems a bit less robust than Gigenet's. It depends. With Softlayer, I'd go for local storage on the cloud stuff.

@Ghan_04:

Gigenet Cloud has a datacenter in Chicago and a datacenter in Los Angeles and they do support private IP traffic between the two for free. It should be possible to setup such a config, but I warn you that their reliability (in my personal experience) is FAR lower than that of Linode and their support isn't quite as good, either.

Softlayer also allows private IP traffic between datacenters, but their cloud seems a bit less robust than Gigenet's. It depends. With Softlayer, I'd go for local storage on the cloud stuff.

Thanks for your response, but I don't want to switch providers. I was hoping on just been able to lease an IP address from somewhere which will act as bridge between linode data centers.

That's not how the Internet works.

When a network operator wants to receive traffic to a particular range of addresses, it announces its route to that 'prefix' to the other networks it connects to. This propagates across the Internet to every other router, which updates its internal routing table accordingly. As such, every single route announced to the Internet has a memory and performance penalty on every router that gets the full routing table.

For this reason, announcements for IPv4 prefixes longer than /24 (that is, less than one /24's worth of addresses) get dropped. So, the minimum address block that may be announced is a /24.

Most IP-based multi-location load-balancing is based on anycasting, which is the announcement of the same addresses from multiple locations. This works out pretty well, but it is an expensive proposition: first, you need to reliably interconnect with multiple networks in multiple locations using BGP, and you need someone to properly engineer the network. Then, you need to be able to deal with any packet from any user going to any of the locations at any time. This pretty much rules out TCP, unless you have strong interconnectivity between the multiple locations and an architecture that can deal with this in a resilient fashion. (UDP, and particularly DNS itself, is very suited for Anycast.)

This requirement for interconnectivity between locations is a serious weakness. Part of how Linode, etc, excel is by not having their own global backbone connecting the various locations. This means that (barring human or meta-architectural failure) a fault that affects one location cannot affect the other locations. You can find providers that do have connectivity between their locations, but you'll inevitably find they're either really expensive and "carrier-grade" (i.e. "Certainly! Our minimum buy is 8x 42U two-post racks, and we have a loading dock right next to the freight elevator… what's a UPS?"), or they have very limited geographic coverage (i.e. "We have a redundant global network with a datacenter in Minneapolis and another in St. Paul!"), or they suck.

Long story short: DNS-based load balancing can be done today very cheaply, and because clients will be sticky to one IP address for a particular TCP connection, it gets around the whole TCP state problem. It's the way to go, and you'd probably end up having to do it anyway.

Oh, and no matter what, database replication cannot occur faster than the speed of light. Welcome to my hell.

Thanks for the clarification. I guess since I don't want to change providers, I'm stuck with DNS round robin load distribution.

It's still possible to do actual load balancing with DNS round-robin. There's obviously a delay in changes to the DNS records taking effect (even with a low TTL), but you can dynamically update the list of which servers are returned from the DNS based on load. If one server is seeing much higher load than others, it can be removed from the rotation until the load drops back down.

This works because clients cache DNS information themselves (irrespective of TTL, I believe), and will stay on a server for the rest of their session once they hit it.

Also note that "real" IP load balancing/failover (i.e. one prefix announced from multiple locations via BGP) is nowhere near instant, nor is it perfect. The 180-second hold timer creates the longest 180 seconds you've ever experienced…

That's why the pros use DNS.

@Guspaz:

This works because clients cache DNS information themselves (irrespective of TTL, I believe), and will stay on a server for the rest of their session once they hit it.
Depends on the client. There's no guarantee of that. You need to handle clients switching from one IP to another, or some of them will have problems.

@mnordhoff:

@Guspaz:

This works because clients cache DNS information themselves (irrespective of TTL, I believe), and will stay on a server for the rest of their session once they hit it.
Depends on the client. There's no guarantee of that. You need to handle clients switching from one IP to another, or some of them will have problems.

Right, but my point was more that the load on a server won't instantly drop to zero when you remove it from the DNS-RR pool. There will still be cached DNS servers serving that up, and clients doing business direct with that server.

Considering that it's a gradual process, then, playing with the DNS records should be a decent enough method of smoothly balancing load.

You can even get trickier than that, and weight how often given IPs appear in a certain order in the list.

from what I have gathered it would then make sense to configure DNS server to use a database e.g. postgresql with pgpool-II so that each server can add and remove themselves automatically from the distributed database which is used to serve DNS info.

I think this solution makes more sense since we wouldn't have to restart dns server and each machine will be in-control of DNS entries.

So tools I'm going to employe will be:

www.postgresql.org

www.pgpool.net/mediawiki/index.php/Main_Page

www.isc.org/software/bind + postgres binding

I will post out comes here, if anyone wants me to try out something else please share.