Load-balancer with multiple servers and shared database

You've raised a few good questions here, Divyansh!

• How can multiple Linodes access a PostgreSQL database?

You can set up remote access for your PostgreSQL database, so that your other Linodes can access the database even though it's hosted on a different Linode. This StackOverflow post has some information that can get you started.

• How will each Linode access user-uploaded files? Block Storage or Object Storage? Can I attach a single volume to all of the Linodes?

A Block Storage volume can only be attached to one Linode at a time. Based on what you've written here, it will likely be most effective to attach a Block Storage volume to your database Linode, and allow your other Linodes to write to the database remotely. This will depend on how quickly you anticipate needing to write to the database, though. Block Storage is not always the best solution for high-performance databases. Using a larger Linode plan with more storage included may be a better option. Object Storage could be a great place to host user-uploaded files as long as your configuration will allow for accessing the files via HTTP.

• What about HAProxy?

HAProxy can be used for load balancing. You could also consider using a NodeBalancer.

• What if two Linodes are writing to a database at once?

PostgreSQL has a nice resource here discussing the things you'll need to keep in mind when multiple Linodes are writing to the same database.

I hope this helps to get you started! Other members of the Linode Community might also be able to share their thoughts and help further.

Hi Jyoo,

Thanks for your detailed response.

Few follow-up queries:

Option 1 - Object storage (ruled out in absence of http urls to access files)
Object storage seems to be out of option for our application (we will not have http urls to access the files).

Option 2 - Block storage (The "database" linode will become bottleneck, which is associated to block storage)
Block storage seems to be an option but the linode to which it will be attached must always be "up". Otherwise, other linodes will not be able to "access" the "files stored on the linode", which is associated with block storage

Option 3 - Linode with higher memory
This option will work till the application exceeds the limit of the maximum memory provided by 1 linode. Even with this option, how do we expand in future to provide load-balancing using multiple linodes. Will this "primary" linode, which is hosting the file-system become a bottle-neck.

Apologies if my questions are naive (and probably already explained in the response). I just want to get the design of the linode right so that I can apply the concept and ensure that the hosted application is future-extensible in terms of both CPU processing (By adding more linodes using load-balancer) and memory (By adding more memory, which is accessible by each of the linodes).

Thanks
Divyansh

Hi Divyansh,

Option 1 - Object storage (ruled out in absence of http urls to access files)
Object storage seems to be out of option for our application (we will not have http urls to access the files).

Can you elaborate on what you mean by “we will not have http urls to access the files“? When you upload a file to Object Storage, you will automatically get a HTTP URL to that file.

Based on your original request:

Application also requires to store user uploaded files (which can be retrieved for download/view purpose).

Object storage is perfect for this use-case. Your users browser can upload to the application running on your Linode, then upload the file to Object Storage. You can then store the file’s URL (on Object Storage) in your database.

If you’re worried about permissions, you can upload the file as “private” then your application can generate time-limited pre-signed URLs for the user to download/view the file directly from the Object Storage.

Hi @jyoo & @andysh

Thanks for your detailed response.

I will try to explore this option, prototype and see how it works and will incorporate into deployment if it works fine.

For my complete understanding, this is the way to go:

1 Load-balancer (which will load balance the traffic across multiple linode servers)

n linode server (which will service the request received from the load-balancer)

Object storage - All "n" linode server will write the files to object storage (which will generate unique url for each created file) and they can store the url in the postgresql database (corresponding to each file being referenced).

postgresql DB - Hosted on a separate linode (Database linode), so that all of the above n linode will interact and write it to DB

This will ensure that I will be able to add as may linodes to "n" linodes for increasing the processing power of the application.

At the same time, I will be able to increase the object storage (associated to database linode) for increasing memory requirements.

Bottlenecks
1 Load balancer (Not sure if they have redundancy architecture for having multiple load balancer)

1 database linode (Since object storage can be associated to only 1 linode and maybe even postgresql can be hosted only on 1 linode)

Can you please confirm I have understood it right?

Thanks
Divyansh

Bottlenecks
1 Load balancer (Not sure if they have redundancy architecture for having multiple load balancer)

I have read somewhere (but can’t find suitable reference now) that the NodeBalancers themselves are highly available.

Have a read of this article where it quotes:

With our NodeBalancer, you don’t have to worry about what would happen if a server fails. We’ve gone the extra mile and built automatic failover into every NodeBalancer. If it were ever to go down, another NodeBalancer is immediately ready to manage your content.

I don’t know how this is done, but effectively you shouldn’t need to worry about the NB’s availability, Linode manage that for you.

1 database linode (Since object storage can be associated to only 1 linode and maybe even postgresql can be hosted only on 1 linode)

Block storage can only be accessed by 1 Linode, Object Storage by any (even non-Linodes.) If you provide a document’s URL on Object Storage to the browser (via an “a” link or “img” tag, if it’s an image) your Linodes won’t have to do any work to serve the document (download/view) - the request will go straight to the Object Storage cluster.

You’re right that PostgreSQL could be a single point of failure, but again there are options with that - hot-spares, replication etc.

@andysh

Appreciate your quick response on this aspect.

Let me reserve 4 linodes and prototype the above concept.

I will update back the thread with the experience (both in case of any further questions or post information useful to other linode users).

Thanks
Divyansh