What's the best way to transfer Block Storage volume data across data centers?
I want to transfer my Block Storage volume to another data center. I know that I cannot transfer the Block Storage volume itself, but I am reading from this guide about how to manually to transfer this data.
I feel pretty good about performing this transfer, but do you have any additional tips or pointers to help me with this process? Thank you!
1 Reply
The linked routine is definitely quite straightforward, although it is nevertheless possible to consolidate some of its steps by creating your target Block Storage volume ahead of time.
This way, you can transfer your data directly between each of your Block Storage volumes. While this will likely result in a longer overall transfer due to Block Storage's slower speeds compared to your Linode's SSD, this method does avoid the need to perform any further internal data transfers.
This direct transfer method will also allow you to avoid filling up your Linode's SSD storage, which may be necessary if you have more Block Storage data than can fit on your Linode's SSD storage.
Some additional tips for your interest:
While a Nanode is perfectly capable of performing this transfer, you may want to use a slightly larger Linode for this transfer operation. The greater bandwidth of these larger plans will make the transfer complete sooner. You may view the bandwidth for our plans under the "Network Out" column on our Pricing page.
rsync is a more powerful utility than SCP, supporting data verification, permissions retention, and sophisticated progress tracking. Just like SCP, it works over SSH, making it relatively easy to use with your existing setup.
From a security standpoint, it is best to create a limited user for SSH access, which includes SCP and rsync. However, a limited user will likely not be able to properly set and retain the ownership and permissions data of your Block Storage volume's files.
To get around this problem, you may first want to archive and compress your data with a permissions-aware archiving format, such as GNU Tar. Depending on the nature of your data, compression may also significantly reduce its size, making the transfer complete sooner.
Once you transfer the data, you can then use the receiving Linode's
root
user to decompress and unarchive the data with the correct permissions. Of course, since your source Linode and destination Linode most likely have a different set of users and groups, it would still be prudent to manually validate the accuracy of these settings for the Linode which will be most actively utilizing this Block Storage volume data.This Community Site post has some excellent tips on how to transfer data between two different Linodes. In particular, generating an SSH key for key-based authentication between your Linode is a solid security practice that I emphatically recommend.
I hope that this information provides you some good additional assistance with transferring your Block Storage Volume data between data centers! If you have any need for further clarification, please don't hesitate to ask.