Host a Media Server using Object Storage
What I am trying to achieve is: Host a media server as a Linode (using Plex for this discussion) and use RSS feeds to download content via torrents while storing this data affordably. This is my current setup:
Object Storage
I've transferred all my data to object storage. To access the media on it I have mounted to my media server as /media/blob/ using fuse and s3fs. At this point I have 2.8TB of data.
Plex
Directories in Plex reference the s3fs mount. The quality and speed of streaming is satisfactory. File listing ls is sub-optimal but not unbearable and this is moot once Plex has indexed the files.
Torrents
Using Transmission and Transmission web I am able to achieve fetching torrents. I have Transmission set to use /media/blob/torrents/… for its required directories (DownloadDir, IncompleteDir etc.)
Using this method is costing me more than I expected in terms of Oject Storage usage and seems to eat my monthly transfer limits MUCH faster than when I was using block storage. So, questions:
Am I using a suitable file store for my purposes?
Does mounting Object Storage on a Linode mean that bandwidth is counting against the monthly transfer limit when:
2.1 Transmission writes directly to /media/blob/torrents
2.2 Plex fetching media to stream from the mounted s3fs and again when
2.3 I stream to my local machine?If the answers to 2 is YES what would be a more efficient method to achieve what I want to do?
I hope I've described my situation sufficiently. Any suggestions on how to achieve having an affordable media download and streaming service are welcome.
2 Replies
rgerke
Linode Staff
Hello!
I've not tried doing this myself, but I will do my absolute best to answer your questions. Let's start with this one:
Am I using a suitable file store for my purposes?
I would say yes to this, because one of the outlined use cases for our Object Storage is media storage. The link provided is to an e-book, but there is some info on that page that talks about good ways of putting Object Storage to use.
Does mounting Object Storage on a Linode mean that bandwidth is counting against the monthly transfer limit when:
2.1 Transmission writes directly to /media/blob/torrents
2.2 Plex fetching media to stream from the mounted s3fs and again when
2.3 I stream to my local machine?
This is really summed up well by another post here in the Linode Community -
Bandwidth Allowance - Linode to Object Storage
If you upload to your object storage from your Linode using IPv6, you won't be charged provided that the Linode and the Object Storage instance are located in the same data center.
For specifics, please take a look at the Transfer Quotas section of this guide:
If the answers to 2 is YES what would be a more efficient method to achieve what I want to do?
My suggestion is to read over the documentation and decide what might be the best setup for your purposes. Keep in mind that you do get 1TB of transfer for your Object Storage for each full month that it is active (and prorated if active for less than a month).
I'll also leave this open for the rest of the Linode Community to take a look at - and if someone has built a similar set-up, feel free to give your suggestions.
Cool, thank you for the feedback.
I would say yes to this, because one of the outlined use cases for our Object Storage is media storage. The link provided is to an e-book, but there is some info on that page that talks about good ways of putting Object Storage to use.
Yes, it is a wonderfully versatile and affordable utility.
I've had great success with rclone to access my buckets as a mounted drive on my local machine. To stream media takes a little bit of customizing but when you find the sweet-spot it's awesome. This, of course, depletes transfer quota.
Mounting my buckets to a Linode using fuse and s3fs has also been useful. Here I have no idea if the traffic is using up transfer quota?
If you upload to your object storage from your Linode using IPv6, you won't be charged provided that the Linode and the Object Storage instance are located in the same data center.
The Linode(s) and the Object Store are in the same data center - so I'm good on that front. I am however unsure whether my transfers are using IPv6 and am struggling to find a comprehensive guide on how to ensure traffic between Linode(s) and storage is using IPv6.
When I do eventually settle on a setup I'll be sure to post it here. I am sure there are others that have a similar use case.
Thanks again for the feedback and I welcome any other thoughts from the community.