How do I use s3fs on Linode Object Storage?

This has been updated to reflect contributions by @mattwatts and @jyoo.

Mounting a Linode Object Storage bucket on your Linode

Installation

We want to install s3fs-fuse:

Depending on your distribution would do (as root):

Example Notes

This example will use mybucket as our bucket name, us-east-1 as our region and /mount/s3 as our mount point.

USEYOURREALACCESSKEY123 will be our access key, and
USEyourREALsecretKEYhereITisLONGER456 will be our secret key.

Wherever these values appear, feel free to substitute your own.

Create a mount point

mkdir -p /mount/s3

Edit your /etc/fstab file

Add the following on a single line:

mybucket /mount/s3 fuse.s3fs _netdev,allow_other,use_path_request_style,url=https://us-east-1.linodeobjects.com/ 0 0

Credentials

Put your credentials in /etc/passwd-s3fs separated by a colon:

USEYOURREALACCESSKEY123:USEyourREALsecretKEYhereITisLONGER456

Set the passwd-s3fs file's permissions:

chmod 600 /etc/passwd-s3fs

Mount your bucket

mount -a

Limitations

From: https://github.com/s3fs-fuse/s3fs-fuse

Generally S3 cannot offer the same performance or semantics as a local file system. More specifically:

• random writes or appends to files require rewriting the entire file
• metadata operations such as listing directories have poor performance due to network latency
• no atomic renames of files or directories
• no coordination between multiple clients mounting the same bucket
• no hard links
• inotify detects only local modifications, not external ones by other clients or tools

Following these instructions I am able read the contents of my s3 bucket. I can drop files into s3 via the linode interface and immediately see them on my mount. However, when I do 'touch test.txt' in the mounted folder the file is created but does not propagate to s3. Is this the expected behavior?

nobrega7121: You may need to change the permissions on the file in order to view them from other locations. The s3fs GitHub has some information about this. You could also use s3cmd by running s3cmd put example.txt s3://my-example-bucket -P, though this is different than creating a file in the mounted directory.

Hey there,

I was attempting to follow these instructions on my Ubuntu 20.04 machine and I ran into a couple of issues that I wanted to share with the community.

Essentially, when following these instructions verbatim, I received the error below:

terminate called after throwing an instance of 'std::invalid_argument'
what(): s3fs_strtoofft
Aborted (core dumped)

I took a look at the s3fs-fuse repo and noticed some discrepancies in the way that they recommend formulating /etc/fstab and what Harry has above. Essentially I needed to remove the s3fs# at the beginning of the line, change the fuse option after the mount point to fuse.s3fs, and remove the ,uid=userid,gid=groupid after the Linode Object Storage URL.

My /etc/fstab entry now looks like this:

{mybucket} {/path/to/mountpoint}  fuse.s3fs _netdev,allow_other,use_path_request_style,url=https://{region}.linodeobjects.com/ 0 0

After making this update I was receiving an error regarding my credentials that I had in ${HOME}/.passwd-s3fs and determined that I needed to move them to /etc/passwd-s3fs. After making that change, I was able to run the mount -a command and get this bucket mounted \o/.

Anyway, I hope this information helps anyone who ends up getting stuck with that same error that I was seeing.

I was messing around with this recently and figured it might be helpful for anyone looking at it, while you should still use the corrections mwatts noted for a normal mount there are still some situations you might want to use the uid= and similar options such as when you need a service user to be able to read files in the mounted directory.

You can include these in the fstab entry or add them to the s3fs mount command with -o

Here's an example of each that gives a specific user permissions, you can do multiple users (from what I can tell they have to all have the same permissions though) or use gid= for a group too if you like. umask= is for configuring the specific permissions you want.

s3fs mount:

sudo s3fs BUCKETNAME /PATH/TO/MOUNT/ -o passwd_file=/PATH/TO/FILE -o url=https://REGION.linodeobjects.com -o use_path_request_style -o allow_other -o uid=xxxx -o umask=xxxx

fstab:

BUCKETNAME /PATH/TO/MOUNT/ fuse.s3fs _netdev,allow_other,uid=xxxx,umask=xxxx,use_path_request_style,url=https://REGION.linodeobjects.com/ 0 0

Note for the fstab you may need to specify passwd_file= if you aren't using /etc/passwd-s3fs to store it.