How do I use my Linode to watch for a file to change in my S3 bucket?

Does a Linode have any built-in way of kicking off an action when new files come in to an s3 bucket?

While there is no 'built-in' way of doing this, it should be straight forward enough to accomplish. We just need to break the problem down into two parts:

Mounting an s3 bucket on your Linode

Just install s3fs:

On Ubuntu you would just do:

sudo apt-get update && \
sudo apt-get upgrade && \
sudo apt-get install fuse s3fs

Edit your /etc/fstab file and add the following line (replace {mybucket} and {/path/to/mountpoint} as appropriate:

s3fs#{mybucket} {/path/to/mountpoint} fuse allow_other,use_cache=/tmp/cache,uid=userid,gid=groupid 0 0

or

{mybucket} {/path/to/mountpoint} fuse.s3fs _netdev,allow_other 0 0

and put your credentials in /etc/passwd-s3fs or ~/,passwd-s3fs in this format:

AWS_ACCESS_KEY_ID:AWS_SECRET_ACCESS_KEY

Set the passwd-s3fs file's permissions:

sudo chmod 600 /etc/passwd-s3fs

To mount the bucket, do:

mount -a

Watching for a file to change

For local file systems, inotify would be the way to go, but since the directory to be watched is remote, the kernel will not receive change notifications. In this case, I would recommend gamin to do this.

Be sure to use the 'poll' method when setting up gamin.

Details about gamin can be found here. and info about FAM which gamin is based on can be found here

You can install it on Ubuntu with:

sudo apt-get install gamin

Additionally if you want to do development with perl or Python you can install one or more of the following:

sudo apt-get install python-gamin
sudo apt-get install libsys-gamin-perl

Connection Performance and Cost

What can you tell me about data speed, connections, and price when I connect to s3 from a Linode Server?

Your connection's performance to your bucket could vary from anywhere between < 1ms to over 300ms latency depending upon which data center your Linode is located in and where your bucket is. For close proximity (i.e. same state) I would expect < 1ms to 10ms latency.

The cost of connecting to s3 (on Linode) would only include outbound data transfer from your Linode. Polling the bucket should produce minimal traffic if you are only checking it once a minute or so, so it is unlikely you would significantly effect your Linode's transfer allotment.