Solution: for Laravel sites who want to transfer files from S3 to Linode (object storage on S3 protocol)

(This was written in another post, reposted here because it will help a lot of new Linode customers)

For Laravel users, a helper to move your files from S3 to Linode

It might work for Laravel 5 and 6 but I can't test it. It certainly works for L7 and L8.

How does it work? It runs on your server. You can also run it on your testserver but beware that the speed of execution will depend on your home/office internet speed. It's a artisan command that will copy your files from S3 to Linode (S3 compatible object storage). The folderstructure stays the same.

How to implement it?

Step 1
Set up your config/filesystem.php file accordingly as you add this:

     'linode'      => [
        'driver'     => 's3',
        'key'        => env('LINODE_S3_ACCESS_KEY'),
        'secret'     => env('LINODE_S3_SECRET'),
        'endpoint'   => env('LINODE_S3_ENDPOINT'),
        'region'     => env('LINODE_S3_REGION'),
        'bucket'     => env('LINODE_S3_BUCKET'),
        'url'        => env('LINODE_S3_BUCKET_URL'),
        'visibility' => 'public',
    ],

And set your .env file with the necessary data. for more info, check this stackoverflow answer

step 2
Run the command 'php artisan make:command LinodeCopy' It will create a file at 'app/Console/Commands/LinodeCopy.php'

Replace all code in the file with this gist I've uploaded

step 3
Upload to your production server.

step 4
Run the command php artisan help linode-copy to see the options. For example php artisan linode-copy avatars is a request to copy the /avatars folder from S3 to Linode. If you want all the subdirectories included, you can run php artisan linode-copy -D avatars

It will show the amount of files. If the number is high, you opt out. The terminal will ask if you want to continue. If you have a lot of files, I recommend you sync folder by folder, not all at once.

If the server runs out of memory, you will notice you know exactly at what #file it broke. You can add --counter 2577 to your artisan command if it hang on 2578. The reason is to avoid running over the previous 2577 files on Linode. You already know they exist. It bypasses them.

The command php artisan linode-copy will copy only files on your root while php artisan linode-copy --D will copy all files, including every subfolder.

Remark 1: This is will probably not work for big files. You may have to tweak the code to stream files instead of copy them. Your server will quickly run out of memory.

Remark 2: This software is experimental, use at your own risk and responsibility. It is designed for my personal usage of the S3 to Linode conversion which are pictures. I can't test it for big files.

Update I've tested it with over 30.000 files. It runs out of memory after a while. Just rerun the same command. All files already on Linode are ignored and untouched on S3. It simply continues where it left off. Especially if you add --counter 9999 to it, wherever you left off. It speaks for itself. The good news is: it works very fast, at about 7 to 10 files a second!

Good luck!