Upload large files (over 10 GB)
Hi, I tried to upload a file with 36GB to Object Storage via linode-cli.
This is the command:
linode-cli obj put filename.extension bucket-name/Directory_1/Directory_2/Directory_3
The process start and says:
filename (36 parts)
Part 1
|---------------------------------------------------------------| 13.1%
After several attempts got this error:
Upload failed! Cleaning up!-------------------------------------------------------------| 13.1%
Any Ideas ????
Thanks in advance.
1 Reply
jhartman
Linode Staff
As noted in our Object Storage "Getting Started", 5GB is the maximum uploaded file size. This is a hard limit for the platform, so specifically when uploading with the CLI (a wrapper of our backend API), uploads larger than 5GB will fail.
In order to get around those limitations, you should use tools like S3cmd or Cyberduck to do multi-part uploads:
I found some additional StackOverflow posts about how to troubleshoot failed multipart uploads (and delete those partial files) as well as how to incorporate multipart uploads into NodeJS scripts: