Download from object storage using SSE-C & signed URLs
I’m really struggling to understand how to download an object from storage that’s been encrypted with SSE-C using boto3.
Uploading works perfectly. Getting the signed URL works okay.
But when I perform a request with the necessary headers I always get the SignatureDoesNotMatch
response.
Here’s my Python code:
import datetime
import hashlib
import boto3
from botocore.config import Config
encryption_key = "32characterslongpassphraseneeded"
encryption_key_md5 = hashlib.md5(encryption_key.encode("utf-8")).hexdigest()
print(encryption_key, encryption_key_md5)
client = boto3.client(
"s3",
{
"aws_access_key_id": "ACCESS_KEY",
"aws_secret_access_key": "SECRET_KEY",
"endpoint_url": "https://us-east-1.linodeobjects.com",
},
config=Config(signature_version="s3v4", region_name="us-east-1"),
)
client.upload_file(
Filename="TEST.pdf",
Bucket="MY_BUCKET",
Key="TEST.pdf",
ExtraArgs={
"Expires": datetime.datetime.now() + datetime.timedelta(hours=2),
"SSECustomerKey": encryption_key,
"SSECustomerAlgorithm": "AES256",
},
)
url = client.generate_presigned_url(
"get_object",
Params={
"Bucket": "MY_BUCKET",
"Key": "TEST.pdf",
"SSECustomerKey": encryption_key,
"SSECustomerAlgorithm": "AES256",
},
ExpiresIn=3600,
)
CURL request:
curl --request GET \
--url 'https://us-east-1.linodeobjects.com/BUCKET_NAME/TEST.pdf?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=CREDENTIAL&X-Amz-Date=20220405T005252Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host%3Bx-amz-server-side-encryption-customer-algorithm%3Bx-amz-server-side-encryption-customer-key%3Bx-amz-server-side-encryption-customer-key-md5&X-Amz-Signature=36496105641e9ee0fa81780e703ccd2d2ff189ec38cab7276f507a12739030f8' \
--header 'x-amz-server-side-encryption-customer-algorithm: AES256' \
--header 'x-amz-server-side-encryption-customer-key: 32characterslongpassphraseneeded' \
--header 'x-amz-server-side-encryption-customer-key-md5: bbb6434b1d1e3eb63fabd2491625f5fd'
If I’m reading the URL correctly it says that a host
header is part of the signed headers, but I have no idea what the host header should be, I’ve tried all my application hostnames, localhost
, 127.0.0.1
, 0.0.0.0
, etc. And I don’t know how to set that with boto3 so I can ensure it doesn’t change.
Anyone have any ideas?
1 Reply
I tested it out using the default script provided in our How to Use Server-Side Encryption with Linode Object Storage guide, and it successfully uploaded, then downloaded and deleted the file. I know this isn't exactly what you were trying to do, though I just wanted to give it a go to make sure our documentation wasn't misleading in some way :)
Here are a few resources that might help troubleshoot the "SignatureDoesNotMatch" error: