Set Get permission on a sub directory without making it public

I'm having trouble giving access a user to a subdirectory of my bucket using his iam:ID. Here's the info output of bucket:

s3://orisha/ (bucket):
Location: default
Payer: BucketOwner
Expiration Rule: none
Policy: {
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Allow access de deb packages",
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam:::1d3f9948-47e1-4d11-9ef7-e63925f84e6a"
]
},
"Action": "s3:GetObject",
"Resource": ["arn:aws:s3:::orisha/packages/*"]
}]
}

This seem to allow my user to access object stored in /packages, but it also seems to set my files to public. I can access them using curl. Is it possible remove public access without removing my user access ?