November 8, 2019 / Nirav S
When object uploaded to s3 from third-party media, sometimes the object ownership considered as an anonymous. It’s difficult to do any object level operations on it when the ownership is anonymous and consider you have a huge amount of objects like in TB, it’s very, very difficult to change anonymous permission one by one for all the objects, it’s near to impossible. So what’s the solution !
The Issue that we faced.
The Solution
Link to install AWS cli
https://docs.aws.amazon.com/cli/latest/userguide/install-linux.html
Script to list and change ownership of objects in s3 bucket
First of all run a command that will output only the names of the object in a particular directory in s3 bucket and pipe its output into a file.
$ aws s3 ls s3://BUCKET_NAME/FOLDER_NAME/ | perl -pe 's/^(?:\S+\s+){3}//' >> FILE_NAME.txt
Now, we have a file(FILE_NAME.txt) which contains the list containing only names of the objects that are in a buckets’ sub-directory.
Now, we use that files’s list of names as an input for our script that will change the ownership.
$ vim script.sh
#!/bin/bash
input="/FILE_PATH/FILE_NAME.txt"
while IFS= read -r line
do
aws s3 ls s3://BUCKET_NAME/FOLDER_NAME/$line --recursive | awk '{cmd="aws s3api put-object-acl --acl bucket-owner-full-control --no-sign-request --bucket BUCKET_NAME --key "$4; system(cmd)}' &&\
aws s3 cp --acl bucket-owner-full-control s3://BUCKET_NAME/FOLDER_NAME/$line s3://BUCKET_NAME/FOLDER_NAME/$line --storage-class STANDARD
Save the above lines in a file and make it executable.
Create another file which will store logs of the above script.
$ touch task.log
Now, execute the command as follows.
$ nice -n 15 sh script.sh &>> task.log &\
> disown -h %1
The Conclusion
Have queries about your AWS project ideas and concepts? Please drop in your project details to discuss with our AWS experts, professionals and consultants.