Add --no-overwrite option to aws s3 cp/mv
Problem
It would be nice to have a convenience function `--no-overwrite` for `aws s3 cp/mv` commands, which would check the target destination doesn't already exist before putting a file into an s3 bucket. Of course this logic couldn't be guaranteed by the AWS API (afaik...) and is vulnerable to race conditions, etc. But it would be helpful to prevent unintentional mistakes!
Unverified for your environment
Select your OS to check compatibility.
2 Fixes
Implement --no-overwrite Option for aws s3 cp/mv
The current aws s3 cp/mv commands do not have a built-in mechanism to prevent overwriting existing files in the target location. This can lead to unintentional data loss if a user mistakenly uploads a file that already exists in the S3 bucket. The proposed --no-overwrite option would check for the existence of the target file before proceeding with the upload or move operation.
Awaiting Verification
Be the first to verify this fix
- 1
Define the --no-overwrite Option
Add a new command-line option --no-overwrite to the aws s3 cp and mv commands. This option will trigger a check to see if the target file already exists in the specified S3 bucket.
bashaws s3 cp localfile.txt s3://mybucket/ --no-overwrite - 2
Implement Existence Check Logic
In the command implementation, add logic to check if the target file exists in the S3 bucket when the --no-overwrite option is specified. If the file exists, return an error message and abort the operation.
bashif aws s3 ls s3://mybucket/localfile.txt; then echo 'Error: File already exists. Use --overwrite to replace.'; exit 1; fi - 3
Update Documentation
Update the AWS CLI documentation to include the new --no-overwrite option, detailing its usage and behavior. Ensure users are aware of how to utilize this feature to prevent accidental overwrites.
- 4
Test the New Feature
Create unit tests to verify that the --no-overwrite option correctly prevents overwriting existing files. Ensure that both positive and negative test cases are covered.
pythondef test_no_overwrite(): assert aws_s3_cp('--no-overwrite') raises Error
Validation
To confirm the fix worked, perform a test upload using the --no-overwrite option to a file that already exists in the S3 bucket. The command should return an error message without overwriting the existing file. Additionally, check the AWS CLI documentation to ensure the new option is correctly described.
Sign in to verify this fix
1 low-confidence fix
Implement --no-overwrite Option for aws s3 cp/mv Commands
The AWS CLI does not currently provide a built-in mechanism to prevent overwriting existing files in S3 when using the cp or mv commands. This can lead to unintentional data loss if a user mistakenly uploads a file with the same name as an existing one. The proposed --no-overwrite option would check for the existence of the target file before proceeding with the upload, thus mitigating the risk of accidental overwrites.
Awaiting Verification
Be the first to verify this fix
- 1
Create a Shell Script Wrapper
Develop a shell script that wraps the aws s3 cp/mv commands and checks for the existence of the target file in the S3 bucket before executing the command.
bash#!/bin/bash if aws s3 ls "s3://$2"; then echo "Error: Target file already exists. Use --overwrite to proceed."; exit 1; else aws s3 $1 $2; fi - 2
Add --no-overwrite Option to the Script
Modify the script to accept a --no-overwrite flag that triggers the existence check. If the flag is not provided, the command should execute normally.
bash#!/bin/bash while [[ "$#" -gt 0 ]]; do case "$1" in --no-overwrite) NO_OVERWRITE=true; shift; ;; *) COMMAND="$1"; shift; ;; esac done if [ "$NO_OVERWRITE" = true ] && aws s3 ls "s3://$2"; then echo "Error: Target file already exists. Use --overwrite to proceed."; exit 1; else aws s3 $COMMAND $2; fi - 3
Test the Script
Run the script with both the --no-overwrite option and without it to ensure it behaves as expected. Verify that it prevents overwriting when the option is used and allows the operation when it is not.
bashbash my_s3_wrapper.sh cp localfile.txt s3://mybucket/remote_file.txt --no-overwrite - 4
Document the Usage
Update the documentation to include information about the new --no-overwrite option, including examples of how to use the script effectively.
bash# Usage: my_s3_wrapper.sh [cp|mv] <local_file> <s3_uri> [--no-overwrite]
Validation
To confirm the fix worked, attempt to upload a file to an S3 bucket using the --no-overwrite option when a file with the same name already exists. The script should prevent the upload and display an error message. Additionally, try uploading without the option to ensure it proceeds as expected.
Sign in to verify this fix
Environment
Submitted by
Alex Chen
2450 rep