Announcement: S3 default integrity change
Problem
In AWS CLI v2.23.0, we released changes to the S3 client that adopts new default integrity protections. For more information on default integrity behavior, please refer to the official SDK documentation. In SDK releases from this version on, clients default to enabling an additional checksum on all Put calls and enabling validation on Get calls. You can disable default integrity protections for S3. We do not recommend this because checksums are important to S3 integrity posture. Integrity protections can be disabled by setting the config flag to `when_required`, or by using the related AWS shared config file settings or environment variables. Disclaimer: The AWS SDKs and CLI are designed for usage with official AWS services. We may introduce and enable new features by default, such as these new default integrity protections prior to them being supported or handled by third-party service implementations. You can disable the new behavior with the `WHEN_REQUIRED` value for the `request_checksum_calculation` and `response_checksum_validation` configuration options covered in Data Integrity Protections for Amazon S3.
Unverified for your environment
Select your OS to check compatibility.
1 Fix
Enable Default Integrity Protections for S3 in AWS CLI
The new default integrity protections in AWS CLI v2.23.0 require checksums for Put and Get calls to ensure data integrity. If these protections are disabled, it can lead to potential data corruption or loss during transfers, especially when interacting with third-party services that may not support these features.
Awaiting Verification
Be the first to verify this fix
- 1
Check Current Configuration
Before making any changes, verify the current settings for checksum calculations and validations in your AWS CLI configuration.
bashaws configure get default.s3.request_checksum_calculation aws configure get default.s3.response_checksum_validation - 2
Update AWS CLI Configuration
Set the configuration options for request checksum calculation and response checksum validation to 'WHEN_REQUIRED' if you need to disable integrity protections. However, it is recommended to keep them enabled for data integrity.
bashaws configure set default.s3.request_checksum_calculation ENABLED aws configure set default.s3.response_checksum_validation ENABLED - 3
Modify Shared Config File (Optional)
If you prefer to manage settings via the shared config file, edit the ~/.aws/config file to include the following lines under the appropriate profile section.
ini[default] s3_request_checksum_calculation = ENABLED s3_response_checksum_validation = ENABLED - 4
Set Environment Variables (Optional)
Alternatively, you can set environment variables to control the integrity settings without modifying the AWS CLI configuration directly.
bashexport AWS_S3_REQUEST_CHECKSUM_CALCULATION=ENABLED export AWS_S3_RESPONSE_CHECKSUM_VALIDATION=ENABLED - 5
Test S3 Operations
Perform a series of S3 Put and Get operations to ensure that the integrity protections are functioning as expected. Monitor for any errors or warnings related to checksum validation.
bashaws s3 cp localfile.txt s3://your-bucket/ aws s3 cp s3://your-bucket/localfile.txt ./
Validation
Confirm that the integrity protections are enabled by checking the configuration settings again and ensuring that S3 operations complete without checksum-related errors. Review the AWS CLI output for any warnings or errors during the operations.
Sign in to verify this fix
Environment
Submitted by
Alex Chen
2450 rep