AWS S3 Bucket Creation Date Discrepancy In Master And Other Regions
AWS S3 buckets are among the most commonly used AWS resource. At the same time, S3 buckets are most prone to misconfigurations. All major famous leaks generally have happened with S3 bucket misconfigurations. For this reason, the cloud teams continuously keep track of S3 bucket configurations.
The AWS resource inventory is a common way of tracking the AWS resources. The OSS projects such as CloudQuery, Steampipe and Scout Suite are used by many cloud teams for this purpose.
The AWS resources are either global or region-wise. Services such as IAM are completely global, while resources such as EC2 are regional. The S3 bucket is one weird creature with a global namespace, even spanning beyond the usual AWS account bubble. At the same time, S3 buckets are associated with regions. S3 buckets are created in a region. This is typically done to reduce latency, minimize costs, or address regulatory requirements.
We collect the AWS resources information with AWS service APIs such as Describe/Get/List. S3 buckets are listed with list-buckets API. Since S3 buckets are regional specific we would call this API region-wise. And this is when we noticed one interesting discrepancy in S3 bucket creation time information.
The same bucket amplify-testfigma-staging-164952-deployment shows a different creation time.
More digging into the issue took us to the AWS documentation for this API.
CreationDate -> (timestamp)
Date the bucket was created. This date can change when making changes to your bucket, such as editing its bucket policy.
After some more digging, we stumbled upon the AWS CLI issue reported in github and Mark Sayson’s blog post. An AWS support engineer wrote:
This is expected behavior due to the design of the service. The GET Service call in S3 (s3api list-buckets or s3 ls with no further arguments in the CLI) works differently when being run against different regions. All bucket creations are mastered in us-east-1, then replicated on a global scale — the resulting difference is that there are no “replication” events to the us-east-1 region. The Date Createdfield displayed in the web console is according to the actual creation date registered in us-east-1, while the AWS CLI and SDKs will display the creation date depending on the specified region (or the default region set in your configuration).
When using an endpoint other than us-east-1, the CreationDate you receive is actually the last modified time according to the bucket’s last replication time in this region. This date can change when making changes to your bucket, such as editing its bucket policy.
The S3 global namespace is confined to the AWS partitions. This is also due to the fact that each partition has its own designated master region. Mark Sayson confirmed in his blog post his findings.
- “us-east-1” for the Classic “aws” partition
- “cn-north-1” for the AWS China “aws-cn” partition
- “us-gov-west-1” for the AWS GovCloud “aws-us-gov” partition
Now this is what we call the fun of working on AWS resource inventory service and discovering interesting tidbits.
Originally published at https://www.cloudyali.io.
