AWS S3 Lifecycle Policies - Prep for Deep Archive

AWS recently released a new S3 storage class called Deep Archive. It's an archival data service with pretty low cost for data you need to hold onto, but don't access very often.

Deep Archive is about half the cost of Glacier at $0.00099 per GB per month, but you sacrifice the option to get your data back in minutes, your only retrieval option is hours.


I work for a health care company so we hold onto patient data for years. There are plenty of reasons we might need to retrieve data from years ago, but few of them would have a time limit of less than several weeks. That makes Deep Archive a great fit for our long term data retention.

Setting it up is as simple as changing an existing life cycle transition to Deep Archive, or creating a new S3 Lifecycle transition to glacier

We put together a quick script to find the lifecycle transition rules in our S3 buckets that move data to Glacier already

$buckets = get-s3bucket;

# Iterate through buckets in the current account
foreach ($bucket in $buckets) {
    write-host -foregroundcolor Green "Bucket: $($bucket.BucketName)";

    # Get the lifecycle configuration for each bucket
    $lifecycle = Get-S3LifecycleConfiguration -BucketName $bucket.BucketName;

    # Print a warning if ther eare no lifecycles for this bucket
    if(!$lifecycle) {
        write-host -foregroundcolor Yellow "$($bucket.BucketName) has no life cycle policies";
    } else {
        # Iterate the transition rules in this lifecycle 
        foreach ($rule in $lifecycle.Rules) {
            write-host -foregroundcolor Magenta "$($rule.Id) with prefix: $($rule.Filter.Lifecyclefilterpredicate.Prefix)";
            # Print a warning if there are no transitions
            if(!($rule.Transitions)) {
                write-host -foregroundcolor Yellow "No lifecycle transitions";
            }

            # Iterate the transitions and print the rules
            foreach ($transition in $rule.Transitions) {
                if($transition.StorageClass -eq "GLACIER") {
                    $color = "Yellow";
                } else {
                    $color = "White";
                }
                write-host -foregroundcolor $color "After $($transition.Days) transition to $($transition.StorageClass)";
            }
        }
    }
}

To run this script you'll need the AWS powershell tools, an IAM account setup, and a default region initialized.

When you run the script it will print out your current S3 buckets, the lifecycle rules, and the transitions in each of them, highlighting the transitions to Glacier in yellow.