Saturday, January 5, 2019

Diving Into (and reducing) Your AWS Costs!

AWS uses a "pay as you go" model for most of it's services. You can start using them at any time, you often get a runway of free usage to get up to speed on the service, then they charge you for what you use. No contract negotiations, no figuring out bulk discounts, and you don't have to provision for max capacity.


This model is a double edge sword. It's great when you're

  • First getting started
  • Working with a predictable workload
  • Working with a modern technology stack (i.e. most of your resources are stateless and can be ephemeral
But it has some challenges when
  • Your workload is unpredictable
  • Your stack is not stateless (i.e. you have to provision for max capacity)
  • Your environment is complex with a lot of services being used by different teams

It's easy to have your AWS costs run away from you and you can suddenly find yourself paying much more than you need or want to. We recently found ourselves in that scenario. Obviously I can't show you our actual account costs, but I'll walk you through the process we used to start digging into (and reducing our costs) with one of my personal accounts.

Step 1: AWS Cost Explorer

Cost Explorer is your first stop for understanding your AWS bill. You'll navigate to your AWS Billing Dashboard, and then launch cost explorer. If you haven't been in cost explorer it doesn't hurt to look at some of the alerts on the home page, but the real interesting data is in Costs and Usage


My preference is to switch to "stack view"

I find this helps to view your costs in context. If you're looking to cut costs the obvious place to start is the server that takes up the largest section of the bar. For this account it's ElastiCache

ElastiCache is pretty straight forward to cut costs for - you either cut your nodes or node size - so let's pick a more interesting service like S3.

Once you've picked a service to try to cut costs for add a service filter on the right hand side, and group by service type

Right away we can see that most of our costs are TimedStorage-ByteHrs which translates to S3 Standard Storage, so we'll focus our cost savings on that storage class.

Next we'll go to Cloudwatch to see where our storage in this class is. Open up Cloudwatch, open up metrics, and select S3.


Inside of S3 click on Storage Metrics and search for "StandardStorage" and select all buckets.


Then change your time window to something pretty long (say, 6 months) and your view type to Number

This will give you a view of specific buckets and how much they're storage. You can quickly skim through to find the buckets storing the most data.

Once you have your largest storage points you can clean them up or apply s3 lifecycle policies to transition them to cheaper storage classes.

After you're done with your largest cost areas, you rinse and repeat on other services.

This is a good exercise to do regularly. Even if you have good discipline around cleaning up old AWS resources costs can still crop up.

Happy cost savings!

No comments:

Post a Comment