So you’re new to AWS Lambda and secrets management? Maybe you’ve just joined a team that’s using KMS and you want to know more about how KMS and Lambda work, or maybe you’re looking to use KMS as your preferred choice for Lambda secrets management.
Whatever your reasoning for investigating AWS KMS with Lambda, today we’re going to cover the in’s and out’s of how the two technologies work together, and show you how you can use them.
By the end of this article you’ll understand what KMS is, how KMS works with AWS Lambda and the alternatives to using KMS for AWS Lambda functions.
To start our conversation let’s get on the same page about what secrets management is and why we need it…
When building applications, a diligent software engineer makes sure that all of the secrets and access credentials for their application are safely stored. Exposed access credentials pose a security risk to our application and may cause trouble the company we work for.
One method that we can use to ensure our secrets are stored safely is through the process of encryption. Once encrypted secrets are no longer readable without access to the decryption method also, this gives us added protection in the advent of a security breach.
So it makes sense that you’d be here investigating how to keep your AWS Lambda functions secure using KMS and the concepts of secrets management and encryption. AWS gives us a few managed services that help with secrets management, and the main one we’ll focus on today is KMS.
But what exactly is KMS? And what does it do?
Let’s take a look at that now…
What is KMS?
The easiest way to explain KMS is to look at the two (very broad) ways that we can interact with the service. We can use KMS either to store our own keys, or let AWS handle the keys.
- AWS keys — AWS generates and stores our keys, and we can never access the keys directly.
- Your keys — You import your own encryption keys to be used then with KMS functionality.
One of the main things to keep in mind about KMS is that KMS only stores encryption keys, not the secrets themselves.
Storing encryption keys is pretty much all KMS does. It’s quite a straight-forward service. The real value-add of KMS is how deeply integrated into many other of the AWS services it is. KMS is used for many areas like encrypting S3 buckets, or RDS backups which all work pretty seamlessly.
But when it comes to understanding KMS, one of the main areas you’ll need to master is IAM. Why? Because IAM is used to apply our keys permissions, defining who, what, where and when the keys can be accessed.
For instance we can grant only certain users, teams or applications access to create, and decrypt with KMS keys. By applying such fine-grained permissions we add another layer of security to our application.
Okay, so we’ve learned that KMS is used for storing encryption keys, and that IAM is important when it comes to understanding KMS. Let’s get into the second part of what we came here to discuss today: how KMS integrates with Lambda. Let’s now take a look at that now…
Why Use KMS & Lambda?
The biggest use case for KMS and Lambda is for decrypting environment variables. Environment variables are typically credentials that grant access to databases or other API’s. Environment variables in Lambda are encrypted at rest but if you log into the console you’d still see them in plain text, and we ideally want to encrypt them, like so…
Lambda Environment Variables Encrypted By KMS
In order to use KMS and Lambda together, we need to encrypt values before we store them as environment variables, and then decrypt them at run time of our Lambda. But this approach isn’t necessarily perfect, and there are some downsides that you should be aware of…
KMS & Lambda Limitations
When it comes to using KMS and Lambda, there are some pain points and difficulties that you should be aware of. And they are…
- Secret Rotation Is Hard(er) — Changing secrets in Lambda can be quite painful. If the secret changes and needs re-encrypting that means you’ll need to manually. This can be made especially painful if you have many different keys per environment.
- No Automatic Decryption — The encrypted secrets will also be useless to your application if not decrypted. But, if you want to decrypt secrets with KMS, you’ll have to manually add code into your application, which adds clutter and run-time performance cost.
- KMS Doesn’t Store Secrets — KMS doesn’t store the secrets themselves, so if you’re using the same secret in many places it may end up scattered across your codebases which can make it hard to find and update them later if you need.
But wait, it’s not all doom and gloom! There are ways that we can mitigate some of these limitations when using KMS with AWS Lambda, and that’s by leveraging different AWS services.
That said, for today’s article we’ll keep our focus on KMS. But nevertheless, let’s take at what these other services are, so that you’re at least aware for future!
Other Secret Managing AWS Services (Parameter Store & Secrets Manager)
The two main tools you can use with (or to replace of KMS) are parameter store and secrets manager. Both of these tools allow you to store secrets themselves, which helps to mitigate the issues of key rotation and coupling secrets to your Lambda functions.
But you might be wondering what are the different between the two services? And the answer is: They’re very similar. But they do have some subtle differences. For instance Secrets Manager doesn’t keep a history of changes to your secrets, whereas parameter store does. And Secrets Manager does allow you to do secrets rotation.
If you’re interested, be sure to take a look at these two AWS services as options for helping you manage secrets, if you have the flexibility to do so. But, since we’re talking about KMS today let’s get back on topic and take a look at how KMS and Lambda actually works in code…
Setting up KMS and Lambda
Now that we’ve laid the groundwork of understanding what KMS is let’s go ahead and apply that newly found knowledge by taking a look at the hard details of implementing KMS and Lambda.
Note: In the next sections I’ll show examples of the infrastructure we need using Terraform. But Terraform isn’t mandatory, it’s just an easy way to describe the resources. You can use any infrastructure as code tool you like, or even create resources manually if you’re only experimenting.
1. Create A KMS Key
The first thing you’ll want to do is create an actual KMS key resource.
One thing to note here is that KMS keys are per region. Since we’ll manually decrypt (and therefore explicitly specify the region of the key) the region we create our key in doesn’t matter too much for us. But, when using KMS with other managed AWS services sometimes the region does matter.
In the above example you might also note that we create both a key and an alias. KMS aliases allow you to reference a key indirectly. But what is a KMS alias and why do we need it?
KMS aliases work much like domain names for websites. Rather than accessing a website via IP directly, we access by domain name. By using a domain the website IP address can change without the user knowing. And that’s exactly how KMS aliases work, too. So we’ll want to create one of those, too.
2. Grant Your Lambda KMS Permissions
Now that we’ve got our key setup we need to setup permissions that grant access to it. And we do that by creating an IAM policy.
In this example we’re going to grant access to our Lambda resource to perform only the kms:Decrypt action. As we’re only going to be decrypting using our Lambda. We’re also applying the permission directly to our Lambda, which means no other resources (except admin users) can have key access.
You may notice that I’m referencing the Lambda ARN directly. If you can reference your Lambda ARN directly that will make sense, as it means that if your ARN changes, so does your policy. But if you’re just tinkering, you can simply copy/paste your ARN into the policy.
Once you’ve created your policy, you will also need to go back and update your KMS key to reference the policy, as our policy is currently only a stand alone policy, and it’s not attached to our KMS key. In Terraform that’s done by adding the policy property to your KMS key.
3. Decrypt at Run Time
Ah, and we’re nearly there! We should now have a key that is accessible by our Lambda function, but in order to use it we’ll need to modify our Lambda code and decrypt our strings at run time.
The snippet above is actually the precise way that AWS recommend you use the KMS SDK for decryption. But there’s a few different things going on, so let’s break down what’s going on here…
Since we’re decrypting at run-time that means our decryption might run on every single Lambda request. However, in order to improve performance we can decrypt only once on our first Lambda call, and cache our response. You can see in the above code how it checks if the decrypted key is available, if it is then the code will not decrypt the key again.
One other thing to note is that you do not specify the KMS key when decrypting. This can be confusing as you might think: surely the decrypt function needs to know the key!? But… AWS works out from your encrypted string which key to use, so you don’t need to reference the KMS key name inside your Lambda function.
And on that note you should now have a Lambda running with it’s own KMS key setup, with encrypted secrets and method of decrypting them in a performant way, at run time! Huzzah!
Encrypt All the Things (In Lambda)
Getting up and running with Cloud Native tools like Lambda isn’t easy. It sometimes feels like on every turn there’s another technology, managed service, concept, or principle that we need to understand in order to get going.
And secrets management is one of those areas. That said, you should now have taken another step forward in your understanding of these areas by learning a bit more about how to keep secrets safe in Lambda using KMS.
Speak soon Cloud Native friend!
- 2023 Summary: Data Driven Stories About The Cloud - December 31, 2023
- 2022 Summary: The Open Up The Cloud System - January 1, 2023
- Open Up The Cloud Newsletter #30 (January Recap 2022) - March 1, 2022