If you’re new to Terraform, you might have started experimenting creating resources. Before long it’s likely that all your Terraform files are inside one large file, or even many large files.
After a certain amount of time this process will start to break down and become hard to maintain. And that’s where Terraform modules come in.
By the end of this article you’ll understand the basics of Terraform modules, and know how to break down large Terraform files into modules.
What are Terraform Modules?
For those unfamiliar — Terraform is an infrastructure as code tool. With Terraform you can create and manage your infrastructure directly in code. But, just as with any code it needs managing and watching closely so that it doesn’t get out of hand and become painful to work with. If you’re new to Terraform, i’d suggest you first check out the six fundamentals you need to know about Terraform before reading up on modules.
A Terraform module is a way that you can break down a Terraform codebase into easier to understand and manage chunks of logic. A Terraform module can be thought of very much like a programming function, wherever there is repeated logic, there is opportunity to abstract and re-use the logic. Terraform modules are the feature within Terraform that allows you to re-use infrastructure code. If you’re interested on reading the fine details about Terraform modules, the Terraform docs are a good place to start.
Do you need Terraform modules?
A common use case for Terraform modules could be if you have different environments: production and testing, for instance. Rather than copy/pasting your code for both environments, you could put your shared code into a Terraform module and have it re-used easily.
Some other reasons you might want Terraform modules are…
- Managing Complexity — If your terraform files are large and long, modules can help you encapsulate and group resources.
- For Re-use — If you’re duplicating the same code in multiple places, Terraform modules can help you prevent the duplication.
So if you’re into the idea of Terraform modules, let’s take a look.
Your First Terraform Module
In order to showcase how you can create a local Terraform module, we’re going to walk through an example, from scratch.
In this case we’re going to refactor a simple S3 bucket. Why S3? Because it’s a simple and cheap resource to provision!
And if you’re curious, the steps we’ll need to take are:
- Creating an S3 bucket (as a regular Terraform resource)
- Creating a Terraform module
- Moving your resource into a Terraform module
- Updating your Terraform state from the old resource to the new module
If you want to follow along, here’s the repository with setup instructions.
Step 1: Create A Regular Resource
In this case, we’re just going to create a vanilla S3 bucket. If you’re not familiar with S3 buckets they’re highly available AWS resources that allow you to upload, and retrieve files. Kinda like a file system, but a lot more flexible. But, we don’t need to know the specifics for today — just imagine you need one!
If you’re following along, go ahead and copy the following resource.
You’ll likely want to change the bucket name as they have to be globally unique (it’s likely that resource name is already taken!)
Step 2: Run Terraform Plan
Next up, run terraform plan
This command performs a dry-run and Terraform will tell you what would happen were you to apply these changes.
(Hopefully your internet is faster than mine 🙃)
Here Terraform is simply telling us that it would create the bucket. Perfect, that’s what we asked for.
Step 3: Apply The Resource
So now that you’re happy with the plan, let’s go ahead and apply those changes.
Run terraform apply, check that Terraform is only creating your bucket, and type yes to authorise the resource creation.
You’re then going to see that resource created in AWS…
Step 4: Create Your First Module
Right, now we can get started with our module refactor. Let’s imagine we’ve got many of these S3 resources that are all configured the same way (according to some company policy). What we will want to do is consolidate these resource definitions into a single place, so that we can re-use them. Just like a re-usable function.
In order to create a new module, all we have to do is create a directory and move our resource. So let’s create a directory (but in our case, we’re going to add three files). These three files form the Terraform Standard Module Structure, which is simply a recommendation on how to structure your modules. We’ll not cover this at to much length, feel free to read the docs for the reasoning behind the module structure.
Anyway, I digress! Let’s go ahead and make the module and it’s files.
mkdir bucketmodule — To make your module.
cd bucketmodule — Move into the new directory.
touch main.tf — Makes a new resources file.
touch variables.tf — Creates a variables file (we won’t use this today).
touch outputs.tf — Creates an output file (we also won’t use this today).
And that’s your new blank module created, as simple as that!
Step 5: Move your code into your module
Well, not quite as simple as that as we still need to put some code into our module! So let’s go ahead and do that. Copy your existing resource from your top level main.tf file (from step 1) into your new main.tf module file.
If you were to run a terraform plan at this point — Terraform would try and delete your S3 bucket. Which is not what we want. Why is Terraform trying to delete our resource? Because we’ve moved the code out to the module, but Terraform doesn’t know that our module exists yet, we need to reference it.
Let’s fix that.
Step 6: Reference your module
To reference your newly created module, add the following code to your top level main.tf. Here we’re creating a new module (which will create all the resources encapsulated inside it) and we’ve told Terraform the location of the module, in our case it’s the path to our created directory.
Then, let’s run terraform init to instantiate the module. We instantiate our module
Step 7: Run a Terraform Plan
Now when you run terraform plan, you’ll see that Terraform is trying to delete your old resource, to create a new one.
That’s because Terraform thinks the module is a new resource, Terraform doesn’t know that the old one and the new module resource are related.
In order to clean this up we need to tell Terraform that our new module is the same as our original resource, and we can do that with the terraform state mv command.
Step 8: Link Old State To A New Resource
When we’ve got the output of our Terraform plan, we should see the new resource, which is identified by the the module.resource_type.unique_name and the previous resource, which is shown as resource_type.unique_name so all we need to do is tell Terraform that these are the same. Terraform state will update to link the new module resource with the old non-module resource state.
Step 9: Check Terraform Plan
And lastly we run terraform plan one last time to see that it reports no changes.
Voila! That’s your Terraform moved to a module, and the state migrated.
Don’t forget: Before you run off in jubilation to refactor all of your Terraform, do remember to run terraform destroy if you don’t want the old resources otherwise you’ll get charged.
Going Further With Terraform Modules
And that’s all there is to it. In order to move a resource into a terraform module, you simply need a new file, move the resource and copy over the state from your old resource to your new module one — that really is all there is to it.
If you want to take modules farther, you can pass in values, like function arguments. Or even store modules remotely which you can then pull in from different code bases.
But now you’ll have a solid basis to take on these more advanced module features. I hope that helped make the idea of Terraform modules a little more concrete, and removed some of the fog!
Speak soon, Cloud Native friend!
- 2023 Summary: Data Driven Stories About The Cloud - December 31, 2023
- 2022 Summary: The Open Up The Cloud System - January 1, 2023
- Open Up The Cloud Newsletter #30 (January Recap 2022) - March 1, 2022