3 Steps To Migrate Existing Infrastructure To Terraform

When everyone keeps talking about Infrastructure As Code you might feel stuck and frustrated because a lot of your cloud infrastructure was created manually. Infrastructure As Code feels like a million miles away for you…

With Terraform it’s possible to bring existing infrastructure under code management in a safe, and incremental way. And today we’re going to go through the three steps you’ll need to take if you want to apply Terraform Infrastructure As Code to your existing infrastructure.

By the end of this article you’ll understand the 3 steps to get started with Terraform Infrastructure As Code on existing infrastructure.

Before we jump in, if you’re not familiar with Infrastructure As Code, check out this article first: Infrastructure As Code: A Quick And Simple Explanation. Or if you want a birds eye view of Terraform first, check out: Learn The 6 Fundamentals Of Terraform — In Less Than 20 Minutes.

Terraform in under 200 words

Before we get to the step for migrating manually created infrastructure, it’s important we’re on the same page about Terraform and what it is. Because Terraform is the tool we’ll use today to bring our existing infrastructure into management by code.

Terraform is an open source declarative Infrastructure As Code tool. It’s a third party tool, so it operates on all cloud providers: AWS, GCP and a whole host of other cloud tools. Terraform is distributed as a CLI, so you simply install it on your machine or a remote build server in order to use it. And lastly, Terraform operates with a concept of state. State is your representation of the cloud resources you want to manage with Terraform.

Because Terraform can be installed on any machine, I’m pretty sure you can use it. And because it works on all cloud providers — even your own data centers — there simply should not be many reasons why you can’t implement Terraform. And that’s why I’ve chosen it as the tool to help you refactor your existing infrastructure.

Okay, and with that out of the way, let’s get to it! Here are the three steps…

Step 1: Begin To Understand Terraform

Terraform Up & Running Book

Firstly you’ll need to clear the mist of uncertainty. We’ll need to know what Terraform is, and how it’s capabilities help us manage our existing manually created infrastructure. At this point we’re trying to move from “I don’t know what I don’t know” to “I know what I don’t know”. We’re not trying to be an expert, but we should first get a lay-of-the-land by familiarising ourselves with the concepts of Terraform.

At a bare minimum, I’d suggest that you become familiar with: state, the language of Terraform (HCL), how Terraform manages resource dependencies, the plan and apply commands, file structure and providers. If you’re keen to jump into this straight away, I have already written these concepts up in the article: Learn The 6 Fundamentals Of Terraform — In Less Than 20 Minutes.

But whilst the article gives you a birds-eye view, there’s only so much I can teach you in a single article. For a more comprehensive view I’d suggest you grab yourself a copy of Terraform Up & Running. The book was written by Yevgeniy Brikman, founder of Gruntwork and it’s a genuinely very good walk through of all the Terraform concepts. You don’t need to read it cover to cover, but at least start to familiarise yourself with the terms and concepts.

Note: If you’re into books, learning and all that jazz, be sure to check out: The Big List Of Cloud Native Engineering Resources.

And once you’ve started to wrap your head around those topics you’ll be in a great place to move to step two…

Step 2: Practice Terraform On Greenfield

Planning a Terraform resource

“Give me six hours to chop down a tree and I will spend the first four sharpening the ax.” — Abraham Lincoln

I know at this point it’ll be really, really tempting to simply go ahead and apply what you learned in step one straight away and try to get it working with your existing infrastructure. But, I do recommend at this point you apply a little more patience, and instead start practicing on greenfield infrastructure.

Why? Because learning most technology can be tricky, and it’s important that you start to understand the nuances, including how to un-pick any situations you get yourself into. That’s why I suggest you start by building out on greenfield first, just to get a feel for how Terraform works.

To do this, pick some infrastructure that you already have manually configured, and start trying to provision new resources using Terraform. But I’m not simply suggesting you have a play, you’ll want to pay specific attention to the following areas, as they’re what’s important when it comes to refactoring your existing infrastructure later.

Understand: How, When & Why Of State

State is how Terraform knows what resources you want to manage. Terraform will convert code into state when you apply your changes. The concept of state is the most important area you’ll need to know for refactoring your existing infrastructure. So take note of when Terraform alters state and why.

Understand: The Terraform Syntax

As you’re building out your project, pay attention to the Terraform syntax. Pay special attention to how the resource blocks are configured, and try to notice the similarities in how different resources are configured. Start to imagine how the syntax will apply to your infrastructure.

Understand: Resource Dependencies

And the last big thing is: you’ll want to get comfortable with is how Terraform handles dependency resolution. Try to create at least a few resources that link to each another. For instance, create a load balancer that references a server, or a domain name that references an asset. Again, start to imagine the dependencies your current manually created infrastructure has.

Generally speaking, at this point you want to make some mistakes. You’ll want to push boundaries and see what the repercussions are. Be curious. Try, for instance to cancel a Terraform apply mid way through and see what happens to your state file. You want to be building confidence that you can get yourself out of sticky situations. Which will be important when we come onto applying these ideas to your existing infrastructure.

And by the end of step two you should understand the main concepts and be pretty confident with using Terraform to create basic infrastructure. Now it’s time to move onto the part that you’ve been waiting for, and that’s bringing your existing manually created infrastructure under management with Terraform.

Step 3: Import Terraform State

Terraform Docs For Importing An S3 Resource

Terraform manages infrastructure through state. If resources are recorded in state, Terraform assumes you want to manage them, if they’re not included in state Terraform assumes you don’t care about them. When you’re starting out you won’t have any resources managed under Terraform state. So the first step with migrating infrastructure into Terraform is to “import” them.

How To Import Terraform State

If you head to the Terraform docs for your favourite resource (honestly, at this point I feel me writing about creating S3 buckets will be a meme…). You’ll see that at the bottom of the document it shows you how to import that specific resource.

Importing is simply a command that is made to your resource, to fetch down the resource metadata and store it in state. The state that would have been created if you’d provisioned with Terraform in the first place. All you need is the resource name, and it’s unique identifier. The identifier will change though dependent on what cloud you’re operating on, for instance.

The Two Requirements To Import State

But wait, there is something I forgot to tell you. Importing state requires two things before you get started (the keen eyed among you might have spotted the two arguments in the above command image). And those two things are:

A resource block — A definition of the resource you’re importing
State — The state of the resource

Before you can begin importing state, you must create a resource block.

Why? Because your resource block tells terraform which piece of your code to assign to that state. Imagine you have two servers in Terraform, it won’t know which server is the one for which you’re importing state, so you’ll have to tell the import command explicitly.

For instance, if you were importing a static S3 bucket, called mywebsite you’d need to have your terraform config setup with a resource corresponding to the name mywebsite which you’re importing. Having your named resource allows Terraform knows to assign the imported state to that resource.

Tip: Don’t worry about writing the resource and it’s attributes perfectly. Just create the resource block. For instance, if you’re importing a server, just add a server resource block. Don’t bother configuring size, tags, etc. Because you can run terraform plan after importing your state, which will tell you the differences. And you can simply copy any differences from the output into your code. Trying to guess the attributes up front is hard, time consuming and inefficient.

The Process Of Importing State

I appreciate that might be a lot of new concepts to digest. So in summary the process of importing state looks like this…

Write a rough resource configuration block for the resource (in code)
Import your resource with the import terraform import command
Use terraform plan to check the differences between your code and state
Make changes to align your resource attributes with the existing resource

Tip: This little tool, called Terraforming lists out all your resource configuration blocks, which can save you time!

Terraform All The Infrastructure!

And that’s it, those are your three steps you for moving your existing infrastructure under management by Terraform. You’ll start by understanding Terraform, then using Terraform on greenfield before finally using Terraform to import state. Follow these three steps and you’ll be up and running with Terraform in no time.

I hope that helped put you on the right track and gets you on the road to implementing Infrastructure As Code. When you get the right setup it becomes far easier to manage your infrastructure this way. As soon as you start, you’ll wish you did it sooner — I bet you.

If you’re keen to know more about Infrastructure As Code, be sure to check out the Infrastructure As Code Ultimate Guide, and also check out the free email course, Terraform Kick Start.

Speak soon Cloud Native friend!

Author
Recent Posts

Lou Bichard

Hey I'm Lou! I'm a Cloud Software Engineer From London. I created Open Up The Cloud to help people get their start and grow their careers in cloud. I edit the monthly Open Up The Cloud Newsletter which I think you'd really like. You can find me on Twitter or LinkedIn , I'm always happy to chat. When I'm not writing you can usually find me picking up heavy things and putting them down, or riding two wheels (sometimes with an engine, sometimes not).