Understanding Terraform & state management

This post covers what Terraform is and its benefits over manual processes. It also provides a step-by-step example of deploying and removing a simple AWS S3 bucket using Terraform, along with a brief introduction to Terraform state management.

Understanding Terraform

Terraform simplifies infrastructure management by enabling you to define and manage resources through code, removing the need for manual processes across various cloud providers.

Comparison: Manual vs Terraform

To illustrate, let’s compare manually creating an AWS S3 bucket with using Terraform code:

Creating an AWS S3 Bucket via the AWS Console:

Manually creating an S3 bucket

Now, creating an AWS S3 Bucket using Terraform:

resource "aws_s3_bucket" "bucket" {
  bucket = "my-unique-bucket-name-2024" 
}

Here:

• resource defines the type of infrastructure.
• "aws_s3_bucket" specifies the AWS S3 resource type.
• "bucket" is a local reference within the .tf file to this specific S3 bucket resource instance.

Deploying the AWS S3 resource with Terraform is as simple as running terraform apply (once your project has been successfully initialised).

Benefits of using Terraform

• Simplicity: Terraform simplifies resource management, including IAM roles and permissions, through code. This ensures scalability and consistency, especially crucial as projects scale in complexity.
• Visibility and Control: With Terraform, resources are defined in code, providing greater visibility and control. Cleanup is as easy as removing code, ensuring no lingering or orphaned resources.

Full example: Deploying an AWS S3 bucket using Terraform

Ensure that you have Terraform installed, an AWS account and your credentials securely set up.

File organisation

In Terraform, configuration files are typically written with the .tf extension. When you run Terraform commands, it processes all .tf files within the same directory, allowing you to organise your Terraform configuration into multiple .tf files.

Terraform project directory files

Initial configuration

Create a file named main.tf to hold the main configuration:

# Configure global settings for Terraform
terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 4.18.0"
    }
  }
}

# Configure the AWS provider
provider "aws" {
  region = "eu-west-1"
}

The terraform { } block specifies the required cloud provider, AWS in this case. The required_providers { } indicates the use of the official AWS provider maintained by HashiCorp and the desired version. The provider "aws" { } block configures AWS as the provider for Terraform and the AWS region to work with (e.g., eu-west-1), ensuring Terraform interacts with resources in the designated AWS region.

Defining an S3 bucket

Create a s3.tf file to define the S3 infrastructure:

resource "aws_s3_bucket" "bucket" {
  bucket = "aws-terraform-example-bucket-2024"
}

Initialising your Terraform project and deploying your resource

After setting up the main configuration and defining your S3 resource, run the following Terraform commands:

• terraform init: Initialises the project, downloading provider plugins, and setting up the working directory.

• terraform plan: Generates an execution plan, displaying proposed changes without applying them.

If you encounter an error Error: error configuring Terraform AWS Provider: no valid credential sources for Terraform AWS Provider found, ensure you have configured your AWS credentials with your access key and secret key that allows you to connect to AWS from your local machine.

• terraform apply: Applies the planned changes to the infrastructure, prompting for confirmation before making modifications. Using terraform apply -auto-approve skips the confirmation prompt.

• terraform fmt(optional): Formats the Terraform configuration files by the defined style.

Your AWS S3 resource should now be successfully deployed.

S3 bucket created using Terraform

Deleting Your S3 Bucket

To delete your S3 bucket, comment out the code or delete the s3.tf file and rerun terraform plan and terraform apply.

Understanding Terraform state management

When running the apply command, you’ll see new files created, including the Terraform state and lock file.

Terraform uses a file named terraform.tfstate to track information about the resources it has created. It's a JSON mapping of the resources defined in your configuration and those that already exist in your infrastructure, helping Terraform know what changes to make without redundantly redeploying.

By default, Terraform stores its state file in the local directory where it was run. While this is fine for personal projects, it becomes problematic when multiple developers collaborate on the same Terraform project leading to concurrency and versioning issues.

AWS S3 and DynamoDB for state management

Storing the Terraform state in the cloud is highly recommended, leveraging services like S3 and DynamoDB. S3 securely stores the project’s state file, while DynamoDB ensures exclusive access, preventing concurrent modifications by multiple individuals through state locking. This approach enables versioning, allowing the rollback to a previous state file if it becomes corrupted. Additionally, it provides encryption and enhances collaboration when multiple developers are simultaneously working on the same Terraform project.

Next: Configuring AWS S3 and DynamoDB for Terraform state management.