Create Terraform pre-requisites for AWS using AWS CLI in 3 easy steps

Generally speaking, when we work with Terraform to provision resources in AWS Cloud, we have a few pre-requisites. These are  -a remote backend to store the Terraform state file, a lock table, and IAM user credentials that Terraform will require to provision the resources. I say “generally speaking” because you can get away with the remote backend bit by pursuing the default (local) option. However, that is not recommended from a collaboration, automation, and security standpoint. Since this setting-up-of-the-prerequisites is a one-time activity, there is no need to automate that bit beyond a point. Yes, that is true that in an organization, over some time, there will be active multiple Terraform configuration projects. And in such a situation, having an automated process to configure the pre-requisites is a step in the right direction. Using the AWS CLI to create a backend -S3 bucket and DynamoDB table and an IAM user should be reasonably easy from a repeatability and consistency objective.

Step 1: Install AWS CLI on your local and log-in.
Download the installer and install it on your local. Once done, open a new windows command prompt and run “aws configure”. If this is a new setup, you will be requested an access key, secret access, etc. There is detailed information on that at AWS CLI Configuration basics docs.

Step 2(a): Create an S3 bucket
The Terraform configuration uses an S3 bucket to store the remote terraform.tfstate file. There are two steps to this process – (a) create an S3 bucket and (b) encrypt the bucket.
On the AWS authenticated command prompt, key in the following statement, where “skundu-terraform-remote-state-two” is the bucket’s name. On successful completion, the below statement in {} appears. Please note that the bucket name has to be globally unique. More info at aws-cli-create-bucket.

# bucket name: skundu-terraform-remote-state-two
aws s3api create-bucket –bucket skundu-terraform-remote-state-two –region us-east-2 –create-bucket-configuration LocationConstraint=us-east-2
#output
{
    "Location": "http://skundu-terraform-remote-state-two.s3.amazonaws.com/"
}

After creating the bucket, I added server-side encryption using the information available at aws-cli-put-bucket-encryption.

# bucket name: skundu-terraform-remote-state-two
aws s3api put-bucket-encryption –bucket skundu-terraform-remote-state-two –server-side-encryption-configuration "{\"Rules\": [{\"ApplyServerSideEncryptionByDefault\":{\"SSEAlgorithm\": \"AES256\"}}]}"
# no output if bucket encryption is successfully applied

Note: If you struggled as I did with error messages then check out possible issues at Stack-Overflow.

Step 2(b): Create a DynamoDB Table
A Terraform configuration uses the DynamoDB table to store the lock to prevent concurrent access to the terraform.tfstate file.
I followed the steps listed here to create the command to use.

# table name: Terraform-backend-lock
aws dynamodb create-table –table-name Terraform-backend-lock –attribute-definitions AttributeName=LockID,AttributeType=S –key-schema AttributeName=LockID,KeyType=HASH –provisioned-throughput ReadCapacityUnits=5,WriteCapacityUnits=5
#output
{
"TableDescription": {
"AttributeDefinitions": [
{
"AttributeName": "LockID",
"AttributeType": "S"
}
],
"TableName": "Terraform-backend-lock",
"KeySchema": [
{
"AttributeName": "LockID",
"KeyType": "HASH"
}
],
"TableStatus": "CREATING",
"CreationDateTime": "2021-04-01T06:51:56.076000-05:00",
"ProvisionedThroughput": {
"NumberOfDecreasesToday": 0,
"ReadCapacityUnits": 5,
"WriteCapacityUnits": 5
},
"TableSizeBytes": 0,
"ItemCount": 0,
"TableArn": "arn:aws:dynamodb:us-east-2:$(AWSAccountNumber):table/Terraform-backend-lock",
"TableId": "9f2312ba-01ae-4f9a-9ea2-b179ffa780b2"
}
}

Step 3: Create an IAM user with the required permissions
Creating an IAM user for Terraform configuration is a multipart process that comprises of the following:
Create a policy from a JSON file.
Create a user.
Create an access key associated with the user.
Attach the policy to the user.
All these steps are listed below.
Here is an example policy file to provide restricted access to Terraform to only communicate with the S3 and DynamoDB table.

{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"dynamodb:PutItem",
"dynamodb:DeleteItem",
"dynamodb:GetItem"
],
"Resource": "arn:aws:dynamodb:$(Region):$(AWSAccountNumber):table/$(BackendLockTableName)"
},
{
"Effect": "Allow",
"Action": "s3:ListBucket",
"Resource": "arn:aws:s3:::$(BackendBucketName)"
},
{
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:PutObject"
],
"Resource": "arn:aws:s3:::$(BackendBucketName)/$(PathToTFStateFile)"
}
]
}

view raw
backend-policy.json
hosted with ❤ by GitHub

I store the same in a backend-role-policy.json file and use it in the step below

# The code for the backend-role-policy.json is available at https://gist.github.com/kunduso/bf94f1aa5e683ed66539458a9a44138d
# create a policy with name "Custom-Terraform-Policy-Backend-April"
# https://docs.aws.amazon.com/cli/latest/reference/iam/create-policy.html
aws iam create-policy –policy-name Custom-Terraform-Policy-Backend-April –policy-document file://backend-role-policy.json
#output
{
"Policy": {
"PolicyName": "Custom-Terraform-Policy-Backend-April",
"PolicyId": "ANPAZIAA3LP6OBWQHE5E6",
"Arn": "arn:aws:iam::$(AWSAccountNumber):policy/Custom-Terraform-Policy-Backend-April",
"Path": "/",
"DefaultVersionId": "v1",
"AttachmentCount": 0,
"PermissionsBoundaryUsageCount": 0,
"IsAttachable": true,
"CreateDate": "2021-04-01T18:45:25+00:00",
"UpdateDate": "2021-04-01T18:45:25+00:00"
}
}
# Create a user with name Terraform-User
# https://docs.aws.amazon.com/cli/latest/reference/iam/create-user.html
aws iam create-user –user-name Terraform-User
#output
{
"User": {
"Path": "/",
"UserName": "Terraform-User",
"UserId": "AIDAZIAA3LP6FKZZCFBD5",
"Arn": "arn:aws:iam::$(AWSAccountNumber):user/Terraform-User",
"CreateDate": "2021-04-01T18:54:27+00:00"
}
}
# Create access key for the user with name Terraform-User
# https://docs.aws.amazon.com/cli/latest/userguide/cli-services-iam-create-creds.html
aws iam create-access-key –user-name Terraform-User
#output
{
"AccessKey": {
"UserName": "Terraform-User",
"AccessKeyId": "AKIAZIAA3LP6LVULLJX7",
"Status": "Active",
"SecretAccessKey": "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",
"CreateDate": "2021-04-01T18:56:47+00:00"
}
}
# Attach a policy with policy ARN to a user with name Terraform-User
# https://docs.aws.amazon.com/cli/latest/reference/iam/attach-user-policy.html
aws iam attach-user-policy –policy-arn arn:aws:iam::$(AWSAccountNumber):policy/Custom-Terraform-Policy-Backend-April –user-name Terraform-User
# No output if user successfully attached to a policy

After executing the above steps, we will have access to a few required variables while working with Terraform configurations. These are the S3 bucket name and location, the DynamoDB table name, and the IAM user’s access-key and secret-access.
These values will be referred to in the backend.tf file and while executing “terraform init”, “terraform plan”, and “terraform destroy” steps.
Below is a descriptive example of a backend.tf configuration

terraform {
backend "s3" {
bucket = "$(BackendBucketName)" # the name of the S3 bucket that was created
encrypt = true
key = "$(PathToTFStateFile)" # the path to the terraform.tfstate file stored inside the bucket
region = "$(BucketRegion)" # the location of the bucket
dynamodb_table = "$(BackendLockTableName)" # the name of the table to store the lock
}
}

view raw
backend.tf
hosted with ❤ by GitHub

As you can see in the code above, I replaced the values of $(BackendBucketName) with the name of the S3 bucket, $(BucketRegion) with the region where the bucket’s region, and $(BackendLockTableName) with the DynamoDB table’s name. The value in $(PathToTFStateFile) is relative to the s3 bucket. E.g., if we prefer to store the terraform.tfstate file at the root of the bucket, the value of $(PathToTFStateFile) will be “terraform.tfstate”.

Here is an example of the IAM access key used in “terraform init”

powershell: |
terraform init -backend-config="bucket=skundu-terraform-remote-state" -backend-config="key=tf/ADO-TF-VPC-Int/terraform.tfstate" -backend-config="region=$(region)" -backend-config="access_key=$(access_key)" -backend-config="secret_key=$(secret_key)" -no-color
workingDirectory: $(build.sourcesdirectory)
displayName: 'terraform init'

view raw
azure-pipelines.yaml
hosted with ❤ by GitHub

Note: The above Terraform command belongs to an azure-pipelines.yaml file. I am using Powershell to run them.

And that brings us to the end of creating AWS resources referred to in a Terraform project. I hope you found this post useful. Please let me know if you have any questions or suggestions. Until then, happy terraforming!

Note Two: In the example above, the IAM user has access only to the resources required to manage the backend. The IAM user’s access needs to be set accordingly depending upon what resources need to be provisioned in a Terraform configuration. E.g., suppose the Terraform configuration is to create a VPC, subnet, route table, etc… In that case, the IAM user will require additional permissions to manage those resources.

Note Three: make sure that the resources do not exist already in your AWS account. Also, log out of the AWS console, once the resources are created.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s