Any Terraform project configuring resources in Azure has pre-requisites. These are (i) a storage account, a container in the storage account, and the access key to the storage account, and (ii) a service principal credential to be able to communicate with Azure to create-update-delete resources. In this post, I describe the process to set up Azure cloud resources using Azure CLI that is required before using/referring them in Terraform configuration.
Step 1: Install Azure CLI on your local laptop and log in.
Azure CLI docs: The Azure command-line interface (Azure CLI) is a set of commands used to create and manage Azure resources. The Azure CLI is available across Azure services and is designed to get you working quickly with Azure, with an emphasis on automation. More info at Azure CLI.
Once installed, open the “Windows Azure SDK Environment” command prompt and key in “az login”. That opens up a web browser requesting you to log in, and once done, the command prompt displays the subscription details.
Step 2: Create a storage account and storage container.
Before we create a storage account, we create a resource group to host that. I have the commands that need to be run to create a resource group, storage account, and storage container.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| az group create –name Terraform-Remote-State-Group –location "East US" | |
| # the above command creates a resource group and displays the result in below format | |
| { | |
| "id": "/subscriptions/$(SubscriptionNumber)/resourceGroups/Terraform-Remote-State-Group", | |
| "location": "eastus", | |
| "managedBy": null, | |
| "name": "Terraform-Remote-State-Group", | |
| "properties": { | |
| "provisioningState": "Succeeded" | |
| }, | |
| "tags": null, | |
| "type": "Microsoft.Resources/resourceGroups" | |
| } | |
| # Storage account name must be between 3 and 24 characters in length and use numbers and lower-case letters only. | |
| # The below command creates a storage account with the name "terraformremoteskundu". The storage account name has to be globally unique | |
| # else you get the message (StorageAccountAlreadyTaken) The storage account named terraform is already taken. | |
| az storage account create –resource-group "Terraform-Remote-State-Group" –name "terraformremoteskundu" –sku Standard_LRS –encryption-services blob | |
| # Once the storage account is created, list the access key with the below command. The output of the command is the access key | |
| az storage account keys list –resource-group "Terraform-Remote-State-Group" –account-name "terraformremoteskundu" –query [0].value -o tsv | |
| # The below command creates a storage container with the name "tfstate" | |
| az storage container create –name "tfstate" –account-name "terraformremoteskundu" –account-key "$(copy_access_key_here)" |
Note: Store the access key to the storage account securely. That key is required when configuring the Azure backend to use the Terraform remote state.
Step 3: Create a service principal with required permissions
There is a single line command to create a service principal that will be sufficient to provision resources in Azure. Below is the command to do so from the Azure CLI.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # az ad sp create-for-rbac –name "$(Service-Principal-Name)" –role "Contributor" –scope "/subscriptions/$(SubscriptionNumber)" | |
| az ad sp create-for-rbac –name "Terraform-User-March-2021" –role "Contributor" –scope "/subscriptions/$(SubscriptionID)" | |
| # I am replacing the tenant and subscription value with variable for security reasons | |
| # Output from the commandline console: | |
| Changing "Terraform-User-March-2021" to a valid URI of "http://Terraform-User-March-2021", which is the required format used for service principal names | |
| Creating 'Contributor' role assignment under scope '/subscriptions/$(SubscriptionID)' | |
| The output includes credentials that you must protect. Be sure that you do not include these credentials in your code or check the credentials into your source control. For more information, see https://aka.ms/azadsp-cli | |
| { | |
| "appId": "3a08aff0-9708-455b-855c-1747fcf7434d", | |
| "displayName": "Terraform-User-March-2021", | |
| "name": "http://Terraform-User-March-2021", | |
| "password": "OJ-6-uMZt4a1v~ZjLl3EdzogUnkzn9GZga", | |
| "tenant": "$(SubscriptionTenantID)" | |
| } |
That command attaches a “contributor” role to the service principal, and the output is presented below that. As per the console output, these are credentials that you must protect. These credentials will be used by Terraform (in Terraform plan and destroy steps) to communicate with the Azure cloud to provision resources.
That is all that is required to set up before using Terraform. On the Terraform project side, we refer to these pre-requisites in the “terraform init”, “terraform plan”, and if applicable “terraform destroy” commands. I say “if applicable” because we do not “generally speaking” destroy all the resources; we alter them via the Terraform configuration as the project matures and evolves.
Below is an example of a Terraform backend configuration in Azure.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| terraform { | |
| backend "azurerm" { | |
| resource_group_name = "$(ResourceGroupToStoreTerraformBackendResources)" | |
| storage_account_name = "$(UniqueStorageAccountName)" | |
| container_name = "$(StorageContainerName)" | |
| key = "terraform.tfstate" | |
| access_key = "$(StorageAccountAccessKey)" | |
| } | |
| } |
Below is the “terraform init” command required to set up a remote backend in Azure.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| – powershell: | | |
| terraform init -backend-config="access_key=$(storage_access_key)" -no-color | |
| workingDirectory: $(build.sourcesdirectory) | |
| displayName: 'terraform init' | |
| # where $(storage_access_key) is a secret variable stored in the Azure pipeline | |
| # variable or a Library variable group. The value of the variable is the storage | |
| # access key to the storage account where the terraform remote state file is stored |
Below is the “terraform plan” command with the secured credentials being passed via the command line.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| – powershell: | | |
| terraform plan -var client_id="$(client_id)" -var client_secret="$(client_secret)" -var subscription_id="$(subscription_id)" -var tenant_id="$(tenant_id)" -out application.tfplan -no-color | |
| workingDirectory: $(build.sourcesdirectory) | |
| displayName: 'terraform plan' | |
| # where $(client_id) is the appID of the service principal, | |
| # $(client_secret) is the password of the service princial, | |
| # $(subscription_id) is the Azure subscription to which the service principal belongs, and | |
| # $(tenant_id) is the tenant value of the service principal. | |
| # These values can be stored in Azure pipeline variables or as a variable group in the Azure DevOps project. |
As an organization continues to use Terraform to automate the environment provisioning process, there is a tendency to reuse these resources like the storage account and the service principal. And, right at the beginning, every possible attempt should be made to keep the pre-requisites of a Terraform configuration project as isolated as possible.
I am not suggesting that we keep resources isolated just for the sake of doing so. If we are careful, we can manage the reuse of these resources. We can use the same resource group to host all the storage accounts belonging to different Terraform configuration projects. We can even reuse the same storage account to host the state file of various Terraform configuration projects. However, the container inside the storage account must be different for different Terraform configuration projects. The service principal “generally speaking” should also be unique per Terraform configuration project. I’ll write a detailed note later on that.
And that brings us to the end of how to create Azure resources referred to in a Terraform project. I hope you found this post useful. Please let me know if you have any questions or suggestions. Until then, happy terraforming!
Note: The above Terraform commands are steps extracted out of an azure-pipelines.yaml file. I am using Powershell to run them.
Another Note: Please end the cloud session once the storage and service principal is created with the command “az logout”.
My Devops Journal 