Automating infrastructure provisioning with Terraform in Azure DevOps CI/CD: a comprehensive tutorial

Overview

The automation of resource provisioning within cloud infrastructure contributes a lot to the consistency, scalability, and speed of any Azure DevOps project. Terraform is one of the most used open-source IaC tools to define, manage, and provision a cloud environment across several providers.

Together with Azure DevOps - a powerful continuous integration / continuous deployment platform - Terraform can further drive automation of your infrastructure and, as a result, your entire development workflow.

Why use Terraform in Azure

Terraform.io simplifies the task of provisioning and managing infrastructure in Azure, primarily because it is treated as a codebase. Virtual machines, storage accounts, and even network resources — all can be managed with a simple, understandable syntax of Terraform. Since infrastructure changes are made declaratively, deployments become repeatable, consistent, and auditable.

• Infrastructure as a Code: The infrastructure is described in human-readable configuration files, which makes version control and collaboration easy.
• Multi-cloud support: While in this tutorial we focus on Terraform Azure, this software can also provision resources across other cloud providers, such as AWS, GCP, and more.
• State management: It maintains a state file to track the resources it manages in order to make sure the infrastructure is consistent.

Prerequisites

1. Terraform software must be installed on your local machine. Note that it is compatible with major operating systems such as macOS, Windows, and Linux.
2. You must have a valid Azure subscription with permission to create resources. A free trial or pay-as-you-go subscription will work, too.
3. A project in Azure DevOps must be set up to manage your Azure CI/CD pipelines and source code.
4. Since we’ll be using version control, ensure that Git is installed on your local machine as well.

Step 1: setting up Terraform for Azure

First, the process of automating infrastructure provisioning starts with configuring Terraform to communicate with your Azure account. To do this, you need to create a service principal on the platform for Terraform to authenticate and manage resources over there:

1. Sign in to the portal.
2. Navigate to Azure Active Directory > App registrations > New registration.
3. Then, set the name and redirect URI (you can also leave the redirect URI empty).
4. After registering, copy the Application (Client) ID, Directory (Tenant) ID, and Client Secret (found under Certificates & Secrets).
5. Assign Contributor or Owner permissions to this service principal for the subscription or resource group you want to manage.

Once you have the service principal credentials, configure Terraform to authenticate using them. Create a file named provider.tf in your Terraform project directory and add the following configuration:

hcl
Copy
1provider "azurerm" {
2  tenant_id       = "YOUR_TENANT_ID"
3  client_id       = "YOUR_CLIENT_ID"
4  client_secret   = "YOUR_CLIENT_SECRET"
5  subscription_id = "YOUR_SUBSCRIPTION_ID"
6}

Step 2: writing Terraform configuration files

Once authentication is configured, let’s define the infrastructure we want to provision. In the example below, we will create a resource group and a virtual machine on Azure. For this, create a main.tf file to define your resources.

• It will create a resource group, virtual network, and subnet in Azure.
• Feel free to modify the configuration to include other resources, such as VMs, databases, and storage accounts.

hcl
Copy
1resource "azurerm_resource_group" "example" {
2  name     = "example-resources"
3  location = "East US"
4}
5
6resource "azurerm_virtual_network" "example" {
7  name                = "example-vnet"
8  location            = azurerm_resource_group.example.location
9  resource_group_name = azurerm_resource_group.example.name
10  address_space       = ["10.0.0.0/16"]
11}
12
13resource "azurerm_subnet" "example" {
14  name                 = "example-subnet"
15  resource_group_name  = azurerm_resource_group.example.name
16  virtual_network_name = azurerm_virtual_network.example.name
17  address_prefix       = "10.0.1.0/24"
18}

Step 3: integrating Terraform with Azure DevOps

Next, integrate your configuration into a DevOps pipeline.

1. Create a new repository in your Azure DevOps project.
2. Clone the repository to your local machine and add your configuration files (main.tf, provider.tf, etc.).
3. Push the files to Azure DevOps.

1. To add a new pipeline, navigate to Pipelines in Azure DevOps.
2. Select the repository you just pushed the Terraform files to as the source.

Choose the YAML pipeline template.

Step 4: configuring the CI/CD pipeline

The core of automating infrastructure provisioning is the configuration of a pipeline that will run the Terraform commands automatically.

• Terraform installation
• Initialization of its configuration
• Running terraform plan (to preview the changes)
• Running terraform apply (to apply the changes to the infrastructure)

To create an azure-pipeline.yml file:

Create an azure-pipeline.yml file in the root of your repository with the following content:

yaml
Copy
1trigger:
2  branches:
3    include:
4      - main
5
6pool:
7  vmImage: 'ubuntu-latest'
8
9variables:
10  TF_VERSION: '1.3.5'
11  AZURE_SUBSCRIPTION: 'YOUR_AZURE_SUBSCRIPTION_ID'
12
13stages:
14  - stage: Terraform_Apply
15    jobs:
16      - job: Apply_Terraform
17        steps:
18          - task: UseTerraform@0
19            inputs:
20              version: $(TF_VERSION)
21          - script: |
22              terraform init
23              terraform plan
24              terraform apply -auto-approve
25            displayName: 'Terraform Apply'

• trigger - Whenever changes are pushed to the main branch, the pipeline is triggered.
• pool - Virtual machine image that will be used to run the pipeline.
• variables - Key variables, such as the Terraform version and Azure subscription.
• steps - Steps to initialize the project, such as initializing the working directory, creating an execution plan, and applying the configuration.

Now you need to set up a service connection to Azure, which will later be used by the pipeline to authenticate with your Azure account:

1. First, navigate to Project Settings > Service Connections.
2. Then add a new Azure Resource Manager connection.

Step 5: running and managing the pipeline

With the pipeline now set up, it will automatically run and provision the infrastructure defined in your Terraform configuration every time changes are pushed to the repository.

• The working directory is initialized.
• An execution plan for the infrastructure changes is displayed.
• The necessary changes are applied to the infrastructure.

Step 6: monitoring and troubleshooting

While Terraform and Azure DevOps do most of the automation, it is always a good practice to monitor the deployment process itself. If there are any issues with the configuration or pipeline, you can check the pipeline logs for what exactly went wrong.

In addition, you can log in to the Azure portal to confirm that the resources were successfully provisioned.

Conclusion

Using the Terraform tool for infrastructure provisioning in Azure DevOps will give you smoother cloud resource management, reduced chances of human error, as well as consistency and repeatability of your environments. Terraform YAML files and DevOps pipelines enable you to fully leverage Terraform Azure DevOps integration which will automate both the infrastructure and application deployment.

This tutorial provides a good foundation for implementing a scalable and repeatable continuous integration/continuous deployment pipeline for a DevOps project or automating the provisioning of Azure resources.