DevOps Shack

TERRAFORM COMPLETE NOTES

Batch-7 | DevSecOps & Cloud DevOps Bootcamp Starts 2nd November

1. Introduction to Terraform

What is Terraform?

Terraform, developed by HashiCorp, is an open-source Infrastructure as Code (IaC) tool that allows
users to define, provision, and manage infrastructure resources through configuration files. These
resources can be physical or virtual components like servers, databases, networking components,
storage, or even Kubernetes clusters. The configurations are written in a declarative language known
as HashiCorp Configuration Language (HCL), which is both easy to learn and highly expressive.

The beauty of Terraform lies in its ability to abstract infrastructure management into a simplified,
declarative model. Instead of manually provisioning resources through cloud provider dashboards, or
writing scripts to automate them, you can simply describe what your infrastructure should look like,
and Terraform handles the rest.

Terraform is widely used because it can manage infrastructure across various cloud providers such as
AWS, Azure, Google Cloud, and even on-premises environments using providers like VMware and
OpenStack. With Terraform, you can manage all your infrastructure in one place, regardless of the
provider.

Infrastructure as Code (IaC)

To better understand Terraform, it’s important to grasp the concept of Infrastructure as Code (IaC).
IaC allows you to automate the provisioning and management of infrastructure in a declarative or
imperative manner using code. Instead of configuring each resource manually (via cloud consoles or
command-line interfaces), you describe the desired state of your infrastructure, and the IaC tool (like
Terraform) will provision it for you.

Here are some key benefits of IaC:

• Consistency: Ensures the same infrastructure setup across multiple environments

(development, staging, production).

• Version Control: Infrastructure configurations can be stored in version control systems like
Git, making it easy to track changes over time, roll back if necessary, and collaborate with
other team members.

• Automation: Infrastructure can be deployed or modified in seconds or minutes, reducing the

risk of human error.
DevOps Shack
How Terraform Works

Terraform operates using providers to interact with infrastructure APIs (like AWS or Azure APIs).
Here's a simplified view of how Terraform works:

1. Configuration Files: You define the desired state of your infrastructure in .tf files using HCL.

2. Terraform Plan: Terraform compares your desired configuration with the current
infrastructure state to generate a plan of action.

3. Terraform Apply: Terraform applies the changes defined in the plan by interacting with cloud
provider APIs to provision, update, or destroy resources.

4. State File: Terraform tracks the resources it manages through a state file (terraform.tfstate).
This state file acts as a record of what Terraform has done, allowing it to manage future
changes incrementally.

2. Why Use Terraform?

a. Multi-Cloud Support

One of the standout features of Terraform is its support for multiple cloud providers. Unlike many
IaC tools that are locked to a single provider (e.g., AWS CloudFormation for AWS), Terraform allows
you to manage infrastructure across multiple cloud environments. This is particularly useful for
organizations adopting multi-cloud or hybrid cloud strategies.

For example, you can manage AWS EC2 instances for your core infrastructure, Google Cloud Storage
for your data storage, and Azure Kubernetes Services (AKS) for your microservices deployment – all
within a single Terraform configuration. This flexibility enables teams to avoid vendor lock-in and
utilize the best features from each cloud provider.

Here’s an example of a simple multi-cloud setup with Terraform:

provider "aws" {
region = "us-east-1"
}

provider "google" { project =

"my-google-project"
region = "us-central1"
}

resource "aws_instance" "my_aws_instance" {

ami = "ami-0c55b159cbfafe1f0"
instance_type = "t2.micro"
}

resource "google_storage_bucket" "my_bucket" {

name = "my-storage-bucket"
location = "US"
}
DevOps Shack
b. Declarative Syntax

Terraform uses a declarative model for defining infrastructure. This means you describe what you
want your infrastructure to look like, and Terraform handles how to achieve that state. For example,
rather than manually specifying every step to create an EC2 instance, you simply define the end
result, and Terraform will handle the underlying logic.

This contrasts with imperative approaches, where you have to specify step-by-step instructions for
every task. With Terraform's declarative nature, your configuration files are easier to understand,
more concise, and less error-prone.

c. Modular Infrastructure

Terraform promotes the use of modules, which are self-contained components of infrastructure that
can be reused across different projects. This makes your infrastructure modular and scalable. A
module is essentially a directory that contains .tf files, and it can be used to encapsulate a set of
resources.

For example, you can create a module for a Virtual Private Cloud (VPC), another module for EC2
instances, and a third for load balancers. You can then reuse these modules across multiple
environments (development, staging, production), without duplicating the code.

Here’s an example of how to define and use a module in Terraform:

# Defining the module (modules/vpc/main.tf)

module "vpc" {
source = "./modules/vpc"
cidr_block = "10.0.0.0/16"
}

# Using the module in your main configuration

module "my_vpc" {
source = "./modules/vpc"
cidr_block = "10.0.0.0/16"
}
By using modules, you can easily organize and
standardize your infrastructure across different projects.

d. Infrastructure State Management

Terraform tracks the state of your infrastructure through a state file (terraform.tfstate). This state file
serves as a snapshot of your current infrastructure and allows Terraform to know what resources
already exist, what resources need to be modified, and what resources should be deleted.

The state file enables Terraform to perform incremental updates to your infrastructure, ensuring
that only the necessary changes are applied. This avoids the need for redeploying the entire
infrastructure, saving time and reducing the risk of errors.

The state file can be stored:

• Locally: The state file is stored on the machine where Terraform is executed.
DevOps Shack
• Remotely: The state file can be stored remotely in services like Amazon S3 or Azure Blob
Storage. This is particularly useful for teams where multiple members are collaborating on
the same infrastructure.

3. Key Features of Terraform

a. Execution Plans

Terraform’s execution plan feature allows you to preview changes before they are applied. When you
run terraform plan, Terraform compares the current state of your infrastructure (stored in the state
file) with the desired state defined in the configuration files. It then generates a plan that shows
what actions Terraform will take, such as creating new resources, updating existing ones, or deleting
obsolete resources.

Example of generating an execution plan:

terraform plan

Output Example:

+ aws_instance.my_instance

ami: "ami-0c55b159cbfafe1f0"

instance_type: "t2.micro"

Plan: 1 to add, 0 to change, 0 to destroy.

This feature provides an opportunity to review changes before applying them, reducing the risk of
accidental changes to production environments.

b. Resource Graph

Terraform builds a resource dependency graph that represents the relationships between your
resources. By understanding the dependencies, Terraform can determine which resources can be
created in parallel and which ones need to be created in a specific order.

For instance, if you're provisioning an EC2 instance and an S3 bucket that don’t depend on each
other, Terraform can create both in parallel. However, if your EC2 instance relies on a VPC being
created first, Terraform will ensure that the VPC is provisioned before the EC2 instance.

c. Multi-Provider Ecosystem

Terraform has a vast ecosystem of providers. Providers are responsible for interacting with cloud
providers and other APIs. Some of the most popular providers include:

• AWS Provider: Manages resources in Amazon Web Services (AWS), such as EC2 instances, S3
buckets, and RDS databases.

• Azure Provider: Manages resources in Microsoft Azure, including Virtual Machines, App
Services, and Azure Kubernetes Services (AKS).
DevOps Shack
• Google Cloud Provider: Manages resources in Google Cloud Platform (GCP), such as
Compute Engine, Cloud Storage, and Google Kubernetes Engine (GKE).

• Kubernetes Provider: Manages Kubernetes clusters and resources, allowing you to manage
Kubernetes infrastructure as code.

d. Modularity and Reusability

As mentioned earlier, Terraform supports modules, which promote reusability and help you organize
your infrastructure code. Modules are especially useful for large-scale infrastructure where you need
to manage hundreds or thousands of resources.

For example, a module that defines a VPC can be reused across multiple environments, ensuring
consistent network configurations for all environments.

e. Collaboration and Version Control

Terraform configurations can be stored in version control systems like Git. This enables teams to
collaborate on infrastructure in the same way they collaborate on application code. Every change to
the infrastructure configuration is tracked in the version control history, making it easy to audit
changes, revert problematic updates, and collaborate across teams.

4. Terraform Installation
Terraform can be installed on various platforms such as Linux, macOS, and Windows. The installation
process is fairly straightforward and involves downloading the binary and setting it up in your
system’s path.

a. Installing Terraform on Linux

Here are the steps to install Terraform on a Linux-based system:

1. Download the Terraform Binary:

o Visit the official Terraform downloads page to download the latest version for Linux,
or use the wget command as shown below:

wget https://releases.hashicorp.com/terraform/1.0.0/terraform_1.0.0_linux_amd64.zip

2. Unzip the Binary:

o After downloading the binary, unzip the file to access the Terraform executable.

unzip terraform_1.0.0_linux_amd64.zip

3. Move the Binary to /usr/local/bin/:

o Move the unzipped binary to the /usr/local/bin/ directory so that it can be executed
from any terminal session.

sudo mv terraform /usr/local/bin/

DevOps Shack
4. Verify the Installation:

o To ensure that Terraform was installed correctly, run the following command:

terraform --version

You should see the installed Terraform version displayed, confirming the installation was successful.

b. Installing Terraform on macOS

If you're using macOS, you can install Terraform using the package manager Homebrew.

1. Install Homebrew (if you haven't already):

o Homebrew is a popular package manager for macOS, and it simplifies the installation
of many development tools, including Terraform. You can install Homebrew by
running:

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

2. Install Terraform using Homebrew:

o Once Homebrew is installed, you can install Terraform by running:

brew install terraform 3. Verify

the Installation:

o Verify that Terraform has been installed correctly by checking the version:

terraform --version

c. Installing Terraform on Windows

For Windows, you can manually download the Terraform binary or use Chocolatey, a Windows
package manager.

1. Using Chocolatey:

o If you already have Chocolatey installed, you can install Terraform by running the
following command in an Administrator-level command prompt:

choco install terraform

2. Manual Installation:

o Alternatively, you can download the Windows binary from the official website.

o Extract the zip file and place the terraform.exe in a directory that’s added to your
PATH environment variable, such as C:\Windows\System32.

3. Verify the Installation:

o Open a Command Prompt window and type the following to check the installed
version of Terraform:

terraform --version
DevOps Shack

5. Terraform Configuration Basics

Once Terraform is installed, the next step is to create a basic configuration to start managing
infrastructure. Terraform uses configuration files written in HashiCorp Configuration Language (HCL)
to describe the desired state of infrastructure.

a. Structure of a Terraform Configuration

A basic Terraform configuration consists of three main components:

• Providers: Define the cloud provider or infrastructure platform.

• Resources: Define the infrastructure resources to be managed.

• Variables and Outputs: Variables allow for flexibility in configurations, and outputs provide
useful information after deployment.

Let’s walk through a simple example where we’ll set up an AWS EC2 instance using Terraform.

b. Defining a Provider

A provider is responsible for creating and managing infrastructure resources. Terraform supports
many cloud providers, including AWS, Azure, and Google Cloud. Before you can provision resources,
you need to define the provider for the platform you are working with. Example: Defining an AWS
Provider provider "aws" { region = "us-west-1" access_key = "your-access-key" secret_key =
"your-secret-key"

• The provider block defines the configuration required to interact with AWS.

• You can either specify your access keys directly in the configuration (not recommended for
security reasons) or use environment variables like AWS_ACCESS_KEY_ID and
AWS_SECRET_ACCESS_KEY for authentication.

c. Defining Resources

Once the provider is defined, you can start defining resources. Resources are the fundamental
components of infrastructure, such as virtual machines, storage, and networks.

Example: Defining an AWS EC2 Instance

resource "aws_instance" "example" {

ami = "ami-0c55b159cbfafe1f0" # Amazon Linux AMI
instance_type = "t2.micro"
}
DevOps Shack
• The resource block defines an EC2 instance on AWS. The AMI (Amazon Machine Image)
specifies the OS for the instance, and instance_type defines the instance size.

• The resource name aws_instance.example is used internally by Terraform to manage this

resource.

d. Variables in Terraform

Terraform supports the use of variables to make configurations more dynamic and reusable. Instead
of hardcoding values like instance types or regions, you can define variables.

Example: Defining a Variable for instance_type

variable "instance_type" {
default = "t2.micro"
}

resource "aws_instance" "example" {

ami = "ami-0c55b159cbfafe1f0"
instance_type = var.instance_type
}
• In the above example, we define a variable instance_type, and in the resource block, we
reference the variable using var.instance_type.

Variables can also be passed as command-line arguments during terraform apply, allowing you to
dynamically set values without editing configuration files.

e. Outputs in Terraform

Outputs are used to extract values from your infrastructure once it has been provisioned. This can be
useful for passing data between configurations or displaying important information like IP addresses,
resource IDs, etc.

Example: Defining an Output for EC2 Instance IP Address

output "instance_ip" { value =

aws_instance.example.public_ip

• After applying the Terraform configuration, the public IP address of the EC2 instance will be
displayed.

You can access output values using terraform output after the infrastructure has been provisioned:

terraform output instance_ip

f. Initializing a Terraform Project

Before running any Terraform commands, you need to initialize the project. The terraform init
command downloads the necessary provider plugins and sets up the working environment.
DevOps Shack
Example: terraform

init

This command is run from the directory containing your Terraform configuration files (.tf files). After
initialization, Terraform is ready to manage your infrastructure.

g. Terraform Plan and Apply

Once your configuration is ready, the next step is to preview the changes Terraform will make. This is
done using the terraform plan command.

Example:

terraform plan

The output will show what resources will be created, modified, or destroyed. This is a critical step to
avoid accidentally modifying production infrastructure.

Once you’re satisfied with the plan, you can apply the changes to create or update resources using
terraform apply. Example:

terraform apply

This will trigger Terraform to communicate with the cloud provider (AWS in this case) and provision
the resources defined in the configuration.

h. Terraform State Management

Terraform keeps track of the resources it manages using a state file. The state file (terraform.tfstate)
contains the current state of your infrastructure, and it’s critical for Terraform’s ability to manage
incremental changes.

By default, the state file is stored locally in the working directory, but it can also be stored remotely
using services like Amazon S3, Azure Blob Storage, or Google Cloud Storage. Remote state is useful
when multiple team members are working on the same infrastructure.

To inspect the current state, you can use:

terraform show

i. Cleaning Up with Terraform Destroy

When you’re finished with your infrastructure and want to clean up the resources, you can use the
terraform destroy command. This will remove all resources that were created by Terraform.

Example:

terraform destroy

Terraform will prompt for confirmation before proceeding with the deletion of resources.
DevOps Shack

j. Organizing Terraform Configurations

As your infrastructure grows, it’s important to organize your Terraform configurations into modules
or separate files. By splitting resources into multiple .tf files or using modules, you can keep your
code organized and reusable.

For example:

/project

├── main.tf

├── variables.tf

├── outputs.tf

├── modules

│ ├── vpc

│ │ ├── main.tf

│ │ ├── variables.tf

│ │ ├── outputs.tf

This separation allows different teams to work on different aspects of the infrastructure without
conflict.

6. Conclusion

In this module, we’ve walked through how to install Terraform on various platforms, create a basic
configuration using providers, resources, variables, and outputs, and manage your infrastructure
with commands like terraform plan, apply, and destroy.

7. Terraform Providers
A provider in Terraform is responsible for managing the lifecycle of resources. Terraform uses
providers to interact with different APIs and cloud platforms like AWS, Azure, Google Cloud, and
many others. Providers serve as the bridge between Terraform and the infrastructure it manages.

a. What are Terraform Providers?

DevOps Shack
Terraform providers are plugins that allow Terraform to communicate with external systems. These
systems can be cloud platforms (like AWS, Azure, or Google Cloud), SaaS platforms (like GitHub or
PagerDuty), or infrastructure services (like Kubernetes or OpenStack).

Providers define the set of resources and data sources that Terraform can manage. For example, the
AWS provider allows you to manage AWS resources like EC2 instances, S3 buckets, RDS databases,
etc.

Each provider exposes certain resources that can be created or managed. For example:

• The AWS provider exposes resources like aws_instance, aws_s3_bucket, aws_vpc, etc.

• The Azure provider exposes resources like azurerm_virtual_machine,

azurerm_storage_account, azurerm_app_service, etc.

• The Google Cloud provider exposes resources like google_compute_instance,

google_storage_bucket, etc.

Terraform providers must be configured before you can use them to provision resources. Provider
configurations typically include authentication credentials (like access keys or tokens) and regional
settings.

b. Configuring a Provider

To configure a provider, you define a provider block in your Terraform configuration. For example, if
you're working with AWS, you would configure the AWS provider like this:

provider "aws" { region =

"us-west-1" access_key =
"your-access-key"
secret_key = "your-secret-key"
}
In this example:

• region: Specifies the AWS region where Terraform will provision resources (e.g., us-west-1).

• access_key and secret_key: These are the AWS credentials that Terraform uses to
authenticate API requests. You can either hard-code them (not recommended for security
reasons) or use environment variables like AWS_ACCESS_KEY_ID and
AWS_SECRET_ACCESS_KEY.

c. Managing Multiple Providers

Terraform supports managing infrastructure across multiple providers in the same configuration. For
example, you could manage both AWS and Google Cloud resources in a single project.

Here’s how you can configure multiple providers in a Terraform configuration:

# AWS Provider

provider "aws" {

region = "us-west-1"
DevOps Shack
}

# Google Cloud Provider

provider "google" { project

= "my-gcp-project" region

= "us-central1"

# AWS EC2 Instance resource "aws_instance"

"my_aws_instance" { ami = "ami-

0c55b159cbfafe1f0" instance_type =

"t2.micro"

# Google Cloud Storage Bucket resource

"google_storage_bucket" "my_bucket" { name

= "my-storage-bucket" location = "US"

In this example:

• The AWS provider is used to provision an EC2 instance in AWS.

• The Google Cloud provider is used to create a storage bucket in Google Cloud.

d. Provider Versioning

Terraform allows you to specify the version of the provider you want to use. This is important to
ensure that the same version of the provider is used across different environments, preventing
breaking changes that might occur due to provider updates.

Here’s how to specify the version of the AWS provider:

provider "aws" { version = "~> 2.0" # Use version 2.x

of the provider region = "us-west-1"

The version argument ensures that Terraform uses version 2.x of the AWS provider. You can also pin
exact versions or specify version ranges.
DevOps Shack

e. Using Provider Aliases

If you need to interact with multiple accounts or regions within the same provider, you can use
provider aliases. For example, you may want to manage resources in multiple AWS regions.

Here’s how you can define provider aliases for AWS:

# Default AWS Provider (us-west-1)

provider "aws" {
region = "us-west-1"
}

# Additional AWS Provider (us-east-1) using an alias

provider "aws" { alias = "us_east" region = "us-
east-1"
}

# Resource in default region (us-west-1)

resource "aws_instance" "west_instance" {
ami = "ami-0c55b159cbfafe1f0"
instance_type = "t2.micro"
}

# Resource in additional region (us-east-1)

resource "aws_instance" "east_instance" {
provider = aws.us_east ami =
"ami-0c55b159cbfafe1f0" instance_type
= "t2.micro"

}
In this example, the provider block with the alias "us_east" configures a second AWS provider for the
us-east-1 region, while the default provider uses us-west-1. The EC2 instance east_instance is
created in the us-east-1 region by referencing the aliased provider.

8. Terraform Resources

A resource is the fundamental building block of Terraform. It represents the infrastructure

components that Terraform manages. Examples of resources include virtual machines, databases,
storage buckets, and load balancers.

Resources are defined in resource blocks within the configuration files. Each resource block specifies
the type of resource to be managed, the provider that will manage it, and the configuration for that
resource.

a. Defining Resources

To define a resource in Terraform, you use the resource block, which has the following format:

resource "<PROVIDER>_<RESOURCE_TYPE>" "<RESOURCE_NAME>" {

DevOps Shack
<CONFIGURATION>

• <PROVIDER>: The cloud provider that manages the resource (e.g., aws, azurerm, google).

• <RESOURCE_TYPE>: The type of resource being managed (e.g., instance, bucket, vpc).

• <RESOURCE_NAME>: A user-defined name for the resource within Terraform.

• <CONFIGURATION>: A set of arguments that specify how the resource should be configured.

Example: Provisioning an AWS EC2 instance: resource

"aws_instance" "my_instance" { ami = "ami-

0c55b159cbfafe1f0" # Amazon Linux AMI instance_type =

"t2.micro"

In this example:

• The resource type is aws_instance, which manages an EC2 instance in AWS.

• The resource name is my_instance, which is how Terraform identifies the resource in its
internal state.

• The ami and instance_type are arguments used to configure the EC2 instance.

b. Resource Attributes

Each resource in Terraform exposes a set of attributes that can be used to configure the resource or
extract information from it.

For example, an EC2 instance exposes the following attributes:

• ami: Specifies the Amazon Machine Image to use.

• instance_type: Defines the size of the instance (e.g., t2.micro).

• public_ip: The public IP address assigned to the instance (output-only attribute).

You can reference resource attributes in other parts of the configuration. For example:

output "instance_ip" { value = aws_instance.my_instance.public_ip

c. Managing Dependencies

Terraform automatically manages the dependencies between resources. It understands that certain
resources depend on others, and it ensures that resources are created in the correct order.

For example, if you're creating an EC2 instance that depends on a VPC, Terraform will know that the
VPC must be created first.

Example: Creating a VPC and an EC2 instance that depends on the VPC:
DevOps Shack
resource "aws_vpc" "my_vpc" {

cidr_block = "10.0.0.0/16"

resource "aws_instance" "my_instance" {

ami = "ami-0c55b159cbfafe1f0" instance_type = "t2.micro"

subnet_id = aws_vpc.my_vpc.id # EC2 instance depends on VPC

In this example, the EC2 instance will not be created until the VPC has been successfully provisioned.

d. Resource Lifecycle Management

Terraform allows you to control the lifecycle of resources using the lifecycle block. This is useful for
customizing how Terraform handles resource creation, update, or deletion.

Here are some common lifecycle arguments:

• create_before_destroy: Ensures that a new resource is created before the old resource is
destroyed.

• prevent_destroy: Prevents Terraform from destroying a resource.

• ignore_changes: Tells Terraform to ignore specific changes to resource attributes.

Example: Ensuring an EC2 instance is created before the old one is destroyed:

resource "aws_instance" "my_instance" {

ami = "ami-0c55b159cbfafe1f0"

instance_type = "t2.micro"

lifecycle {

create_before_destroy = true

In this case, Terraform will create a new EC2 instance before destroying the old one, minimizing
downtime.

e. Resource Provisioners

Provisioners allow you to execute scripts or commands on the resource after it has been created.
Provisioners can be useful for bootstrapping resources or performing configuration tasks.
DevOps Shack
Example: Running a shell script on an EC2 instance after it is created:

resource "aws_instance" "my_instance" {

ami = "ami-0c55b159cbfafe1f0"

instance_type = "t2.micro"

provisioner "remote-exec" {

inline = [

"sudo apt-get update",

"sudo apt-get install -y nginx"

connection {
type = "ssh" user =

"ubuntu" private_key =

file("~/.ssh/id_rsa")

host = self.public_ip

In this example, the remote-exec provisioner is used to run commands on the EC2 instance after it is
created. The connection block specifies how Terraform should connect to the instance.

9. Conclusion

In this module, we covered the concepts of Terraform Providers and Terraform Resources in great
detail. Providers are essential for interacting with external systems, and resources are the
fundamental units of infrastructure that Terraform manages. We explored how to configure
providers, define resources, manage dependencies, and control resource lifecycles.

In the next module, we will dive into Variables, Outputs, and Data Sources, which are essential for
making Terraform configurations more dynamic and reusable.
DevOps Shack

10. Variables in Terraform

Variables in Terraform allow you to parameterize your configuration files. Instead of hardcoding
values, you can define variables and use them throughout your configuration. This makes your code
more modular and reusable, especially when working across different environments such as
development, staging, and production.

a. Declaring Variables

Variables in Terraform are declared using the variable block. Each variable can have the following
attributes:

• default: The default value of the variable.

• type: The type of the variable (e.g., string, list, map).

• description: A brief description of what the variable represents.

• sensitive: Marks the variable as sensitive, ensuring it is not displayed in logs or outputs.

Here’s an example of declaring a simple string variable for an AWS region:

variable "aws_region" { default = "us-west-1"

description = "The AWS region to deploy resources in."

In this example, we define a variable named aws_region with a default value of "us-west-1". The
description provides context about what the variable does.
DevOps Shack

b. Variable Types

Terraform supports several types of variables, including:

• String: A single value, such as a region or instance type.

• List: A collection of values. Useful for defining multiple instances, subnets, etc.

• Map: A collection of key-value pairs. Useful for specifying configuration options in a

structured format.

• Boolean: A true or false value.

Example of a list variable:

variable "instance_types" {

type = list(string)

default = ["t2.micro", "t2.small", "t2.medium"]

Example of a map variable:

variable "ami_ids" {

type = map(string)

default = {

us-east-1 = "ami-123456" us-

west-1 = "ami-654321"

With these variable types, you can create dynamic configurations that can adapt to different input
values.

c. Using Variables in Configuration

Once you’ve declared variables, you can reference them in your resource definitions using the var.
prefix. Example:

resource "aws_instance" "my_instance" {

ami = var.ami_ids[var.aws_region]

instance_type = var.instance_type

In this example:

• var.ami_ids[var.aws_region] dynamically selects the appropriate AMI based on the region.

DevOps Shack
• var.instance_type references the instance type from the list of instance types.

d. Passing Variables to Terraform

There are several ways to pass values to variables in Terraform:

1. Default Values: If no value is provided, Terraform will use the default value defined in the
variable block.

2. Command Line Flags: You can pass variables directly through the terraform apply or
terraform plan command using the -var flag.

terraform apply -var="aws_region=us-east-1"

3. Environment Variables: Variables can be set using environment variables. For example, to
pass the AWS region, you can use:

export TF_VAR_aws_region=us-east-1

terraform apply

4. Terraform .tfvars Files: Variables can be defined in a separate .tfvars file and passed to
Terraform during runtime. Example of a terraform.tfvars file:

aws_region = "us-east-1" instance_type

= "t2.medium"

Run Terraform with:

terraform apply -var-file="terraform.tfvars"

e. Sensitive Variables

Sometimes, you may need to pass sensitive information, such as API keys, database passwords, or
SSH private keys. Terraform allows you to mark variables as sensitive, ensuring they are not exposed
in logs or outputs.

Example: variable

"db_password" {

type = string

sensitive = true

By setting the sensitive flag to true, Terraform will ensure that the value of db_password is not
displayed during terraform apply or terraform plan.

11. Outputs in Terraform

DevOps Shack
Outputs in Terraform allow you to extract and share data from your infrastructure. They can be used
to display essential information after resources are provisioned or to pass values between Terraform
modules.

a. Defining Outputs

You can define an output using the output block. Each output block contains:

• value: The value to output (typically, the result of a resource attribute).

• description: A brief description of the output (optional).

• sensitive: If set to true, the value will not be displayed in the Terraform output logs.

Example: Defining an output for the public IP address of an EC2 instance:

output "instance_public_ip" { value =

aws_instance.my_instance.public_ip description = "The

public IP address of the EC2 instance."

In this example:

• value extracts the public_ip attribute from the aws_instance resource.

• description provides context for what this output represents.

b. Viewing Outputs

After applying the configuration, you can view outputs by running the terraform output command:

terraform output

This command will display all the outputs defined in your configuration. To view a specific output,
you can run:

terraform output instance_public_ip

This will display only the value of the instance_public_ip output.

c. Using Outputs Across Modules

One of the most powerful use cases for outputs is sharing data between Terraform modules. When
working with multiple modules, outputs from one module can be used as input for another module.

Example: Passing an output from one module to another:

DevOps Shack
# Module 1: VPC Module

module "vpc" { source =

"./modules/vpc"

# Module 2: EC2 Module that depends on the VPC module

module "ec2" { source = "./modules/ec2"

vpc_id = module.vpc.vpc_id # Using output from the VPC module

subnet_ids = module.vpc.subnet_ids

In this example, the vpc_id and subnet_ids from the vpc module are passed to the ec2 module as
inputs.

d. Sensitive Outputs

Similar to sensitive variables, Terraform allows you to mark outputs as sensitive. This ensures that
sensitive information, such as passwords or private keys, is not displayed in the output logs.

Example: output "db_password" { value =

aws_db_instance.my_db.password sensitive

= true

12. Data Sources in Terraform

Data sources allow you to query and reference existing infrastructure outside of Terraform. This is
useful for situations where you need to use resources that are managed by another team, project, or
process but still want to integrate them into your Terraform configuration.

a. What are Data Sources?

A data source in Terraform allows you to read information about existing infrastructure components.
Data sources do not create new resources; instead, they retrieve data about resources that already
exist.

For example, you might want to query an existing AWS VPC to retrieve its ID and use it in your
configuration.

b. Using Data Sources

DevOps Shack
To use a data source, you define a data block in your configuration. The format of a data source block
is similar to that of a resource block, but it only retrieves information instead of creating or modifying
resources.

Example: Querying an existing AWS VPC:

data "aws_vpc" "default" {

default = true
}

resource "aws_instance" "my_instance" {

ami = "ami-0c55b159cbfafe1f0"
instance_type = "t2.micro"
subnet_id = data.aws_vpc.default.id # Using VPC data source
}
In this example, the data block queries the default VPC in AWS and retrieves its ID. This ID is then
used to create an EC2 instance in the same VPC.

c. Common Data Sources

Terraform provides a wide range of data sources across different providers. Here are a few common
data sources:

• AWS:

o aws_vpc: Retrieves information about an existing VPC. o aws_ami: Queries

available AMIs (Amazon Machine Images).

o aws_subnet: Retrieves information about an existing subnet.

• Azure:

o azurerm_resource_group: Retrieves information about an existing resource group.

o azurerm_virtual_network: Retrieves information about an existing virtual network.

• Google Cloud:

o google_compute_network: Retrieves information about an existing Google Cloud

network.

o google_compute_subnetwork: Retrieves information about an existing subnetwork.

d. Example: Using Data Sources in a Multi-Cloud Environment

Let's look at a more advanced example where we query information from both AWS and Google
Cloud using data sources.

# Data Source: Querying default AWS VPC

data "aws_vpc" "default" {
default = true
}
DevOps Shack
# Data Source: Querying a Google Cloud storage bucket
data "google_storage_bucket" "my_bucket" {
name = "my-existing-bucket"
}

# Using AWS VPC data to create an EC2 instance

resource "aws_instance" "my_instance" {
ami = "ami-0c55b159cbfafe1f0"
instance_type = "t2.micro"
subnet_id = data.aws_vpc.default.subnets[0].id
}

# Using Google Cloud bucket data in an output output

"bucket_location" {
value = data.google_storage_bucket.my_bucket.location
}
In this example:

• The default AWS VPC is queried using the aws_vpc data source, and the subnet ID is used to
provision an EC2 instance.

• The existing Google Cloud Storage bucket is queried using the google_storage_bucket data
source, and its location is output after the infrastructure is applied.

e. Benefits of Using Data Sources

Data sources are incredibly useful for making your Terraform configurations more dynamic and
flexible. Here are a few benefits:

• Seamless Integration with Existing Infrastructure: Data sources allow you to work with
resources that are managed outside of Terraform, reducing the need to recreate them.

• Cross-Project Reusability: You can reference shared infrastructure resources between teams
or projects without managing them directly.

• Avoids Redundant Resource Creation: You can query existing resources to avoid the
accidental duplication of infrastructure components.

13. Conclusion

In this module, we explored the concepts of variables, outputs, and data sources in Terraform. These
are critical components for making your infrastructure-as-code more dynamic, reusable, and flexible.
By using variables, you can parameterize your configuration for different environments. Outputs
allow you to extract and share valuable data from your infrastructure, while data sources enable you
to query and integrate with existing infrastructure outside of Terraform's direct control.

• Variables make configurations more dynamic, allowing for flexibility across environments.

• Outputs help expose critical information about your resources, which can be passed to other
modules or systems.
DevOps Shack
• Data Sources allow you to query existing infrastructure, helping you integrate Terraform with
infrastructure managed by other teams or providers.

In the next module, we will delve into State Management, Remote State, and Backends, which are
critical for ensuring that Terraform manages infrastructure consistently across team members and
environments.

14. Terraform State Management

a. What is Terraform State?

Terraform uses a state file (terraform.tfstate) to keep track of the resources it manages. This state file
serves as a snapshot of your infrastructure's current configuration, allowing Terraform to know what
resources it has created, updated, or destroyed.

Without state, Terraform would have to query your cloud provider’s API every time to check the
current status of resources. Instead, the state file allows Terraform to incrementally manage
resources by comparing the current infrastructure state (stored in the state file) with the desired
configuration (defined in .tf files).

For example, when you run terraform apply, Terraform reads the state file to determine:

• What resources exist.

• Which resources need to be created, modified, or destroyed.

b. How State Works

When Terraform creates or modifies resources, it updates the state file to reflect the changes. For
example, when you provision an EC2 instance on AWS, Terraform saves information about that
instance (like the instance ID and public IP) in the state file.

Here’s how Terraform uses state:

1. State Initialization: When you first run terraform init, Terraform creates an empty state file
or fetches an existing one if it’s stored remotely.

2. State Refresh: Terraform refreshes the state by querying the provider's API to make sure the
state file matches the actual state of resources.

3. Plan: When you run terraform plan, Terraform compares the state file to the configuration
files to generate a plan of what needs to be changed.

4. Apply: After reviewing the plan, terraform apply updates the infrastructure and the state file
accordingly.
DevOps Shack
5. Destroy: When running terraform destroy, Terraform reads the state file to identify and
delete the existing resources.

c. Where is the State File Stored?

By default, Terraform stores the state file locally in the directory where you run Terraform
(terraform.tfstate). However, for collaborative environments or projects, it's better to store the state
file in a remote backend (like Amazon S3 or Azure Blob Storage) to avoid conflicts when multiple team
members work on the same infrastructure.

d. What is Stored in the State File?

The state file contains information about all the resources Terraform manages. This includes:

• Resource IDs: Unique identifiers for the resources created, such as EC2 instance IDs or S3
bucket names.

• Resource Attributes: Additional information like IP addresses, security group rules, or subnet
IDs.

• Outputs: Any values specified in output blocks, such as public IP addresses or database
endpoints.

Sensitive information, such as passwords or access keys, may also be stored in the state file. This is
why it’s important to protect the state file and ensure it’s not accessible to unauthorized users.

e. Why State is Important

State is critical to Terraform’s operation because:

• Tracking Resource Changes: The state file acts as the "source of truth" for Terraform. It
allows Terraform to know what changes need to be made when you run terraform apply.

• Performance: Without a state file, Terraform would need to query the cloud provider’s API
for every resource on each execution, slowing down operations.

• Incremental Changes: The state file allows Terraform to make incremental changes to
resources, updating only the components that need modification, rather than re-deploying
everything.

15. Remote State Management

In collaborative environments, storing the state file locally can lead to conflicts if multiple people are
working on the same infrastructure. To solve this, Terraform supports remote state, which allows the
state file to be stored in a shared location where it can be accessed by multiple users.

Remote state storage also provides features like state locking, preventing multiple users from
applying changes at the same time and corrupting the state file.
DevOps Shack

a. What is Remote State?

Remote state refers to storing the Terraform state file in a remote location, such as an S3 bucket,
Azure Blob Storage, Google Cloud Storage, or HashiCorp's Terraform Cloud. By doing so, multiple
team members can access the same state file, ensuring consistency and collaboration.

Remote state also provides:

• State Locking: Ensures that only one user can modify the state at a time.

• State Encryption: Protects sensitive data within the state file by encrypting it.

• Versioning: Remote storage like S3 can version state files, allowing you to recover previous
versions in case of accidental changes or corruption.

b. Configuring Remote State with Amazon S3

To store your state file in an Amazon S3 bucket, you can configure Terraform to use an S3 backend.

Here’s an example configuration for remote state with S3:

terraform { backend "s3" { bucket =

"my-terraform-state" key =
"global/s3/terraform.tfstate" region
= "us-west-1"
encrypt = true
}
}
In this example:

• bucket: Specifies the S3 bucket where the state file will be stored.

• key: Defines the path to the state file within the S3 bucket.

• region: Specifies the AWS region where the bucket is located.

• encrypt: Ensures the state file is encrypted using S3 server-side encryption.

c. Remote State Locking with DynamoDB

To avoid multiple users attempting to modify the state file simultaneously, you can enable state
locking using DynamoDB. This ensures that only one user can run terraform apply at a time,
preventing state corruption.

Here’s how to configure DynamoDB state locking with S3:

terraform { backend "s3" { bucket =

"my-terraform-state" key =

"global/s3/terraform.tfstate" region

= "us-west-1" encrypt = true

DevOps Shack
dynamodb_table = "terraform-lock"
}

In this example, the dynamodb_table argument specifies the table that Terraform will use to lock the
state file while changes are being applied.

d. Remote State with Terraform Cloud

Another option for remote state storage is Terraform Cloud, which provides native support for
storing Terraform state, locking, and collaboration features. Terraform Cloud offers:

• State management: A secure, remote backend for storing state files.

• State versioning: Automatic versioning of state files to ensure recoverability.

• Integrated state locking: Prevents multiple users from making changes simultaneously.

• Collaborative workflows: Terraform Cloud integrates with version control systems (like
GitHub) and supports team-based collaboration.

Example configuration for using Terraform Cloud as a backend:

terraform {
backend "remote" {
organization = "my-org"

workspaces {
name = "my-workspace"
}
}
}

16. Backends in Terraform

A backend in Terraform defines where and how the state file is stored and how Terraform operations
(like apply, plan, destroy) are executed. By default, Terraform uses a local backend, but it can be
configured to use a variety of remote backends, including Amazon S3, Azure Blob Storage, Google
Cloud Storage, Terraform Cloud, and others.

Backends allow you to:

• Store State Remotely: For collaboration and disaster recovery.

• State Locking: Prevent multiple users from changing the state file at the same time.

• State Encryption: Ensure sensitive data in the state file is protected.

• Run Terraform in Remote Environments: Some backends, like Terraform Cloud, allow
Terraform to execute plans and apply changes remotely, rather than on your local machine.

a. Types of Backends
DevOps Shack
Here are a few common backends used in Terraform:

1. Amazon S3:

o Widely used for storing state files remotely in AWS. o Supports state locking with

DynamoDB.

o Supports server-side encryption.

2. Azure Blob Storage:

o Used for storing Terraform state files in Azure.

o Supports state locking with Azure Cosmos DB.

3. Google Cloud Storage:

o Used for storing state files in Google Cloud.

o Supports versioning and encryption.

4. Terraform Cloud:

o Native backend provided by HashiCorp for state storage and remote operations.

o Includes integrated state locking and collaboration features.

b. Configuring a Backend

Here’s how you can configure Terraform to use Azure Blob Storage as a backend:

terraform { backend
"azurerm" {
storage_account_name = "myterraformstorage"
container_name = "tfstate"
key = "terraform.tfstate"
}
}
In this example:

• storage_account_name: The name of the Azure storage account where the state file is
stored.

• container_name: The container within the storage account where the state file is stored.

• key: The name of the state file.

c. Benefits of Using Remote Backends

Using a remote backend provides several advantages:

• Collaboration: Teams can share the same state file, ensuring that everyone is working from
the same source of truth.
DevOps Shack
• Security: Remote backends like S3 or Terraform Cloud can encrypt the state file to protect
sensitive information.

• State Locking: Prevents multiple users from running terraform apply simultaneously, which
could result in conflicts or state corruption.

• Disaster Recovery: Remote backends store state files in highly available and durable storage,
ensuring that state files are not lost in case of local machine failure.

17. Conclusion

In this module, we covered the critical concept of state management in Terraform, including how
Terraform tracks the state of resources using the state file, and how remote state and backends
enable collaboration, security, and locking in multi-user environments.

State management is essential for ensuring that Terraform can accurately manage your
infrastructure, especially as your projects grow in size and complexity. By storing the state file
remotely, using state locking, and encrypting the state file, you can create a more secure, reliable,
and scalable infrastructure management process.
DevOps Shack

18. Terraform Modules

a. What are Terraform Modules?

A module in Terraform is a container for multiple resources that are used together. A module
encapsulates one or more resources into a single unit that can be reused across projects and
environments. Think of a module as a way to package infrastructure code so that it can be easily
maintained, shared, and reused.

Modules provide several benefits:

• Reusability: Once a module is created, it can be reused across different projects or

environments.

• Abstraction: Modules can abstract away complex infrastructure details, making it easier for
users to consume them.

• Maintainability: By organizing infrastructure into modules, you can manage large, complex
projects more effectively.

• Consistency: Modules ensure that infrastructure is created consistently across different

environments.

In Terraform, every configuration is part of a module. The configuration in your working directory is
called the root module, and you can call other modules from within the root module.

b. Defining a Module

A module consists of the following components:

1. main.tf: Defines the resources managed by the module.

2. variables.tf: Defines the input variables the module accepts.

3. outputs.tf: Defines the outputs the module provides to the calling configuration.

Example: A simple module for creating an AWS EC2 instance.

Step 1: Create the Module Directory Structure

/modules

/ec2

main.tf

variables.tf

outputs.tf
DevOps Shack

Step 2: Define the main.tf File (Resources)

resource "aws_instance" "my_instance" {

ami = var.ami
instance_type = var.instance_type
tags = {
Name = var.instance_name
}
}
In this main.tf file, an EC2 instance is created using the AMI, instance type, and instance name
provided via variables.

Step 3: Define the variables.tf File (Inputs)

variable "ami" {
description = "The AMI to use for the EC2 instance"
}

variable "instance_type" {
description = "The instance type to use"
default = "t2.micro"
}

variable "instance_name" {
description = "The name to assign to the EC2 instance"
default = "example-instance"
}
Here, three variables are defined: ami, instance_type, and instance_name. These are passed to the
module by the calling configuration. Step 4: Define the outputs.tf File (Outputs) output
"instance_id" { value = aws_instance.my_instance.id

output "instance_public_ip" { value =

aws_instance.my_instance.public_ip

The outputs.tf file defines outputs that expose the instance ID and public IP of the EC2 instance
created by the module.

c. Using a Module
DevOps Shack
Once the module is defined, you can call it from your root module (i.e., the main configuration file in
your working directory). Here’s how you would call the EC2 module created above.

Example: Using the EC2 module in your root configuration.

module "my_ec2" { source =

"./modules/ec2" ami = "ami-
0c55b159cbfafe1f0" instance_type =
"t2.micro"
instance_name = "my-terraform-instance"
}
In this example:

• source: Specifies the location of the module. In this case, it is a local path (./modules/ec2).

• ami, instance_type, and instance_name: These are the input variables required by the
module.

When you run terraform apply, Terraform will use the module to provision an EC2 instance with the
specified AMI, instance type, and name.

d. Using Remote Modules

In addition to local modules, Terraform allows you to use remote modules hosted in version control
systems like GitHub, or in the Terraform Registry. This makes it easy to share and reuse modules
across different teams and projects. Example: Using a module from GitHub.

module "vpc" { source = "github.com/terraform-aws-

modules/terraform-aws-vpc"

version = "2.0.0" cidr =

"10.0.0.0/16" azs = ["us-west-1a",

"us-west-1b"]

In this example:

• The source argument points to a GitHub repository containing the module.

• The version argument specifies which version of the module to use.

• The module accepts inputs like cidr and azs to define the VPC configuration.

e. Best Practices for Terraform Modules

1. Use Modules for Reusable Infrastructure: Break down your infrastructure into reusable
modules. For example, create separate modules for VPC, EC2 instances, databases, and so
on.

2. Version Your Modules: If you're using modules from version control or the Terraform
Registry, always specify the module version to avoid breaking changes.
DevOps Shack
3. Modularize Your Projects: Organize your project into separate modules for easier
maintenance. For example, you can have a module for networking, another for compute
resources, and a third for monitoring.

4. Document Your Modules: Provide clear documentation for the inputs, outputs, and purpose
of each module. This helps other team members understand how to use the modules
correctly.

19. Terraform Workspaces

Terraform workspaces allow you to manage multiple environments (e.g., development, staging,
production) from the same configuration. Each workspace has its own separate state file, which
means that changes in one workspace won’t affect the resources in another workspace.

a. What are Terraform Workspaces?

By default, Terraform operates in a single workspace called default. However, you can create
additional workspaces to manage different environments using the same configuration files.

Workspaces are useful when:

• You want to maintain separate infrastructure environments (e.g., dev, staging, prod) without
duplicating configuration code.

• You need to create isolated copies of the infrastructure for testing or QA purposes.

Each workspace has its own state, so the resources created in one workspace are independent of
those in another.

b. Creating and Managing Workspaces

You can create a new workspace using the terraform workspace new command, and switch between
workspaces using the terraform workspace select command. Step 1: Create a New Workspace
terraform workspace new dev

This creates a new workspace called dev. Terraform will automatically create a new state file for this
workspace.

Step 2: Switch to an Existing Workspace terraform

workspace select dev

This command switches Terraform to the dev workspace. Any resources created or modified while in
this workspace will be isolated from other workspaces.

Step 3: List All Workspaces terraform

workspace list
DevOps Shack
This command lists all workspaces in the current project. You’ll see an output like:

* default

dev prod

The * symbol indicates the active workspace.

c. Using Workspaces in Configuration

You can reference the active workspace in your configuration using the terraform.workspace variable.
This is useful for customizing resource names or tags based on the workspace.

Example: Customizing resource names based on the workspace.

resource "aws_instance" "my_instance" {

ami = "ami-0c55b159cbfafe1f0"

instance_type = "t2.micro"

tags = {

Name = "my-instance-${terraform.workspace}"

In this example, the EC2 instance name will be suffixed with the workspace name (e.g., my-
instancedev or my-instance-prod).

d. Using Workspaces for Multiple Environments

Workspaces are ideal for managing multiple environments like development, staging, and
production, without duplicating code. Each environment can have its own workspace, and resources
in each environment will be completely isolated.

Example workflow:

1. Create a workspace for each environment:

terraform workspace new dev terraform

workspace new staging terraform

workspace new prod

2. Deploy resources to each environment:

terraform workspace select dev terraform apply # Apply

changes to the dev environment

DevOps Shack
terraform workspace select staging terraform apply # Apply

changes to the staging environment

By using workspaces, you can avoid duplicating configurations for each environment while keeping
the infrastructure state isolated.

e. Limitations of Workspaces

While workspaces are useful for managing environments, they have some limitations:

• Limited to State Management: Workspaces only isolate the state, not the configuration
itself. If you need to manage different configurations (e.g., different AMI IDs for dev and
prod), you’ll need to handle that logic using variables.

• Not Suitable for Every Use Case: For large-scale projects with many environment-specific
differences, using separate workspaces might become complex. In such cases, it may be
better to create separate Terraform projects for each environment.

20. Conclusion

In this module, we covered Terraform Modules and Terraform Workspaces, both of which are crucial
for organizing and managing infrastructure in a scalable and maintainable way.

• Modules enable you to create reusable, modular components that simplify infrastructure
management and promote consistency across projects.

• Workspaces allow you to manage multiple environments using the same configuration,
isolating state files for each environment.

By using modules and workspaces effectively, you can reduce duplication, improve maintainability,
and ensure consistent deployments across different environments.

21. Conditional Expressions in Terraform

Conditional expressions in Terraform allow you to make decisions in your configuration based on
input values or variables. This enables you to adapt your configuration dynamically depending on the
environment, region, or other factors.

a. Basic Conditional Syntax

DevOps Shack
The syntax for conditional expressions in Terraform is similar to the ternary operator found in many
programming languages. The format is:

condition ? true_value : false_value

• condition: The condition to evaluate (e.g., var.environment == "production").

• true_value: The value to use if the condition is true.

• false_value: The value to use if the condition is false.

b. Example: Conditional Instance Size

Suppose you want to provision different EC2 instance types based on the environment (e.g., use
t2.micro in development and t3.large in production). You can use a conditional expression to achieve
this.

Example:

variable "environment" {

default = "dev"

resource "aws_instance" "my_instance" {

ami = "ami-0c55b159cbfafe1f0" instance_type = var.environment

== "production" ? "t3.large" : "t2.micro"

In this example:

• The instance_type is determined by the value of var.environment. If the environment is

production, Terraform will use t3.large; otherwise, it will use t2.micro.

c. Example: Conditional Resource Creation

You can also use conditional expressions to decide whether or not to create a resource. This is useful
for controlling optional resources, such as adding a load balancer only in production environments.

Example:

resource "aws_elb" "my_elb" {

count = var.create_elb ? 1 : 0

name = "my-load-balancer"
availability_zones = ["us-west-1a", "us-west-1b"]
}
In this example:
DevOps Shack
• The count argument determines whether the resource is created. If var.create_elb is true,
Terraform will create one load balancer; if false, Terraform will create none.

22. For Loops in Terraform

Terraform provides for loops to iterate over lists or maps and create multiple resources or dynamic
configurations. Loops are incredibly useful when you need to provision multiple instances of the
same resource or dynamically configure settings based on a list of values.

a. Basic For Loop Syntax

Terraform supports for loops in two main contexts:

1. List iteration: Used for iterating over a list of values.

[for item in list : item_expression]

2. Map iteration: Used for iterating over a map's keys and values.

{for key, value in map : key_expression => value_expression}

b. Example: Creating Multiple EC2 Instances Using a For Loop

If you want to create multiple EC2 instances based on a list of instance types, you can use a for loop
to iterate over the list and create resources dynamically.

Example:

variable "instance_types" {
default = ["t2.micro", "t2.small", "t2.medium"]
}

resource "aws_instance" "my_instances" {

count = length(var.instance_types)

ami = "ami-0c55b159cbfafe1f0"
instance_type = var.instance_types[count.index]
}
In this example:

• The count parameter creates an EC2 instance for each element in the instance_types list.

• count.index is used to reference the current element in the list.

c. Example: Dynamic Tags Using a For Loop

You can also use for loops to dynamically generate resource attributes, such as tags.

Example:
DevOps Shack
variable "tags" {
type = map(string)
default = {
Name = "example-instance"
Owner = "admin"
Env = "dev"
}
}

resource "aws_instance" "my_instance" {

ami = "ami-0c55b159cbfafe1f0"
instance_type = "t2.micro"

tags = {for key, value in var.tags : key => value}

}
In this example:

• The for loop iterates over the tags map and dynamically applies the tags to the EC2 instance.

• Each key-value pair in the tags map becomes a tag for the instance.

d. Example: Looping Through Multiple Availability Zones

You can use a for loop to create resources in multiple availability zones.

Example:

variable "availability_zones" { default

= ["us-west-1a", "us-west-1b"]
}

resource "aws_subnet" "my_subnets" {

count = length(var.availability_zones)

vpc_id = aws_vpc.my_vpc.id cidr_block =

cidrsubnet("10.0.0.0/16", 8, count.index)
availability_zone = var.availability_zones[count.index]
}
In this example:

• The for loop iterates through the list of availability zones and creates a subnet in each one.

23. Dynamic Blocks in Terraform

Dynamic blocks in Terraform allow you to generate multiple nested blocks within a resource,
dynamically controlling how many blocks are created and their configuration. This is particularly
useful when dealing with resources that require nested blocks, such as security groups, load
balancers, and network ACLs.
DevOps Shack

a. What are Dynamic Blocks?

A dynamic block allows you to conditionally add multiple nested blocks within a resource based on a
variable or a loop. It consists of three parts:

• for_each: Defines the collection to iterate over (a list or map).

• content: Defines the content that will be generated for each item in the loop.

b. Example: Dynamic Security Group Rules

Suppose you need to create multiple security group rules for different ports. You can use a dynamic
block to iterate through a list of ports and create a rule for each one.

Example:

variable "allowed_ports" {
default = [22, 80, 443]
}

resource "aws_security_group" "my_sg" {

name = "example-sg"

dynamic "ingress" {
for_each = var.allowed_ports

content {
from_port = ingress.value
to_port = ingress.value
protocol = "tcp" cidr_blocks
= ["0.0.0.0/0"]
}
}
}
In this example:

• The dynamic block iterates over the allowed_ports variable and creates a separate ingress
block for each port.

• ingress.value references the current port in the loop.

c. Example: Dynamic Block for Autoscaling Policies

You can also use dynamic blocks to create multiple autoscaling policies for an application.

Example:

variable "scaling_policies" {
DevOps Shack
default = { scale_up = { adjustment = 2,
cooldown = 300 } scale_down = { adjustment = -1,
cooldown = 300 }
}
}

resource "aws_autoscaling_policy" "my_policy" {

for_each = var.scaling_policies

name = "${each.key}-policy"
scaling_adjustment = each.value.adjustment cooldown
= each.value.cooldown adjustment_type =
"ChangeInCapacity"
autoscaling_group_name = aws_autoscaling_group.my_asg.name
}
In this example:

• A dynamic block creates multiple autoscaling policies (scale-up and scale-down) for an
autoscaling group based on the scaling_policies variable.

• Each policy has different values for scaling_adjustment and cooldown.

d. When to Use Dynamic Blocks

Dynamic blocks are useful when:

• You need to create multiple nested blocks of the same type (e.g., security group rules,
autoscaling policies).

• The number of blocks or their configuration is dynamic, depending on input variables.

Dynamic blocks help reduce redundancy in your configuration and make it more scalable by
eliminating the need to manually define multiple similar blocks.

24. Conclusion

In this module, we covered some of Terraform's most powerful features: Conditional Expressions,
For Loops, and Dynamic Blocks. These features allow you to create dynamic, flexible, and scalable
configurations that adapt to different environments and requirements. By using these tools, you can
write more efficient infrastructure code that reduces redundancy and improves maintainability.

25. Importance of Testing in Terraform

Testing Terraform configurations ensures that your infrastructure behaves as expected before
deploying to production. Since Terraform deals with real resources like servers, databases, and
DevOps Shack
networking components, errors can lead to misconfigured or missing infrastructure, which can have
significant consequences.

Terraform testing is typically divided into three categories:

1. Unit Testing: Testing individual components (e.g., modules) to verify their functionality.

2. Integration Testing: Verifying that different parts of the infrastructure interact correctly.

3. Acceptance Testing: Ensuring that the entire infrastructure is provisioned correctly and
functions as expected in real-world scenarios.

26. Unit Testing Terraform Modules with terraform validate

Terraform has built-in commands to validate and lint your configurations. The terraform validate
command checks whether the configuration files are syntactically valid and internally consistent. a.
Running terraform validate

Before applying a configuration, it's essential to validate it:

terraform validate

• This command ensures that the syntax of the Terraform configuration files (.tf) is correct.

• It does not create or destroy any resources but ensures the Terraform code is ready to be
applied.

If the configuration is valid, you will see an output similar to:

Success! The configuration is valid.

If there are issues, Terraform will provide an error message indicating what went wrong.

27. Terraform Plan: Checking for Changes

The terraform plan command is another key component in testing your configuration. It allows you
to preview the changes Terraform will make before applying them. This helps you verify whether the
desired changes are correct and expected.

a. Running terraform plan

terraform plan

• terraform plan generates an execution plan that shows what Terraform will do to achieve the
desired state of the infrastructure.

• The plan lists which resources will be created, modified, or destroyed.

• This is useful for testing the logic in your configuration to ensure it behaves as expected.

Example output:

+ aws_instance.my_instance
DevOps Shack
ami: "ami-0c55b159cbfafe1f0"

instance_type: "t2.micro"

...

Plan: 1 to add, 0 to change, 0 to destroy.

In this example:

• The + symbol indicates that Terraform will add a new resource (an EC2 instance).

• The plan gives you a chance to review the actions Terraform will take before applying them.

28. Testing with terraform apply in a Sandbox Environment

While terraform plan is useful for reviewing the changes, it's important to test the actual application
of changes in a non-production environment (such as a sandbox, dev, or staging environment). a.
Running terraform apply

Once you're satisfied with the plan, you can apply the changes:

terraform apply

This will actually provision the resources in the target environment. Running this in a sandbox
environment allows you to test whether the resources are correctly created and configured. If
everything works as expected, you can apply the same changes to the production environment.

Example output:

aws_instance.my_instance: Creating...

aws_instance.my_instance: Creation complete after 30s [id=i-0abcd1234]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

29. Automated Testing with Terratest

Terratest is a popular Go-based framework for writing automated tests for Terraform code. It allows
you to write tests that automate the execution of Terraform configurations, deploy infrastructure,
and verify that it works as expected. It’s especially useful for integration and acceptance testing. a.
What is Terratest?

Terratest enables you to:

• Write tests in Go that can verify your Terraform code.

• Provision infrastructure using Terraform and validate its correctness by running real-world
checks (e.g., pinging an EC2 instance, checking that a load balancer is correctly set up).

• Automatically destroy the infrastructure after the tests have completed.

b. Example: Testing with Terratest

DevOps Shack
Here’s an example of a basic Terratest test to check if an EC2 instance was successfully created:

package test

import (
"testing"
"github.com/gruntwork-io/terratest/modules/aws"
"github.com/gruntwork-io/terratest/modules/terraform"
)

func TestTerraformEC2Instance(t *testing.T) {

terraformOptions := &terraform.Options{
// Path to the Terraform code
TerraformDir: "../examples/terraform-aws-ec2-instance",
}

// Run `terraform init` and `terraform apply`

terraform.InitAndApply(t, terraformOptions
)
// Get the instance ID of the newly created EC2 instance
instanceID := terraform.Output(t, terraformOptions, "instance_id")

// Verify that the EC2 instance is running

aws.AssertInstanceIsRunning(t, instanceID, "us-west-1")

// Clean up resources with `terraform destroy`

defer terraform.Destroy(t, terraformOptions)
}
In this example:

• The test provisions an EC2 instance using Terraform and then verifies that the instance is
running.

• After the test is completed, Terratest automatically destroys the infrastructure using
terraform destroy.

c. Installing and Running Terratest

To install and run Terratest, follow these steps:

1. Install Go: Ensure you have Go installed on your system.

brew install go # macOS

sudo apt-get install golang-go # Ubuntu

2. Set Up Terratest: Create a new Go project and import Terratest.

go mod init my_terraform_tests go get

github.com/gruntwork-io/terratest/modules/terraform

3. Run the Tests:

DevOps Shack
go test -v

Terratest will provision the infrastructure, run the tests, and clean up afterward.

30. Debugging Terraform Configurations

Sometimes, things go wrong when running Terraform. Whether it’s a failed terraform apply, incorrect
resource creation, or unexpected changes, debugging your Terraform configuration is crucial to
identify and resolve issues.

a. Enabling Debug Logs

Terraform allows you to enable detailed logging to help with debugging. You can set the TF_LOG
environment variable to DEBUG to enable debug logs.

Example:

export TF_LOG=DEBUG

terraform apply

With TF_LOG set to DEBUG, Terraform will output detailed information about what it’s doing at each
step. This can help you pinpoint issues in the configuration or identify API errors when interacting
with cloud providers.

b. Reviewing the Terraform State

The state file (terraform.tfstate) is critical to Terraform’s operation. If Terraform behaves

unexpectedly, it’s important to inspect the state file to ensure it reflects the actual infrastructure.

You can view the state with the following command:

terraform show

This command shows the contents of the state file, allowing you to inspect the resources Terraform is
managing and their current state. If a resource is missing or misconfigured, it will likely show up in
the state file.

c. Debugging Resource Dependencies

Terraform automatically manages resource dependencies, ensuring that resources are created,
modified, or destroyed in the correct order. However, if you suspect a dependency issue, you can use
terraform graph to visualize the resource graph.

Example:

terraform graph | dot -Tsvg > graph.svg

This command generates a graph of dependencies between resources, which can help you identify
issues related to resource ordering or dependencies.

d. Common Errors and How to Fix Them

1. Resource Already Exists: If Terraform tries to create a resource that already exists, check
whether the resource is in the state file. If not, import it using:

terraform import <resource> <ID>

DevOps Shack
2. Timeout Errors: Terraform operations can time out if the infrastructure takes too long to
provision. You can increase the timeout setting in the resource configuration:

resource "aws_instance" "my_instance" {

ami = "ami-0c55b159cbfafe1f0"

instance_type = "t2.micro"

timeouts {

create = "10m"

31. Conclusion

In this module, we explored Terraform Testing and Debugging Techniques to ensure that your
infrastructure-as-code is robust and functions as expected. Testing is crucial in any software
development workflow, and Terraform provides several tools, such as terraform validate and
terraform plan, for validating configurations. More advanced testing can be automated using
Terratest, which integrates real-world testing into your Terraform workflows. Additionally, Terraform’s
debugging tools, such as debug logs and state management, help identify and fix issues when things
go wrong.

32. Terraform Best Practices

Following best practices in Terraform development helps improve collaboration, security, and
maintainability. Let’s explore some key practices you should adopt in your Terraform projects.

a. Modularize Your Terraform Code

DevOps Shack
As your infrastructure grows, it's essential to split your Terraform configuration into modules.
Modules promote reusability and separation of concerns, allowing you to manage different parts of
your infrastructure independently.

Here are some guidelines for creating effective modules:

• Single Responsibility Principle: Each module should be responsible for one component (e.g.,
VPC, EC2, RDS). This keeps your modules focused and reusable.

• Use Inputs and Outputs: Define variables for user inputs and outputs for passing information
between modules.

• Store Modules in Version Control: Store modules in a Git repository or the Terraform
Registry to ensure version control and easy sharing between teams.

Example structure:

/modules
/vpc
main.tf
variables.tf
outputs.tf
/ec2
main.tf
variables.tf
outputs.tf
This structure makes it easy to reuse modules across projects, reducing duplication and ensuring
consistency.

b. Use Terraform State Backends

As discussed in Module 5, storing the state file remotely is a best practice, especially for teams
working in a collaborative environment. Use a backend like Amazon S3, Azure Blob Storage, or
Terraform Cloud to ensure the state is consistent across team members and protected with features
like encryption and locking.

Example: S3 Backend Configuration

terraform { backend "s3" { bucket =

"my-terraform-state" key =
"global/s3/terraform.tfstate" region
= "us-west-1"
encrypt = true
}
}
Using a remote backend helps avoid state corruption, enables state locking, and adds an extra layer
of security.

c. Use Version Control and Tagging

Always store your Terraform configurations and modules in a version control system (such as Git).
This allows you to track changes, collaborate with other developers, and roll back to previous
versions if something goes wrong.
DevOps Shack
Key practices for version control:

• Commit Often: Make small, incremental commits with meaningful commit messages.

• Use Branches: Separate your development, staging, and production environments using Git
branches.

• Tag Versions: Tag releases of your infrastructure code to provide easy rollback options.

Example:

git add .

git commit -m "Add Terraform configuration for VPC"

git tag v1.0.0 git

push --tags

By tagging your Terraform configurations, you can track infrastructure versions, making it easier to
identify what version is running in production.

d. Use .terraformignore

Similar to .gitignore, the .terraformignore file allows you to exclude specific files or directories from
being uploaded when using remote backends like Terraform Cloud.

Example .terraformignore file:

*.tfstate

*.log

.secret/*

This prevents sensitive or unnecessary files from being uploaded to the backend.

33. Advanced Terraform Techniques

Terraform provides several advanced techniques to help manage complex infrastructure, secrets, and
workflows. Let’s explore some of these advanced features.

a. Managing Secrets in Terraform

Managing secrets like API keys, passwords, and SSH keys can be challenging in Terraform. It’s
important to avoid hardcoding sensitive information in your configuration files. There are several
ways to manage secrets securely in Terraform:

1. Environment Variables: Store sensitive information in environment variables and pass them
to Terraform at runtime.

Example: export

TF_VAR_db_password="mysecretpassword"
DevOps Shack
terraform apply

2. Terraform Cloud/Enterprise: Terraform Cloud allows you to store sensitive information as

workspace variables, which are securely encrypted and protected.

3. Secrets Management Tools: Integrate with secrets management tools like HashiCorp Vault,
AWS Secrets Manager, or Azure Key Vault to dynamically inject secrets into your Terraform
configurations.

Example: Using AWS Secrets Manager in Terraform data

"aws_secretsmanager_secret" "db_password" { name =

"my-db-password"

resource "aws_db_instance" "my_db" {

allocated_storage = 10 engine = "mysql" instance_class =

"db.t2.micro" name = "mydb" username = "admin"

password = data.aws_secretsmanager_secret.db_password.value

In this example, the database password is securely retrieved from AWS Secrets Manager, rather than
being hardcoded in the Terraform configuration.

b. Remote Execution with Terraform Cloud

With Terraform Cloud, you can execute Terraform commands remotely, rather than running them on
your local machine. This is especially useful for larger teams and organizations that want to centralize
infrastructure management.

Terraform Cloud provides:

• Remote State Management: Automatically stores and locks the state file.

• Remote Plan and Apply: Executes terraform plan and apply in a remote environment,
ensuring consistency across team members.

• Collaboration Tools: Allows multiple team members to work on the same infrastructure.

Example: Configuring Terraform Cloud as a Backend terraform { backend "remote" {

organization = "my-org"

workspaces { name =

"my-workspace"

}
DevOps Shack
}

By using Terraform Cloud, you can also integrate CI/CD pipelines, automate the application of
Terraform plans, and manage approvals.

c. Using terraform import to Manage Existing Resources

If you have existing resources that were created manually (e.g., through a cloud provider’s console),
you can bring those resources under Terraform management using the terraform import command.
This is useful when you want Terraform to manage resources that were not initially provisioned by
Terraform.

Example: Importing an AWS EC2 Instance terraform

import aws_instance.my_instance i-0abcd1234

• In this example, the EC2 instance with the ID i-0abcd1234 is imported into Terraform’s state.

• After importing, you need to define the resource in your configuration (e.g., main.tf) so that
Terraform can manage it going forward.

d. Using terraform workspace for Multiple Environments

Terraform workspaces are useful for managing multiple environments (e.g., dev, staging, production)
from a single configuration, as discussed in Module 6.

Example Workflow with Workspaces:

1. Create and Switch Workspaces:

terraform workspace new dev terraform

workspace new prod

terraform workspace select dev

2. Customize Resource Names: resource "aws_instance"

"my_instance" { ami = "ami-0c55b159cbfafe1f0" instance_type

= "t2.micro"

tags = {

Name = "my-instance-${terraform.workspace}"

By using workspaces, you can manage different environments without duplicating configuration files.
DevOps Shack
e. Using depends_on to Manage Complex Dependencies

Terraform automatically manages dependencies between resources. However, in some complex

scenarios, you may need to explicitly declare dependencies using the depends_on argument. This
ensures that resources are created or destroyed in the correct order. Example: Explicit
Dependency with depends_on resource "aws_instance" "my_instance" {

ami = "ami-0c55b159cbfafe1f0"

instance_type = "t2.micro" depends_on =

[aws_s3_bucket.my_bucket]

resource "aws_s3_bucket" "my_bucket" {

bucket = "my-bucket"

In this example, Terraform will ensure that the S3 bucket is created before the EC2 instance, even if
there is no direct connection between the two resources.

34. Conclusion

In this module, we covered Terraform Best Practices and Advanced Techniques to help you manage
infrastructure-as-code efficiently, securely, and at scale. By adopting best practices like modularizing
your code, using remote backends for state management, and storing sensitive information securely,
you can improve the maintainability and security of your Terraform projects. Additionally, advanced
techniques like using workspaces, remote execution, and importing existing resources help
streamline complex workflows and infrastructure management.

35. Benefits of CI/CD Integration with Terraform

Integrating Terraform with CI/CD provides several key benefits:

• Automation: Infrastructure is provisioned automatically based on code changes, reducing

the manual intervention required.

• Consistency: Enforces a repeatable, reliable process for managing infrastructure, ensuring

that each deployment is consistent.

• Collaboration: Team members can collaborate effectively using version control systems (like
Git), ensuring that all changes are tracked and reviewed.

• Testing and Validation: CI/CD pipelines can validate the Terraform configuration (e.g., using
terraform plan and terraform validate) before applying changes, reducing the risk of
misconfigurations.
DevOps Shack
• Rollback: CI/CD integration can help roll back infrastructure changes if something goes
wrong during deployment.

36. Terraform with GitHub Actions

GitHub Actions is a powerful CI/CD tool that can automate workflows directly in your GitHub
repository. You can use GitHub Actions to trigger Terraform runs whenever a change is made to the
infrastructure code (for example, after a pull request is merged).

a. Setting Up GitHub Actions for Terraform

1. Create a GitHub Actions Workflow: GitHub Actions workflows are stored in the
.github/workflows/ directory of your repository. Example workflow file:
.github/workflows/terraform.yml

name: Terraform

on: push:
branches: -
main
pull_request:
branches:
- main

jobs:
terraform:
runs-on: ubuntu-latest

steps:
- name: Checkout repository
uses: actions/checkout@v2

- name: Set up Terraform

uses: hashicorp/setup-terraform@v1

- name: Initialize Terraform

run: terraform init

- name: Terraform Plan

run: terraform plan

- name: Terraform Apply

if: github.event_name == 'push'
run: terraform apply -auto-approve
In this example:
DevOps Shack
• The workflow is triggered by pushes or pull requests to the main branch.

• It checks out the repository, sets up Terraform, and runs terraform init and terraform plan.

• On a push to the main branch, terraform apply is automatically executed to apply the
changes.

2. Storing Secrets in GitHub: Sensitive information like access keys or API tokens should be
stored securely using GitHub Secrets. In your repository, go to Settings → Secrets and
variables → Actions and add your secrets.

b. Automating Plan and Apply

In a typical GitHub Actions workflow, terraform plan can run for every pull request to validate the
changes without actually applying them. terraform apply can run only when the changes are merged
into the main branch.

Example of separating plan and apply:

- name: Terraform Plan
run: terraform plan
if: github.event_name == 'pull_request'

- name: Terraform Apply run:

terraform apply -auto-approve if:
github.event_name == 'push'
This structure ensures that infrastructure changes are applied only after code has been reviewed and
merged.

37. Terraform with Jenkins

Jenkins is another popular CI/CD tool that can be used to automate Terraform workflows. You can
use Jenkins pipelines to validate, plan, and apply Terraform configurations.

a. Setting Up Jenkins for Terraform

1. Install the Terraform Plugin: Jenkins has a Terraform plugin that makes it easy to integrate
Terraform with Jenkins pipelines.

o Go to Manage Jenkins → Manage Plugins → Available, search for "Terraform", and

install the plugin.

2. Creating a Jenkins Pipeline: In Jenkins, create a new pipeline job and define your pipeline in
the Jenkinsfile.

Example Jenkinsfile for Terraform:

pipeline {
agent any

stages {
DevOps Shack
stage('Checkout') {
steps {
checkout scm
}
}
stage('Terraform Init') {
steps {
sh 'terraform init'
}
}
stage('Terraform Plan') {
steps {
sh 'terraform plan'

}
}
stage('Terraform Apply') {
when {
branch 'main'
}
steps {
sh 'terraform apply -auto-approve'
}
}
}
}
In this example:

• The pipeline checks out the code from the repository.

• It initializes Terraform using terraform init and validates the configuration with terraform
plan.

• terraform apply is only executed on the main branch.

3. Storing Credentials in Jenkins: Use Jenkins Credentials to securely store access keys and
tokens for your cloud provider. Go to Manage Jenkins → Manage Credentials and add your
credentials there. You can reference these credentials in the pipeline using environment
variables.

38. Terraform with GitLab CI

GitLab CI is the built-in CI/CD tool in GitLab, which can also be used to automate Terraform
workflows.

a. Setting Up GitLab CI for Terraform

1. Create a .gitlab-ci.yml File: GitLab CI configurations are defined in the .gitlab-ci.yml file located
in the root of your repository.

Example .gitlab-ci.yml file:

DevOps Shack
image: hashicorp/terraform:latest

stages: -
validate
- plan
- apply

before_script:
- terraform init

validate:
stage: validate
script:

- terraform validate

plan:
stage: plan
script:
- terraform plan
only:
- merge_requests

apply:
stage: apply
script:
- terraform apply -auto-approve
only:
- master
In this example:

• The pipeline has three stages: validate, plan, and apply.

• terraform validate runs in the validate stage.

• terraform plan runs only for merge requests to preview the changes.

• terraform apply runs only on the master branch, ensuring that changes are applied only after
being merged.

2. Managing Secrets in GitLab CI: You can store sensitive credentials using GitLab CI variables.
Go to your project’s Settings → CI/CD → Variables and add your cloud provider credentials
as variables.

39. Best Practices for CI/CD Pipelines with Terraform

a. Use Separate Environments for Dev, Staging, and Production

When deploying infrastructure via CI/CD, use separate pipelines or workspaces for different
environments (e.g., dev, staging, production). This ensures that changes can be tested in a staging
environment before being deployed to production.
DevOps Shack
Example: Using workspaces in Terraform: - name: Terraform Apply (Staging)

run: terraform workspace select staging && terraform apply -auto-approve

- name: Terraform Apply (Production) run: terraform workspace select

production && terraform apply -auto-approve

b. Implement Approval Gates

For production environments, consider adding an approval gate before running terraform apply. This
ensures that infrastructure changes are reviewed by a team member before being applied.

Example using GitHub Actions with manual approval:

jobs:
terraform:
runs-on: ubuntu-latest

steps:
- name: Terraform Apply
run: terraform apply -auto-approve
if: github.event_name == 'workflow_dispatch'
This configuration requires a manual trigger to apply changes, ensuring that production changes are
not applied automatically.

c. Use Automated Testing

Before applying infrastructure changes, run automated tests to validate the configuration. Use tools
like Terratest (as discussed in Module 8) or InSpec to ensure the infrastructure behaves as expected.

40. Conclusion

In this module, we covered how to integrate Terraform with CI/CD pipelines using GitHub Actions,
Jenkins, and GitLab CI. These tools enable you to automate infrastructure provisioning, ensure
consistency, and reduce manual intervention. By incorporating CI/CD into your Terraform workflows,
you can implement best practices such as automated testing, environment isolation, and approval
gates, making your infrastructure-as-code processes more reliable and efficient.

In the next module, we will cover Terraform Cost Estimation and Optimization Techniques, which
will help you ensure that your infrastructure is both cost-efficient and scalable.
DevOps Shack

45. Benefits of Multi-Cloud Deployments

Using multiple cloud providers can offer several advantages:

• Avoiding Vendor Lock-In: By relying on multiple providers, you are not locked into a single
cloud vendor, reducing risks such as price hikes or outages.

• Leveraging Best Features: Different cloud providers excel in specific areas (e.g., AWS for
compute, Azure for enterprise services, Google Cloud for machine learning). Multi-cloud
allows you to pick the best services from each provider.

• Resilience: Multi-cloud deployments improve disaster recovery and high availability by

distributing infrastructure across different cloud environments, ensuring minimal downtime
in the event of an outage in one provider.

However, managing multiple cloud environments can add complexity, which is why Terraform's
provider model simplifies the process by allowing consistent infrastructure management across
clouds.

46. Key Concepts for Multi-Cloud Deployments in Terraform

Terraform’s ability to work with multiple cloud providers simultaneously is powered by providers.
Each cloud provider (such as AWS, Azure, or Google Cloud) has its own Terraform provider plugin that
allows Terraform to interact with its APIs.

a. Multiple Providers in a Single Configuration

In a multi-cloud setup, you define multiple provider blocks in your Terraform configuration, each
corresponding to a specific cloud provider. You can then use resources from each provider in the
same configuration.

b. Provider Aliases

Terraform allows you to create provider aliases if you need to work with multiple regions, accounts,
or subscriptions within the same provider. This is particularly useful when deploying resources to
different environments (e.g., different AWS regions or Azure subscriptions).

47. Example: Deploying Resources Across AWS and Azure

Let’s explore a basic example where we deploy an EC2 instance on AWS and a virtual machine (VM)
on Azure, all from a single Terraform configuration.

a. Configuration Structure

Your directory structure might look like this:

/multi-cloud

main.tf

variables.tf

outputs.tf
DevOps Shack

b. Provider Configuration

In the main.tf file, you define the providers for both AWS and Azure. You can also specify multiple
accounts or regions using provider aliases.

Step 1: Configure AWS and Azure Providers

# AWS Provider Configuration
provider "aws" {
region = var.aws_region
access_key = var.aws_access_key
secret_key = var.aws_secret_key
}

# Azure Provider Configuration provider

"azurerm" {
features {}
subscription_id = var.azure_subscription_id
client_id = var.azure_client_id client_secret
= var.azure_client_secret
tenant_id = var.azure_tenant_id
}
In this example:

• We are using both the AWS and Azure providers.

• The aws provider deploys resources to the specified AWS region.

• The azurerm provider deploys resources using credentials for an Azure subscription.

c. Defining Resources in AWS and Azure

Once the providers are defined, you can declare resources from both providers. For example, let’s
create an EC2 instance in AWS and an Azure Virtual Machine. Step 2: Create an AWS EC2
Instance resource "aws_instance" "my_aws_instance" {

ami = "ami-0c55b159cbfafe1f0"

instance_type = "t2.micro"

tags = {

Name = "AWS-Instance"

Step 3: Create an Azure Virtual Machine

DevOps Shack
resource "azurerm_resource_group" "my_rg" {
name = "myResourceGroup"
location = "East US"
}

resource "azurerm_virtual_network" "my_vnet" { name

= "myVNet" address_space = ["10.0.0.0/16"] location
= azurerm_resource_group.my_rg.location
resource_group_name = azurerm_resource_group.my_rg.name
}

resource "azurerm_network_interface" "my_nic" {

name = "myNIC"
location = azurerm_resource_group.my_rg.location
resource_group_name = azurerm_resource_group.my_rg.name

ip_configuration {
name = "myNICConfig"
subnet_id = azurerm_virtual_network.my_vnet.subnets[0].id
private_ip_address_allocation = "Dynamic"
}
}

resource "azurerm_windows_virtual_machine" "my_vm" {

name = "myWindowsVM"
resource_group_name = azurerm_resource_group.my_rg.name
location = azurerm_resource_group.my_rg.location
size = "Standard_DS1_v2"
admin_username = "adminuser"
admin_password = "P@ssw0rd123!"
network_interface_ids = [azurerm_network_interface.my_nic.id]
os_disk {
caching = "ReadWrite"
storage_account_type = "Standard_LRS"
}
}
In this example:

• An AWS EC2 instance is created in the aws_instance block.

• A Windows Virtual Machine is created on Azure, including resources like a resource group,
virtual network, and network interface.

d. Outputting Resource Information

Once the resources are created, you can use the outputs block to display relevant information, such
as the public IP addresses of both instances.
DevOps Shack
output "aws_instance_public_ip" { value =

aws_instance.my_aws_instance.public_ip

output "azure_vm_public_ip" { value =

azurerm_windows_virtual_machine.my_vm.public_ip_address

e. Apply the Configuration

To deploy these resources to both AWS and Azure, run the following commands:

terraform init # Initialize the providers

terraform plan # Review the plan terraform

apply # Deploy the infrastructure

This command will simultaneously deploy the EC2 instance on AWS and the virtual machine on
Azure.

48. Managing Multi-Cloud Credentials

Managing credentials for multiple cloud providers is critical for a secure multi-cloud deployment. You
should avoid hardcoding sensitive credentials in your Terraform configuration. Instead, use
environment variables or secure credential storage services such as AWS Secrets Manager, Azure
Key Vault, or Google Secret Manager.

a. Using Environment Variables

You can pass cloud provider credentials securely through environment variables.

Example for AWS and Azure: export

AWS_ACCESS_KEY_ID="your-aws-access-key" export

AWS_SECRET_ACCESS_KEY="your-aws-secret-key" export

ARM_CLIENT_ID="your-azure-client-id" export

ARM_CLIENT_SECRET="your-azure-client-secret" export

ARM_TENANT_ID="your-azure-tenant-id"

Terraform will automatically pick up these credentials when the provider blocks are executed.

b. Using Cloud Secrets Management

DevOps Shack
Alternatively, you can integrate Terraform with secrets management tools like AWS Secrets Manager
or Azure Key Vault to fetch sensitive information dynamically. Example: Retrieving secrets from AWS
Secrets Manager:

data "aws_secretsmanager_secret_version" "db_password" {

secret_id = "db_password"

resource "aws_db_instance" "my_db" {

allocated_storage = 20

engine = "mysql" instance_class = "db.t2.micro" password =

data.aws_secretsmanager_secret_version.db_password.secret_string

49. Best Practices for Multi-Cloud Deployments

a. Consistency Across Cloud Providers

When managing multi-cloud environments, aim to maintain consistency in the way resources are
named, tagged, and configured. This helps with resource management, cost tracking, and
automation.

Example: Using consistent tagging across providers:

resource "aws_instance" "my_instance" {

tags = {

Environment = "production"

Project = "multi-cloud-demo"

resource "azurerm_virtual_machine" "my_vm" {

tags = {

Environment = "production"

Project = "multi-cloud-demo"

}
DevOps Shack
}

b. Monitoring and Observability

Ensure that you have a unified monitoring and observability solution across cloud providers. Many
third-party tools, like Prometheus, Grafana, and Datadog, support multi-cloud monitoring. c.
Centralized Logging

Use tools like ELK Stack (Elasticsearch, Logstash, Kibana), Azure Monitor, or AWS CloudWatch to
collect and analyze logs from different cloud providers in a single location. This simplifies
troubleshooting in multi-cloud environments.

50. Conclusion

In this module, we explored how to manage Multi-Cloud Deployments with Terraform. By leveraging
Terraform's provider architecture, you can manage infrastructure across multiple cloud providers like
AWS, Azure, and Google Cloud from a single configuration. We discussed key concepts such as
multiple providers, managing credentials securely, and deploying resources across clouds with
consistency. Multi-cloud deployments offer flexibility, resilience, and reduced vendor lock-in, but
they also require careful management and planning

51. Importance of Disaster Recovery

Disaster recovery focuses on ensuring business continuity during disasters such as data center
outages, network failures, natural disasters, or cyber-attacks. A well-defined DR strategy minimizes
downtime and data loss, ensuring that services are restored quickly after a failure. Key aspects of DR
include:

• High Availability (HA): Ensures services are always available by distributing them across
multiple regions or zones.

• Data Backup and Restore: Regular backups of databases, file systems, and application data
allow recovery in case of corruption or data loss.

• Failover and Failback: The ability to switch from a failed primary system to a secondary
system (failover) and back (failback) once the issue is resolved.

Terraform can automate many of these tasks, ensuring that infrastructure and services are easily
replicated or restored in case of failure.

52. Key Concepts for Disaster Recovery with Terraform

a. Multi-Region and Multi-Zone Deployments

One of the most common approaches to disaster recovery is to deploy resources across multiple
regions or availability zones. In case of a failure in one region, traffic is redirected to another region
where the same infrastructure is running.

b. Data Backup and Replication

Regular data backups are essential for disaster recovery. Terraform can manage cloud-native backup
services such as Amazon S3, Azure Backup, and Google Cloud Storage to automatically back up your
data to secure storage locations.
DevOps Shack
c. Automated Failover

Automated failover ensures that if a primary system fails, traffic is automatically redirected to a
backup system. This can be achieved using tools like AWS Route 53, Azure Traffic Manager, or
Google Cloud Load Balancing to direct traffic based on availability.

53. Multi-Region Deployment with Terraform

In a multi-region setup, you deploy the same infrastructure in multiple regions to ensure that
services remain available even if one region experiences an outage.

a. Example: Deploying a Multi-Region Setup in AWS

Let’s consider an example where we deploy an application in two AWS regions. We will use Route 53
for DNS failover, ensuring that traffic is directed to the available region in case one goes down.

Step 1: Define AWS Providers for Multiple Regions

provider "aws" {
region = "us-east-1"
alias = "us_east"
}

provider "aws" {
region = "us-west-1"
alias = "us_west"
}
In this example, we define two AWS providers with aliases (us_east and us_west) to handle resources
in different regions.

Step 2: Deploy EC2 Instances in Multiple Regions

# EC2 Instance in US-East resource

"aws_instance" "web_us_east" {

provider = aws.us_east ami = "ami-

0c55b159cbfafe1f0" instance_type =

"t2.micro"

tags = {

Name = "WebServer-East"

}
DevOps Shack
# EC2 Instance in US-West resource

"aws_instance" "web_us_west" {

provider = aws.us_west ami = "ami-

0c55b159cbfafe1f0" instance_type =

"t2.micro"

tags = {
Name = "WebServer-West"

In this example:

• An EC2 instance is deployed in both the US-East and US-West regions.

• Both instances serve the same application, ensuring high availability.

Step 3: Configure Route 53 for DNS Failover

resource "aws_route53_zone" "example" {

name = "example.com"
}

resource "aws_route53_record" "primary" {

zone_id = aws_route53_zone.example.zone_id
name = "www.example.com"
type = "A"

alias {
name = aws_instance.web_us_east.public_ip
zone_id = aws_instance.web_us_east.id
evaluate_target_health = true
}

failover_routing_policy {
type = "PRIMARY"
}
}

resource "aws_route53_record" "secondary" {

zone_id = aws_route53_zone.example.zone_id
name = "www.example.com" type = "A"

alias {
name = aws_instance.web_us_west.public_ip
zone_id = aws_instance.web_us_west.id
evaluate_target_health = true
}
DevOps Shack
failover_routing_policy {
type = "SECONDARY"
}
}
In this example:

• Route 53 is configured with failover routing.

• The primary instance in US-East is used by default, and if it becomes unhealthy, traffic is
automatically redirected to the secondary instance in US-West.

54. Data Backup and Replication with Terraform

Backup and replication are key elements of a disaster recovery strategy. Cloud providers offer built-in
tools for backing up data, such as AWS RDS snapshots, Azure Backup, and Google Cloud Persistent
Disk Snapshots.

a. Automating Backups with Terraform

Let’s automate the process of backing up an AWS RDS database using Terraform. Example:

Creating RDS Snapshots

resource "aws_db_instance" "mydb" {

allocated_storage = 20 engine
= "mysql" instance_class =
"db.t2.micro"
name = "mydb"
username = "admin"
password = "password"
backup_retention_period = 7 # Retain backups for 7 days
backup_window = "01:00-03:00"
}

resource "aws_db_snapshot" "mydb_snapshot" {

db_instance_identifier = aws_db_instance.mydb.id
db_snapshot_identifier = "mydb-snapshot-${timestamp()}"
}
In this example:

• An RDS instance is created with automatic backups enabled, retaining them for 7 days.

• A manual RDS snapshot is also created on-demand using Terraform, and the snapshot is
timestamped.

b. Automating Storage Backups (Amazon S3)

For object storage, such as Amazon S3, you can use lifecycle rules to automatically transition data to
cheaper storage classes (e.g., Glacier) or delete old data after a certain period. Example: S3 Lifecycle
Rules
resource "aws_s3_bucket" "my_bucket" {
bucket = "my-backup-bucket"
DevOps Shack

lifecycle_rule {
enabled = true

transition {
days
= 30
storage_class
= "GLACIER" #
Move data to
Glacier after 30
days
}

expiration {
days = 365 # Delete objects after 365 days
}
}
}

In this example:

• S3 lifecycle rules automatically transition data to Glacier for long-term, low-cost storage after
30 days.

• Objects are deleted after 365 days to reduce storage costs and ensure data is not kept
indefinitely.

55. Implementing Disaster Recovery for Kubernetes with Terraform

Kubernetes clusters are often used for deploying containerized applications, and disaster recovery is
crucial to ensure cluster availability. Terraform can be used to create highly available Kubernetes
clusters across multiple regions or zones.

a. Example: Deploying Kubernetes Cluster Across Multiple AWS Availability Zones Step
1: Create EKS Cluster in Multi-AZ
resource "aws_eks_cluster" "my_cluster" {
name = "my-eks-cluster"
role_arn = aws_iam_role.eks_role.arn

vpc_config {
subnet_ids = aws_subnet.my_subnets[*].id
}
}
DevOps Shack
resource "aws_eks_node_group" "my_node_group" {
cluster_name = aws_eks_cluster.my_cluster.name
node_role = aws_iam_role.node_role.arn
subnet_ids = aws_subnet.my_subnets[*].id
scaling_config { desired_size = 3 max_size = 5
min_size = 1
}
}
In this example:

• A highly available EKS cluster is deployed across multiple AWS availability zones, ensuring that
the Kubernetes control plane and worker nodes remain available in case of failure in one zone.

56. Best Practices for Disaster Recovery with Terraform

a. Regularly Test Your DR Plan

Ensure that your disaster recovery plan is tested regularly by simulating outages and ensuring that
failover, backups, and restores work as expected.

b. Automate Backup Processes

Automate the backup process using cloud-native tools and Terraform. Regular, automated backups
reduce the risk of human error and ensure that the latest data is available for recovery.

c. Monitor and Optimize Costs

Monitor the costs associated with disaster recovery, especially in multi-region deployments. Use cost
management tools like AWS Budgets or Azure Cost Management to ensure DR solutions are
optimized without overspending.

57. Conclusion

In this module, we covered how to implement Disaster Recovery Strategies using Terraform. We
explored the importance of multi-region deployments, data backup and replication, and failover
mechanisms to ensure high availability and minimize downtime during disasters. By automating DR
strategies with Terraform, you can create a robust and scalable infrastructure that can withstand
outages and recover quickly.

In the next module, we will explore Terraform in Hybrid Cloud Deployments, focusing on how to
integrate on-premises infrastructure with cloud environments.

58. Understanding Hybrid Cloud Deployments

A hybrid cloud consists of a mix of on-premises data centers, private clouds, and public clouds (such
as AWS, Azure, or Google Cloud). Organizations use hybrid cloud architectures to:

• Maintain Control over sensitive data or regulatory requirements by keeping certain

workloads on-premises.
DevOps Shack
• Leverage Cloud Resources for workloads that need to scale dynamically, such as
customerfacing applications or burst compute needs.

• Optimize Costs by using cloud services for non-critical workloads and on-premises resources
for core business systems.

Managing hybrid cloud environments can be complex, but Terraform simplifies this by providing a
unified approach to infrastructure-as-code across cloud and on-premises environments.

59. Key Concepts in Hybrid Cloud Deployments

a. Extending On-Premises Infrastructure to the Cloud

One of the key goals of a hybrid cloud is to extend on-premises resources into the cloud. For
example, you can extend an on-premises network into AWS or Azure using VPN or Direct Connect. b.
Consistent Management

Terraform allows you to use consistent configurations across cloud providers and on-premises
infrastructure, providing a single pane of glass for managing the entire infrastructure. c.
Multi-Cloud Integration

In many hybrid cloud setups, different parts of the infrastructure may be hosted in different clouds,
such as AWS for computing power and Azure for database services. Terraform’s multi-provider
support makes it easy to manage these resources within a single configuration.

60. Hybrid Cloud Architecture Using Terraform

In a typical hybrid cloud architecture, we might have the following components:

• On-Premises Data Center: Hosts legacy applications or systems with strict security or
compliance requirements.

• Cloud Provider (AWS/Azure): Hosts scalable workloads like web applications or cloud-native
services.

• Network Integration: Connects the on-premises network with the cloud provider’s network
via VPN, Direct Connect, or ExpressRoute.

a. Example: Extending On-Premises Network to AWS

Let’s walk through an example of setting up a hybrid cloud where an on-premises network is
extended to AWS using AWS Site-to-Site VPN.

Step 1: Define the On-Premises Network

For this example, we assume that an on-premises network with a private subnet exists, and we want
to extend this network to AWS by creating a VPN connection between the on-premises environment
and an AWS VPC.

Step 2: AWS Provider Configuration

We begin by configuring the AWS provider in our Terraform configuration.

DevOps Shack
provider "aws" {

region = "us-west-2" }

Step 3: Create an AWS VPC

resource "aws_vpc" "my_vpc" {

cidr_block = "10.0.0.0/16" tags

={

Name = "HybridCloudVPC"

resource "aws_subnet" "my_subnet" {

vpc_id = aws_vpc.my_vpc.id

cidr_block = "10.0.1.0/24"

availability_zone = "us-west-2a"

In this configuration:

• We create a VPC with a CIDR block of 10.0.0.0/16.

• A subnet is created in the VPC for the application workloads. Step 4: Create

a Virtual Private Gateway (VGW) resource "aws_vpn_gateway" "my_vgw" {

vpc_id = aws_vpc.my_vpc.id

A Virtual Private Gateway (VGW) is created and attached to the AWS VPC. This acts as the
connection point for the VPN.

Step 5: Define the VPN Connection to On-Premises

resource "aws_customer_gateway" "on_prem_gateway" {

bgp_asn = 65000
ip_address = "1.2.3.4" # On-premises gateway public IP
type = "ipsec.1"
}

resource "aws_vpn_connection" "vpn_conn" {

customer_gateway_id = aws_customer_gateway.on_prem_gateway.id
type = "ipsec.1" static_routes_only = true

tags = {
Name = "OnPremToAWS-VPN"
DevOps Shack
}
}

resource "aws_vpn_connection_route" "route" {

vpn_connection_id = aws_vpn_connection.vpn_conn.id
destination_cidr_block = "192.168.1.0/24" # On-premises CIDR block
}
In this configuration:

• The Customer Gateway defines the on-premises VPN gateway with its public IP address and
ASN.

• The VPN connection is established between the AWS VPC and the on-premises network.

• A VPN connection route is created to route traffic between the AWS VPC and the
onpremises CIDR block.

b. Example: Extending On-Premises Network to Azure Using ExpressRoute

In this example, we extend an on-premises network to Azure using Azure ExpressRoute, which
provides a private connection between your on-premises network and Azure datacenters.
Step 1: Azure Provider Configuration provider "azurerm" {

features {}

Step 2: Create Azure Virtual Network resource

"azurerm_virtual_network" "my_vnet" {

name = "my-vnet"

address_space = ["10.1.0.0/16"]

location = "West US"

resource_group_name = azurerm_resource_group.my_rg.name

This creates an Azure Virtual Network (VNet) that will host workloads in Azure.

Step 3: Create ExpressRoute Circuit resource "azurerm_express_route_circuit"

"my_circuit" {

name = "my-expressroute-circuit"

location = "West US"

resource_group_name = azurerm_resource_group.my_rg.name

service_provider_name = "Equinix"
DevOps Shack
peering_location = "Silicon Valley"

bandwidth_in_mbps = 1000

sku { tier =

"Standard"

family = "MeteredData"

In this example:

• An ExpressRoute circuit is created with a service provider (e.g., Equinix) to

connect the onpremises network to Azure.

• The circuit has a bandwidth of 1 Gbps and uses metered data. Step 4:

Configure VPN Gateway and Connection resource

"azurerm_virtual_network_gateway" "vpn_gateway" {

name = "vpn-gateway"

location = "West US"

resource_group_name = azurerm_resource_group.my_rg.name

type = "ExpressRoute"

vpn_type = "RouteBased"

sku = "Standard"

This creates a Virtual Network Gateway to handle the ExpressRoute connection on Azure.

61. Managing Hybrid Cloud Resources with Terraform

a. Managing On-Premises Resources

Terraform has providers that support on-premises resources, such as VMware vSphere, OpenStack,
and Kubernetes. These providers allow you to define and manage on-premises VMs, storage, and
networking using the same approach as for cloud resources.

Example: Managing a VMware vSphere VM:

provider "vsphere" {
vsphere_server = "vsphere.local"
user = "myuser"
DevOps Shack
password = "mypassword"
allow_unverified_ssl = true
}

resource "vsphere_virtual_machine" "vm" {

name = "terraform-vm"
resource_pool_id = data.vsphere_resource_pool.pool.id
datastore_id = data.vsphere_datastore.datastore.id

num_cpus = 2
memory = 4096
guest_id = "rhel7_64Guest"

network_interface {
network_id = data.vsphere_network.network.id
adapter_type = "vmxnet3"
}

disk {
label = "disk0"
size = 40
thin_provisioned = true
}
}
In this example:

• Terraform manages a VMware vSphere VM, including its CPU, memory, storage, and network
configuration.

• This resource can be integrated into a broader hybrid cloud strategy by linking it with cloud
providers.

62. Best Practices for Hybrid Cloud with Terraform

a. Centralized Logging and Monitoring

Ensure that logging and monitoring for both on-premises and cloud resources are centralized. Tools
like AWS CloudWatch, Azure Monitor, and Prometheus can help collect and analyze logs from both
environments.

b. Consistent Tagging and Resource Management

Apply consistent tagging and naming conventions across cloud and on-premises environments to
simplify resource tracking, cost management, and automation.

c. Secure Network Connections

Ensure that VPN or direct connections between on-premises and cloud environments are secure,
using encryption protocols and firewalls to protect data in transit.

63. Conclusion
DevOps Shack
In this module, we explored how to manage Hybrid Cloud Deployments using Terraform. We
covered how to extend on-premises networks to the cloud using VPN and ExpressRoute, how to
manage on-premises resources using providers like VMware vSphere, and best practices for hybrid
cloud environments. By using Terraform to manage both cloud and on-premises infrastructure, you
can ensure consistency, scalability, and security across your hybrid cloud architecture.

64. Challenges of Running Terraform in Production

When using Terraform to manage production infrastructure, there are several key challenges to
address:

• State Management: Terraform state contains sensitive information and should be handled
securely.

• Security: Access controls, secret management, and audit logs are critical to protecting
production infrastructure.

• Collaboration: Teams need a way to collaborate without risking conflicts in infrastructure

updates.

• Automation: Production environments require automation to reduce human error and

enforce consistency in deployments.

• Monitoring and Logging: Monitoring changes and logging activities is important to track
issues and ensure accountability.

Terraform provides tools and integrations to help overcome these challenges, enabling safe and
scalable production deployments.

65. State Management in Production

Terraform’s state file tracks the current state of your infrastructure and allows Terraform to
determine what changes need to be applied. In a production environment, managing the state
securely and ensuring consistency across teams is crucial.

a. Remote State Storage

In production, Terraform state should be stored remotely using backends like Amazon S3, Azure Blob
Storage, or Terraform Cloud. Remote state allows for better collaboration, versioning, locking, and
security.

Example: Storing State in Amazon S3 terraform

{
DevOps Shack
backend "s3" { bucket = "my-
terraform-state-prod" key =
"global/s3/terraform.tfstate" region
= "us-west-1" encrypt = true
dynamodb_table = "terraform-lock"
}
}
In this configuration:

• The state is stored in an S3 bucket with encryption enabled.

• DynamoDB is used to implement state locking, preventing multiple users from making
changes to the state at the same time.

b. Locking and Versioning

State locking prevents simultaneous operations on the state file, ensuring that only one Terraform
execution can modify the infrastructure at any given time. This is crucial in production environments
to avoid race conditions or conflicts.

Versioning allows you to roll back to previous states if needed, helping to recover from mistakes or
unexpected issues.

In the above example:

• DynamoDB is used for locking.

• S3 supports versioning, allowing you to track changes to the state file.

66. Security Considerations in Terraform Production

a. Role-Based Access Control (RBAC)

In a production environment, it's essential to implement Role-Based Access Control (RBAC) to

restrict who can apply changes to the infrastructure. This can be done using cloud provider-specific
IAM policies or Terraform Enterprise’s RBAC features.

Example: AWS IAM Policy for Terraform Users

{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:PutObject",
"s3:ListBucket"
],
"Resource": "arn:aws:s3:::my-terraform-state-prod/*"
},
DevOps Shack
{
"Effect": "Allow",
"Action": "dynamodb:*",
"Resource": "arn:aws:dynamodb:us-west-1:123456789012:table/terraform-lock"
}
]
}
This IAM policy grants a Terraform user permission to interact with the S3 bucket that stores the state
file and the DynamoDB table used for state locking.

b. Secret Management

Secrets such as API keys, passwords, and credentials should never be hardcoded in your Terraform
configuration. Instead, you can use cloud-native secret management solutions like AWS Secrets
Manager, Azure Key Vault, or HashiCorp Vault to store and retrieve sensitive information. Example:
Retrieving Secrets from AWS Secrets Manager

data "aws_secretsmanager_secret_version" "db_password" {

secret_id = "my-database-password"
}

resource "aws_db_instance" "example" {

allocated_storage = 20 engine
= "mysql" instance_class =
"db.t2.micro" name =
"exampledb" username =
"admin"
password = data.aws_secretsmanager_secret_version.db_password.secret_string
}
In this example, the database password is retrieved dynamically from AWS Secrets Manager, ensuring
that sensitive information is not exposed in the Terraform code.

67. Collaboration and Version Control in Terraform

In production environments, multiple teams often collaborate on infrastructure changes. To avoid

conflicts and ensure accountability, it’s essential to use version control systems like Git along with
remote state backends and locking mechanisms.

a. Git Workflow for Terraform

Using Git for version control ensures that changes to infrastructure are tracked, reviewed, and
approved before being applied.

Example Workflow:
DevOps Shack
1. Create a Feature Branch: Create a new branch for any infrastructure changes.

git checkout -b add-new-ec2-instance

2. Make Changes to Terraform Configuration: Add new resources or modify existing ones.

3. Run terraform plan: Generate an execution plan to see what changes will be applied.

4. Submit a Pull Request: Once the changes are ready, submit a pull request for review.

5. Merge and Apply: After approval, merge the pull request into the main branch and apply the
changes using Terraform.

68. CI/CD Pipelines for Terraform

Automating Terraform workflows through CI/CD pipelines ensures that infrastructure is deployed
consistently and with minimal human intervention. You can integrate Terraform into CI/CD pipelines
using tools like Jenkins, GitHub Actions, or GitLab CI.

a. GitHub Actions for Terraform

Here’s an example of a GitHub Actions workflow that automates the Terraform plan and apply
process.

name: Terraform

on:
pull_request:
branches: -
main push:
branches:
- main

jobs:
terraform:
runs-on: ubuntu-latest

steps:
- name: Checkout repository
uses: actions/checkout@v2

- name: Set up Terraform

uses: hashicorp/setup-terraform@v1

- name: Initialize Terraform

run: terraform init
DevOps Shack
- name: Terraform Plan
run: terraform plan

- name: Terraform Apply if: github.ref == 'refs/heads/main' run:

terraform apply -auto-approve In this workflow:

• terraform plan is run for every pull request to validate changes.

• terraform apply is triggered when changes are pushed to the main branch.

69. Monitoring and Logging in Production

Monitoring and logging are critical for tracking the status of your infrastructure and identifying
potential issues in production. Many cloud providers offer native monitoring tools that can be
integrated with Terraform.

a. Monitoring with AWS CloudWatch

You can configure AWS CloudWatch to monitor key metrics and send alerts when infrastructure
behaves unexpectedly.

Example: Creating a CloudWatch Alarm for EC2

resource "aws_cloudwatch_metric_alarm" "cpu_alarm" {

alarm_name = "high-cpu-usage"
comparison_operator = "GreaterThanThreshold"
evaluation_periods = 2
metric_name = "CPUUtilization"
namespace = "AWS/EC2"
period = 60 statistic
= "Average" threshold
= 80
alarm_description = "This metric monitors high CPU usage on the EC2 instance."
dimensions = {
InstanceId = aws_instance.my_instance.id
}

alarm_actions = [aws_sns_topic.my_topic.arn]
}
In this example, an AWS CloudWatch alarm is created to monitor EC2 CPU utilization. If CPU usage
exceeds 80%, an alert is sent to an SNS topic.

70. Best Practices for Running Terraform in Production

a. Infrastructure as Code (IaC) Principles

Follow best practices for Infrastructure as Code (IaC):

• Idempotency: Ensure that running the same Terraform code multiple times produces the
same result.

• Versioning: Use Git to track changes to your Terraform code and enable rollbacks.
DevOps Shack
• Modularity: Break your Terraform configuration into reusable modules to simplify
management and ensure consistency.

b. Automated Testing

Automate testing of your Terraform code using tools like Terratest to validate that infrastructure
behaves as expected before it is deployed to production.

71. Conclusion

In this module, we covered how to run Terraform in Production. We discussed critical aspects such as
state management, security considerations, collaboration through version control, and automation
using CI/CD pipelines. Implementing these practices helps ensure that your production infrastructure
is deployed and managed securely, consistently, and with minimal downtime.

72. Understanding Compliance in Infrastructure as Code

Compliance in Infrastructure as Code (IaC) involves ensuring that the infrastructure you deploy
adheres to various security and regulatory requirements. This includes:

• Security Compliance: Ensuring infrastructure is secure by enforcing best practices such as

network segmentation, encryption, and access controls.

• Regulatory Compliance: Adhering to legal and industry-specific regulations such as GDPR,

HIPAA, SOC 2, PCI DSS, and others.

• Operational Compliance: Implementing internal policies related to resource management,

cost control, and operational efficiency.

Terraform provides mechanisms to enforce compliance rules through code, helping to automate and
audit compliance efforts across cloud environments.

73. Key Compliance Challenges in Terraform

a. Configuration Drift

Configuration drift occurs when the actual state of your infrastructure deviates from the desired state
defined in your Terraform code. This can happen when changes are made manually outside of
Terraform’s control, leading to compliance violations.

b. Access Control and Secret Management

Ensuring that access to sensitive resources (e.g., databases, S3 buckets) is restricted and that secrets
(e.g., API keys, credentials) are securely stored and managed is a key compliance requirement. c.
Regulatory Reporting

Many regulations require organizations to generate reports demonstrating compliance with specific
policies. Keeping track of infrastructure changes and ensuring that they meet regulatory
requirements can be difficult without automation.

74. Enforcing Compliance with Terraform

DevOps Shack
a. Policy as Code (PaC)

Policy as Code (PaC) is a methodology where compliance policies are written in code, much like
Infrastructure as Code (IaC). This approach allows you to enforce compliance rules across your
infrastructure automatically.

Terraform integrates with several Policy as Code tools to ensure that infrastructure adheres to
security, operational, and regulatory standards.

b. Using Sentinel for Policy as Code

Sentinel is HashiCorp’s Policy as Code framework designed to work with Terraform. It allows you to
write policies that are evaluated before any infrastructure changes are applied. Example: Sentinel
Policy to Restrict AWS Instance Sizes

import "tfplan/v2" as tfplan

main = rule {
all tfplan.resource_changes as _, rc {
rc.change.after.instance_type is "t2.micro" or
rc.change.after.instance_type is "t2.small"
}
}
In this example:

• The Sentinel policy ensures that only t2.micro or t2.small instance types are allowed.

• If any attempt is made to deploy an instance with a larger size, the policy will block the
change.

Sentinel policies can be used to enforce a wide range of compliance requirements, from restricting
resource types to enforcing tagging standards or ensuring encryption.

c. Open Policy Agent (OPA) for Terraform

Open Policy Agent (OPA) is another popular open-source policy engine that can be used to enforce
compliance rules for Terraform. OPA is more general-purpose than Sentinel and can be used with
many tools and systems.

Example: OPA Policy to Require Encryption for S3 Buckets

package terraform.s3

deny[msg] {
input.resource_type == "aws_s3_bucket"
input.config.server_side_encryption_configuration == null
msg = "S3 bucket must have encryption enabled."

}
This OPA policy denies any S3 bucket that does not have server-side encryption enabled, ensuring
compliance with security best practices.
DevOps Shack

75. Automating Compliance Checks in CI/CD Pipelines

You can integrate compliance checks into your CI/CD pipelines using Terraform. By automating these
checks, you ensure that infrastructure changes are reviewed against compliance policies before they
are applied.

a. GitHub Actions Example for OPA Compliance

Here’s how you can integrate OPA into a GitHub Actions pipeline to check for compliance before
applying changes.

name: Terraform Compliance Check

on:
pull_request:
branches:
- main

jobs:
terraform:
runs-on: ubuntu-latest

steps:
- name: Checkout repository
uses: actions/checkout@v2

- name: Set up Terraform

uses: hashicorp/setup-terraform@v1

- name: OPA Policy Check

run: opa eval --data policy.rego --input terraform-plan.json --format pretty
In this example:

• The pipeline evaluates Terraform plan outputs using OPA policies.

• If the policies are violated (e.g., an S3 bucket without encryption), the pipeline will fail,
preventing the change from being applied.

76. Managing Sensitive Data and Secrets in Terraform

Managing sensitive data, such as API keys, database credentials, and access tokens, is critical for
maintaining compliance. Terraform integrates with various secret management tools to ensure that
sensitive information is securely stored and accessed.

a. AWS Secrets Manager Example

DevOps Shack
Using AWS Secrets Manager, you can store sensitive data securely and retrieve it dynamically during
Terraform runs.

data "aws_secretsmanager_secret" "db_password" {

name = "my-database-password" }

resource "aws_db_instance" "my_db" {

allocated_storage = 20 engine
= "mysql" instance_class =
"db.t2.micro"
password = data.aws_secretsmanager_secret.db_password.secret_string
}
This ensures that the password is not stored in the Terraform configuration but retrieved securely at
runtime.

77. Logging and Auditing for Compliance

To meet regulatory requirements, organizations often need to keep detailed logs of infrastructure
changes and access to sensitive resources. Terraform can integrate with logging and auditing tools to
provide the necessary visibility.

a. AWS CloudTrail for Terraform Logging

AWS CloudTrail can be used to log all API requests made by Terraform, including who applied the
changes and what resources were modified.

resource "aws_cloudtrail" "my_trail" {

name = "my-trail"
s3_bucket_name = aws_s3_bucket.my_bucket.id
include_global_service_events = true is_multi_region_trail
= true
enable_logging = true
}
This ensures that all Terraform activity is logged, providing an audit trail that meets compliance
requirements for regulations like SOC 2 and PCI DSS.

b. Azure Monitor for Auditing

On Azure, Azure Monitor can track and log infrastructure changes, helping to ensure compliance
with security and operational policies.
resource "azurerm_monitor_diagnostic_setting" "example" {
name = "example-setting"
target_resource_id = azurerm_storage_account.example.id

log {
category = "Administrative"
enabled = true
retention_policy { enabled =
true
days = 365
DevOps Shack
}
}
}
This Azure Monitor configuration logs all administrative actions (e.g., resource creation, deletion)
and retains them for 365 days.

78. Best Practices for Terraform Compliance

a. Implement Continuous Compliance

Ensure that compliance checks are integrated into every stage of your infrastructure lifecycle. By
running automated compliance checks in CI/CD pipelines and during infrastructure updates, you
reduce the risk of non-compliant resources being deployed.

b. Use Policy as Code (PaC) Across Teams

Standardize compliance rules using Policy as Code frameworks like Sentinel or OPA. These policies
should be reviewed regularly and updated as compliance requirements evolve.

c. Regular Audits and Monitoring

Set up regular audits of your infrastructure using logging and monitoring tools like CloudTrail, Azure
Monitor, or Google Cloud Logging. Regular audits ensure that your infrastructure adheres to
compliance standards and that any unauthorized changes are detected early.

79. Conclusion

In this module, we covered how to ensure Compliance with Terraform. By integrating compliance
checks through Policy as Code frameworks like Sentinel and OPA, managing sensitive data securely,
and setting up automated logging and auditing, you can ensure that your infrastructure adheres to
security, regulatory, and operational standards. Automating compliance checks and continuously
monitoring your infrastructure helps prevent violations, keeping your environment secure and
compliant.

80. Importance of Disaster Recovery and High Availability

Disaster Recovery (DR) and High Availability (HA) are critical to ensuring that your infrastructure is
resilient and can continue to function during outages or failures. These strategies focus on
minimizing downtime, data loss, and service disruptions in various failure scenarios:

• Disaster Recovery (DR): Ensures that critical infrastructure and data can be restored in the
event of a catastrophic failure (e.g., natural disasters, data center outages).

• High Availability (HA): Ensures that applications and services remain accessible, even when
parts of the system fail, by using redundancy and failover mechanisms.

Terraform allows you to define your DR and HA strategies as code, ensuring consistency and
repeatability when deploying resilient infrastructure.

81. Key Concepts in Disaster Recovery and High Availability

DevOps Shack
a. Multi-Region Deployments

One of the core strategies for DR and HA is deploying infrastructure across multiple regions. This
ensures that if one region goes down, traffic can be routed to another region where the same
infrastructure is deployed.

b. Replication and Failover

To ensure data integrity and service continuity, resources like databases, storage, and applications
need to be replicated across multiple regions. Failover mechanisms are put in place to automatically
switch to backup systems if the primary system fails.

c. Backup and Restore

Automating backups of critical infrastructure, such as databases and storage, ensures that in the
event of a disaster, data can be restored quickly and accurately. Terraform can help manage and
automate backup schedules and recovery.

82. Multi-Region Deployment with Terraform

To achieve high availability and disaster recovery, deploying your infrastructure across multiple
regions is a common approach. In this section, we will create a multi-region setup using AWS as an
example, deploying resources in two regions and setting up failover mechanisms.

a. AWS Multi-Region Architecture

In a multi-region setup, we’ll deploy an application across two AWS regions: US-East and US-West.
The infrastructure will include an EC2 instance in each region, a RDS database, and Route 53 for DNS
failover.

Step 1: Define AWS Providers for Multiple Regions

provider "aws" {
region = "us-east-1"
alias = "us_east"
}

provider "aws" {
region = "us-west-2"
alias = "us_west"
}
In this example, we define two AWS providers with aliases for the US-East and US-West regions.

Step 2: Deploy EC2 Instances in Multiple Regions

# EC2 instance in US-East

resource "aws_instance" "web_us_east" {
provider = aws.us_east
ami = "ami-0c55b159cbfafe1f0"
instance_type = "t2.micro" tags = {
Name = "WebServer-East"
DevOps Shack
}
}

# EC2 instance in US-West

resource "aws_instance" "web_us_west" {
provider = aws.us_west
ami = "ami-0c55b159cbfafe1f0"
instance_type = "t2.micro" tags = {
Name = "WebServer-West"
}
}
In this configuration:

• An EC2 instance is created in both US-East and US-West regions.

• Both instances host the same application, ensuring high availability across regions.

Step 3: Configure Route 53 for DNS Failover

resource "aws_route53_zone" "example" {

name = "example.com" }

resource
"aws_route53_record" "primary" { zone_id =
aws_route53_zone.example.zone_id name =
"www.example.com"
type = "A"

alias {
name = aws_instance.web_us_east.public_ip
zone_id = aws_instance.web_us_east.id
evaluate_target_health = true
}

failover_routing_policy {
type = "PRIMARY"
}
}

resource "aws_route53_record" "secondary" {

zone_id = aws_route53_zone.example.zone_id

name = "www.example.com"
type = "A"
DevOps Shack
alias {
name = aws_instance.web_us_west.public_ip
zone_id = aws_instance.web_us_west.id
evaluate_target_health = true
}

failover_routing_policy {
type = "SECONDARY"
}
}
In this configuration:

• Route 53 is used to create DNS failover routing.

• If the primary EC2 instance in US-East becomes unhealthy, traffic is automatically routed to
the secondary EC2 instance in US-West.

83. Automating Backups for Disaster Recovery

Automating backups of critical data ensures that you can recover from data loss or infrastructure
failures. Terraform can manage backup policies for cloud resources such as databases and storage. a.
Automating RDS Backups in AWS

Let’s automate the backup process for an RDS database in AWS, ensuring that daily backups are
taken and stored securely.

resource "aws_db_instance" "mydb" {

allocated_storage = 20 engine
= "mysql" instance_class =
"db.t2.micro"
name = "mydb"
username = "admin"
password = "password"
backup_retention_period = 7 # Retain backups for 7 days
backup_window = "01:00-03:00"
}
In this configuration:

• An RDS instance is created with automated backups that are retained for 7 days.

• The backups are taken daily during the specified backup window.

b. Automating S3 Backups

In addition to databases, other critical data, such as files stored in S3, need to be backed up. You can
use S3 lifecycle policies to automate data backup and retention.

Example: S3 Lifecycle Policy for Backup and Archival

resource "aws_s3_bucket" "my_bucket" {

bucket = "my-backup-bucket"
DevOps Shack

lifecycle_rule {
enabled = true

transition {
days = 30
storage_class = "GLACIER" # Move data to Glacier for long-term storage after 30 days
}

expiration {
days = 365 # Delete data after 365 days
}
}
}
In this example:

• Data in the S3 bucket is automatically transitioned to Glacier for long-term archival after 30
days.

• Objects are deleted after 365 days to manage storage costs.

84. Automated Failover and Replication

Automated failover ensures that when the primary infrastructure fails, traffic is seamlessly routed to
the backup infrastructure. Services like AWS RDS, Azure SQL, and Google Cloud SQL support
automated failover for databases.

a. Multi-AZ Failover with RDS

AWS RDS offers Multi-AZ deployments, which automatically replicate data across availability zones
and provide failover support in case of hardware or network failures. Example: Enabling Multi-AZ
for RDS

resource "aws_db_instance" "mydb" {

allocated_storage = 20 engine
= "mysql" instance_class =
"db.t2.micro"
multi_az = true # Enable Multi-AZ failover
name = "mydb" username =
"admin"
password = "password"
}

In this example:

• The Multi-AZ option is enabled, ensuring that RDS automatically replicates the database to
another availability zone.

• In the event of an outage, AWS automatically fails over to the standby instance in another
availability zone.

85. Monitoring and Alerts for DR and HA

DevOps Shack
Monitoring your infrastructure is critical for ensuring high availability and disaster recovery.
Terraform can configure monitoring tools like CloudWatch, Azure Monitor, and Google Cloud
Monitoring to alert you to potential failures.

a. Monitoring EC2 Instances with CloudWatch

resource "aws_cloudwatch_metric_alarm" "cpu_alarm" {

alarm_name = "high-cpu-usage"
comparison_operator = "GreaterThanThreshold"
evaluation_periods = 2
metric_name = "CPUUtilization"
namespace = "AWS/EC2"
period = 60
statistic = "Average"
threshold = 80
alarm_actions = [aws_sns_topic.my_sns_topic.arn]

dimensions = {
InstanceId = aws_instance.web_us_east.id
}
}
In this configuration:

• A CloudWatch alarm is set to trigger when CPU utilization exceeds 80%.

• The alarm sends a notification to an SNS topic to alert administrators.

86. Best Practices for DR and HA with Terraform

a. Use Multi-Region and Multi-Zone Setups

Ensure that critical applications are deployed in multiple regions or availability zones to improve
resilience and reduce the risk of downtime.

b. Automate Failover and Backup

Automate failover mechanisms and regularly test them to ensure that they work as expected.
Similarly, automate data backups and retention to ensure recoverability in case of disasters.

c. Monitor and Test Disaster Recovery Plans

Set up regular disaster recovery drills to test your recovery processes. Ensure that your monitoring
and alerting systems provide timely notifications in case of failures.

87. Conclusion

In this module, we explored Disaster Recovery (DR) and High Availability (HA) strategies using
Terraform. We covered how to deploy multi-region infrastructure, automate backups, and

implement failover mechanisms to ensure that your applications and data remain available and
recoverable in case of failure. By automating these processes with Terraform, you can improve
resilience and minimize downtime, ensuring business continuity.
DevOps Shack

88. Importance of Cost Management in Cloud Infrastructure

Effective cost management ensures that organizations maintain control over cloud spending while
leveraging the flexibility and scalability of cloud services. Key reasons why cost management is critical
include:

• Budget Control: Prevents unexpected cloud cost overruns.

• Resource Optimization: Ensures that resources are right-sized and utilized efficiently.

• Scaling and Flexibility: Allows infrastructure to scale without incurring unnecessary costs.

• Cloud Financial Operations (FinOps): Aligns infrastructure operations with financial goals,
optimizing for both performance and costs.

By integrating cost management strategies into your Terraform workflows, you can ensure that
resources are used efficiently and cloud costs remain predictable.

89. Key Cost Management Strategies Using Terraform

a. Right-Sizing Resources

Right-sizing refers to ensuring that resources are appropriately sized for their workloads.
Overprovisioning resources such as compute, memory, and storage can result in unnecessary
costs. Terraform can help automate the process of adjusting resource sizes based on usage
patterns. Example: Right-Sizing AWS EC2 Instances resource "aws_instance" "example" {

ami = "ami-0c55b159cbfafe1f0" instance_type = "t3.micro" # Adjust

to smaller instance type based on usage

tags = {

Name = "example-instance"

In this example, the instance type is adjusted from a larger instance (e.g., t3.large) to a smaller one
(t3.micro) to optimize costs.

b. Use Reserved Instances and Savings Plans

Cloud providers like AWS, Azure, and Google Cloud offer Reserved Instances (RIs) or Savings Plans,
which provide significant cost savings in exchange for a commitment to using a certain amount of
compute resources over a specific time period.

Example: Reserved Instances in AWS

You can manage reserved instances in AWS using Terraform to ensure long-term infrastructure cost
savings.
DevOps Shack
resource "aws_instance" "example" {

ami = "ami-0c55b159cbfafe1f0" instance_type = "t3.micro"

instance_lifecycle = "reserved" # Define the instance as a reserved instance

Reserved instances can provide discounts of up to 75% compared to on-demand pricing, making
them an effective strategy for cost management when workloads have predictable usage.

c. Automating Resource Shutdown to Save Costs

Idle resources can incur significant costs, especially in non-production environments like
development and testing. Terraform can automate the shutdown or scaling down of idle resources to
save costs during off-hours.

Example: Scheduling EC2 Instance Shutdown

You can use AWS Lambda and CloudWatch Events to schedule the shutdown of EC2 instances during
non-business hours to save costs.

resource "aws_cloudwatch_event_rule" "shutdown_rule" {

name = "shutdown-instances" schedule_expression =

"cron(0 18 * * ? *)" # Every day at 6 PM

resource "aws_lambda_function" "shutdown_ec2" {

filename = "lambda_shutdown.zip"

function_name = "shutdownEC2Instances"

handler = "lambda_function.lambda_handler"

runtime = "python3.8" role =

aws_iam_role.lambda_exec.arn

resource "aws_cloudwatch_event_target" "shutdown_target" {

rule = aws_cloudwatch_event_rule.shutdown_rule.name

target_id = "EC2Shutdown" arn =

aws_lambda_function.shutdown_ec2.arn

In this example, a Lambda function shuts down EC2 instances every evening at 6 PM, helping reduce
costs by stopping idle resources during non-peak hours.
DevOps Shack

90. Cost Monitoring and Alerts with Terraform

Monitoring cloud costs in real time is essential to prevent unexpected cost overruns. Terraform can
help automate the setup of monitoring tools like AWS Budgets, Azure Cost Management, or Google
Cloud Billing to track and control cloud spending.

a. Setting Up AWS Budgets for Cost Tracking

AWS Budgets can be used to set cost thresholds and receive alerts when spending exceeds a
specified limit. Terraform allows you to automate the creation and management of AWS Budgets.
Example: AWS Budgets with Terraform resource "aws_budgets_budget" "monthly_budget" {
budget_type = "COST" limit_amount = "500" limit_unit = "USD"

time_unit = "MONTHLY"

cost_filters = {

Service = "AmazonEC2"

notification {

comparison_operator = "GREATER_THAN"

threshold = 80

threshold_type = "PERCENTAGE" notification_type

= "ACTUAL" subscriber_email_addresses =

["[email protected]"]

In this example:

• A monthly budget of $500 is set for EC2 services.

• An alert is triggered when 80% of the budget is used, ensuring that cost overruns are
addressed early.

b. Azure Cost Management with Terraform

Azure provides Cost Management and Billing tools that allow you to set budgets, track spending, and
optimize costs across your resources. Terraform can be used to automate cost management in Azure.
DevOps Shack

Example: Azure Cost Management Budget resource

"azurerm_cost_management_budget" "example" {

name = "monthly-budget" scope

= azurerm_subscription.primary.id amount

= 500 time_grain = "Monthly"

time_period_start = "2024-01-01"

time_period_end = "2024-12-31"

notification { threshold = 0.8 # 80%

threshold contact_emails =

["[email protected]"]

In this example, an Azure Cost Management Budget is set up with a limit of $500 for the year 2024,
and a notification is sent when 80% of the budget is reached.

91. Cost Optimization Using S3 Lifecycle Policies

Storage costs, especially for long-term data retention, can be optimized using lifecycle policies that
automatically transition data to cheaper storage tiers, such as Amazon S3 Glacier or Azure Blob
Storage Archive Tier.

a. Example: S3 Lifecycle Policy for Cost Optimization

resource "aws_s3_bucket" "my_bucket" { bucket =

"my-data-bucket"

lifecycle_rule {

enabled = true

transition {

days = 30

storage_class = "GLACIER" # Move objects to Glacier after 30 days

}
DevOps Shack
expiration { days = 365 # Delete objects

after 365 days

In this example:

• Data in the S3 bucket is automatically transitioned to Glacier for low-cost archival storage
after 30 days.

• Objects are deleted after 365 days to avoid unnecessary long-term storage costs.

92. Terraform for Autoscaling and Cost Optimization

Autoscaling helps to manage cloud costs by automatically adjusting resource usage based on
demand. Terraform can automate the configuration of autoscaling groups to ensure that you only pay
for the resources you need.

a. Autoscaling EC2 Instances Based on Load

By configuring an autoscaling group, you can dynamically add or remove instances based on resource
usage patterns, optimizing both performance and cost.

Example: EC2 Autoscaling Group with Terraform

resource "aws_autoscaling_group" "example" {

desired_capacity = 1 max_size =5

min_size =1

launch_configuration = aws_launch_configuration.example.id

tag { key = "Name" value

= "autoscaled-instance"

propagate_at_launch = true

resource "aws_launch_configuration" "example" {

image_id = "ami-0c55b159cbfafe1f0"
DevOps Shack
instance_type = "t3.micro" security_groups =

[aws_security_group.instance.id]

lifecycle {

create_before_destroy = true

In this example:

• The autoscaling group automatically adjusts the number of EC2 instances based on demand,
reducing costs during periods of low usage while maintaining performance during peak
periods.

93. Best Practices for Cost Management with Terraform

a. Regularly Review and Optimize Resources

Set up regular reviews of your Terraform configurations to ensure that resources are right-sized and
that you are using cost-saving options like Reserved Instances or Savings Plans.

b. Automate Cost Alerts

Automate the setup of cost alerts to notify you when spending thresholds are exceeded. This helps
catch unexpected cost increases early and allows for corrective action.

c. Implement Autoscaling

Use autoscaling groups to dynamically adjust resources based on traffic and usage patterns, ensuring
that you only pay for the resources you need at any given time.

94. Conclusion

In this module, we explored how to use Terraform for Cost Management and Optimization. We
covered key strategies such as right-sizing resources, using reserved instances, automating resource
shutdowns, setting up cost alerts, and integrating autoscaling to manage cloud costs effectively. By
leveraging Terraform to automate cost management processes, you can maintain control over your
cloud expenses while ensuring that your infrastructure scales efficiently.

95. Why Use Multi-Cloud Deployments?

Many organizations adopt a multi-cloud strategy for several reasons:

• Vendor Independence: By using multiple cloud providers, organizations avoid being locked
into a single vendor's ecosystem, increasing flexibility and negotiating power.
DevOps Shack
• Resilience and Redundancy: Multi-cloud environments offer better disaster recovery options
by distributing workloads across multiple clouds, reducing the risk of complete outages.

• Cost Optimization: Different cloud providers offer varying pricing models, and multi-cloud
strategies allow organizations to use the most cost-effective services for specific workloads.

• Leveraging Best Features: Each cloud provider excels in specific areas (e.g., AWS for
compute, Azure for enterprise services, Google Cloud for machine learning), and multi-cloud
deployments allow organizations to take advantage of each provider's strengths.

96. Key Concepts in Multi-Cloud Deployments with Terraform

a. Terraform Providers

Terraform’s provider architecture makes multi-cloud deployments possible. Each cloud provider (e.g.,
AWS, Azure, GCP) has its own Terraform provider plugin that allows Terraform to interact with its
APIs. You can define multiple providers within the same Terraform configuration to deploy resources
across different cloud platforms.

b. Provider Aliases

In a multi-cloud or even multi-region deployment, Terraform supports provider aliases, allowing you
to use different regions or cloud accounts for specific resources within the same configuration. This
provides flexibility and control when managing multiple clouds.

97. Multi-Cloud Deployment Example: AWS and Azure

Let’s explore an example where we deploy an EC2 instance on AWS and a virtual machine on Azure,
demonstrating how to manage resources across multiple clouds within a single Terraform
configuration.

a. Setting Up Providers for AWS and Azure

In this example, we will configure both AWS and Azure providers to manage resources in each cloud.

Step 1: Define AWS and Azure Providers

provider "aws" {
region = "us-west-2"
alias = "aws_us_west"
}

provider "azurerm" {
features {}
alias = "azure_west_us"
}
In this configuration:

• The AWS provider is set to the US-West region.

• The Azure provider is defined for managing resources in Azure's West US region.
DevOps Shack

b. Deploying Resources in AWS and Azure

Step 2: AWS EC2 Instance Deployment

We’ll create an EC2 instance on AWS as part of our multi-cloud setup.

resource "aws_instance" "web_server_aws" {

provider = aws.aws_us_west ami =
"ami-0c55b159cbfafe1f0" instance_type =
"t2.micro" tags = {
Name = "WebServer-AWS"
}}
This creates a basic EC2 instance in AWS's US-West region.

Step 3: Azure Virtual Machine Deployment

Next, we’ll create a virtual machine in Azure.

resource "azurerm_resource_group" "my_rg" {
provider = azurerm.azure_west_us
name = "myResourceGroup"
location = "West US"
}

resource "azurerm_virtual_network" "my_vnet" {

provider = azurerm.azure_west_us
name = "myVNet" address_space = ["10.0.0.0/16"]
location = azurerm_resource_group.my_rg.location
resource_group_name = azurerm_resource_group.my_rg.name
}

resource "azurerm_network_interface" "my_nic" {

provider = azurerm.azure_west_us
name = "myNIC"
location = azurerm_resource_group.my_rg.location
resource_group_name = azurerm_resource_group.my_rg.name

ip_configuration {
name = "myNICConfig"
subnet_id = azurerm_virtual_network.my_vnet.subnets[0].id
private_ip_address_allocation = "Dynamic"
}
}

resource "azurerm_linux_virtual_machine" "my_vm" {

provider = azurerm.azure_west_us name =
"myLinuxVM" location =
azurerm_resource_group.my_rg.location
resource_group_name = azurerm_resource_group.my_rg.name
DevOps Shack
network_interface_ids = [azurerm_network_interface.my_nic.id]
size = "Standard_DS1_v2" admin_username =
"adminuser"
admin_password = "P@ssw0rd123!"

os_disk {
caching = "ReadWrite"
storage_account_type = "Standard_LRS"
}
}
In this example:

• We create an Azure Resource Group, Virtual Network, Network Interface, and finally a Linux
Virtual Machine.

• This demonstrates how Terraform can manage resources in Azure alongside resources in
AWS.

c. Outputting Multi-Cloud Resource Information

To see the results of our multi-cloud deployment, we can output the public IP addresses of both
instances.
output "aws_instance_public_ip" {
value = aws_instance.web_server_aws.public_ip
}

output "azure_vm_public_ip" {
value = azurerm_linux_virtual_machine.my_vm.public_ip_address
}

d. Apply the Multi-Cloud Configuration

To deploy the infrastructure across both AWS and Azure, use the following Terraform commands:

terraform init # Initialize providers terraform

plan # Review the execution plan terraform

apply # Deploy infrastructure

This process will deploy resources across both AWS and Azure based on the configurations we
provided.

98. Managing Multi-Cloud Resources with Terraform

Terraform provides several features to make managing multi-cloud resources easier.

a. Managing State Across Multiple Providers

DevOps Shack
In a multi-cloud environment, it's crucial to manage Terraform state properly. You can use remote
state backends (e.g., S3, Azure Blob Storage, or Google Cloud Storage) to store Terraform state files,
ensuring consistency and enabling collaboration across teams. Example: Storing State in Azure Blob
Storage

terraform { backend
"azurerm" {
storage_account_name = "mystorageaccount"
container_name = "tfstate"
key = "terraform.tfstate"
}
}
This stores the Terraform state file in an Azure Blob Storage container, ensuring that state is remotely
managed and accessible.

b. Managing Multi-Cloud Credentials

For security reasons, it’s essential to manage multi-cloud credentials securely. Rather than
hardcoding credentials in your Terraform configurations, use environment variables or secret
management tools such as AWS Secrets Manager, Azure Key Vault, or Google Cloud Secret
Manager.

Example: Using Environment Variables for AWS and Azure Credentials

export AWS_ACCESS_KEY_ID="your-aws-access-key" export

AWS_SECRET_ACCESS_KEY="your-aws-secret-key" export

ARM_CLIENT_ID="your-azure-client-id" export

ARM_CLIENT_SECRET="your-azure-client-secret" export

ARM_SUBSCRIPTION_ID="your-azure-subscription-id"

Terraform will automatically pick up these environment variables when executing the provider blocks.

99. Multi-Cloud Networking Considerations

When deploying across multiple clouds, networking between cloud providers can become complex.
You need to ensure secure and reliable connections between resources in different clouds.

a. VPN Connectivity Between AWS and Azure

You can set up site-to-site VPN connections between AWS and Azure to enable secure
communication between the two environments.

Example: VPN Setup Between AWS and Azure

1. Create a Virtual Private Gateway (VGW) on AWS: resource "aws_vpn_gateway" "my_vgw" {

vpc_id = aws_vpc.my_vpc.id

}
DevOps Shack
2. Create a Customer Gateway (CGW) on Azure:

resource "azurerm_virtual_network_gateway" "azure_vpn_gw" {

name = "azure-vpn-gateway"

location = "West US"

resource_group_name = azurerm_resource_group.my_rg.name

type = "Vpn"

vpn_type = "RouteBased"

sku = "Standard"

3. Establish the VPN Connection: Once the gateways are in place, you can create a VPN
connection between AWS and Azure, allowing resources in each cloud to communicate
securely.

100. Best Practices for Multi-Cloud Deployments

a. Consistent Resource Tagging

Ensure that resources in all cloud providers are consistently tagged. This helps with cost
management, monitoring, and compliance. Example: Applying Consistent Tags

resource "aws_instance" "web_server" {

tags = {
Environment = "production"
Project = "multi-cloud"
}
}

resource "azurerm_virtual_machine" "my_vm" {

tags = {
Environment = "production"
Project = "multi-cloud"
}
}

b. Use Modular Terraform Code

For large multi-cloud environments, use Terraform modules to organize and reuse code. This helps
reduce duplication and ensures consistency across cloud deployments.

101. Conclusion

In this module, we explored how to manage Multi-Cloud Deployments with Terraform. We covered
the key concepts, benefits, and challenges of using multiple cloud providers, and provided examples
DevOps Shack
for deploying resources across both AWS and Azure. By using Terraform, you can manage multi-cloud
environments consistently, securely, and efficiently, avoiding vendor lock-in and improving resilience

102. Why Use Terraform for Kubernetes Management?

Managing Kubernetes clusters and workloads using Terraform offers several advantages:

• Automation: Terraform can automate the creation of Kubernetes clusters across multiple
cloud providers (e.g., AWS EKS, Azure AKS, Google GKE).

• Infrastructure as Code (IaC): Terraform allows you to define Kubernetes clusters and
workloads declaratively, making it easier to track changes and ensure consistency.

• Multi-Cloud Support: Terraform can manage Kubernetes clusters across different cloud
providers, allowing for multi-cloud Kubernetes management from a single toolset.

• Scaling and Flexibility: Terraform can integrate with Kubernetes to dynamically scale clusters
and workloads based on resource requirements.

103. Key Concepts in Kubernetes Management with Terraform

a. Terraform Providers for Kubernetes

Terraform uses providers to interact with cloud platforms and services. The Kubernetes provider
enables Terraform to manage Kubernetes resources, while cloud-specific providers (like AWS, Azure,
and Google Cloud) are used to create the infrastructure for hosting Kubernetes clusters.

b. Kubernetes Cluster Management

Terraform can automate the creation of Kubernetes clusters on various platforms. For example, you
can use the AWS EKS provider to create an EKS cluster, the Azure AKS provider to create an AKS
cluster, and the Google GKE provider to create a GKE cluster.

c. Kubernetes Resource Management

Once a Kubernetes cluster is created, Terraform can also manage Kubernetes resources such as
Deployments, Services, Namespaces, and Ingresses. This provides a unified workflow for managing
both the infrastructure and workloads.

104. Example: Deploying a Kubernetes Cluster on AWS EKS

Let’s walk through an example of deploying a Kubernetes (EKS) cluster on AWS using Terraform,
followed by the deployment of a simple Kubernetes workload.

a. AWS EKS Cluster Deployment

First, we will create an AWS EKS cluster using Terraform. We’ll configure the EKS cluster, define the
worker nodes, and set up networking components such as VPC and subnets. Step 1: Define the
AWS Provider and EKS Configuration
DevOps Shack
provider "aws" {
region = "us-west-2"
}

module "eks" {
source = "terraform-aws-modules/eks/aws"
cluster_name = "my-eks-cluster"
cluster_version = "1.21"
subnets = ["subnet-123456", "subnet-789012"]
vpc_id = "vpc-12345678"
node_groups = {
eks_nodes = {
desired_capacity = 2
max_capacity = 3
min_capacity = 1
instance_type = "t3.medium"
}
}
}
In this configuration:

• We define the AWS provider for the US-West-2 region.

• The EKS module is used to create an EKS cluster, specifying the VPC and subnets in which the
cluster will reside.

• We define an EKS node group that will manage the worker nodes for the cluster.

b. Deploying Kubernetes Workloads Using Terraform

Once the Kubernetes cluster is deployed, we can manage Kubernetes resources (such as
Deployments, Services, and Ingresses) using the Kubernetes provider in Terraform. Step
2: Configure the Kubernetes Provider

provider "kubernetes" {
host = module.eks.cluster_endpoint
cluster_ca_certificate = base64decode(module.eks.cluster_certificate_authority_data)
exec {
api_version = "client.authentication.k8s.io/v1alpha1"
args = ["eks", "get-token", "--cluster-name", module.eks.cluster_name]
}
}
In this configuration:

• The Kubernetes provider is configured to connect to the newly created EKS cluster using the
cluster endpoint and certificate authority.
DevOps Shack
Step 3: Deploy a Kubernetes Workload (Nginx)

resource "kubernetes_deployment" "nginx" {

metadata {
name = "nginx-deployment"
namespace = "default"
}

spec {
replicas = 2
selector {
match_labels = {
app = "nginx"
}
}
template {
metadata {
labels = {
app = "nginx"
}
}
spec {
container {
name = "nginx"
image = "nginx:latest"
ports {
container_port = 80
}
}
}
}
}
}

resource "kubernetes_service" "nginx_service" {

metadata {
name = "nginx-service"
namespace = "default"
}

spec {
selector = {
app = "nginx"
}
port {
port = 80
target_port = 80
}
type = "LoadBalancer"
}
}
In this example:
DevOps Shack
• A Kubernetes deployment is created for the Nginx application, with two replicas.

• A Kubernetes service of type LoadBalancer is created to expose the Nginx deployment

externally.

c. Apply the Configuration

To deploy the EKS cluster and Kubernetes workload, run the following Terraform commands:

terraform init # Initialize providers and modules

terraform plan # Review the plan terraform apply

# Deploy the cluster and workload This will create

the EKS cluster, configure the worker nodes, and

deploy the Nginx workload on Kubernetes.

105. Managing Kubernetes Resources with Terraform

Terraform allows you to manage a wide variety of Kubernetes resources beyond deployments and
services. You can use Terraform to create and manage Namespaces, ConfigMaps, Ingresses,
Persistent Volumes, and more.

a. Example: Creating a Kubernetes Namespace resource

"kubernetes_namespace" "my_namespace" { metadata { name = "my-

app-namespace"

This creates a new namespace called my-app-namespace for isolating Kubernetes resources.

b. Example: Managing Kubernetes ConfigMaps

ConfigMaps allow you to store configuration data for your applications in key-value pairs.
DevOps Shack
resource "kubernetes_config_map" "my_config_map" {
metadata {
name = "app-config"
namespace = "default"
}

data = {
app_environment = "production"
app_version = "v1.0.0"
}
}
This ConfigMap can be referenced in a Kubernetes deployment to provide environment-specific
configuration data.

106. Kubernetes Autoscaling with Terraform

One of Kubernetes' key features is its ability to automatically scale workloads based on demand.
Terraform can be used to configure Horizontal Pod Autoscalers (HPA) for dynamic scaling of
workloads.

a. Example: Configuring a Horizontal Pod Autoscaler (HPA)

resource "kubernetes_horizontal_pod_autoscaler" "nginx_autoscaler" {
metadata {
name = "nginx-hpa"
namespace = "default"
}

spec {
scale_target_ref {
kind = "Deployment"
name = kubernetes_deployment.nginx.metadata[0].name
}
min_replicas = 2
max_replicas = 10
target_cpu_utilization_percentage = 50
}
}
In this example:

• The HPA will scale the Nginx deployment between 2 and 10 replicas based on CPU
utilization.

• The target CPU utilization is set to 50%, meaning that the autoscaler will add or remove
replicas to maintain this level of resource usage.

107. Best Practices for Managing Kubernetes with Terraform

a. Use Modular Code

DevOps Shack
For larger Kubernetes environments, break your Terraform configuration into reusable modules. This
allows you to manage resources in a scalable and maintainable way.

b. Automate Cluster and Workload Scaling

Automate the scaling of both Kubernetes clusters and workloads using autoscaling mechanisms like
HPA for workloads and cluster autoscaler for nodes.

c. Secure Kubernetes Resources

Ensure that Kubernetes resources such as Secrets, Ingresses, and RBAC policies are properly
managed to avoid security risks. Terraform can be used to enforce security policies through RBAC.

108. Conclusion

In this module, we explored how to use Terraform for Kubernetes Management. We covered key
concepts such as deploying Kubernetes clusters on AWS EKS, managing Kubernetes resources like
deployments and services, and configuring autoscaling with Terraform. By integrating Terraform into
your Kubernetes workflow, you can automate cluster provisioning, workload management, and
scaling, ensuring that your Kubernetes environment is efficient, secure, and scalable.

109. What is Serverless Computing?

Serverless computing is a cloud computing model in which the cloud provider dynamically manages
the allocation of resources. This model allows developers to focus on writing code and eliminates the
need to manage underlying infrastructure, scaling, and availability. Some key benefits of serverless
computing include:

• Automatic Scaling: Functions automatically scale up or down based on demand.

• Cost Efficiency: You only pay for the compute resources you actually use (e.g., function
execution time), rather than provisioning servers that may remain idle.

• Reduced Operational Complexity: The cloud provider handles server provisioning, scaling,
and patching, which reduces operational overhead.

Common serverless platforms include:

• AWS Lambda

• Azure Functions

• Google Cloud Functions

110. Key Concepts in Serverless Architectures

a. Functions as a Service (FaaS)

DevOps Shack
Serverless platforms like AWS Lambda, Azure Functions, and Google Cloud Functions allow
developers to deploy individual functions that execute in response to triggers (e.g., HTTP requests,
database events, or message queues). This model is often referred to as Functions as a Service
(FaaS).

b. Event-Driven Architecture

Serverless functions are typically triggered by events such as:

• HTTP requests (via API Gateway or HTTP triggers)

• Database changes (e.g., DynamoDB Streams, Cosmos DB triggers)

• File uploads (e.g., Amazon S3 events)

• Scheduled tasks (e.g., AWS CloudWatch Events, Azure Timer triggers)

111. Deploying AWS Lambda Functions with Terraform

In this section, we will deploy an AWS Lambda function using Terraform. AWS Lambda is one of the
most widely used serverless computing platforms, allowing developers to run code in response to
events without provisioning or managing servers.

a. AWS Lambda Function Example

Let’s walk through an example where we create a simple AWS Lambda function that processes
requests via API Gateway. Step 1: Define the AWS Provider provider "aws" { region = "us-
west-2"

This defines the AWS provider and specifies the region where the Lambda function will be deployed.

b. Step 2: Define the Lambda Function

resource "aws_lambda_function" "my_lambda" {

function_name = "my-lambda-function" handler
= "lambda_function.lambda_handler"
runtime = "python3.8"
role = aws_iam_role.lambda_role.arn

filename = "lambda_function.zip" # The zip file containing the Lambda code

source_code_hash = filebase64sha256("lambda_function.zip")
}

In this example:

• The Lambda function is written in Python 3.8 and packaged as a ZIP file
(lambda_function.zip).

• The handler specifies the entry point for the function (lambda_function.lambda_handler).
DevOps Shack
• The IAM role (lambda_role) provides the Lambda function with the necessary permissions to
execute.

IAM Role for Lambda Function

resource "aws_iam_role" "lambda_role" {
name = "lambda-execution-role"

assume_role_policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "sts:AssumeRole",
"Effect": "Allow",
"Principal": {
"Service": "lambda.amazonaws.com"
}
}
]
}
EOF
}

resource "aws_iam_role_policy" "lambda_policy" {

role = aws_iam_role.lambda_role.id

policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "logs:CreateLogGroup",
"Resource": "arn:aws:logs:*:*:*"
},
{
"Effect": "Allow",
"Action": "logs:CreateLogStream",
"Resource": "arn:aws:logs:*:*:*"
},
{
"Effect": "Allow",
"Action": "logs:PutLogEvents",
"Resource": "arn:aws:logs:*:*:*"
}
]
}
EOF
}
In this example:
DevOps Shack
• An IAM role is created that allows the Lambda function to interact with Amazon CloudWatch
Logs to store logs.

c. Step 3: Integrate Lambda with API Gateway

To make the Lambda function accessible via HTTP, we’ll set up an API Gateway trigger.hcl resource

"aws_api_gateway_rest_api" "my_api" {

name = "My API" description = "API Gateway

for my Lambda function" }

resource "aws_api_gateway_resource" "lambda_resource" {

rest_api_id = aws_api_gateway_rest_api.my_api.id parent_id =

aws_api_gateway_rest_api.my_api.root_resource_id path_part =

"lambda"

resource "aws_api_gateway_method" "get_lambda" {

rest_api_id = aws_api_gateway_rest_api.my_api.id resource_id

= aws_api_gateway_resource.lambda_resource.id http_method

= "GET" authorization = "NONE"

resource "aws_lambda_permission" "api_gateway_invoke" {

statement_id = "AllowAPIGatewayInvoke" action =

"lambda:InvokeFunction" function_name =

aws_lambda_function.my_lambda.arn principal =

"apigateway.amazonaws.com"

In this example:

• An API Gateway is set up with a GET method that triggers the Lambda function.

• A Lambda permission is created to allow API Gateway to invoke the Lambda function.

d. Apply the Terraform Configuration

DevOps Shack
To deploy the Lambda function and API Gateway, run the following commands:

terraform init terraform

plan terraform apply

This will create the Lambda function, configure its IAM role, and expose it via API Gateway.

112. Deploying Azure Functions with Terraform

Next, we’ll explore how to deploy an Azure Function using Terraform. Azure Functions is Microsoft’s
serverless computing platform, allowing developers to deploy functions that can be triggered by
HTTP requests, events, and more.

a. Azure Function Example Step

1: Define the Azure Provider

provider "azurerm" {

features {}

Step 2: Create an Azure Function App

resource "azurerm_resource_group" "example_rg" {

name = "example-resources"
location = "West US"
}

resource "azurerm_storage_account" "example_storage" {

name = "examplestorageacct"
resource_group_name = azurerm_resource_group.example_rg.name
location = azurerm_resource_group.example_rg.location account_tier
= "Standard"
account_replication_type = "LRS"
}

resource "azurerm_app_service_plan" "example_plan" {

name = "example-appserviceplan" location =
azurerm_resource_group.example_rg.location
resource_group_name = azurerm_resource_group.example_rg.name
kind = "FunctionApp" sku {
tier = "Dynamic"
size = "Y1"
}
}

resource "azurerm_function_app" "example_function" {

name = "examplefunction"
location = azurerm_resource_group.example_rg.location
resource_group_name = azurerm_resource_group.example_rg.name
DevOps Shack
app_service_plan_id = azurerm_app_service_plan.example_plan.id
storage_account_name = azurerm_storage_account.example_storage.name
storage_account_access_key = azurerm_storage_account.example_storage.primary_access_key
os_type = "Linux" version = "~3" }

In this example:

• We create an Azure Function App within a resource group, storage account, and app service
plan.

• The Function App uses the Linux OS and version 3.x of Azure Functions runtime.

b. Deploy the Azure Function

After defining the infrastructure, you can deploy the Azure Function with:

terraform init terraform

plan

terraform apply

113. Best Practices for Managing Serverless Architectures with Terraform

a. Use Modular Code for Functions

Use Terraform modules to organize serverless functions, making it easier to manage large numbers
of functions across multiple environments (e.g., development, staging, production).

b. Manage Function Secrets Securely

For sensitive data like API keys or database credentials, use cloud-native secret management tools
such as AWS Secrets Manager or Azure Key Vault. This ensures that sensitive information is not
hardcoded into your Terraform code.

114. Conclusion

In this module, we covered how to use Terraform for Serverless Architectures. We explored
deploying AWS Lambda functions and Azure Functions, integrating with services like API Gateway,
and managing serverless functions using Terraform. By using Terraform to provision and manage
serverless architectures, you can automate infrastructure deployment, reduce operational
complexity, and ensure consistency across multiple environments.
DevOps Shack

115. Why Monitoring and Logging Matter

Effective monitoring and logging are critical to ensuring that your infrastructure is running smoothly
and that any issues are detected before they impact users. Some key reasons why monitoring and
logging are important include:

• Performance Monitoring: Tracking the performance of applications and services helps

ensure that they meet performance SLAs.

• Error Detection: Logs provide critical information for troubleshooting errors and issues.

• Security Monitoring: Logging security-related events, such as access attempts and policy
changes, helps detect and prevent security breaches.

• Automated Alerts: Alerts ensure that teams are notified immediately when predefined
thresholds or anomalies are detected.

116. Key Concepts in Monitoring and Logging with Terraform

a. Infrastructure Monitoring

Terraform can configure infrastructure monitoring tools such as AWS CloudWatch, Azure Monitor,
and Google Cloud Monitoring to track the performance and health of your resources. Metrics such
as CPU usage, memory utilization, and network traffic can be automatically collected and analyzed. b.
Logging

Logs provide detailed records of events that occur within your infrastructure. Terraform can
configure logging solutions such as AWS CloudWatch Logs, Azure Log Analytics, and Google Cloud
Logging to capture, store, and analyze logs from various infrastructure components. c. Alerts

Setting up alerts based on predefined thresholds allows teams to respond to issues before they
escalate. Terraform can configure alerting mechanisms across platforms, enabling automated
notifications when critical thresholds are exceeded.

117. Example: AWS CloudWatch for Monitoring and Alerts

Let’s walk through how to use Terraform to configure AWS CloudWatch to monitor an EC2 instance’s
CPU usage and set up an alert if usage exceeds a certain threshold.

a. AWS CloudWatch Monitoring

resource "aws_cloudwatch_metric_alarm" "high_cpu_alarm" {
alarm_name = "HighCPUAlarm"
comparison_operator = "GreaterThanThreshold"
evaluation_periods = 2
metric_name = "CPUUtilization"
namespace = "AWS/EC2"
period = 300
statistic = "Average"
threshold = 80
DevOps Shack
alarm_actions = [aws_sns_topic.alert_topic.arn]

dimensions = {
InstanceId = aws_instance.my_instance.id
}
}
In this example:

• AWS CloudWatch is used to monitor the CPUUtilization of an EC2 instance.

• An alarm is triggered if the average CPU usage exceeds 80% for two consecutive evaluation
periods (5 minutes each).

b. Setting Up AWS SNS for Alerts

To notify teams when the CPU threshold is breached, we use AWS Simple Notification Service (SNS).

resource "aws_sns_topic" "alert_topic" {

name = "cpu-alerts-topic" }

resource
"aws_sns_topic_subscription" "email_subscription" {
topic_arn = aws_sns_topic.alert_topic.arn protocol = "email"
endpoint = "[email protected]" # Replace with the email address you want to receive alerts
}
In this configuration:

• An SNS topic is created, and an email subscription is added.

• When the CPUUtilization alarm is triggered, an alert is sent to the specified email address.

118. Example: Azure Monitor for Metrics and Alerts

Azure Monitor is a comprehensive solution for monitoring infrastructure performance in Microsoft

Azure. Let’s set up Azure Monitor to track the performance of a Virtual Machine (VM) and configure
an alert based on CPU usage.

a. Azure Monitor Metrics for a VM

resource "azurerm_monitor_metric_alert" "cpu_alert" {

name = "HighCPUAlert"
resource_group_name = azurerm_resource_group.example.name
scopes = [azurerm_linux_virtual_machine.example.id] description
= "Alert when CPU usage is greater than 80%"

criteria {
metric_name = "Percentage CPU"
DevOps Shack
aggregation = "Average"
operator = "GreaterThan"
threshold = 80
dimensions = []
time_grain = "PT1M"
metric_namespace = "microsoft.compute/virtualmachines"
}

action {
action_group_id = azurerm_monitor_action_group.example.id
}
}
In this example:

• Azure Monitor tracks the CPU usage of a virtual machine.

• An alert is triggered if the CPU usage exceeds 80% on average for a specific time interval.

b. Azure Monitor Action Group for Alerts

Action groups define how Azure alerts are delivered.

resource "azurerm_monitor_action_group" "example" {

name = "my-action-group"
resource_group_name = azurerm_resource_group.example.name
short_name = "alertgrp"

email_receiver { name
= "emailalerts"
email_address = "[email protected]"
}
}
In this example, an action group is set up to send email alerts to a specific address whenever the CPU
alert is triggered.

119. Logging Infrastructure with Terraform

Terraform can also automate the setup of logging services, which capture events and performance
data across your infrastructure.

a. AWS CloudWatch Logs

CloudWatch Logs capture system and application logs for AWS resources like EC2, Lambda, and RDS.

resource "aws_cloudwatch_log_group" "my_log_group" {

name = "/aws/lambda/my_lambda_logs"
retention_in_days = 7
}
DevOps Shack
resource "aws_lambda_function" "my_lambda" {
function_name = "MyLambdaFunction" handler
= "lambda_function.lambda_handler"
runtime = "python3.8"
filename = "lambda_function.zip"

environment {
variables = {
LOG_GROUP_NAME = aws_cloudwatch_log_group.my_log_group.name

}
}
}
In this example:

• CloudWatch Logs is set up to capture logs from an AWS Lambda function.

• Logs are retained for 7 days before being deleted, helping manage storage costs.

120. Best Practices for Monitoring and Logging

a. Automate Monitoring and Logging

Use Terraform to automate the setup of monitoring and logging across your infrastructure. This
ensures that all critical resources are monitored, and logs are captured consistently.

b. Set Up Alerts for Critical Events

Ensure that alerts are set up for critical metrics such as CPU usage, memory utilization, and network
traffic. Use tools like AWS SNS, Azure Monitor Action Groups, or Google Cloud Alerts to notify teams
when issues arise.

c. Centralize Logs for Easy Access

Centralize logs using platforms like AWS CloudWatch Logs, Azure Log Analytics, or Google Cloud
Logging. Centralized logging makes it easier to troubleshoot issues and analyze system performance.

121. Conclusion

In this final module, we covered how to integrate Monitoring and Logging with Terraform. We
explored examples of configuring AWS CloudWatch and Azure Monitor for infrastructure metrics,
setting up alerts, and capturing logs. Monitoring and logging are essential for ensuring that your
infrastructure remains healthy, secure, and performant.

This concludes our comprehensive documentation on Terraform. Over the course of these modules,
we’ve explored how Terraform can be used to automate, manage, and optimize cloud infrastructure
across multiple platforms, from provisioning basic resources to managing serverless architectures
and Kubernetes clusters.

Key Takeaways:

• Infrastructure as Code (IaC): Terraform provides a declarative and consistent way to define,
provision, and manage infrastructure across multiple cloud providers.
DevOps Shack
• Automation: Terraform enables the automation of infrastructure provisioning, scaling, and
management, reducing operational overhead.

• Best Practices: Using Terraform modules, remote state, and version control helps ensure
infrastructure is scalable, maintainable, and secure.