Practical CI/CD Guide to Deploying AWS Infrastructure through Terraform - Multi Environment Deployment - Part 3

This is my third blog post in this series. In my first part of this series, I have given an introduction to tools/service I will be using in my project. In the second part, I have explained terraform directory structure. In this part, I will give details of terraform code and terraform cloud configurations.

Terraform Cloud

Terraform Cloud is HashiCorp’s managed service offerings. Terraform Cloud is an application that manages Terraform runs in a consistent and reliable environment instead of on your local machine. .Terraform Cloud removes the complexities of maintaining your own Terraform state files in a multi-team, collaborative Terraform environment. Terraform Cloud runs Terraform on disposable virtual machines in its own cloud infrastructure.

learn.hashicorp.com/collections/terraform/c..

Features

• Run Terraform securely and remotely and collaborate on infrastructure with your team.

• Store your Terraform state file securely with encryption at rest.

• Terraform Cloud can automate a run as soon as a pull request is merged.

• Review and comment on plans prior to executing any change to infrastructure.

• Set up a private module registry that stores all of your modules.

• Sentinel makes it possible to create hard and soft provisioning rules across your organization.

• Cost estimation shows the hourly and monthly costs for any Terraform run, and budget policies can be enforced with Sentinel.

• Set up different access levels for admins, DevOps operators, and developers consuming Terraform resources.

• Export audit logs to external systems via an API, or export their outputs into Splunk for better visualization.

• Automate Terraform Cloud functionality into your CI/CD Pipeline

• Send notifications via email, Slack, or via webhooks when a Terraform run is completed.

Terraform Cloud Components

• Organizations: Organizations are a shared space for teams to collaborate on workspaces in Terraform Cloud. You can share remote_state configurations within your organizations. For Example, you can create organizations per project or product. For example, you are trying to build an Apple product, you can name it like "apple". For my project, I have named like "nitheeshp"

• WorkSpaces: Terraform Cloud manages infrastructure collections with workspaces instead of directories. Workspace is similar to your environments like dev, stage, prod. the workspace contains terraform configurations, Variable values, state files associated with an environment. Each workspace retains backups of its previous state files.

In my project, I have created a workspace for each aws services. You can associate each workspace to a git branch or git repo. In my case, i am running it through GitOps Method

When you create a workspace you have the option to design you're terraform workflow in three ways

• Version Control Workflow

• CLI-Driven Workflow

• API-Driven Workflow

If you see my terraform directory strtuture I have not added default values to my variables. I have associated variables in terraform workspace settings

If you look at main.tf I use the same terraform cloud-config for all environments. you might be wondering how I am applying changes to a specific workspace. I am making use of the workspace prefix.

terraform {
  required_version = "~> 0.12"
  backend "remote" {
    hostname     = "app.terraform.io"
    organization = "nitheeshp"
    workspaces { prefix = "vpc-" }
  }
}

When you add workspace in your config, it will ask for a prompt to select a workspace. For example.

$ terraform init

Initializing the backend...

Successfully configured the backend "remote"! Terraform will automatically
use this backend unless the backend configuration changes.

The currently selected workspace (default) does not exist.
  This is expected behaviour when the selected workspace did not have an
  existing non-empty state. Please enter a number to select a workspace:

  1. dev
  2. stage
  3. prod

  Enter a value:

When you are using terraform in CI/CD, Set the TF_WORKSPACE environment variable to the workspace name you wish to be selected.

export TF_WORKSPACE="dev"`

Deploying Security Groups

As explained above I am using separate repo and different workspace for deploying security groups. For Deploying security groups you need a vpc id. That's where terraform data source comes in handy.

Data sources allow data to be fetched or computed for use elsewhere in Terraform configuration. The use of data sources allows a Terraform configuration to make use of information defined outside of Terraform, or defined by another separate Terraform configuration.

data "terraform_remote_state" "vpc" {
  backend = "remote"

  config = {
    organization = "nitheeshp"
    workspaces = {
     name = "vpc-${var.env}"
    }
  }
}

provider "aws" {
  region = "ap-south-1"
}


module "elb_sg" {
  source = "terraform-aws-modules/security-group/aws"

  name        = "${var.env}-elb-sg"
  description = "elb security group."
  vpc_id      = data.terraform_remote_state.vpc.outputs.vpc_id

  egress_with_cidr_blocks = [
    {
      from_port   = 0
      to_port     = 65535
      protocol    = "all"
      description = "Open internet"
      cidr_blocks = "0.0.0.0/0"
    }
  ]

if you see here I am fetching vpc_id from the vpc-dev workspace

vpc_id      = data.terraform_remote_state.vpc.outputs.vpc_id

In my next blog, I will explain GitOps Pipeline Deployment