Practical CI/CD Guide to Deploying AWS Infrastructure through Terraform - Multi Environment Deployment - Part 1

Recently I am thinking to build a complete end to end green field DevOps personal lab project.Greenfield Software Project refers to developing a system for a new product and requires development from a clean slate – no legacy code around. It is an approach used when you're starting fresh and with no restrictions or dependencies. You have a golden opportunity to implement a solution from scratch. The project is open to modern tools and architectures.

I have been searching on the internet to get some idea on the setting CI/CD pipeline for terraforming deployment. But I couldn't find any practical end to end terraform deployment guide. Most of the guide or blog posts I found explains the deployment pipeline for single (Prod) environments. So I have decided to create my personal lab project and convert into a blog post . In this blog post series I will explain end to end terraform deployment pipeline from Development to Production environments. I will divide this blog into multiple posts. In this blog post, I will introduce concepts and tools I will be using in my lab project.

Why I choose to terraform as my infrastructure as a code tool, there are two reasons. The first one is I have been using cloud formation for a long time and have good experience in it, wanted to get some experience with terraform. The second reason for choosing to terraform is that it is a green filed DevOps project; I can choose a modern tool and experiment with it. I will explain some features of terraform in a later blog.

Infrastructure Tools

Cloud Vendor: AWS

You are free to choose your favourite cloud provider like Azure, Google Cloud. I have chosen AWS

Infrastructure as Code(IaC) : Terraform Cloud

Traditionally Infrastructure (Servers, Load Balancer, Database) was provisioned using a combination of scripts and manual processes. Some person executes these scripts or following through the run-books. If these scripts or runbooks are not updated frequently, they can potentially become a show-stopper in deployments. IaC enables the creation, deployment and maintenance of infrastructure in a programmatic, descriptive, and declarative way.

There are many options you can provision your Infrastructure in an automated way. Like cloud formation,ansible,chef,terraform .

Configuration Management Tools

I have liked to terraform for its support for Multi-Cloud Deployment and reusable modules. A Terraform module is a set of Terraform configuration files in a single directory. Modules allow you to group resources together and reuse this group later. It is 60-70 % compact than aws cloud formation.

Terraform Modules

You can find more details about terraform modules from bellow link

terraform.io/docs/language/modules/develop/..

There is a good blog post that explains why 1password chooses cloud formation over terraform

blog.1password.com/terraforming-1password

Terraform is open-source provisioning Tools. Terraform cross-platform and it can run on Windows, Linux or macOS.There are three ways you can use Terraform

• Terraform OSS (Free)

• Terraform Cloud (Paid - Saas Model)

• Terraform Enterprise (Paid - Self Hosted)

Terraform Offerings

Why Terraform Cloud

I am using terraform cloud for my lab project.Terraform OSS is great when you are using it in a small team and as your team grows complexity of managing terraform also grows .Terraform Cloud is a commercial SaaS product developed by HashiCorp.

Terraform Cloud Features

• Terraform Cloud offers a team-oriented remote Terraform workflow
• VCS Connection (GitHub, GitLab, Bitbucket)
• State Management (Storage, History, and Locking)
• Full User Interface
• Single sign-on (SSO) integrated with Okta.
• Terraform Cloud acts as a remote backend for your Terraform state
• Terraform Cloud embeds the Sentinel policy-as-code framework, which lets you define and enforce granular policies for how your organization provisions infrastructure. You can limit the size of compute VMs, confine major updates to defined maintenance windows, and much more.
• Terraform Cloud can display an estimate of its total cost, as well as any change in cost caused by the proposed updates.

Deployment : GitHub Actions

You can deploy infrastructure from your laptop using terraform cli or through terraform console. If you are a single team member this might work for you for some time. When your team size grows this method won't be able to scale. You need to deploy from the centralised place where everyone has visibility, control and rollback options.

There are many tools available to deploy from a centralised place (CI/CD). I wanted to experiment with terraform pipeline deployment with the "GitOps" method.GitOps uses a Git repository as the single source of truth for infrastructure definitions.GitOps uses merge requests as the change mechanism for all infrastructure updates. When new code is merged, the CI/CD pipeline enacts the change in the environment. Any configuration drift, such as manual changes or errors, is overwritten by GitOps automation.

I will be using GitHub Actions for my deployment .GitHub Actions help you automate tasks within your software development life cycle. GitHub Actions are event-driven, meaning that you can run a series of commands after a specified event has occurred. For example, every time someone creates a pull request for a repository, you can automatically run a command that executes a testing script,plan script and apply script.This enables you to include Continues Integration (CI) and continuous deployment (CD) capabilities and many other features directly in your repository.

Some of the features of GitHub listed bellow

• Github Actions are completely integrated into Github
• Github Actions is fully integrated into Github and it can be managed in the same place as all your other repository related features like pull requests and issues.
• Supports docker container
• Github Actions are free for every repository and include 2000 free build minutes per month for all your private repositories

You can learn more about GitHub Actions from this link

docs.github.com/en/actions/learn-github-act..

In the next blog post, i will cover in details each service