Terraforming Google Cloud Platform with MongoDB using Packer

Use Case

• Importing a Guest operating system (Ubuntu 16.04) and install Mongo DB on top of it.
• Provisioning MongoDB image as VM using infrastructure as code.

Baking Vs Frying

• Baked images are previously prepared with software and configuration. They are usually bigger as it is bundled with installations and it’s dependencies. Baked images empowers “Immutable Architecture” because most of the time they don’t need extra intervention after instantiation. In case of failure, it’s better to recreate it, rather than repair. In case of ‘Autoscaling’ baked images are preferred.
• Frying is known as provisioning over raw images. In order to be ready-to-use, these lightweight images must be provisioned with software and configurations after instantiation. Concern about fried provisioning, is “How to avoid breaking it ?” when executed repeatedly. The package manager like apt usually install latest copy of packages unless the version is not specified. Unexpected behaviour can happen with untested latest version of packages.

Benefits

• Maintenance and Recovery were easy in case of failure conditions.
• The total cost of ownership was also less due to the reduced need for infrastructure.

Platform Components (GCP)

Compute Engine

• To configure your Google Cloud Platform infrastructure

Cloud storage

• The list of GCS paths, e.g. ‘gs://mybucket/path/to/file.tar.gz’, where the image will be exported.

Tools

Machine Image

• It is a static unit that has pre-configured operating system and installed software that can quickly create new running machines. Different platforms has their own machine formats like, AMI for EC2, VMDK/VMX files for VMware or Compute Engine Images for GCP etc.
• In this example we are using base image of Ubuntu 16.06. On base image we’ll install MongoDB server and bake the bundle to form a new image.

Packer

Introduction

• HashiCorp Packer made it easy to automate and use any type of machine image. It promotes configuration management using automated scripts to install and configure software in packer made images.

Why use packer ?

• It is an OpenSource tool that provides a single source configuration which can be use to configure machine images from different providers.
• Can create multiple images for different platforms in parallel.

Advantages

• Lightning fast infrastructure deployment.
• Inter provider portability
• Testable: Post building the image, smoke test can be executed to check if things are working fine.

Code

• image.json

{
  "variables": {
    "project_id": "project_id",
    "zone": "us-central1-a"
  },
  "builders": [
    {
      "type": "googlecompute",
      "account_file": "service-account.json",
      "project_id": "{{user `project_id`}}",
      "source_image": "ubuntu-1604-xenial-v20190913",
      "ssh_username": "root",
      "zone": "{{user `zone`}}"
    }
  ],
  "provisioners": [{
    "type": "shell",
    "scripts": ["tzdata.sh", "mongo.sh"]
  }],
  "post-processors": [
    {
      "type": "googlecompute-export",
      "paths": [
        "gs://dummy-bucket/file1.tar.gz"
      ],
      "keep_input_artifact": true
    }
  ]
}

• mongo.sh

#!/usr/bin/env bash
sudo apt-get install apt-transport-https -y
wget -qO - https://www.mongodb.org/static/pgp/server-4.2.asc | sudo apt-key add - ;
echo "deb [ arch=amd64 ] https://repo.mongodb.org/apt/ubuntu xenial/mongodb-org/4.2 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-4.2.list ;
sudo apt-get update -y;
sudo apt-get install -y mongodb-org;
echo "mongodb-org hold" | sudo dpkg --set-selections ;
echo "mongodb-org-server hold" | sudo dpkg --set-selections ;
echo "mongodb-org-shell hold" | sudo dpkg --set-selections ;
echo "mongodb-org-mongos hold" | sudo dpkg --set-selections ;
echo "mongodb-org-tools hold" | sudo dpkg --set-selections ;

• tzdata.sh

#!/usr/bin/env bash
sudo adduser --disabled-password --gecos "packer" packer;
su packer;
export DEBIAN_FRONTEND=noninteractive;
sudo apt-get update -y;
ln -fs /usr/share/zoneinfo/America/New_York /etc/localtime;
sudo apt-get install -y tzdata;
sudo dpkg-reconfigure --frontend noninteractive tzdata;

Execute Packer

• packer validate image.json
• packer build image.json

Terraform

Introduction

• OpenSource tool for building, changing and versioning infrastructure.

Why to use Terraform ?

• Build by same company as Packer, HashiCorp, Terraform is based on the same principle as of Packer.
• Terraform lets you to manage infrastructure on GCP using single configuration file in TF format.

Advantages

• Empower Infrastructure as Code
• Provide Execution Plan
• Generate Resource Graph

Code

• provider.tf

provider "google" {
    credentials = "${file("service-account.json")}"
    project     = "project-name"
    region      = "us-central1"
  } 

• instance.tf

resource "google_compute_instance" "mongo-server" {
    name         = "mongo-server"
    machine_type = "n1-standard-1"
    zone         = "us-central1-a"
  
    tags = ["mongo"]
  
    boot_disk {
      initialize_params {
        image = "packer-1569120732"
      }
    }
  
  
    network_interface {
      network = "default"
  
      access_config {
        // Ephemeral IP
      }
    }
  
  
    metadata_startup_script = "sudo service mongod start"
  
    service_account {
      scopes = ["userinfo-email", "compute-ro", "storage-ro"]
    }
  }

Execute Terraform

• terraform init
• terraform plan
• terraform apply -auto-approve

Conclusion

• Our work is done, the image and instance is successfully created using Packer and Terraform on GCP.

Source Code

• https://github.com/rajat965ng/techdown2k19/tree/packer/terraform