Cloudification

My sample backlog for creating a cloud-enabled platform. My answer to ‘From where to start ?’ situation.

It took me just 17 hrs over a weekend to create a sample of the cloud-enabled platform, completely on my own using this backlog.

Dynasty Warriors — Landscape, Army, Horse, Spear

While having almost 9 years of work experience in writing code for backend and frontend applications, from the past 4 years I’ve been also working on cloud migration projects. Despite the organizational designation, I performed in the equal capacity of Senior Dev, Tech lead and Architect at the same time in almost every other assignment. I got a chance to interact with people having designations like Engineering Managers, Solution Architects, Principal Consultants, and Evangelists. Some of them came from the background of core infrastructure and platform provisioning, while others have prior experience of application development and later on they worked in platform engineering as well. I’m penning down, my thoughts on creating an open backlog for the cloudification of any existing architecture based on my experience and knowledge exchange with such people.

The intent of this write up is to have a high-level product backlog to create sprint stories and kick-start development.

Identify cloud platform: eg. AWS, GCP, Azure, etc.

8 criteria to select a cloud provider.

1. Certifications & Standards
2. Technologies & Service Roadmap
3. Data Security, Data Governance and Business policies
4. Service Dependencies & Partnerships
5. Contracts, Commercials & SLAs
6. Reliability & Performance
7. Migration Support, Vendor Lock in & Exit Planning
8. Business health & Company profile

Identify tools

Container orchestration

Eg. AKS, GKE, Openshift, etc.

Solution for securing the network

Eg. OpenVPN, Bastion host (aws, gcp) etc.

Source code repository management tools

Eg. GitLab, Gogs, Bitbucket, etc.

CI/CD pipeline

Eg. Jenkins, Teamcity, GitLab, CircleCI, etc.

Infrastructure as a code

Eg. Pulumi, Terraform, AWS CloudFormation, Ansible, Chef, Puppet Enterprise, Google Cloud Deployment Manager, Azure Automation, SaltStack, etc.

Image baking

Eg. Packer (for VM), Docker (for Container)

Setup Artifactory/ Repository

Eg. Nexus, JFrog, etc.

Artifact scanning

Eg. Nexus Auditor, Jfrog Xray, Qualys, etc.

Secrets Manager

Eg. Vault, Google KMS, AWS Secrets Manager, etc.

Logging

Eg. ELK stack, Splunk, Fluentd, StackDriver, CloudWatch, etc.

Monitoring

Eg. Prometheus, Grafana, etc.

Cloud security and compliance risks

Eg. Google Apigee Sense, Amazon VPC PrivateLink, RedLock Cloud, Duo Security, etc.

Areas of concern across all tools

There is no optimum strategy for the following areas. Every team needs to formulate it on there own depending upon cost and time allocated.

1. High Availability: Availability can be measured relative to "100% operational" or "never failing."
2. Disaster Recovery: To maintain or quickly resume mission-critical functions following a disaster.
3. Backups: Creating and storing copies of data that can be used to protect tool against data loss 
4. On-boarding process: Process of introducing new users to your product.
5. RBAC (Role Back Access Control): Access rights only to the information users need to do their jobs and prevents them from accessing information that doesn't pertain to them

Primary tasks

Identify the “Big Bang” solution.

How to plant the first seed of your platform in a cloud provider? 
  It can be done either through a local machine or by some external build pipeline eg. Google cloud build, Azure DevOps, AWS Code pipeline, etc.

Implement VPN or Jump Servers.

1. IP Whitelisting.
2. Implement cloud security and compliance risk tools.

Setup your DEV environment

 1.  Automate infrastructure provisioning.
 2.  Create shared VPC.
 3.  Create VPC peering in the case of multiple VPCs.
 4.  Active Directory's LDAP to/from Synchronization with the cloud platform.
 5.  Set up CI/CD tools.
 6.  Provision artifactory.
 7.  Create a backend for the terraform state. Eg. cloud storage, Terraform Enterprise, etc.
 8.  Writing custom Terraform providers (if not available) for any tool.
 9.  Create CI/CD pipelines.
 10. Implement Sentinel policies if eligible.
 11. Form and implement IAC testing strategy.
 12. Bake VM images.
 13. Publish docker gold images.
 14. Implement monitoring and alerting solutions for every tool.

Subject to maturity of team and project requirements these tasks can be modified. Many times developers face ‘From where to start ?’ situation when brainstorming together in their formative discussions. Hopefully, this high-level open backlog can help them to kick-start the discussion and transform it into achievable user stories.