Building an Automated Serverless Deployment Pipeline

The Challenge

Though microservices can be developed and deployed in isolation, the client needed all modules to be built and deployed together to have a more integrated approach. The different environments/stages and deployment process needed to be separated without any dependency between these environments. The deployment process was required to be very flexible and the implementation needed to be as simple as possible to facilitate the creation of a new environment.

• Project Environments

  
  

In this project, there were three different environments (DEV/QA/ PROD) identified. 

1. DEV – Development environment
2. QA – Quality Assurance environment
3. PROD – Production environment

The process should be flexible enough to create a new environment at any given point too.

How 1 Billion Tech helped

The 1 Billion Tech development team helped the client to automate the complete development and deployment process.

The Solution

The solution was fully automated based on AWS DevOps Services such as AWS CodeCommit, AWS CodeBuild, AWS Code Pipeline and the Open Source Serverless Framework. This stack helped the team with Source Control.

• Source Control

The application used Git version control system for managing the source code and used its branching strategy to separate our source code version to multiple environments. As illustrated, there are three different environments and we maintained three different Git branches for each environment (DEV/QA/PROD). All developments are managed by the origin/DEV branch. When there is a feature development, the developer needs to create a new branch based on the origin/DEV branch. After completion of the new feature development, the feature development branch needs to be merged with the origin/DEV branch as well (see Figure 1).

Figure 2 below shows you how to manage separate branches for different environments.

• Code Build

This application fully utilizes the AWS serverless architecture and many of its DevOps features such as CodeCommit, CodeBuild and CodePipeline. While CodePipeline orchestrated the complete DevOps channel, CodeCommit and CodeBuild helped to maintain the code build process. 

However, CodeDeploy was skipped in the CodePipeline, allowing the Serverless Framework to take over the task (see Code Deployment section).

The code pipeline consisted of two main stages (see Figure 3).

1. Setting up the Source Provider (CodeCommit/GitHub/S3).
2. Setting up the Code Build Process (CodeBuild) - This stage provides the capability to listen to source code changes for the specified branches.

The Code Pipeline execution strategy used a source code change event. With this approach, any commits pushed to the Git branch will trigger a pipeline execution (see Figure 4).

As the “Dev” branch is usually busy with feature commits, bug fixes, experimental code changes, etc., developers may not need to execute the pipeline for every change that they push into it. In order to tackle this, we added another approval stage for the “Dev” environment Code Pipeline, which required a manual approval to continue with the pipeline execution (see Figure 5).

• The .env file

As mentioned above, this project can have multiple running environments with a single source code repository. Therefore, we had to maintain a file (.env file) that contains all the environment-specific details, which can be used to fetch configurations for the running environment (see Figure 6). 

• DevOps Files

  
  

When we consider microservice level deployments, each microservice was built as a BuildProject. The BuildProject requires a build specification file that contains details about the build environment and command execution. Hence, a separate DevOps file is maintained here (see Figure 7).

• Code Deployment

  
  

After completing the code build process with AWS CodePipeline, the deployment process was managed with the help of the Serverless Framework. With this, all related Lambda functions can be updated using its built-in CloudFormation templates.

BENEFITS TO THE CLIENT

1. A fully automated code build and deployment process
2. New releases can be easily deployed
3. Ability to introduce new microservices to the project with minimal changes
4. All pipeline services are fully managed by AWS
5. Minimum effort to create a new environment