Continuous Integration and Delivery pipeline for the Department of Transport’s On-Demand Transport platform

With the adoption of a DevOps approach, The Department of Transport for Western Australia needed to speed up the deployment of the On-Demand transport application and services throughout its environments, using a scalable, low cost solution.

The Scenario

The Western Australian Department of Transport have adopted Amazon Web Services as their Cloud platform for the On-Demand Transport suite of applications using a microservices architecture to take advantage of the agility and flexibility that this model provides.

One of the tenets of DevOps is automation; to provide robust and repeatable processes. An area commonly addressed is that of a continuous integration and delivery pipeline to deploy your application code and infrastructure across all your environments in a consistent manner.

Continuous Integration also provides fast feedback to the team on build and code quality issues.

The WA Dept. Transport turned to Akkodis to solve this.

The Solution

There are several tooling options that you can use when selecting a CI/CD pipeline and the chosen solution for this environment was Jenkins. It is a widely popular, open source, very well integrated into AWS and provides many plugins to make it a flexible solution.

The solution was required to be highly available, scalable, cost effective, and expandable to other projects and teams. It also needed to provide a mechanism for manual decision gates to manage the deployment of the software into the controlled environments. The defined pipelines should also follow the same infrastructure as code principle that is used to define the application’s infrastructure and be maintained in the source code repository of the application.

There are several ways that Jenkins can be deployed on AWS but to meet the requirements defined above, the following architecture was used:

  • A single Jenkins controller/primary is deployed to an EC2 instance with an Auto Scaling Group to provide the required high availability.
  • A spot fleet is used for running the Jenkins build agents responsible for building the software artefacts, providing a cost effective and scalable compute platform.
  • The EC2 deployment agents are provisioned by Jenkins, allowing the deployment to the various environments. These agents run with an instance profile that allows the agent to assume a cross account role to deploy the infrastructure and application artefacts only to the environment in which it is entitled.
Pipeline Pipeline

The pipeline is triggered by a pull request to the Master branch of the source code repository, where the build and unit tests are executed. A static code analysis is also performed and only if these steps pass and the appropriate quality gates are met, is the code deployed automatically to the development environment.

Each stage in the pipeline calls out to “hooks” using a standard naming convention to allow the stages to be customised for each project or component.

Build Pipeline Build Automated Development Pipeline

A deployment to the test environment is then triggered following a successful deployment to the development environment where the pipeline awaits an approval before continuing. This process is repeated in the remaining environments until it eventually reaches the production environment.

Build Approved Deployment Pipeline Build Approved Deployment Pipeline

The Outcome

The time and effort required to deploy the application components has been dramatically reduced and any inconsistencies introduced as part of previous manual steps have been eliminated.

The result is a pipeline that can easily be adopted by any project for the Department using a standard template, which calls specific build and deployment tasks to meet the individual needs of the project.