Infrastructure as code in a virtual machine environment

I’m a senior DevOps engineer working in the AWS Virtual Machine Environment-Replacement (VME-R) team within the Health and Working Age programme. We’re a very close team who work together to support the VME-R project.

The VME-R project is a key priority for DWP Digital. Its aim is to remove DWP’s dependence on our existing VME estate, and onto a modern distributed technical platform. With more than 80,000 people in DWP, it’s a huge project.

DevOps is a methodology and set of principles within engineering that speeds up development. It also allows incremental improvements by delivering code changes more frequently and reliably. This is called continuous integration (CI) and continuous deployment (CD), and is implemented through a CI/CD pipeline.

At DWP Digital, DevOps is a growing culture. We have teams of DevOps engineers applying CI/CD techniques alongside colleagues working in Agile methods.

We use a wide range of tools, technologies and techniques to support the VME-R project and have a fairly robust toolbox.

One of the key techniques we use is ‘Infrastructure as Code’ (IaC). Its meaning isn’t always clear, but IaC has been with us since the beginning of the DevOps culture.

IaC is the name given to the techniques used to describe, provision and manage IT infrastructure. This includes the compute, storage, network layers, and other resources as part of the deployment of modern applications.

This is essentially the practice of treating any infrastructure as if it were code, rather than through standard operating procedures and manual processes.

Using IaC lets us adopt beneficial practices such as source and version control, peer code reviews, automated unit testing, functional testing, integration testing, release tagging, release promotion. And it means we practice continuous integration and continuous deployment.

It gets us away from long, manual, error-prone processes, and helps us to reduce costs, remove human error and therefore improves efficiencies.

Within the AWS VME-R team, we employ IaC to build and maintain our infrastructure using tools such as Terraform by HashiCorp.

Using Terraform, the infrastructure for an application can be quickly created, updated, destroyed, and recreated based on a known configuration that is version controlled.

We also use Terraform Workspaces, which is a feature of Terraform. This allows Terraform to use something called a state file, this saves the state of any given environment, such as production, development and test environments.

Using these Workspace state files allows Terraform to have different configurations. We can use the same Terraform code, but have slightly different configurations for each environment, rather than having to duplicate all the Terraform code for each environment.

For example, if we were to create several virtual machines for development, we would change Terraform to use the development workspace, and make changes to the configuration of these virtual machines. Then, when we apply the changes, this will only affect the development state file, and only make changes to the development environment.

The same would be true if switching to the test workspace, where we could have a completely different configuration, but using one set of Terraform configuration code.

Following this approach allows the re-use of the same Terraform code, but with specific variables for either workspace, rather than duplicating code for multiple environments, and having to maintain this code.

The key benefit of having your Infrastructure as Code is that there is no configuration creep. Everything is documented, it is created exactly the same every time, the code is in a stable known working state, and is re-usable by other projects.

Another huge benefit is the lack of documentation, since the code itself will document the state of the environment. This is particularly powerful because it means, for the first time, that infrastructure documentation is always up to date.

And, as any change is pushed to GitLab, any environmental changes are logged, as GitLab is used to host the Terraform code.

We also have a GitLab CI/CD pipeline, which runs when any new commits are pushed to the GitLab repository. This checks the quality, formatting and syntax of the code prior to the code being applied to any environment.

This also allows environments to be destroyed and re-created at will, in very little time.

Gone are the days of having to provision a virtual machine manually and having different configurations for each machine created, if changes to the specifications of machines was required, having to amend each manually, or even creating manual scripts.

This can be achieved by simply changing a parameter with the Terraform code, committing the code to GitLab and applying the change to the infrastructure.

Want to find your digital career? Have a look at our digital jobs.