We developed these tools to run, manage and automate our applications. We’ll explain how they help save us time, maintain security, and make our deployment pipeline run with unbelievable efficiency. So be warned if infrastructure isn’t your forte, this blog will get a bit technical.

Previously, we’d preconfigure servers with Ansible, (an engine for automating IT projects) and create an Amazon Machine Image (AMI) on Amazon Web Services (AWS) using Packer. We’d then run a CodeDeploy deployment on an EC2 instance (a virtual server running in AWS Cloud) upon startup. 

It worked well, but we wanted to minimise the difference between production and development environments. The reason? An application might work perfectly on a developer’s machine, but not on the server. So we started looking at alternatives.

Docker looked like a promising way of getting around this problem. It would allow us to run containers from pre-baked images, with each container isolated and bundled with its own sets of libraries and tools. And importantly, the same image could be used for development as on servers.

However, we lacked an orchestration service that would allow us to easily start, stop, and replace containers. We also needed a way of making sure the containers were working correctly. So when Amazon launched Elastic Container Service (ECS) – which handled all of these tasks – we knew the time was right to consider switching to Docker.

We already had the know-how to build containers – and did it successfully with CircleCI – but we still had two problems:

1. How do we easily (and securely) configure applications running on Amazon ECS?
   
2. How do we automatically configure deployed docker applications?
   

We set out to overcome both of these obstacles by developing our own solutions.

Our goal was to automate as much app configuration as possible. We wanted environments to be almost self-provisioning.

First of all, applications needed to be configurable via environment variables, according to the 12-factor methodology. So we enabled this by making a few small changes to our applications. Some of the frameworks we use – Django, Ruby on Rails, NodeJs and Silverstripe – have native support for 12-factor, but some had to be modified to enable it.

Secondly, on ECS, there are task definitions that have environment variables set. But we had a few problems with them, including:

• Anyone with read-only access to ECS can read all environment variables, including secrets.
  
• It’s so tightly coupled that to change one variable we’d have to change the whole task definition and redeploy the service.
  

To overcome these limitations, we came up with a solution: use AWS Systems Manager (SSM) Parameter store. It’s basically a key-value store that can keep keys in a tree-like structure which can be read via one API call.

We wrote ssm-parent, a custom Docker entrypoint, to fetch all required keys from the SSM Parameter store. It exposes parameters as environment variables for the application. Our aim was to avoid modifying applications to integrate them with the SSM Parameter store.

Plus, most of the parameters are automatically generated by Terraform – database, cache endpoints and other resources. Developer-provided credentials (such as API keys) are stored in an encrypted file and put in the same SSM Parameter store by Terraform.

Before deploying the code, we may need to run some pre-deployment commands – such as running a database migration. We may also want it to run automatically when a special deployment git branch is pushed to the repository. And we need an event trail.

This all becomes possible with our new ecs-tool. It does all of the above, plus it can:

• run anywhere because it’s just a single binary
  
• deploy multiple services in parallel
  
• provide ECS Service events while deploying
  
• run commands, and print the output.
  

We’ve found these tools useful in our everyday work – we hope you’ll find them useful too!  Our tools are open source, so we welcome you to contribute, suggest features, or give feedback. Just raise an issue or send a pull request to either: 

• https://github.com/springload/ecs-tool or
  
• https://github.com/springload/ssm-parent.
  

And, feel free to check our other open source projects!  

Need a hand automating your workload?

We have certified AWS engineers on our team who can help with your DevOps or cloud workloads. For a friendly chat, get in touch.