Creating a Slash Command endpoint with Cloudformation and ECS

Introduction

Nowadays, the term ChatOps, where enhanced communication tools enable DevOps teams to manage their systems, is becoming more common. Probably the most popular, Slack features Integrations, which includes Slash Commands: by typing “/” (slash) followed by specific commands, the user is capable of probing a service, listing the number of current users in a given platform, or filtering EC2 instances by given name.

Building a Slash Command is not hard, but it involves some pieces of software and infrastructure that must be glued together to provide a decent and scalable service. When using AWS, we have different ways of achieving this: having a plain EC2 instance with an EIP attached, or an Auto Scaling group of EC2 instances with an ELB and a Lambda function proxied by an API Gateway route are just some of the options.

In this guide, we’ll go through another option: building a Slash Command endpoint using Docker running on ECS cluster created by Cloudformation. The Slash Command is very straightforward: it will receive a string as a parameter and use it to filter the list of EC2 instances that an AWS account currently has, returning the EC2 instance ID, full name tag and its private IP.

Getting Started

For this guide, you will need:

  • A valid AWS account
    • If you follow the steps within this guide, you should spend less than USD 2, if you run the examples just for a couple of hours. Tip: Always remember to create a billing alarm in your AWS Account, so that you don’t be surprised with unwanted billings.
  • An IAM user with valid credentials to create the mentioned resources
  • Slack account
  • Text editor
  • Git setup, if you want to commit
  • Docker setup

CONCEPTS

In this guide, we’ll mainly cover two AWS products: Cloudformation and ECS.

Cloudformation is the Infrastructure as Code solution by AWS. Using Cloudformation gives you the ability to reproduce and version resources on your AWS infrastructure by using text files, just like developing code. You may customize parameters and outputs, create dependencies between resources, and deploy updates on your environment, creating a robust and modular set of components.

Cloudformation is based on templates, which are text files formatted as JSON or YAML data. Each template has a list of resources and may have parameters that you input to customize your environment and output pieces of information useful for keeping your operations up and running. Once you run your template (through AWS CLI or AWS Console Management), Cloudformation validates your file and creates your stack, which is all the resources defined in your text managed together. It is worth it to remember that Cloudformation itself is available at no additional charge, as AWS only charges for the resources created with the templates.

EC2 Container Service (ECS) is the AWS container management service which helps you run, monitor, deploy, and scale containers on EC2. It schedules the placement of containers easily on clusters of Amazon Linux or your custom EC2.

ECS is formed by clusters, services, and tasks definitions. Clusters are just groups of EC2 running the ECS agent and organized to run a specific set of services. You can run the Amazon Linux ECS ready instance or run your own customized EC2 on your cluster, but first you need to install the ECS agent as AWS recommends. Services are a given number of containers for a specific task definition that you want to run in a cluster. And finally, task definitions allow you to detail how ECS should run and maintain your containers, defining their volumes, ports, resources, etc. Most of the time, your application will require multiple task definitions, with multiple containers linked with each other, each of them with their own task definitions, representing a single component of the architecture.

HANDS ON

The Cloudformation templates, the code used to build the Docker image, as well as the Docker Hub repo are available in the end of this guide. Feel free to fork the code, tsuggest features, and pull the image to test. If you are testing locally, remember to pass as environment variables your AWS credentials to the container using “-e AWS_ACCESS_KEY_ID=<YOUR_ACCESS_KEY_ID>” and “-e AWS_SECRET_ACCESS_KEY=<YOUR_SECRET_ACCESS_KEY>”.

First, create your Slash Command on your Slack account. For this example, I used “/ec2search” as the command, but you can obviously set another name that suit your needs. When creating your Slash Command, copy your Slack Token – we’ll need this when creating our Cloudformation stack.

user_54533_594b69e31be9f.png

Next, we’ll create the resources we need on AWS. Open the AWS Console and navigate to Cloudformation. We’ll start with the VPC and subnets. The Cloudformation template vpc.yaml already creates a pair of private subnets (with no direct route to Internet) and a pair of public subnets (direct route to Internet). It also already specifies the CIDR IP ranges as default parameters. You can change those parameters if you need to use different ranges for your stack. Once you upload the file, you’ll be able to list the parameters and change what is needed:

user_54533_594b6a3bd4707.png_800.jpg

Verify the parameters and click next. It is recommended to add more key/value tags if you want to better organize your stack and the created resources. Click next and review your stack details before clicking on next to initiate the creation. After some minutes, your stack will be created and you will see the status as “CREATE_COMPLETE” on the console.

As the next step, we’ll create the ECS cluster. It includes an Application Load Balancer, an Auto Scaling Group, a Launch Configuration, a set of EC2 Container Service resources (Cluster, Service and Task Definition), Security Groups and IAM Roles. Again, once you upload the file, you’ll be able to list the Parameters and change what is needed:

user_54533_594b6a5c1a156.png_800.jpg

Specify the parameters and click next. In this example, I’m using a base Docker image created with PHP and AWS PHP SDK to listen HTTP Post requests, filter EC2 on us-west-2 region with a given string, and return the list to the user. Note that I selected the two Public Subnets created with the VPC stack, a t2.micro (if you think your app will need more resources, just select a larger instance) and pasted the Slack Token copied in the first step. If you have your own Slash Command application endpoint, you can Dockerize it, push to Docker Hub and change the DockerImage Parameter.

Click next and review your stack details before clicking on next to initiate the creation. In this particular stack, you need to grant permissions to AWS to create IAM roles for you – as a security step, you need to check the box in the Capabilities section. This stack will take some minutes more to be finished – you will see the status as “CREATE_COMPLETE” on the console.

When it completes, go to the Outputs tab and copy your ECSALB endpoint and paste it on your Slash Command Integrations Settings page. Remember to include “http://” and the “/slack-command.php” path. It should be like this:

user_54533_594b6a7a6ea99.png_800.jpg

Save your Slash Command Integration Settings and open your Slack app or URL and give it a try. You can open a chat with your own user to test it. If you type /ec2search and hit enter, you’ll get the command help usage. If you type /ec2search “linuxacademy” it will search for all EC2 instances in your default region matching “linuxacademy” on the tag Name. In this case, I only had the EC2 instance used for the own ECS Cluster used in this guide.

user_54533_594b6a997ff5b.png_800.jpg

WRAP UP

In this guide, we went through the setup of two Cloudformation stacks to create a Slack Integration (Slash Command). The first stack creates the VPC and subnets used by the second stack, which creates an Application Load Balancer, an Auto Scaling Group, a Launch Configuration, a set of EC2 Container Service resources (Cluster, Service and Task Definition), Security Groups, and IAM Roles.

The Slash Command example searches the list of EC2 in your main region and lists the matching Instances IDs and their Private IPs. Albeit testing simple PHP code, this guide shows a very easy and replicable way of creating your own Slash Command to operate with AWS API and bootstraping your ChatOps skills. There are many improvements that could be done to apply this stack on real life operations. For instance:

  • Filter the EC2 instances using regular expressions and by tags
  • List Auto Scaling Groups and the EC2 instances that are attached to them
  • Search for EC2 instances in multiple regions at the same time
  • Start/stop EC2 instances

When you finish testing your new Slash Command, remember to get rid of all resources created within this guide. Fortunately, Cloudformation can delete the resources created with it. You just need to navigate to Cloudformation console and delete each of the stacks created with this guide. Please note that as the ECS stack depends on the VPC stack, you must delete the ECS stack first and wait for its deletion to succeed before deleting the VPC stack. Also remember that any additional resources created manually will not be removed by Cloudformation.

Sources / Resources

  1. https://github.com/stasta/slashcommand-ec2
  2. https://hub.docker.com/r/stasta/slackcommand-ec2search
  3. https://github.com/awslabs/ecs-refarch-cloudformation/blob/master/infrastructure/vpc.yaml
  4. http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/quickref-ecs.html

Looking for team training?