cloudformation

CloudFormation @ foodpanda

CloudFormation is a declarative and flexible language in JSON format that describes your AWS infrastructure. You can enumerate what AWS resources, configuration values and interconnections you need in a template and then let AWS Cloudformation do the rest.

Quite simply, it’s your infrastructure in code.

It’s very easy to use and you can start with some sample templates that AWS has made available – https://aws.amazon.com/cloudformation/aws-cloudformation-templates

 

When you deploy your Cloudformation templates, you create a “Stack” in the AWS Region of your choice.

The most difficult thing for us was visualizing our Stacks, keeping GROWTH and Fl3xIb1lity in mind, our Founding Company isn’t called “Rocket Internet” for no reason – https://www.rocket-internet.com

 

Be prepared to make mistakes. A Production Stack cannot easily be destroyed and recreated without downtime, but, it’s not impossible.  

 

Some things that helped us along our journey:

– Set everything up with Autoscaling in mind, you can have an Autoscaling group with Min = 1 and Max = 1.

– Use Tags wherever you can.

– Be aware of Cloudformation limits – http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/cloudformation-limits.html

– Be aware of AWS Service limits, like IAM inline policy limits – http://docs.aws.amazon.com/IAM/latest/UserGuide/reference_iam-limits.html

– Know the language’s intrinsic functions e.g {“Ref” : “AWS::NoValue”} – http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference.html

– Use nested stacks. Although we had the luxury of using 1 template in many regions and stacks, with multiple fleets and many AWS services, 7000 lines of JSON Cloudformation code is not maintainable, certainly not scalable, and so we learn and we become better 🙂

– Consider Costs – Use Reserved instances, Spot instances and most definitely Autoscaling, so that you can scale down when you do not need the instances. You should also try to find the “Sweet Spot” by right sizing your instances.

– You won’t be able to use AWS Services for everything.. bummer. For example Redis Elasticache does not support the SLAVEOF command.. yet, similarly AWS Elasticsearch, does not support custom plugins.. yet 🙂 and AWS Aurora is not available in all AWS Regions.. yet

 

Once you have your infrastructure launched, look into some orchestration software, like:

– Puppet – https://puppetlabs.com

– SaltStack – http://saltstack.com

– Chef – https://www.chef.io

 

Once you have your infrastructure and applications in code, you will want to, quite naturally, do other things like DNS also via APIs.

We use Cloudflare – https://www.cloudflare.com

Cloudflare has a fully featured API that we can use in conjuntion with AWS command line interface (CLI) – https://aws.amazon.com/cli managed via Puppet to create specific resources like DNS CNAMEs, SQS queues, SNS topics, IAM user policies and s3 bucktes.

 

Some of my other presentations may help you:

– A DevOps Guide to Database Infrastructure Automation for eCommerce – http://www.riaannolan.com/a-devops-guide-to-database-infrastructure-automation-for-ecommerce

– Cape Town Linux Users Group (CLUG) – PXE boot + kickstart + puppet – https://archive.org/details/clug-2010-07-27-kickstart-puppet
Please connect with me on Likedin.com – https://de.linkedin.com/in/riaannolan

Posted by on 12 Jan 2016

Director Software Engineering @ foodpanda

Leave a Reply