https://arizona.box.com/v/automate-things/
Mark Fischer from the University of Arizona is presenting on Software Defined Infrastructure with AWS Cloud Formation, Docker, and Jenkins
Mark – 20 years of web app dev, 5 years of infrastructure tools dev, and 2 years of AWS infrastructure dev.
Goals – Codify infrastructure decisions, document deployment process in code, ensure repeatable operations, empower developers and product owners to get more done quickly. Want to move to a place to quickly and reliably can replicate deployments.
Just because you move to AWS doesn’t mean you get automation. When you configure things manually in the console, it can leave lots of things hanging – Security Groups, SSH keys, IAM roles, etc. A three tier web app may have 10+ separate resources that need to be configured.
Automation progression: Manual Infrastructure Provisioning -> Cloud Formation;Manual Environment Configuration (libraries, java versions, etc) -> Docker; Manual code deployment -> Jenkins
CloudFormation
Cloud formation is a text based template and a deployment engine: JSON text document, defines AWS resources and resource relationships, has a parameter system so you can spin things up with different names, etc. CloudFormation tracks resources and can handle de-provisioning of all of the infrastructure pieces. If you don’t like JSON, you can now use YAML. Nicer to work with, and you can have comments.
Configuration as Code: CloudFormation allows you to codify your infrastructure deployments. Can track modifications to templates.
UA CloudFormation catalog. https://bitbucket.org/ua-ecs/service-catlog
Docker
Codifies server configurations in dockerfile. Files can be versioned and managed. There are lots of Docker enabled environments.
Jenkins – DevOps Glue
Can fill gaps where you find yourself typing a couple of commands to paste automations together. Lots of functionality: checkout a git repository, build a Java Project; run shell scripts; integrations with slack, email, etc.
UA migrating financial system to AWS – Jenkins jobs involved in lots of steps. Using AWS OpsWorks (managed Chef environment). Usernames, passwords, keys, can be stored securely in Jenkins and then passed to jobs. Devs can launch environments for new code branches – whole process takes 20-25 minutes.
Jenkins allows restricting access to jobs – can create, can run, can manage secrets. Allows you to abstract AWS deployment capabilities. BAs can perform their own database refreshes (e.g. prod -> dev), DevOps staff can manage Jenkins jobs without knowing all the credential secrets.
Sticking Points
Automation takes more time up front to get right, but subsequent deployments are shorter. IAM permissions can be a pain if you have lots of apps in the same account (making sure that people only have access to work on their app); Persistent file storage is hard. Use S3 and RDS as much as possible. EFS makes this slightly easier.
Oren Sreebny 