Automatic Deployment and Continuous Integration with CircleCI and ElasticBeanstalk

The Workflow

Let me describe the workflow we were trying to accomplish.

We wanted to run tests on every merge into 'master'. If the tests pass, we wanted the code to be automatically pushed to a staging server.

Once we verify staging is working properly, we want to merge that code into the 'production' branch. Tests will be run again and if all is well, the code will automatically be pushed to a production server.

The Tech Stack

We saw plenty of options and decided to go with AWS because of how many services they offer, the amount and quality of their documentation, and they have a AWS Pop-up Loft right across the street from us.

We chose CircleCI in part because of this page describing automatic deployment with CircleCI, Docker, and AWS. We initially wanted to use Docker, partly because we saw the benefit in having the same environment throughout our workflow, but mostly because it seems like a really cool technology to work with. But once we started working with AWS, things were just going smoothly without Docker and we decided to put Docker on the back-burner until we saw a stronger use case.

Workflow + Tech Stack

So, to sum up our workflow using the tech stack we decided on:

Any push to the branch 'master' will trigger tests to run in CircleCI. If those tests pass, CircleCI will use the AWS Elastic Beanstalk CLI to deploy to our staging server.

Once we manually verify everything is working properly on staging (this process is to be automated in the future with navigation scripting/testing utility like CasperJS), we merge the code into the branch 'deployment'. When CircleCI sees a merge to deployment, the tests are run once more and if everything passes, CircleCI pushes the changes to our production server.

Our app also requires a node script to be run for each active user. This node script needs to be run on it's own unique public IP address.

To accomplish this task, we used the AWS-SDK for Javascript in Node.js to spin up and spin down Elastic Beanstalk environments whenever we need.

Below is the circle.yml file we are using to automate our tests and deployment.

machine:
  python:
    version: 3.4.2
  environment:
    DISPLAY: 8000
    MYSQL_DATABASE: 'circle_test'
    MYSQL_DATABASE_USER: 'ubuntu'
    MYSQL_DATABASE_PASSWORD: ''
dependencies:
  pre:
    - pip install awsebcli
    - eb init thesis -p Node.js -r us-west-2
test: 
  override:
    - grunt test
deployment:
  staging:
    branch: master
    commands:
      - eb deploy thesis-staging --timeout 10
  production:
    branch: production
    commands:
      - eb deploy thesis-production --timeout 10

And here is our .ebextensions/config.config which the awsebcli command-line tool uses to configure our Elastic Beanstalk environment.

packages:
  yum:
    git: []
    openssl-devel: []
    expat-devel: []
    gettext-devel: []
    zlib-devel: []
container_commands:
  02_install_node:
    command: "yum install -y nodejs npm --enablerepo=epel"
  03_install_node:
    command: "npm install -g bower"
  05_npm_install:
    command: "npm install"
  07_bower_install:
    env:
      HOME: "/usr/bin"
      PATH: "/usr/bin"
    command: "bower install --allow-root"