Harry's Engineering

Harry's
Engineering
Blog

02.01.14

Dynosaur: A Heroku Autoscaler

By: Andy O'Neill

We are excited to announce the first release of Dynosaur, our Heroku autoscaler that uses the Google Analytics Live API to dynamically provision Heroku dynos based on the number of current users on the site.

Heard you like Heroku

The main autoscaler engine lives in its own gem and can be run on the command line with a config file (e.g. as a Heroku worker process!) but we also wrote a simple web interface for it. Dynosaur-rails can be trivially deployed as a Heroku app, and it allows you to manually tweak the minimum number of dynos at runtime, in case your autoscaler estimates are off.

How it works

Dynosaur uses one or more plugins to determine the estimated number of dynos required, then hits the Heroku API to set the correct value. We err on the side of conservative and wait some fixed amount of time after any change before reducing the number of dynos.

The first (and only!) plugin uses the Google Analytics Live API (currently in private beta) and a configurable ‘users per dyno’ value to estimate the required number of dynos.

This chart shows how the Analytics plugin estimated our required dynos during a test. The blue line is the number of active users on the site. The orange line is the number of dynos required (configured to 10 users per dyno for this test).

Google Analytics Chart

For the same time period, we can see that the combined estimated number of dynos (green line) follows the Analytics plugin, except it has a minimum of 1 dyno and never drops to zero. The actual selected number of dynos (blue line) follows the estimate as it climbs, but as it descends it goes slower, since we have the blackout interval set at 5 minutes.

Heroku Chart

The Web Interface

We made a little Rails app to make it easier to deploy, monitor and configure Dynosaur. It has very few features:

Mmmm, Bootstrap! Have some screenshots:

Status View

Configure Main App

Configure Google Analytics Plugin

Securing the Web Interface

We restrict access to Dynosaur-rails with http basic auth (configured via Heroku environment variables) and an IP whitelist (also configured through environment variables).

Pluggable

Number of active users suits our usage well, but one could equally use app response time from NewRelic or some other core metric for your app to decide how to scale. See the README for more info on extending Dynosaur with new plugins.

When multiple plugins are enabled, Dynosaur will combine them by taking the maximum value, thus each plugin really defines a floor for the number of dynos.

Stats

Dynosaur can optionally use Librato to collect some statistics on its operation. We track the estimated dynos (based on combined estimates from all plugins), and the value we have actually chosen (including delay before downsizing and min/max dyno settings). Also, for each plugin configured we track the value (e.g. ‘current users on site’) and the estimate of dynos required by that plugin. To enable statistics, just create a free Librato account and configure Dynosaur with the account email address and API key.

Awesomeness

Dynosaur is letting us skip that whole rigmarole of wake-up-at-6am-because-we-were-featured-in-the-NYT-and-traffic-is-through-the-roof. More sleep equals happy developers, and this new API from Google is proving very useful to us from an operations perspective. We want you to be happy too, so please use it, write new plugins, fix bugs and open pull requests!