Credit to the organisers of DevCon in Tel Aviv last month for a speakers line up which included Evgeny Goldin of Spotify in Berlin who described his journey with Spotify, how they do continuous integration with Docker and their own open source tool Helios, the orchestration tool for deploying and managing containers across multiple servers.
Prior to Docker’s suite of zero to Docker orchestration services Machine, Swarm and Compose, Helios was built as a solution to make sure that containers are deployed and run as smoothly as possible. This means that Spotify’s jump into building and using it’s own orchestration tool in anger sees it using something which is tailored specifically for their use without using any of the increasing palate of tools entering the marketplace.
Goldin describes Helios, “it’s a system built from a couple of simple blocks. Zookeeper keeps all the data about your deployments and you use a command line tool (CLI) to talk to an entity called master which is just another machine, then the master talks to Zookeeper and Agent (also some other machine where your services will be running) watch for changes and respond to these changes, so it’s very simple. You ask master to deploy a service, this information is stored in Zookeeper and Agent pulls it down and start running the service.”
David Xia from Spotify also explains in this blog post how Helios works with the lifecycle of a Helios deploy command and status command.
Goldin demonstrates examples of how they use Helios both with VirtualBox and with AWS and ultimately describes continuous delivery as a journey, complimented by the evolution of Helios. The Spotify team are working hard on improving reporting and visualisation but in the face of a growing number of tools available like CoresOS Fleet, Docker Swarm & Compose and AWS ECS, Goldin still says it is one of the easiest he has used.
