Ansible - Provision Docker Swarm Mode (1.12)

Lately I have been working quite a bit with the latest Docker Swarm Mode released in Docker 1.12 and so far it has been pretty awesome. The days of spinning up a Docker Swarm cluster with all of the complexity (Consul, Registrator, and etc.) are old-school now. To do a comparison you can checkout a great post by Scott Lowe here and also checkout a Vagrant lab that I put together for learning almost a year ago here. With the latest release of Docker 1.12 they have made the process SO much easier and much less complex. To read up on the functionality and ease head over here. Now the purpose of this post is not to go step by step on how easy it now is to provision a cluster but rather to take it to another level. Let’s automate the provisioning using Ansible instead. This allows for a much more consistent and predictable provisioning method as well as the ability to easily scale your cluster(s). In addition to this post I highly recommend you fork or clone my GitHub repo to spin up a 7-node Docker Swarm Mode cluster completely automated and provisioned using Ansible and Vagrant. You can read more about the usage and some examples within that repo as well. So what I WILL cover in this post is the process along with examples of how to use Ansible to provision a Docker Swarm Mode cluster.

• Assumptions:
  • You have already provisioned your nodes in which will comprise your Docker Swarm Mode cluster
    • OS Installed (Ubuntu 16.04 in my case) as well as Docker Engine installed…Checkout my Ansible role for this as well here
  • Identified the interface which will be used for all Docker Swarm Mode communications and such (enp0s8 in my case)

So the first thing we will ensure is correct is our Ansible inventory and groups. We need to define the overall Docker nodes group (docker-nodes) as well as which nodes are considered managers (docker-swarm-managers) and which nodes are workers (docker-swarm-workers). Below is an example of the Ansible inventory that I use:

[docker-nodes]
node[0:6]

[docker-swarm-managers]
node[0:2]

[docker-swarm-workers]
node[3:6]

Now we need to define some Ansible variables which are required to successfully provision our cluster:

docker_swarm_addr: "{{ hostvars[inventory_hostname]['ansible_' + docker_swarm_interface]['ipv4']['address'] }}"
docker_swarm_cert_expiry: '2160h0m0s' # Validity period for node certificates (default 2160h0m0s)
docker_swarm_dispatcher_heartbeat_duration: '5s' # Dispatcher heartbeat period (default 5s)
docker_swarm_interface: "enp0s8"
docker_swarm_managers_ansible_group: 'docker-swarm-managers'
docker_swarm_networks:
  - name: 'my_net'
    driver: 'overlay'
    state: 'present'
  - name: 'test'
    driver: 'overlay'
    state: 'absent'
docker_swarm_primary_manager: '{{ groups[docker_swarm_managers_ansible_group][0] }}'
# docker_swarm_primary_manager: 'node0'
docker_swarm_task_history_limit: '5' # Task history retention limit (default 5)
docker_swarm_workers_ansible_group: 'docker-swarm-workers'
docker_swarm_port: "2377"

As you can see from the above we can define which node will be considered as the docker_swarm_primary_manager in two different ways. The first (default) method is to use the Ansible inventory group name and choose the first node in that group or the second method would be to just define the actual node name. Some of the other variables are to define/update actual Docker Swarm cluster settings (default values by default) as well as defining some Docker networks to manage.

And now for the actual Ansible playbook that we will be using to make all of the magic happen behind this:

As you can see we are actually capturing both the manager and worker tokens for the cluster as well. This is the key for success here. So how are we doing this? Let’s take a look at the tasks that accomplish this.

First we need to capture the tokens on our docker_swarm_primary_manager:

- name: docker_swarm | Capturing Docker Swarm Worker join-token
  command: "docker swarm join-token -q worker"
  changed_when: false
  register: "docker_swarm_worker_token"
  when: >
        inventory_hostname == docker_swarm_primary_manager

- name: docker_swarm | Capturing Docker Swarm Manager join-token
  command: "docker swarm join-token -q manager"
  changed_when: false
  register: "docker_swarm_manager_token"
  when: >
        inventory_hostname == docker_swarm_primary_manager

If you notice from above we are literally just running the docker swarm command to query our join-token for the manager and worker tokens and registering those facts. Easy as that!

Next we need to define the facts for both join-tokens in order for our additional nodes to successfully join our Swarm cluster. And we do that with the following tasks:

- name: docker_swarm | Defining Docker Swarm Manager join-token
  set_fact:
    docker_swarm_manager_token: "{{ hostvars[docker_swarm_primary_manager]['docker_swarm_manager_token'] }}"
  changed_when: false
  when: >
        inventory_hostname != docker_swarm_primary_manager

- name: docker_swarm | Defining Docker Swarm Worker join-token
  set_fact:
    docker_swarm_worker_token: "{{ hostvars[docker_swarm_primary_manager]['docker_swarm_worker_token'] }}"
  changed_when: false
  when: >
        inventory_hostname != docker_swarm_primary_manager

Now that these facts are registered we can now leverage these on our additional Swarm nodes.

Some additional tasks we are able to do are to also manage our Docker networks including overlay networks for our Swarm cluster. We can also update some additional settings for our Docker Swarm cluster which we do in the last three tasks of the playbook.

So there you have it. An easy way to provision a Docker Swarm (1.12) mode cluster using Ansible allowing for easily scaled out provisioning.

Stay tuned for additional posts coming in the future in regards to some additional Docker Swarm mode goodness.

Enjoy!