Orchestration: What it is and its Importance to DevOps

A symphony, unlike a jazz band, requires orchestration because it has even greater specialization with expertise in using different musical instruments.  (It's been said that, "the violinist section thinks that the french horn section are a bunch of cowboys.") DevOps in an enterprise is in a similar situation (developers, sysadmins, network ops). Each group has difference skill sets and tools. To add to the complexity, within each group there is a variance in tool preference.

Working together effectively

Standardizing the toolset across groups (let's all play tuba) isn't really practical. The people doing the work are effective with the tools they know.  While there can be movement *within* groups toward some standards (the brass section settles on trumpet), selecting a single tool *across* these groups is likely to fail as some tools are better suited for certain contexts than others.

For example, we can't just all use spreadsheets to solve our problems as they don't address the ability to configure the networking domain. Another example is that it's not practical that we all become software developers as the cost is high to build new network management products when there are already apps that address some of our needs. Some tools are well suited to solving one, targeted problem, and it'd be a pity to not use it in those situations.

The sweet spot for devops is in script automation. In this realm, developer practices such as the use of source control can be used to create "communication through code" or a "single source of truth." Most script automation is open source with broad community support and frequently there paid support is available too. Alternatively, macro recording strategies typically target a specific vendor's hardware and their configurations aren't driven from source code.

Script automation without being used with orchestration is subject to the following:

1. hardcoded values
2. applicable to only one piece of hardware
3. hard to maintain beyond the original script developer
4. once there are hundreds of scripts with hundreds of lines, it's difficult to:

• get a view to what scripts are applied to what hardware and 
• how frequently (applied by schedule or continuously or ...)

Points 1, 2, and the second bullet of 4 can be mitigated by the scripts being driven by orchestration.  Points 3 and the first bullet of 4 is aided by the use of source control.

Communication through Code

Just as development teams preserve their efforts in source control so everyone on the team or organization has visibility, placing the configuration of devices into source code should:

• make visible what automation is affecting what production devices
• become a system of record

That resolves the first part of the above fourth bullet.

Orchestration

Wouldn't it be nice if you learned to play ONE instrument well, you could operate thousands of others? Then later, if you had to add a new one, perhaps a voxophone, everyone else who could play piano could immediately GET, at some level, how to work with the voxophone? Then later, if something were to go wrong with the voxophone, everyone knew the scope of the problem and where to start, even if they weren't the voxophone player.

This is orchestration.  One of its benefits is that there is a layer of standardization across a diverse toolset, without ruining the ability to get things done with the best specific tool at hand. 

This ability to "normalize" interacting with a variety of automation (python, tcl, CLI, webAPIs,...) through one or a few well known tools would:

• simplify the effort to get a high level understanding of what's happening
• allow others (those who have less experience with the automation script or tooling) to make orchestration adjustments to a script without requiring digging into the body of the script
• anyone can understand how, when, and what (devices) is being managed
• greater ownership, beyond the script author, of what's happening across all the tools

For example

One way Ansible can orchestrate scripts is through the use of the Scripts module and then Ansible facilitates execution by:

• transfers the script to the remote node for execution
• passing arguments from the playbook to the script
• executes the script in the shell environment on the remote node.