What is Workflow Automation and Why You Should Use It

Section 3.2 of CompTIA'sCloud+ certification exam covers workflow automation. You may be asking yourself what this is and why you need it? After all, can't you just manually spin up resources and be fine? Why do you need complicated processes to spin up these resources? In this post, we'll cover the topics that can answer these questions.

What is Workflow Automation?

Workflow automation is a process by which blueprints are defined for a certain outcome — and steps are taken to achieve that end result. These are commonly referred to as runbooks. Think of this as a recipe for making pizza. Except in the workflow automation world, we use terms like runbooks instead of recipes. This will list the ingredients or requirements such as cheese, sauce, toppings, dough. Your recipe may have you make the dough from scratch or you may opt to get pre-made dough pizza crusts.

This is a great example of the flexibility of workflow automation. It certainly can describe building something from scratch but it can also build upon other pre-made building blocks. For example, if you are spinning up a new Ubuntu virtual machine, you likely won't be building that from scratch. You would typically reference an Ubuntu image made available by your cloud provider.

There are quite a few reasons to use workflow automation. We'll highlight the top three below and expand upon them a bit. There are certainly many other reasons, but these are the major drivers.

1. Consistency

When using automation, one of the key benefits is consistency. When manually setting up environments, it is easy to skip steps or forget them. These mistakes can cause long term issues that aren't easy to track down. When using workflow automation, it creates a reliably duplicable environment and consistency with each instantiation of that environment. When making changes to an environment, these automations can help ensure those changes are consistent across the environments.

Going back to our pizza recipe example. If you don't have a recipe, your pizza may not turn out consistent. You may make them slightly different or use different ingredients. You may not want to go back because sometimes it tastes great and other times it is less desirable. In infrastructure though, consistency is key and provides stability of the environment.

For example, AWS CloudFormation uses declarative configuration management in which you define the end state — and it figures out how to get there — no matter what state you started with. This provides very consistent results whether you are standing up new infrastructure or bringing existing infrastructure into specification.

2. Transparency

Workflow automation and runbooks can usually be stored in a source repository or system where multiple people can review it. This review process is important as everyone can be on the same page in regards to what the deployment should look like. Our pizza reference breaks down a little on this one, but parts of it still hold true.

If you are gluten- or lactose-intolerant, you likely want some transparency into the ingredients so that you know if you can eat the pizza. Also, if you are looking to restrict your diet, but still want to indulge, you may want to know the estimated calories of the meal. In the infrastructure world, transparency goes a long way to help ensure the consistency.

Many automation tools use either JSON or YAML formats which are very easy to read. This allows for greater transparency. Typically, you would even store this in a version control system so that everyone can review at any given time. JSON is fairly human readable but typically requires a viewer to format it while YAML is much easier to read in its native format.

3. Ease of Duplication

If you have an environment or set of changes you want to reproduce easily, workflow automation makes that easier. Spinning up the environment one time may be easy, but what if you have to spin up five instances of it and remember all of the details? Or what if you need to apply a specific patch to multiple environments? With workflow automation, all the hard work happens in the planning phase. Once that is done, you have an easily reproducible set of steps.

Back to the pizza analogy, this is very similar to a recipe. Imagine a restaurant that makes them from scratch without a recipe. Some staff may not know how and may have to ask other employees how to do that. It would be very time consuming to hunt down instructions for making pizza each time.

Using a cloud automation tool, the ease of duplication is unparalleled. In Microsoft Azure, you can use their Blueprints stack and simply apply to each Resource group as needed making the duplication as easy as a few clicks of the mouse or a few strokes of the keyboard.

What Processes Can be Automated?

Nearly every step of the deployment, creation, and management process can be automated. Going through some examples, when spinning up a new resource, any storage, network, memory requirements are allocated. An operating system image is cloned or instantiated against those resources. If there are any network ACLs that need to restrict traffic, those can be applied as well. Even after you have a running resource, the automation does not have to stop there. Workflow automation can cover some first time boot configuration settings.

For example, you may have a Debian Linux instance that needs to have Apache and PHP installed. It may also need SSH enabled and your public keys copied over so that you can access it using key based authentication. Once all that is in place, your application like WordPress may need to be copied over and extracted. This is a simple example of what could be achieved via automation.

It does not even end there though. Perhaps you have some malicious traffic trying to connect to your network and you want to block certain IP addresses. If your security groups have been provisioned via automation they very likely can be updated easily as well.

Code and application deployment is a huge one as well. Automations could be written such that a snapshot or backup is taken first, then any dependencies such as new OS libraries are installed before deploying the new code.

Cloud Automation vs Cloud Orchestration: What's the Difference?

It is easy to get confused by what each of these does or think they are the same thing. Cloud automation tends to be involved with spinning up various individual resources. For example, you may have cloud automation for AWS EC2 instances and a separate cloud automation to pull source from Github — and build it and then deploy over to the EC2 instance. These EC2 instances may go into their own VPC which needs to be created first.

Where cloud orchestration comes in is that it hovers above all these various automations as the umbrella that organizes and orchestrates it. When the dependencies are properly defined, many of these tools can work out all of the order of operations to ensure resources are available for other resources to depend on in the provisioning process. In the above example, it is important to ensure the VPC exists first so that the EC2 instance has a network container to be instantiated into. The EC2 instance has to exist before any code or applications can be deployed to it.

What Cloud Automation Tools Exist?

Cloud providers like AWS and Azure have their native tools because they want to make it as easy as possible for you to spin up your resources as necessary. One of these tools for example is AWS has CloudFormation. It is AWS specific though but is very flexible.

But what if you’re working with cloud-agnostic or multi-cloud environments? You're not left to use each individual platform's tools. There are great 3rd-party tools like Terraform that work with all of the major cloud providers.

We've talked mostly about provisioning but there are some great maintenance automation tools as well, such as RedHat's Ansible and SaltStack. Putting resources in the cloud is not just about spinning up servers and letting them run themselves. They still need to be maintained and updated.

Final Thoughts

Workflow automation can seem overwhelming. There are so many pieces to it and seemingly just as many tools. Always start with your business requirements and find the right tools for those requirements. For example, are you standardizing on AWS? If so you only need to focus on tools that work with AWS. Do you need automations to provision and maintain an environment? If so, you need to look at tools that can do both or two separate tools that can work together on that.

There are typically no wrong answers when it comes to these decisions. Business requirements and cost are usually the main factors and help narrow the field down. If the big picture gets overwhelming, break it down into smaller pieces and then take a step back to make sure they all fit together.