Architecture trade-offs between Designer Workflows and Resource Provider Contracts

Introduction: Automations in the computing and storage industries often require infrastructure that supports the extensibility of logic that is specified by their clients. The infrastructure may support a limited number of predefined workflows that are applicable across clients, but they cannot rule out the customization that individual clients need. There are two ways to support the extensibility of any infrastructure platform. One technique involves the use of a dependency graph of workflows while the other technique involves the use of custom resources that are provisioned by resource providers external to the infrastructure. This article compares the two techniques for their pros and cons.

Description: When an Automation platform supports dependency elaboration via a flowchart, the user dictates the set of steps to be taken in the form of a controlled sequence including conditions on existing built-in workflows. By composing the steps in various modules, a client can write sophisticated logic to perform custom actions that can be re-used across their own modules. This extension allows them to make the infrastructure do tasks that were not available out of the box and the infrastructure records these specifications with the help of tasks and dependencies. The flowchart defined by the automation client gets persisted in the form of a dependency graph on tasks that can be either built-ins or client-defined.

The alternative to this approach is the representation of the result of these activities as resources to be provisioned. This technique generalizes automation as a system that provisions various resources and reconciles their states to the desired configuration defined by its clients. The representation of the resource allows the separation of control and data plane activities for the automation platform enabling many more capabilities for the platform without affecting any of the tasks involved on the data plane. Control and data plane refer to activities about provisioning versus usages of a resource. A resource once provisioned can participate in various data activities without requiring any changes to its provisioning or configuration which eliminates the need to interact with the automation or its client for the usages of the resource. Additionally, control plane activities are subject to governance, management, programmability, and security enhancements that are not easy to specify and manage using logic that gets encapsulated by the automation clients in their customizations without hooks on the automation side.

These two techniques have their own advantages. For example, the dependency-based orchestrations technique provides the following advantages:

1. Coordination and orchestration of activities across workflow boundaries are useful when workflows are componentized into multiple sub-workflows.

2. Orchestration of the workflows’ activities even when they have dependencies on external services which is useful some external service needs to be available for the activity to complete.

3. Enablement of "replay" when upstream artifacts change which prevents rewriting logic when those artifacts change

4. Generating regular and common names in the event catalog provides the ability to namespace, and map existing and new associations and their discoverability.

On the other hand, the resource provisioning architecture supports:

1. A nomenclature and discovery of resources that can be translated with export and import for portability.

2. It provides an opportunity to offload all maintenance to the reconciliation logic in-built into the corresponding operators that the platform maintains.

3. Scope and actions become more granular with export-import capabilities.

4. It improves visibility of all resources in the control plane that makes it easy to manage some or all of them with filters.

Eventually, both techniques require definitions and manifests to be declarative which is wonderful for their testing and validations or for their visualizations in viewers.

Conclusion: The universalization of logic via control and data plane separation enables the automation to be more platform-oriented and increase its portfolio of capabilities with little or no impact on the client's logic. This provides more opportunities for platform development than the dependency graph-based technique.