This is a continuation of previous articles on IaC shortcomings and resolutions. In this section, we focus on automation involving external tools and APIs. Almost all mature DevOps pipelines rely on some automation that is facilitated by scripts and executable rather than IaC resources. The home for these scripts usually turns out to be in pipelines themselves or gravitate to centralized one-point maintenance destinations such as Azure Automation Account Runbooks or Azure DevOps depending on scope and reusability.

While deciding on where to save automation logic, some considerations often get ignored. For example, Runbooks run in sandbox environment or as Hybrid Runbook Worker.

When the executables are downloadable from the internet, either can be used since internet connectivity is available in both. But when local resources need to be managed such as an Azure storage account or on-premises store, they need to be managed via a Hybrid Runbook Worker. The Runbook worker enables us to manage local resources that are not necessarily native to the cloud and bridges the gap between cloud-based automation and on-premises or hybrid scenarios. There are two installation platforms for the Hybrid Runbook Worker: Extension-based (v2) and Agent-based (v1). The former is the recommended approach because it simplifies installation and management by using a VM extension.  It does not rely on the Log Analytics Agent and reports directly to an Azure Monitor Log Analytics workspace.  The V1 approach requires the Log Analytics agent to be installed first.  Both v1 and v2 can coexist on the same machine. Beyond those choices are just limitations and other options such as Azure DevOps might be considered instead. Webhooks and APIs are left out of this discussion, but they provide the advantage that authentication and encryption become part of each request.

 

Azure DevOps aka ADO is a cloud-based service, and it does not have restrictions on its elasticity. The DevOps based approach is critical to rapid software development cycles. The Azure DevOps project represents a fundamental container where data is stored when added to Azure DevOps. Since it is a repository for packages and a place for users to plan, track progress, and collaborate on building workflows, it must scale with the organization. When a project is created, a team is created by the same name. For enterprise, it is better to use collection-project-team structure which provides teams a high level of autonomy and supports administrative tasks to occur at the appropriate level.  

Some tenets for organization from ADO have parallels in Workflow management systems: 

· Projects can be added to support different business units  

· Within a project, teams can be added  

· Repositories and branches can be added for a team  

· Agents, agent pools, and deployment pools to support continuous integration and deployment  

· Many users can be managed using the Azure Active Directory.  

It might be tempting to use GitOps and third-party automation solutions including Jenkins-based automation, but they only introduce more variety. Consolidating resources and automation in the public cloud is the way to go.

As with all automation, it is important to register them in source control so that their maintenance can become easy. It is also important to secure the credentials with which these scripts run. Finally, lockdown of all resources in terms of network access and private planes is just as important as their accessibility for automation.