This is a continuation of a series of articles on hosting solutions and services on Azure public cloud with the most recent discussion on Multitenancy here. The previous articles discussed Azure Arc instances and this one introduces serverless batch processing with durable functions in Azure Container Instances.

Durable Functions is an extension of Azure Functions that lets us write stateful functions in a serverless compute environment.  It involves the use of an orchestration function to orchestrate the execution of other Durable functions within a function app.  Orchestration functions define function workflows using procedural code without declarative schema or designers.  Functions can call other durable functions synchronously and asynchronously and output can be saved to variables.  They are durable and reliable because the execution is automatically checkpointed when the function awaits or yields. Local state is not lost during reboots or failures. They can be long running.  Durable functions might also involve entity functions which define operations for reading and updating small pieces of state, known as durable entities. Like orchestrator functions, entity functions are functions with a special trigger type and manage the state of an entity explicitly, rather than implicitly representing state via control flow.  Entities provide a means for scaling out applications by distributing the work across many entities, each with a modest sized state. This extension manages state, checkpoints and restarts behinds the scenes.

Durable functions can be used to schedule, manage and deploy serverless batch processing jobs in Azure Container Instances. Containers are popular for packaging, deploying and managing code hosted on orchestration frameworks such as Azure Kubernetes Service and Azure Service Fabric. 

The scenarios for using multitenant serverless batch processing are ones where workloads are simple and use only one container image. Another use case might be the case where computing needs vary depending on each individual job. Multi-tenant scenarios where some tenants need large computing power and other tenants have small computing requirements represent hybrid requirements and serverless batch processing can help in this regard.

The architecture almost always involves a series of durable functions used with a Managed Service Identity.  The batch processing job is packaged into a container image stored in a Azure Container registry. An Http Trigger invokes the orchestration function to orchestrate the deployment. An activity function uses the container image stored in the ACR to create an ACI container in a container group.  An orchestration function uses the container URL to call and start the batch processing job and to monitor the job progress. When the job completes, the orchestration function raises an external event and provides job status Completed or Failed. Depending on the job status, the orchestration function stops, restarts or deletes the container group. Variations of this architecture involve the use of restart policies to control the container instances and the use of a full-fledged orchestration framework to manage complex multi-container tasks and interactions.