Writing an Azure Custom Resource Provider

 

Introduction: The Azure Custom Resource Provider allows these partners to extend the Azure Control plane to new services for their customers. This is an extensibility platform to Azure. It allows us to define custom APIs that can be used to enrich the default Azure experience. The Custom resource providers are a list of contracts between the Azure and the endpoints. It describes how the Azure should interact with an endpoint.  The resource provider acts like a proxy and forwards requests and responses to and from the specified endpoint.  A resource provider specifies two types of contracts: resourceTypes and actions. The endpoint definition is comprised of three fields: name, routing type, and endpoint.

Before we review the custom resource providers, let us review the Azure Resource Provider. Partners build and deploy native Azure Applications for customers using the resource providers. For example, with Microsoft.KeyVault/vaults as <resource-provider>/<resource-types>, partners can store keys and secrets using the Microsoft.KeyVault secret provider.  All the resource providers in Azure are listed via the Portal and while may be ready to use, most require the users to register them with the click of a button so that they are ready to use. Resource providers come with their own schema, templates, permissions, and the conventions followed via Azure Resource Manager. Notably, the separated resource template and resource provider pattern lowers the barrier for service model composition when deploying Azure services.

Azure offer Kubernetes and containers via services that have resource providers as Microsoft.ContainerService and Microsoft.ContainerInstance respectively. These resources are sufficient to launch an instance of Kubernetes and containers. If we are looking to bring resources directly to the Kubernetes control plane, we will require the Azure OSBA. The Azure OSBA expands the service catalog and enabled Azure resources to be recognized as Kubernetes custom resources. Azure Service operators take it a step forward and enable the resources to be managed via the Kubernetes control plane.

A custom resource provider can extend the ARM API to include internal and external resources. It enables custom scenarios on top of existing Azure workflows. The ARM Templates can be customized for control and effect.  The contract between Azure and an endpoint is defined through a new resource named as the Microsoft.CustomProviders/resourceProviders. This implies there are three parts: the custom resource provider, endpoints and the custom resources. A custom resource is comprised of three fields: name, routingType and endpoint. ResourceTypes describe new custom resources. Supported http methods are PUT, DELETE and GET. Actions can be added to a custom resource or across all resources via Azure. The equivalent CRUD operation via PowerShell includes the New-AzCustomProvider cmdlet and the association with an endpoint is created via the New-AzCustomProviderAssociation.

Writing an Azure Custom Resource Provider

 

Introduction: This is an extensibility platform to Azure. It allows us to define custom APIs that can be used to enrich the default Azure experience. Before we review the custom resource providers, let us review the Azure Resource Provider. Partners build and deploy native Azure Applications for customers using the resource providers. For example, with Microsoft.KeyVault/vaults as <resource-provider>/<resource-types>, partners can store keys and secrets using the Microsoft.KeyVault secret provider.  All the resource providers in Azure are listed via the Portal and while may be ready to use, most require the users to register them with the click of a button so that they are ready to use. Resource providers come with their own schema, templates, permissions, and the conventions followed via Azure Resource Manager. Notably, the separated resource template and resource provider pattern lowers the barrier for service model composition when deploying Azure services.

The Azure Custom Resource Provider allows these partners to extend the Azure Control plane to new services for their customers. The 200+ Azure services categorized as Compute, Storage, Network and PaaS services continue to form the base while other internal and external services are added. The Azure Resource Manager maintains tags, RBAC, ARM Templates, Locks, policies and activity logs for the resources and the create, update, delete of the resources are exposed via the CLI, PowerShell, HTTP request, and Azure Portal.  The Azure Custom provider is in public preview mode, so it is not available to run mission critical SLA based workloads.

The Custom resource providers are a list of contracts between the Azure and the endpoints. It describes how the Azure should interact with an endpoint.  The resource provider acts like a proxy and forwards requests and responses to and from the specified endpoint.  A resource provider specifies two types of contracts: resourceTypes and actions. The endpoint definition is comprised of three fields: name, routing type, and endpoint.

The use of Custom resource providers is not restricted to a cluster-based ecosystem or container-based orchestration frameworks. The Azure Service operator serve those scenarios. Azure Service Operator is an open-source project that exposes Azure Services as Kubernetes operators. Exposing Azure Resources on the Kubernetes control plane is desirable for several reasons and if the exponential growth in popularity of the Kubernetes infrastructure is any indication, then those reasons must hold. Azure resources are also managed in the cloud via the Azure resource manager, so the concepts of resource manifest and state reconciliation are similar. Kubernetes is a portable, extensible, open-source platform for managing containerized workloads and services. It is often a strategic decision for any company because it decouples the application from the hosts so that the same application can work elsewhere with minimal disruption to its use.   

Azure offer Kubernetes and containers via services that have resource providers as Microsoft.ContainerService and Microsoft.ContainerInstance respectively. These resources are sufficient to launch an instance of Kubernetes and containers. If we are looking to bring resources directly to the Kubernetes control plane, we will require the Azure OSBA. The Azure OSBA expands the service catalog and enabled Azure resources to be recognized as Kubernetes custom resources. Azure Service operators take it a step forward and enable the resources to be managed via the Kubernetes control plane.

 

 Azure FHIR Service continued...  

Introduction: This article is a continuation of the series of articles starting with the description of SignalR service which was followed by a discussion on Azure Gateway service, Azure Private Link, and Azure Private Endpoint and the benefit of diverting traffic to the Azure Backbone network. Then we started reviewing a more public internet-facing service such as the Bing API. and the benefits it provided when used together with Azure Cognitive Services. We then discussed infrastructure API such as Provider API, ARM resources, and Azure Pipeline and followed it up with a brief overview of the Azure services support for Kubernetes Control Plane via the OSBA and Azure operator. Then we followed it with an example of Azure integration service for Host Integration Server (HIS). We started discussing the Azure FHIR service next. We were reviewing its search capabilities and now we will view its regulatory compliance and security policies. 

Description:   

The regulatory compliance in Azure Policy provides Microsoft created and managed initiative definitions known as built-ins for the compliance domains and security controls related to different compliance standards. The assignment is not automatic and must be provisioned on the FHIR resource to help it be compliant with a specific standard. The built-ins are organized by responsibility as (Customer, Microsoft, Shared) in its list of controls and compliance domains. 

A regulatory compliance initiative definition has a grouping portion each of which defines a name for the control, a category for the compliance domain, and a reference to the policy metadata that has information about the policy metadata with information about the control. These definitions can be original or copied and edited by the customer. A link to the regulatory compliance initiative is also visible on the Azure security dashboard. The SDK lists a summarize method that counts the number of definitions under the “compliant” and the “non-compliant” categories. 

As an example of compliance, the Cybersecurity Maturity Model Certification is mentioned, and it attempts to prevent the theft of intellectual property and sensitive information from all industrial sectors due to malicious cyber activity.  The FedRamp High and FedRamp moderate are shared responsibilities in the cloud. Azure provides certain compliance by virtue of being a cloud provider. The use of FHIR resources in compliance with these standards must be specified by the user provisioning the resource via the portal. FedRamp High and FedRamp moderate both pertain to account management, monitoring, and role-based access controls and have different impact levels. HIPAA HITRUST 9.2 targets both privilege management and role-based access control. 

There are a number of other standards and versions that can be met with the same technique as above. 

 

Azure FHIR Service continued...

Introduction: This article is a continuation of the series of articles starting with the description of SignalR service which was followed by a discussion on Azure Gateway service, Azure Private Link, and Azure Private Endpoint and the benefit of diverting traffic to the Azure Backbone network. Then we started reviewing a more public internet-facing service such as the Bing API. and the benefits it provided when used together with Azure Cognitive Services. We then discussed infrastructure API such as Provider API, ARM resources, and Azure Pipeline and followed it up with a brief overview of the Azure services support for Kubernetes Control Plane via the OSBA and Azure operator. Then we followed it with an example of Azure integration service for Host Integration Server (HIS). We started discussing the Azure FHIR service next.

Description: 

The Azure Fast Healthcare Interoperability resource or the FHIR service is a managed, standards-based compliant API for clinical health data. It enables anyone working with health data to ingest, manage, and persist protected health information (PHI) in the cloud. 

As with any data access layer, this service provides a flexible data model for the PHI data with standardized semantics and data exchange that enables easy interoperability with other applications. In addition to data access, the FHIR service focuses on all industry compliance requirements for PHI data. 

The data is secured with unparalleled intelligence and customer’s data are isolated by databases providing all controls at the database level. The security features are provided by a layered, in-depth defense and advanced threat-protection system. The service can be used with Digital Imaging Communications in Medicine (DICOM) and Azure Cognitive Services to provide meaningful insights into PHI data. With the Azure IoT connector, the data can be unified in FHIR from diverse data streams such as clinical, imaging, device, and unstructured data

Healthcare ecosystems can be built on this service as the source of truth. With its unparalleled focus on security and audit of PHI data, it becomes invaluable for turn-key production-ready solutions by enabling applications to pursue more business opportunities with AI & ML pipelines for real-world actions.

Data security and privacy guarantees are implied with the number of certifications that Azure has received. Search on FHIR is an understated but big differentiator for PHI data. The search targets the workspace-name and account-name qualified endpoint.  FHIR supports common search parameters as well as resource-specific parameters with a wide range of data types including reference, composite, quantity, and special. Common search parameters include _security, _profile, _query, and _filter. Resource-specific parameters include _content,_id, _profile, _query, _source, and _tag. Both chained search and reverse chained search are supported. A chained search is a search using a parameter on a resource referenced by another resource. For example, finding hits with a patient’s name is a chained search. A reverse chained search is one where resources are retrieved using criteria on other resources that reference them. For example, the medication requests can be searched to find only the ones that include information for a specific patient.

Azure FHIR Service

Introduction: This article is a continuation of the series of articles starting with the description of SignalR service which was followed by a discussion on Azure Gateway service, Azure Private Link, and Azure Private Endpoint and the benefit of diverting traffic to the Azure Backbone network. Then we started reviewing a more public internet-facing service such as the Bing API. and the benefits it provided when used together with Azure Cognitive Services. We then discussed infrastructure API such as Provider API, ARM resources, and Azure Pipeline and followed it up with a brief overview of the Azure services support for Kubernetes Control Plane via the OSBA and Azure operator. Then we followed it with an example of Azure integration service for Host Integration Server (HIS). We now discuss Azure FHIR service.

Description: 

The Azure Integration Service consisted of four components: – namely, APIs, Orchestrations, Messaging, and Events.  They are represented by Azure resource such as API management, Logic Apps, Service Bus, and Event Grid. There is also a fifth component when it comes to healthcare. This is the Fast Healthcare Interoperability resource or the FHIR service, which is a managed, standards-based compliant API for clinical health data. It powers solutions and analytics and makes it easy for anyone working with health data to ingest, manage, and persist protected health information (PHI) in the cloud. It is enterprise grade and demonstrates high performance with low latency. It is accessible by mobile and web applications and has sufficient role based-access control to secure all accesses universally. Audit log has significant emphasis and this service and tracks all activities per data store.

As with any data access layer, this service provides a flexible data model for the PHI data with standardized semantics and data exchange that enables easy interoperability with other applications. Data transfer to FHIR enable universal connectivity and exchange because of the standard definitions. The exchange is rapid due to the power of Azure.

The FHIR service allows the exchange of data via its APIs. As with all Azure APIs, the programmability can be exercised via REST based services, PowerShell, CLI and SDKs.  In addition to data access, FHIR service focuses on all industry compliance requirements for PHI data. 

The data is secured with unparalleled intelligence and customer’s data are isolated by databases providing all controls at the database level. The security features are provided by a layered, in-depth defense and advanced threat protection system.

This service is used by startups, devices such as IoT and application development simply because it is a fully managed data management service for PHI data. Healthcare ecosystems can be built on this service as the source of truth. The FHIR data model is not only suitable for conventional, transactional and analytical purposes but also for deep learning. As part of the PaaS offerings, it becomes invaluable for turn-key production ready solutions.

Whitepaper continued...

 Introduction: 

This is a continuation of the whitepaper on Host Integration Server introduced here. We elaborated on the four components of the overall design – namely, APIs, Events, Messaging, and Orchestration in our subsequent article. In this final article, we review the Azure Security integration.

Description:

There are five technology areas of integration for the Host Integration Server (HIS), and they are:

1) Network Integration that connects application infrastructure to existing IBM mainframes and midrange system network architectures. This service connects desktops, devices, and servers to existing host systems while reducing costs. For example, the print service provides server-based printer emulation.

2) Data integration component offers direct access to data stored in IBM DB2 management systems. It includes multiple data clients and one data service with support for a variety of data providers such as ADO.Net, OLEDB and ODBC.

3) Application Integration is provided by the Transaction Integrator which allows enterprise developers to call business rules in host mainframe. It comprises of a plugin designer, administration tool and runtime components. 

4) Message Integration is provided by WCF channel for IBM WebSphere MQ which allows enterprise developers to send or receive MQ messages between WCF And heterogeneous or native IBM programs.

5) Security Integration which is provided by Enterprise Single-sign on with AD integration to secure IBM host systems. It maps to their host credentials which are stored in a SQL Server. These mappings can be retrieved at runtime from both ESSO SDK and HIS features.

 When these integration areas are reimplemented on the Azure control pane, we can leverage the Azure iPaaS solution which is a set of cloud services that are essential for mission critical enterprise integration. These services provide four core technologies that are required for cloud-based integration – a way to publish and manage application programming interface, a straightforward way to create and run integration /workflow logic with the help of an orchestration, some messaging that facilitates the loose coupling between applications and a technology that supports communication via events.

There are always other services that can be combined from other cloud technologies but the above four iPaaS offerings namely API management, Logic Apps, Service Bus, and Event Grid are sufficient to perform integration for services such as HIS. There is a one-to-one correlation between the integration areas of Host Integration Server and the iPaas offerings.

The security integration is facilitated by Azure Active Directory (AAD) and Role-Based access control (RBAC). In RBAC, authorization is mapped to roles. A user can take different roles. This effectively handles changes in the organization. Since users are not assigned rights directly but only acquire it with roles, management of individual user rights becomes a matter of assigning appropriate roles to user's accounts. The roles are classified based on the set of stabilized duties and responsibilities in the management. There are three primary rules for RBAC.
Role assignment - A subject can exercise a permission only if the subject has been selected or assigned a role.
Role authorization - A subject's active role must be authorized for the subject. i.e User cannot take any or all roles.
Permission authorization - A subject can exercise a permission only if the permission is authorized for the subject's active role. i.e the user can exercise only those permissions assigned to the role.
Roles can be hierarchical in which a higher-level role assumes all that comes with the lower-level role. An Azure login context can be set to a given subscription which can then be used to find the service principal and the role that needs to be allowed access to the resource. With the help of this principal, an application can be added to its operation service role. The addition of principal to role is done with internal security context and not that of the logged-in principal. This rounds up the security integration.

Conclusion: An enterprise integration solution hosted on Azure must make use of the four components of API management, Logic Apps, Service Bus and Enterprise Grid from iPaaS. This allows the Host Integration Server to be hosted on Azure.

 

Whitepaper continued...

 Introduction: This is a continuation of the whitepaper on Host Integration Server introduced here. We now describe the remaining of four components of the overall design – namely, APIs, Events, Messaging, and Orchestration.

Description:

3) Messaging:

Plenty of backend operations are asynchronous by nature requiring background operations. Sometimes, a direct call via API Management is perfect. In other cases, some form of background processing is required. Situations like this call for a message queue.

The Azure Service Bus is well positioned to provide this functionality. It allows non-blocking interactions between different components of software by facilitating message exchanges on queues where the messages are ordered and retried. The messaging works across many kinds of software, including cloud applications, on-premises applications, and Azure services. Service Bus provides all the following:

Queue semantics, including message persistence and strict first-in, first-out ordering of messages, deduplication, dead letters, poison messages and retries.

Transactional guarantees, letting a queue read or write be part of a larger operation that succeeds or fails as a single unit.

Poison message handling so that a message that causes problems won’t put a receiver into an ongoing loop.

High availability for built-in disaster recovery via geo-replication.

Service Bus provides all the features required for enterprise messaging.

4) Events:

Events facilitate the notion of publisher-subscriber so that the polling on messages from a queue can be avoided.

This is exactly what Event Grid allows. Rather than requiring a receiver to poll for new messages, the receiver instead registers an event handler for the event source it’s interested in. Event Grid then invokes that event handler when the specified event occurs. 

Azure Services generate events. The arrival of a new Service Bus message might cause Event Grid to send a message that starts a Logic App running, or the creation of a new blob in Azure Blob Storage might cause a custom cloud application to begin processing the contents of that blob. Using an event-driven approach can simplify application development, and it can also save money, since the receiver needn’t waste cycles polling for new messages.

A blob storage can receive an event by subscribing to a standard topic provided for that service. This is a common use case for the Azure Event Grid, but it is not the only option. Custom topics allow different subscriptions.

Conclusion: An enterprise integration solution hosted on Azure must make use of these four components of API management, Logic Apps, Service Bus and Enterprise Grid.