Service Broker for Azure

Problem Statement: Custom resources in the public cloud can be defined with the help of an Azure Resource Manager manifest and the registration of an Azure Resource provider, usually a service hosted on Azure, to provide resources based on the manifest. This article investigates the role of an external resource provider for Azure where the service is not directly registered with the resource manager for a lookup but resolved indirectly. 

Description:  The use of resource and an orchestration framework to reconcile the desired state for the resource as described by its manifest to the actual state of the resource as visible to a resource manager is a control loop paradigm widely popular across infrastructure frameworks both in the enterprise and in the cloud. For example, the well-known Kubernetes is a container orchestration framework that designates a ‘Kube-controller-manager' that embeds the core control loops shipped with it. This non-terminating loop watches the shared state of the cluster through an ‘apiserver’ and makes changes to the resource to move it from its current state to the desired state declared in its manifest. Some examples of these controllers are replication controller, endpoint controller, namespace controller, and service accounts controller. The same applies to the Azure public cloud with the help of the Azure Resource Manager and the corresponding ARM templates for the resources provisioned in the cloud.  

The difference is in the use of a service broker that allows implementations of the resource provisioning to exist outside the cluster or the cloud for Kubernetes and Azure respectively. Kubernetes service brokers allow the provisioning of services outside the cluster. This enables services to run anywhere and work independently from the cluster. They can even be deployed in the public and private cloud where they can scale across Kubernetes clusters. This is a helpful addition to any Kubernetes cluster. 

 

The service broker architecture also enforces consistency across the provisioning of resources. This is done with the help of a standard set of APIs for all resources and their corresponding services that implement the service broker.  Resources are the way in which Kubernetes recognizes every entity in the cluster. The framework of Kubernetes then takes this inventory of resources and reconciles the state of the cluster to match the definition of the resources. Anything hosted on the Kubernetes clusters can be described with the help of a custom resource. 

 

Service brokers facilitate the resources provisioned by external services to be understood by Kubernetes as custom resources within the cluster.  This bridging of external resource provisioning within-cluster representation of resources has provided significant benefits to users for the use of these resources. 

 

Almost all provisioning of resources translates into well-known operations of creating, update, delete, get and list on the resources. Therefore, the resource operations are aligned with the usage of resources along with most workflows. This makes it handy for use of Kubernetes clusters for many applications and services. 

 

The trend in cloud computing has shifted from service-oriented architecture towards microservices. This has helped the independent provisioning of resources, deployment, and scaling of services, and overhaul or reworking of services. The service broker makes no claim about how the services need to evolve - whether to form a structural composite pattern or a behavior maintaining pattern. This minimal enforcement has worked mutually beneficial for both the services as well as the Kubernetes cluster. 

One example of the use of a resource with a service broker is logging. Logging, although available out-of-box with Kubernetes, can also be considered a service to be provisioned external to the cluster. This is easy to do with a variety of log products that provide service-like functionality. If there is no data loss, most log system users are tolerant to latency. This makes it easier for Logging to be implemented with merely a Kubernetes service broker and alleviating all concerns for logging from the cluster. 

The same could apply to Azure if it could entertain cloud extenders for the customization of resources. Another convenience that comes with this approach is the use of a hierarchical namespace rather than the type definitions we have with Azure Resource Manager templates today.  

Azure supports extensions but the extensions are not a replacement for the Open Service Broker API implementation. There is also a difference between the existing OSBA API and resolving of the namespaces by an existing Azure OSBA. This would be a service that performs like a DNS service. 

Conclusion: This article discusses improvement possibilities that are not yet available in the public cloud. 

 

 Subscription Provisioning Automation Task:

Problem statement: Subscription provisioning automation task is encountered when isolation is required between the billing profiles in the Azure public cloud cost management system.  This example describes how to achieve it

Description: The automation of this task relies on the notion that a subscription within Azure is a resource just like any other Azure Resource Management definition. Usually, it is created by the account owner who signed up with a Microsoft Customer Agreement or registered with Azure via the Azure Portal. A subscription is a way of organizing Azure resources and supports a billing method. Organizations create additional subscriptions when they expand their management group hierarchy to support business priorities. 

The task of creating a subscription can be automated programmatically. It requires the following parameters:

 {

      "name": "SubscriptionProvisioning",

      "displayName": "__GroupSubscription_NAME__",

      "initialOwnerPrincipalId": "__BILLING_initialOwnerPrincipalId__",

      "workload": "DevTest",

      "billing": {

        "scope": "/billingAccounts/__BILLING_costManagement__:__BILLING_billingScope___2019-05-31/billingProfiles/__BILLING_Profile__/invoiceSections/__BILLING_Invoice__",

        "pcCode": "P7007777",

        "costCategory": "FX",

        "airsRegisteredUserPrincipalId": "__BILLING_airsRegisteredOwnerPrincipalId__"

      }

    }

Where the parameters are explained as 

“displayName”: the name with which the account will be displayed for finding it in the list of subscriptions pertaining to the account.

“initialOwnerPrincipalId”: the initial owner for the subscription who can add additional owners as necessary. Usually this is the same principal that is associated with the account in the first place.

“workload”: describes the environment as production or development purposes.

“billing.scope”: refers to the invoice scope and is resolved by the cost management hierarchy comprising of the cost management account, billing scope, billing profile and invoice. All of these are object identifiers in the form of GUIDs. The billing invoice profile guid corresponds to the service identifier in the service tree registrations maintained by Azure.

“airsRegisteredUserPrincipalId”: refers to the service principal who requested the cost management plus billing profile to be setup at https://azuremsregistration.microsoft.com/Request.aspx

 

With these parameters, it is a simple pseudo-resource registration step to provision a subscription automatically.

Conclusion: Changing business priorities can now be handled with isolation of assets via additional subscriptions provisioned with the help of the automation described here.

 This article is a proposal to change the de facto public connectivity methods for the Azure Public cloud resources such as a database, cache, servicebus etc. The preferred connectivity method is a public endpoint with a hardcoded public IP address and an assigned port. There are a few limitations with this approach even though it has proven to work for many usages. First, the address, binding and contract for the resource is now tied directly to the resource and are static which increases security concerns since anyone can access them as many times as they want. Second, the resource does not differentiate between the availability zones that provide redundancy and availability for the resource and consequently provide no control to the user to refer to the endpoints in those zones. Third, there is no load balancing between the connectivity methods of the same resource. Instead, this is a proposal for commissioning of an application gateway or a load balancer that is automatically provisioned with the resource. The gateway provides the much-needed functionality of an http proxy as well as load balancer which the users have had to install themselves. The services provisioning these resources are now free to expand their abilities beyond the limitation of including them behind the same public endpoint.

The idea of using a proxy and load balancer is widely accepted with resource orchestration technology such as Kubernetes with its definition of a Kubernetes Service. Kubernetes is a container framework that enables migration of applications across hosts and provides all abstractions that they need. It introduces the separation of hosts in the form of pods. Container infrastructure layering allows even more scale because it virtualizes the operating system. A Kubernetes Service is no longer a single entity behind an IP address and a port. Instead, it supports auto-scaling and dynamic port assignment.

The Azure resource are provisioned with one of the following three connectivity methods: 1) Public Endpoint, 2) Virtual Networks and 3) Private Endpoint. These methods are by themselves sufficient to reach from all on-premises and cloud clients, but these methods do not provide the benefit of an application gateway. The gateway-sold-separately technique does not automatically address the often-repeated tasks faced by the clients when they have many resources to place behind a load balancer or a proxy. It also imposes restrictions on the service to provide all the features behind the single endpoint.

The use of a gateway-like connectivity method with the Azure resource enables staged migrations as the portfolio of features are expanded and the older dependencies retired. Each new feature could be a swap replacement of an existing functionality. Incremental additions of features help pave the way for testing.

The logging is improved significantly with separation of concerns and enhanced load balancing and auto-scaling. Liveness and readiness probe could also be added to the deployments which improves overall visibility, health and readiness of the resource across availability zones and replicas.

If the user must use an application gateway in addition to provisioning the resource, providing it as a built-in option for connectivity with the resource merely adds much needed convenience to the user and hence this proposal.

Introduction: This article is a continuation of the series of articles starting with the description of SignalR service which was followed by a discussion on Azure Gateway service, Azure Private Link, and Azure Private Endpoint and the benefit of diverting traffic to the Azure Backbone network. Then we started reviewing a more public internet-facing service such as the Bing API. and the benefits it provided when used together with Azure Cognitive Services. We then discussed infrastructure API such as Provider API, ARM resources, and Azure Pipeline. 

Azure Bastion is a fully managed PaaS service, and it is provisioned inside the virtual network created by the Azure user. It provides secure and seamless RDP/SSH connectivity to the virtual machines directly from the Azure portal over the TLS. Since the SSH/RDP access is provided in a software-as-a-service, the virtual machine resource does not need a public IP address, agent, or special software. It provides RDP or SSH connectivity to all the virtual machines in the virtual network so that host-level settings need not be configured on a host-by-host basis.  

The key benefits of this service include the RDP and SSH access to the network, remote session over the TLS, firewall traversal over corporate network, no public IP, no management of network policies and network security groups, network vulnerability avoidance from port scanning, and network edge level protection that dissipates the hardening on each resource internally thwarting zero-day attacks and hardening in one place. 

Bastion hosts do not introduce a new concept to the network. A jump server or relay server has been used for access to production environment virtual machines in traditional enterprise settings. Bastion merely provides a jump server access by sitting at the public side of a perimeter network They are also secure to withstand attacks from the public internet. 

The Bastion host is deployed with its own AzureBastionSubnet that has a minimum /27 prefix which means that there are at least 30 addressable hosts in that subnet. The prefix length is an indication of the number of bits in the IP address that can be used in the mask leaving the remaining bits to vary such that each variation can be the valid IP address for a single host. The number of such variations determines the number of hosts that can be addressed in the network. The user can connect to the Azure Portal for using the Bastion service using any HTML5 browser. 

The number of Bastion hosts can be scaled manually for concurrent RDP/SSH access. It supports up to 50 host instances. Increasing this number increases the number of concurrent connections. Network architects are saved a lot of effort on host and security group settings and instead can delegate all access via this service.

  

As an Azure public cloud service, it comes with its support for ARM-based deployment, command-line interface, REST-API, SDK, and UI-based management and provisioning routines. 

 
 

This article is dedicated to my teacher.

Kusto search queries 

Enterprises realize the productivity impact of searches by thousands of their employees.  They invested in making search available on their internal portals, machine data, log indexes, and collaboration document servers. The last one involves hundreds and thousands of artifacts created and saved daily that are mostly personal. Authors save it on their local disk, external file shares, or the cloud storage depending on the size, but they usually cannot find what they want when they want. Tools like grep, find and locate do not scale with the volume of data often taking longer than acceptable time for searches. File indexing is a technique that works very well for search over offline data. It parses the content of this file to form text tokens using an analyzer and stores them in a repository together with the location of the file.  The analyzer tokenizes the content into a bag of words. This collection is then stored in an index with the help of an index writer. The index keeps track of each word and its occurrence information in sorted order. Search over the index is faster than scanning the document because it is like finding a name in the phonebook. Some searches can also yield scores based on some criteria for a match including learning techniques that can improve results.  

The same analogy applies to machine data such as logs, metrics, and events. Engineers find these useful to search when troubleshooting incidents such as in Software reliability engineering, development, and test of software on large and complex systems, and for the reconstruction of the timeline of events. The tools and expressions have two popular forms – structured query operators and vectorized execution. Public clouds offer some of the largest datasets to search and one of them uses Kusto queries. Some examples of canned queries are listed below: 

1) duration of user logon time between login/logoff events - here the requests for the user may be selectively filtered for specific security events and then the timestamps for the corresponding pair of events may be put in a table.  

source |  
where user = ‘user1’ |  

datetime_diff(hour, logoffTime, loginTime) 

2) potential suspicious activity detection - Here the requests made by the user are compared in their routing paths with the known set for anomalies specifically that do not fall in known workflow sequences and then raised as suspicious  

3) detecting callers - clients and clients identified by the programs they use can help mitigate denial of service attacks mounted by specific clients that do not behave well with others. The number of requests made from the client is compared with the others, in this case, to see if they are repeatedly trying something that they should not.  

source |  

where user = ‘user2’ 

| distinct client_id 

 Azure Virtual Desktop: 

Among the suite of services from Azure that we have been discussing in our previous articles beginning with signal R and more recently with internal facing, but very critical Azure Pipeline, we now review Azure Virtual Desktop service. This service was formerly known as Windows Virtual Desktop, and it enables secure remote work and provides the familiarity and compatibility of Windows 11. Having a computer in the cloud supports a hybrid workforce and it enables the virtual desktop deployments to work side by side with Azure services within the Azure portal. Since the virtual desktop is a resource just like any other Azure resource and since it is a cloud service, it can provide a level of availability and scalability that Azure is known for. 

We describe the usages of the Azure Virtual Desktop resource object in this article after an introduction in the previous article.

This service can be quite popular for the IT of an organization to provision a host pool for all the employees. In such case, they will limit the ability for the employees to create their own host pools as shown by this example: {"details":[{"code":"InvalidTemplateDeployment","message":"{\"deploymentStatusCode\":-1,\"stage\":6,\"expected\":true,\"error\":{\"code\":\"InvalidTemplateDeployment\",\"details\":[{\"code\":\"RequestDisallowedByPolicy\",\"target\":\"ags-hostpool-1\",\"message\":\"Resource 'test-hostpool-1' was disallowed by policy. Policy identifiers: '[{\\\"policyAssignment\\\":{\\\"name\\\":\\\"CSEO Block WVD resource creation\\\",\\\"id\\\":\\\"/providers/Microsoft.Management/managementGroups/<tenantId>/providers/Microsoft.Authorization/policyAssignments/Block WVD creation\\\"},\\\"policyDefinition\\\":{\\\"name\\\":\\\"Deny WVD resource creation\\\",\\\"id\\\":\\\"/providers/Microsoft.Management/managementGroups/<tenantId>/providers/Microsoft.Authorization/policyDefinitions/Deny WVD creation\\\"}}]'.

As with all Azure services, the resource can be provisioned with an Azure Resource Management manifest. Zone specifications can be included in the ARM templates.  Access to the service can be made from a variety of programming languages via REST APIs and SDKs. There is also scope for command-line automations and web portal usages. Role based access control can determine who has access to what at a service level as well as at the resource level. A virtual desktop can be shared or personal. The shared mode allows different users to use the same machine. Automatic assignment is the default type for new personal desktop host pools created in the Azure Virtual Desktop Environment. When users are assigned to the personal desktop host pool so that they can see the desktop in their feed.

Remote app streaming provided by this service lets us run our app in Azure and stream it to a remote device. Azure Virtual Desktop can be used for this experience by creating low literacy performance streaming to work from virtually anywhere on any device. Another key benefit of using this service is the ability to join virtual desktop to Active Directory and create a host pool which is a collection of one or more identical virtual machines also known as “session host”. Each host pool can contain an app group that users can interact with as they could on their local desktop. Users can be automatically assigned by first assigning them to the host pool so that they may see the desktop in their feed and claim the session host. Direct assignment, on the other hand, assigns the user to both the personal desktop host pool and a specific session host. The assignment groups and the assignments are available to view against the host pool in the Windows Azure portal. 

Session hosts can be scaled automatically using the scaling tool. The SessionThresholdPerCPU parameter determines the new VMs that need be started to load-balance the connections. The number of sessions can be restricted with the MaxSessionLimit Parameter.  The pool and the group are way to efficiently manage the resources. Assignment to the group can be controlled with direct or automatic assignments. The pool can be governed by polices and settings that determine the service level for the users. During off-peak usage, cost can be saved by shutting down unused vms and this is controlled by the MinimumNumberOfRDSH parameter. Applications can be published into the Azure Virtual Desktop environment for the convenience of the user.

 Azure Pipeline

Introduction: This article is a continuation of the series of articles starting with the description of SignalR service which was followed by a discussion on Azure Gateway service, Azure Private Link, and Azure Private Endpoint and the benefit of diverting traffic to the Azure Backbone network. Then we started reviewing a more public internet-facing service such as the Bing API. and the benefits it provided when used together with Azure Cognitive Services. We now focus internally with infrastructure API such as Provider API and ARM resources and follow it up with a discussion on the Azure Pipeline. 

Description:    

Azure Pipeline provides the opportunity to allow code to move quickly. Their tools deliver value faster and more transparently. Some customization is inevitable depending on the business purpose of the software, but most pipelines can do with out-of-box features. 

Azure services are fully managed services. Their dependencies, build, test and deployment logic can be quite large, but they are handled very well by the parameters, triggers, jobs, tasks, and stages in a pipeline once the artifacts are generated or become available.

For example, to deploy software to millions of devices, the following steps can be taken. First, the Build module images, and the Push module images task can be edited so that the modules are available in the edge solution. Second, the modules can be built and pushed to Azure Container Registry. Third, the deployment manifests can be uploaded to say an IoT Hub. Finally, the edge devices can get the deployment and pull images from Azure Container Registry.

The targets are one example of how flexible and critical the Azure Pipeline is but there are others.  YAML files can be authored to take advantage of multiple channels. It is also possible to integrate with other CI/CD workflows and a wide range of tasks can be enabled with extensions. Some extensions for common use cases are already available. Some ecosystem support can be enumerated as the support to automatically build and test .NET Core projects, build test and deploy Node.js applications, Python, Java, Go, PHP, and GitHub Actions.

Applications can be built on multiple branches and CI trigger can be created for a topic branch, multiple platforms can be built, pipeline artifacts can be published at any stage, service containers can be used, cross platform scripts can be executed. PowerShell script with the use all Azure and desktop cmdlets can be written, Git commands can be run, the build time can be reduced using caching. Build runs and versions can be incremented, and tasks can be used to increase retention. These are just some of the examples in which the pipeline can be sued.

Conclusion: These are the ways in which the Azure Pipeline can be used, and it is available as just another cloud resource along with the benefits that come with a cloud service.