Introduction:  

This article is a continuation of the series of articles starting with the description of SignalR service. In this article, we begin to discuss Azure services for Containers including the Azure Kubernetes Service and the Azure Container Instance after the last article on Azure Stream Analytics.  AKS provides a Kubernetes cluster in a serverless environment hosted in the cloud which can elastically scale your Kubernetes cluster to meet the challenges of your workload. The workload can be run in the cloud at the edge or as a hybrid with support for running .Net applications on Windows Server containers. Java applications on Linux containers or microservice applications in different languages and environments. Essentially, this provisions a datacenter on top of Azure stack HCI. The hyper-converged infrastructure is an Azure service that provides security performance and feature updates and allows the data center to be extended to the cloud. When the AKS is deployed on a Windows server 2019 data center the cluster is local to the Windows Server but when it is deployed to the Azure stack HCI it can scale seamlessly because the HCI is hosted on its set of clusters.

Well-known container orchestration frameworks such as Kubernetes rose in popularity meteorically with their widespread acceptance on Linux hosts. The challenge with running Kubernetes is that the Kubernetes Control Plane must be hosted on Linux flavor Virtual machines. While it can support workload servers on both Linux and Windows, it is not easy to get much further by trying to run the control plane on Windows Server. Minikube tried to reduce this gap but it could not avoid this requirement either. Once we have a virtual box on a window server, an Ubuntu flavor image can be used to launch a host and then deploying the Kubernetes is easy. Azure Kubernetes Service also requires Linux flavor servers for launching the Kubernetes control plane and it can run worker nodes on either operating system.

Kubernetes uses pods to run an instance of the microservice or user application. The pods, deployments, stateful sets, and daemon sets are the worker node components. The pod hosts, deployment controllers, and Kubernetes scheduler ensures that the replicas are distributed efficiently. When the pods have images that correspond to windows servers and OS flavors, they can run .NET applications. Mixed-mode operating systems can also be utilized for hosting an application by leveraging Kubernetes node selectors, taint, and toleration techniques. 

The issues faced in hosting Kubernetes control plane on Windows with Minikube, MicrosK8s, and other Kubernetes runtimes include the following:

1)      Use of an external insecure docker registry 

a.       The insecure term is used only for HTTP versus HTTPS. It was required because docker and minikube on windows work together by taking an –insecure-registry as a start-up parameter. This is not the case on Linux where we can do without this parameter and have the minikube host its own docker registry. On windows, we install the Docker toolbox and Minikube separately. This gives us two virtual machines by name ‘default’ for docker and ‘minikube’ for the Kubernetes cluster. Both are Linux environments

2)      Sufficient disk space and NAT traversal

a.       The insecure registry requires sufficient disk space. The default size is about 18GB and this is not sufficient when there are multiple images required. The typical size for commercial workloads is at least 30GB. The Docker Toolbox is preferred over other software packages for installing Docker on Windows. It is preferable to have Minikube start with at least 2 CPU and 8 GB memory. Small minikube deployments are expected to have frequent restarts due to low resources. The specification of CPU, memory and disk for the whole host does not necessarily lower the size of various clusters used with workloads. A minimal-dev-values file could help specify the lower end of the number of replicas and containers and their consumption. 

3)      Minikube’s support is usually host-facing as opposed to the external world which is also why it requires an insecure registry.

4)      Security of the overall Minikube installation as opposed to the deployment of the Kubernetes stack on the Linux nodes is a concern because the instance is already completely owned as opposed to a managed and locked-down instance. 

Cloud services do not encounter these issues because they have the luxury of elastic and even hyper-converged resources and they have absolutely no need to host the Kubernetes control plane on Windows.