Service Fabric (continued)     

Part 2 compared Paxos and Raft. Part 3 discussed SF-Ring and Part 4 discussed its architecture. This article describes how to configure Service Fabric clusters

The Service Fabric managed cluster SKU is a starting point for creating a Service Fabric cluster. There are several other ways to configure it. These include:

1.       Adding a virtual machine scale set extension to a node type - Each node type in a service fabric managed cluster is backed by a virtual machine scale set (vmss). It enables us to add vmss extensions that are small applications that provide post-deployment configuration and automation on Azure VMs. Extensions take mandatory parameters and simplify the installation.

2.       Configuring cluster availability zone spanning – Service fabric clusters can span availability zones in the same region which provide low latency network and high availability. Azure availability zones are only available in select regions.

3.       Configuring cluster network settings – Service Fabric managed clusters are created with a default network configuration which consists of a load balancer, public IP, VNet with one subnet allocated, and a NSG configured for essential cluster functionality. The settings of this network can be configured with NSG rules, RDP access, Load balancer config, IPV6, custom virtual network, custom load balancer, accelerated networking, and auxiliary subnets.

4.       Configuring a node type for large vmss – which are created by specifying the multiplePlacementGroups in added to the nodeType definition. Managed cluster node types set this property to false to keep fault and upgrade domains consistent within a placement group but it can be set for large vmss.

5.       Configuring managed identity - which are specified by a property vmManagedIdentity that has been added to node type definitions and contains a list of identities that may be used.

6.       OS and Data encryption – The disk encryption options are selected with the recommendation that the encryption be set at host. This improves on Azure disk encryption by supporting all OS types and images, including custom images.

7.       Autoscaling - which gives great elasticity and enables addition or reduction of nodes on demand on a secondary node type. The rule for the workload can be configured and autoscaling handles the rest.

8.       Scaling a node type – A service fabric managed cluster node type is scaled with portal, ARM template or PowerShell. We can also configure autoscale for a secondary node type if a fully automated solution is needed.

9.       Enabling Automatic OS image upgrades – This is a choice for the user even though the Service Fabric managed clusters manages the vmss resources.

10.   Modifying the OS SKU for a nodeType – This is helpful for scenarios such as migrating from earlier version of Windows to later or from switching one SKU to another.

11.   Configuring placement properties – These are used to ensure that certain workloads run only on certain node types in the cluster

12.   Managing the disk type SKU such that managed disks are used for all storage types so that the disk type and size are not provisioned.

13.    Configuring the cluster upgrade options such as wave deployment for automatic upgrades

This rounds up the configurations for the service fabric cluster.

Service Fabric (continued)    

Part 2 compared Paxos and Raft. Part 3 discussed SF-Ring and Part 4 discussed its architecture. This article describes monitoring and diagnostics.

Service Fabric provides monitoring at various levels – Application Monitoring, Cluster Monitoring, Infrastructure monitoring etc.

Application monitoring allows us to study the performance of features and components of an application. The responsibility of application monitoring is on the users developing an application and its services. It answers questions on how much traffic is flowing to the application, the success of the calls made to the services, the actions taken by the users on the application, if the application is throwing unhandled exceptions and whether the services are running fine within their containers.

Service Fabric makes applications resilient to hardware failures. Failures are rapidly detected, and workloads go through failover to other nodes. Cluster monitoring is critical to this end. There are a comprehensive set of events out of the box. These events can be accessed through the event store or the operational channel. Service Fabric events are available from a single ETW provider with a set of relevant filters to separate them. EventStore provides events available in the Service Fabric Explorer and through the REST API.

These events illustrate activities taken by the platform on different entities such as Nodes, Applications, Services, Partitions, etc. If a node were to go down, the platform would emit a NodeDown event and a tool of choice would use that event to generate notifications. These events are available both on Windows and Linux clusters.

The platform includes a health model which provides extensible health reporting for the status of the entities in a cluster. Each node, application, service, platform, replica or instance has a continuously updatable health status. Whenever the health of a particular entity transitions, a corresponding event would also be emitted. Queries and alerts can be setup on a dashboard with these events.

Users can also override heath for entities. If the application is going through an upgrade and the validation tests were failing, then the events can be written to the Service Fabric Health using the Health API to indicate that the application is no longer healthy and the Service Fabric will automatically rollback the upgrade.

Watchdogs are available as a separate service that watches health and load across the services, pings endpoints and reports unexpected health events in the cluster. This can help prevent errors that may not be detected and are based only on the performance of a single service. Watchdogs are also a good place to host code that performs remedial actions that don’t require user involvement such as archiving older log entries

Infrastructure monitoring is also referred to as performance monitoring since it pertains to system performance and depends on many factors. These are typically measured through performance counters. They can come from a variety of sources including the operating system, the .Net framework, or the service fabric platform itself. Performance counters are also available for Reliable Services and Actors programming models.

Application Insights is used for application monitoring, Diagnostic agent is used for cluster monitoring and Azure monitor logs is used for infrastructure monitoring.

 

Service Fabric Summary continued Part 3. 

Part 1 of Service Fabric Summary focuses on the principle and the purpose behind the design decisions in Service Fabric. Part 2 focuses on usage.  Part 3 in this article focuses on efficiency.

Service Fabric provides an infrastructure to build, deploy, and upgrade microservices efficiently with options for auto scaling, managing state, monitoring health, and restarting services in case of failure. It helps developers and administrators to focus on the implementation of workloads that are scalable, reliable, and manageable by avoiding the issues that are regularly caused by complex infrastructures. The major benefits it provides include deploying and evolving services at very low cost and high velocity, lowering costs to changing business requirements, exploiting the widespread skills of developers, and decoupling packaged applications from workflows and user interactions.   

 

Service Fabric follows an application model where an application is a collection of microservices. The application is described in an application manifest file that defines the different types of service contained in that application, and pointers to the independent service packages. The application package also usually contains parameters that serve as overrides for certain settings used by the services. Each service package has a manifest file that describes the physical files and folders that are necessary to run that service, including binaries, configuration files, and read-only data for that service. Services and applications are independently versioned and upgradable. 

 

A package can deploy more than one application but if one service fails to upgrade, the entire application is rolled back. For this reason, the microservices architecture is best served by multiple packages. If a set of services share the same resources and configuration or have the same lifecycle, then those services can be placed in the same application type. 

Service Fabric programming models can be chosen whether the services are stateful or stateless. 

 

A stateless service is chosen when it must scale, and the data or state can be stored externally.  There is also an option to run an existing service as a guest executable and it can be packaged in a container will all its dependencies. Service Fabric models are both container and executable as stateless services.  

 

Service Fabric is also useful for scenarios that require low-latency reads and writes, such as in online gaming or instant messaging. Applications can be built to be interactive and stateful without having to create a separate store or cache. Gaming and instant messaging are some examples of this scenario. 

Applications that must reliably process events or streams of data run well on Service Fabric with its optimized reads and writes. Service Fabric supports application processing pipelines, where results must be reliable and passed on to the next processing stage without any loss. These pipelines include transactional and financial systems, where data consistency and computation guarantees are essential. 

Stateful applications that perform intensive data computation and require the colocation of processing (computation) and data in applications benefit from Service Fabric as well. Stateful Service Fabric services eliminate that latency, enabling more optimized reads and writes. As an example, real-time recommendation selections for customers that require a round trip-time latency of less than hundred milliseconds are handled with ease. 

Service Fabric also supports highly available services and provides fast failover by creating multiple secondary service replicas. If a node, process, or individual service goes down due to hardware or other failure, one of the secondary replicas is promoted to a primary replica with minimal loss of service. 

Individual services can also be partitioned where services are hosted by different hosts. Individual services can also be created and removed on the fly. Services can be scaled from a few instances on a few nodes to thousands of instances on several nodes and dialed down as well. Service Fabric helps with the complete life cycles. 

 

 

Capacity and Scaling are two different considerations for Service Fabric and must be reviewed individually. Cluster capacity considerations include Key considerations include initial number and properties of cluster node types, durability level of each node type, which determines Service Fabric VM privileges within Azure infrastructure, and reliability level of the cluster, which determines the stability of Service Fabric system services and overall cluster function 

A cluster requires a node type. A node type defines the size, number, and properties for a set of nodes (virtual machines) in the cluster. Every node type that is defined in a Service Fabric cluster maps to a virtual machine scale set aka VMSS. A primary node type is reserved to run critical system services. Non-primary node types are used for backend and frontend services. 

Node type planning considerations depend on whether the application has multiple services or if they have different infrastructure needs such as greater RAM or higher CPU cycles or if any of the application services need to scale out beyond hundred nodes or if the cluster spans availability zones. 

The durability characteristics of the cluster are offered as durability tiers. A durability level designates the privileges that Service Fabric VMs have with the underlying Azure architecture which can allow override of infrastructure requests such as reboot, reimage or migration that impacts the quorum requirements. 

Scalability considerations depend on whether scaling is required for the number of nodes of each node type or if it is required for services. 

Migration plan for a cloud service to Service Fabric:

 

The preparation for the migration involves the following:

1)      Defining the Service Fabric cluster: The planning for a Service Fabric cluster will culminate with an Azure Resource Manager (ARM) template that defines the node types and their numbers.  Sample template for creating a Service Fabric cluster is included here.

2)      Developer workstation requirements: The workstation requirements are important for application development. These include Docker for windows to containerize and test the applications prior to deployment, Visual Studio Docker support for Docker compatibility, and compute resources when workstations do not have 8GB of memory. Azure DevTest Labs service is an option for such compute.

3)      Networking requirements: Service Fabric deployments for large enterprises require careful planning for the following:

a.       Reachability over VPN: Enterprises extend their corporate network with a private address space to an Azure subscription. ExpressRoute or Site-to-Site VPN provides secure on-premises connectivity and must have corresponding resources instantiated for viewing and managing via the Azure Portal.

b.       NSG rules: Enterprises must secure their inbound and outbound network traffic through third-party firewall appliances or with Azure Network Security Group rules or both.

c.       Address Space: The Service Fabric cluster has tight control over address space requirements and subnets. These will also be described under the placement constraints.

4)      Containerizing existing Windows applications: Docker images can run an operating system, an IIS server and the windows application to be migrated to Service Fabric. Proper choice of base line image, Docker layers, customizations and hardening are required to secure the containerized image. A sample Dockerfile has been included here for this purpose. The base image could be a locked-down custom enterprise image that enforces enterprise requirements but often it is an official signed image from a public image repository and as an alpine version at that.

5)      Cluster configuration – node types:  A Service Fabric cluster may have one or mode nodeTypes which loosely map to an Azure Virtual Machine Scale Set (VMSS) each. A VMSS is a set of Virtual Machines (VM)s with the same set of properties. Each cluster must have at least one node type-primary node type, which runs the Service Fabric system services. The application containers cannot be run on the primary node type. The sizing of the primary node type is directly proportional to the size of the cluster. T-shirt sizing of Service Fabric cluster is available and can be referenced via monikers such as Standard_D2s_v3. Application and Data intensive compute can form the other node types. Choice of node types can be determined based on the application that is being migrated. If the Service Fabric cluster starts out with being dedicated to a single application, it can have just two node types – primary and application. Each scale set can be scaled in or out independently and does not require extensive planning and testing because the actual size can grow or shrink driven by monitoring and metrics.

Migration plan for a cloud service to Service Fabric: 

Goal: List some of the considerations towards the migration of a web service hosted on IaaS infrastructure to Service Fabric. A description of the summary and features of Service Fabric are shown here. This article focuses on architectural decisions and recommended practices. 

The migration cost is usually a sliding scale between the extremes of no code changes to the existing service to refactoring for stateless microservices such that they can be run on serverless platforms. Proper choice of migration strategy can ensure a great deal of satisfaction for all stakeholders.

A typical approach to migrate existing workloads is the lift-and-shift strategy. For example, the workload can be provisioned directly to another VMs with network and storage components and deploy the existing applications onto those VMs. Another approach is to move the application to PaaS platform. The drawback of the lift-and-shift strategy is that it often results in overprovisioning and overpaying for compute resources.

The cost-effective way of running applications has been demonstrated by the container orchestration framework. Containerizing an existing application enables it to run on a cluster with other applications. It provides improvements in resource usages, dynamic scaling of instances, shared monitoring and DevOps.

Optimizing and provisioning the resources for containerization is not trivial. Service Fabric allows experimentation by scaling out the instances on demand. Both Windows and Linux applications can be migrated to a runtime platform without changing code and their instances can be scaled without overprovisioning VMs. The result is better density, better hardware use, simplified operations, and overall lower cloud-compute costs.

Even a large set of Windows-based web applications on erstwhile IIS hosting infrastructure can be migrated to Service Fabric with improved density, monitoring, consistency, and DevOps all within a secure extended private network in the cloud. The principle is to use Docker and Service Fabric’s containerization support that package and hosts existing web applications on a shared cluster with preconfigured monitoring and operations. This results in an optimal performance-to-cost ratio.

Migration plan for a cloud service to Service Fabric:

Goal: List some of the considerations towards the migration of a web service hosted on IaaS infrastructure to Service Fabric. A description of the summary and features of Service Fabric are shown here. This article focuses on architectural decisions and recommended practices.

Service Fabric is Microsoft’s distributed platform for building, running, and maintaining microservices applications in the cloud. It is a container orchestrator, and it can provide quality of service to microservice framework models such as stateful, stateless, and actor. It differs from Azure Container Service in that it is an Infrastructure-as-a-Service offering rather than a Platform-as-a-Service offering. There is also a Service Fabric Mesh offering that provides a PaaS service for Service Fabric applications. Service Fabric provides its own specific programming model, allows guest executables, and orchestrate Docker containers. It supports both Windows and Linux, but it is primarily suited for Windows. It can scale to handle Internet-of-Things traffic. It is open to workloads and is technology-agnostic. It relies on Docker and has supported both Windows and Linux containers, but it provides a more integrated feature-rich orchestration that gives more openness and flexibility. 

The table of comparisons includes:

Area	Cloud Services	Service Fabric
Application composition	Roles	Services
Density	One role instance per VM	Multiple services in a single node
Minimum number of nodes	2 per role	5 per cluster, for production deployments
State management	Stateless	Stateless or stateful
Hosting	Azure	Cloud or on-premises
Web hosting	IIS	Self-hosting
Deployment model	Classic deployment model	Resource Manager
Packaging	Cloud service package files (.cspkg)	Application and service packages
Application update	VIP swap or rolling update	Rolling update
Autoscaling	Built-in service	Virtual machine scale sets for auto scale out
Debugging	Local emulator	Local cluster

 

A worker role can be mapped to a Service Fabric stateless service. A web role can also be mapped to a ServiceFabric stateless service. Unlike Web roles, service fabric does not support IIS. A service must move to a web framework that can be self hosted and then the service must be moved to Service Fabric

Worker role and service fabric Service API offer similar entry points. The key differences between the lifecycle of worker roles and Service Fabric services is that the Lifecycle is tied to the VM for a worker role and It is separate from the VM for a service fabric service.

The lifetime of a worker role is determined by when the Run method exits while the RunAsync method for service fabric can run to completion while the instance remains. The Http listeners are described in the service definition files for cloud services while they are specified in the service manifest for service fabric. The configuration for a Service Fabri Service comes from application manifest in the Application package, service manifest in the service package, and settings.xml from the configuration package. The settings can be saved in the setting.xml file, the application manifest can be used to override the setting, and the environment specific settings can be put into the parameter file.

 Service Fabric Notes Summary

ServiceFabric is Microsoft’s distributed platform for building, running and maintaining microservices applications in the cloud. It is a container orchestrator and it is able to provide quality of service to microservice framework models such as stateful, stateless, and actor. It differs from Azure Container Service in that it is a Infrastructure-as-a-Service offering rather than a Platform-as-a-Service offering. There is also a Service Fabric Mesh offering that provides a PaaS service for Service Fabric applications. Service Fabric provides its own specific programming model, allows guest executables and orchestrate Docker containers. It supports both Windows and Linux but it is primarily suited for Windows. It can scale to handle Internet-of-Things traffic. It is open to workloads and is technology-agnostic. It relies on Docker and has supported both Windows and Linux containers but it provides a more integrated feature-rich orchestration that gives more openness and flexibility. 

It is often compared to Kubernetes which is also a container orchestration framework that hosts applications. Kubernetes extends this idea of app+container all the way where the host can be nodes of a cluster. Cluster membership is maintained by requiring kubelets to maintain a connection with the API server to send heartbeats every period. Service Fabric avoids this with a decentralized ring and failure detection. It is motivated by the adage that distributed consensus is at the heart of numerous co-ordination problems, but it has a trivially simple solution if there is a reliable failure detection service.

Leader election differs from algorithm to algorithm. For example, Raft algorithm was proposed to address the long-standing issues with understandability of the widely studied Paxos algorithm. It has a clear abstraction and presentation and can be a simplified version of the original Paxos algorithm. Specifically, Paxos divides terms between servers whereas Raft allows a follower to become a candidate in any term, but followers will vote for only one candidate per term. Paxos followers will vote for any candidate, whereas Raft followers will only vote for a candidate if the candidate’s log is at least as up to date. If a leader has uncommitted log entries from a previous term, Paxos will replicate them in the current term whereas Raft will replicate them in their original term. Raft’s leader election is therefore lightweight when compared to Paxos. Service Fabric organizes nodes in rings and the heartbeats are only sent to a small subset of nodes called the neighborhood. The arbitration procedure involves more nodes besides the monitor but it is only executed on missed heartbeats. A quorum of privileged nodes in an arbitration group helps resolve the possible failure detection conflicts, isolate appropriate nodes, and maintain a consistent view of the ring. Membership and failure detection in Service Fabric relies on two key design principles: 1. A notion of strongly consistent membership and 2. Decoupling failure detection from failure decision. All nodes responsible for monitoring a node X must agree on whether X is up or down. When used in the SF-ring, all predecessors and successors in the neighborhood of a node X agree on its X’s status. The SF-Ring is a distributed hash table. It provides a seamless way of scaling from small groups to large groups. SF-Ring was developed at around the same time as P2P DHTs like Pastry, Chord, and others. SF-Ring is unique because 1) Routing table entries are bidirectional and symmetrical, 2) the routing is bidirectional, 3) Routing tables are eventually convergent, 4) there is decoupled mapping of Nodes and keys and 5) There are consistent routing tokens. 

Service Fabric distinguishes itself with support for strong consistency and support for stateful microservices. Each of the SF components offer strong consistency behavior. There were two ways to do this: provide consistent – build consistent applications on top of inconsistent components or use consistent components from the grounds-up. The end-to-end principle dictates that if performance is worth the cost for a functionality then it can be built into the middle. If consistency were instead to only be built at the application layer, each distinct application will have significant costs for maintenance and reliability. Instead if the consistency is supported at each layer, it allows higher layer design to focus on their relevant notion of consistency and allows both weakly consistent applications and strongly consistent applications to be built on top of Service Fabric. This is easier than building consistent applications over an inconsistent substrate.