The Lessons from enabling search for multitenant applications:

Multitenant search solutions provide search capabilities to any number of tenants who cannot see or share data of any other tenant. Searching often requires content virtualization across different sources where the content can be considered a shared resource. Providing content isolation with search capabilities is the topic of this article.

A search service usually builds an index of the content so that the queries for the content can return results efficiently since the index is far easier and more efficient to lookup than traversing the entire content store. The search service can be considered equivalent to a database service and indeed search can be provided out of box from a database. All the multitenant considerations for database services hold for the search service as well. Each index within a search service has its own schema, which is defined by several customizable fields. Search services, indexes, fields and documents are infrastructure and there is little or no user control.

Some considerations for multitenant applications:

Multitenant applications must effectively distribute resources while preserving privacy between the tenants. A few considerations must be made when designing the architecture for such an application.

-          Tenant Isolation – No tenants must have unauthorized or unwarranted access to the data of other tenants

-          Cloud resource cost – any solution must remain cost-effective.

-          Ease of operation – the impact on the application’s operations and complexity is important.

-          Global Footprint – Applications may need to effectively serve tenants which are distributed across the globe

-          Scalability – applications may need to reconcile between maintaining a sufficiently low level of complexity and scaling to the number and size of workloads.

A uniform scope is assumed where each tenant is a whole instance of an application, but they can handle many smaller scopes. If the service per tenant and index per tenant are not sufficiently small scopes, then the index can be utilized for a finer degree of granularity. A field can be added to an index to make it behave differently to different client endpoints. This helps to achieve separate user accounts, separate permission levels, and even separate applications.

Reference: https://1drv.ms/w/s!Ashlm-Nw-wnWhLMfc6pdJbQZ6XiPWA?e=fBoKcN

 

 The Lessons from enabling search for multitenant applications:

Multitenant search solutions provide search capabilities to any number of tenants who cannot see or share data of any other tenant. Searching often requires content virtualization across different sources where the content can be considered a shared resource. Providing content isolation with search capabilities is the topic of this article.

A search service usually builds an index of the content so that the queries for the content can return results efficiently since the index is far easier and efficient to lookup than traversing the entire content store. The search service can be considered equivalent to a database service and indeed search can be provided out of box from a database. All the multitenant considerations for database services hold for the search service as well. Each index within a search service has its own schema, which is defined by several customizable fields. Search services, indexes, fields and documents are infrastructure and there is little or no user control. A search service can enable tenant isolation via one of the following three methods.

The application developer consumes one or more search services and divide their tenants among services, indexes or both. The methods to provide tenant isolation, scale, cost efficiency, ease of operations and reduced footprint have the following patterns:

1.       One index per tenant: This pattern has an index dedicated to a tenant within a search service that is shared with other tenants.

2.       One service per tenant: This pattern has its own dedicated service offering the highest levels of data and workload separation.

3.       A mix of both: Larger more active tenants are assigned dedicated services and the smaller ones are assigned individual indexes within the shared services.

In an index-per-tenant model, tenants achieve data isolation because all search requests and document operations are issued at an index level. The application layer directs tenant traffic to proper indexes while managing the resources at the service levels across all tenants. This model is simple for the application developers and allows them to oversubscribe the capacity among the application’s tenants. The tradeoff is the restrictions to handle peak load where most or all of the tenants are highly active. The index per tenant model provides the basis for a variable cost model. It may not be the most efficient for an application with a global footprint.

In the service-per-tenant model, each tenant has its own search service. In this model, each application achieves a maximum level of isolation for its tenants. Each service has dedicated storage and throughput for handling search request as well as separate API keys. This pattern is effective when each tenant has a large footprint, or the workload has little variability from tenant to tenant since resources are not shared.

With the mix for both index-per-tenant and service-per-tenant strategies, an application’s largest tenants can occupy dedicated services while the numerous less active, smaller tenants can occupy indexes in a shared service. It ensures that the largest tenants can derive high performance from the service, while protecting the smaller tenants from noisy neighbors. There is some planning required to classify tenants into those that required dedicated and those that can leverage a shared service.

The above patterns assume a uniform scope where each tenant is a whole instance of an application but they can handle many smaller scopes. If the service per tenant and index per tenant are not sufficiently small scopes, then the index can be utilized for a finer degree of granularity. A field can be added to an index to make it behave differently to different client endpoints. This helps to achieve separate user accounts, separate permission levels, and even completely separate applications.

Reference: https://1drv.ms/w/s!Ashlm-Nw-wnWhLMfc6pdJbQZ6XiPWA?e=fBoKcN

 

 The Lessons from the overuse of multitenancy – quotas and reservations:

The computing power and storage matters to the users more than how it is provisioned and where. The ease of requesting the compute, the cleanliness of instantiating and deleting the associated resource, the ability to migrate applications and services far outweigh any consideration of the vendor or the mechanism just so long as the performance, security and costs are comparable.

One of the classic indicators of overengineering is when there are many dials and knobs. There are two specific problems with these. First, it is an enormous burden for the users to figure out when and how to tune them optimally. Second, it increases the chance of oversight, improper use, and mistakes.

Consider the case brought forward by Alphonse Chapanis, a US Army Air Force aero medical lab psychologist who interviewed many pilots of the B-16 ‘Flying fortress’ bomber aircraft and were distressed to have landed a perfectly good plane after their mission only to see it suffer damage on stopping slowly. He determined that the pilots could not distinguish between the switches between the flaps and the landing gear that were co-located and retracted the landing gear when they meant to retract the flaps causing a nosedive and damage to the wings, propeller, and the underside of the aircraft. This incident review resulted in the switches having the landing wheel and wing flap design introduced to tell them apart and even to place them on opposite sides of the controller box. The savings from the prevented aircraft loss thereafter are a testament to the proper placement and use of controls.

Containers are governed by the quotas and limits of the underlying system to manage the shared resources. Solaris introduced partitions for different flavors, quotas and limits for file-systems and soon applications would start failing in one partition because they didn't have enough resources on this as they earlier had on the undivided host. One of the reasons Solaris zones suffered from application migration considerations from the operating system consumers is that they kept running into application failures due to limits. Both Linux containers and Solaris zones must be understood as operating system features and not as standalone products for virtualization technologies.

Linux containers have the advantage that they provide the same environment without the overhead of a separate kernel and hardware. This translates to lightning-fast availability of an isolated shell as well as their reduction of failure points. The hardware is not necessarily partitioned per virtual machine and the separation of kernel does play significantly in terms of performance because the emulation is different from the real especially when compared to a standalone and dedicated instance.

 Quotas often cause resource starvation and non-deterministic behavior of otherwise sound applications. Soft limits complicate the matter even further. This calls for a judicious use of leveraging multitenancy.

 Storage and data considerations for a Multitenant Provider 

The previous articles described the multitenancy in databases using schema-based consolidation and within-database virtualization by partitioning the data dictionary horizontally. These articles also described elastic pools, sharding patterns, row-level security and key management. 

Data for applications grows by leaps and bounds year after year. This article describes the storage and data considerations and requirements.

When the volume of data increases, multitenancy depends on a clear strategy to scale the data and storage resources and to apply automation to their management, 

Elastic pools share compute resources between several databases on the same server. This helps to achieve performance elasticity of each database. The sharing of provisioned resources across databases reduced their unit costs. There are built-in protections against noisy neighbor problems. The architectural approach must meet the levels of the scale expected from the system.

Resource and request quotas must be part of this decision.  If a single storage account is deployed to contain all of the tenant’s data, then exceeding a specific number of storage operations per second will reject the application’s requests and all of the tenants will be impacted. Monitoring and retry for services must be enabled.

When designing a solution that contains multitenant data services, there are different options and levels of data isolation, For example, there can be separate containers between tenants or the database and accounts can be shared between multiple tenants. When using Azure storage for blob data, separate blob containers or separate storage accounts for each tenant can be deployed.  When deploying resources for services they can be in a single shared Azure subscription, or they can be in multiple Azure subscriptions with one per tenant. Isolation must even be higher to meet security and compliance requirements or to avoid noisy neighbor problems. Keeping the architecture as simple as possible while meeting the requirements helps with growth and planning. Tenants will likely require customizations and using feature flags to independently author, test and deploy updates are necessary. Cross tenant management operations such as regular maintenance activities, monitoring and metering of tenants, reporting data from across isolated tenants enforcing a schema and how to deploy schema updates, considering high availability requirements, and migrating tenants needed to move to different types of service, deployment or perhaps another region are all operational considerations that must be planned for in the multitenancy approach.

Higher tenancy density and lower operating costs are desirable but not the only optimization parameter. Patterns can be judiciously chosen or they can be mixed and matched especially for creating service levels. It is a good practice to scale by using deployment stamps. When we work with shared infrastructure, there are several caveats to consider. If a single resource is used, it might impose scale restrictions and limits that might interfere with maximum scale and current and future limits. Measuring and monitoring for a single tenant might be difficult depending on the services used and their tiers. Choosing premium tiers can help overcome this difficulty.

Sharding pattern can scale to a large number of tenants with appealing cost effectiveness. Horizontal, vertical and functional data partitioning can be applied. Other patterns include dedicated storage containers for each tenant and geographically distributed containers patterns. 

Each storage product or its organizational unit might have features to support multitenancy.  For example, the previous article described multitenancy approach with row-level security, tenant-level encryption, resource pooling and governance, sharding and partitioning 

Authentication and authorization strategy must be planned. A valet-key pattern can provide clients with access to storage resources. Finally, consumption must be measured and costs must be assigned to tenants. 

  Databases as a Multitenant Provider 

 

The previous article described the multitenancy in databases using schema-based consolidation and within-database virtualization by partitioning the data dictionary horizontally. This article focuses on elastic pools, sharding patterns, row-level security and key management.

Elastic pools share compute resources between several databases on the same server. This helps to achieve performance elasticity of each database. The sharing of provisioned resources across databases reduced their unit costs. There are built-in protections against noisy neighbor problems.

Resource management in dense elastic pools is achieved by implementing resource governance.

Within a database there can be multiple resource pools and workload groups, with resource limits set at both pool and group levels. User workloads and internal workloads are classified into separate resource pools and workload groups. User workload on the primary and readable secondary replicas into a designated pool and partitioned workload groups for various internal workloads. Other pools and workload groups may be reserved for various internal workloads.

In addition, job objects may be used for process level resource governance and File Server Resource Manager may be reserved for storage quota management.

Resource governance is hierarchical in nature. From top to bottom, limits can be enforced at various levels using their level appropriate mechanisms starting with the operating systems, then the resource pools and the workload groups. Data I/O governance limits both the read and the write physical I/O against data files of a database. IOPS limits are set for each service level to minimize the noisy neighbor effect. This approach allows customers to use dense elastic pools to achieve adequate performance and major cost savings. The only shortcoming with this approach is that dense competition gives rise to significant resource contention which can impact internal processes. One of the following three mitigation actions can be chosen by the customers. First, the query workload can be tuned to reduce resource consumption. Second, the pool density can be reduced by moving some databases to another pool and 3. the pool can be scaled up to get more resources.

The Sharding pattern enables to scale the workloads across multiple databases. Tools provided by the databases support the management of shard maps which track the tenants assigned to each shard. They also initiate and track queries and management operations on multiple shards by using elastic jobs.

These jobs can be periodically executed against one or many databases to run queries and perform maintenance tasks. The scripts must be defined, maintained, and persisted across a group of databases

Row-level security is useful for enforcing tenant level isolation in sharded tables. Group memberships or execution contexts are used to control access to the rows in a database table. It simplifies the design and coding of security in the application and implements restrictions on data row access. This access restriction logic is based out of the database tier rather than one spanning the application tier. This system is made more reliable and robust by reducing the surface area of the security system.

End-to-end encryption of data at rest and in transit is achieved through encryption keys and separation of databases for each tenant and always enabling the always encrypted feature.

Storage and data approaches for multitenancy must consider scale, performance predictability, data isolation, complexity of implementation, complexity of management and operations, costs, patterns and anti-patterns and best practices.

 

 

 

Databases as a Multitenant Provider

Persistence of data mandates a database for the guarantees of Atomicity, Consistency, Isolation and Durability as well as for querying. Tenants can be served their own databases in which case the database becomes a multi-tenant data platform and the technology stack for handling the data might vary from tenant to tenant.  Groups of database can be put into a container to manage them as a single unit. The benefits of a multitenant database include the following:

1.       High consolidation density where containers of database makes it easier to manage databases than the conventional single database per tenant

2.       Rapid provisioning and cloning using SQL where the database can be created, populated and cloned using SQL scripts.

3.       Rapid patching and upgrades where the patching of individual databases inside a container can be accomplished by plugging out and plugging in the database. The patching of the container is also fast.

4.       Converting and consolidating existing databases into containers so that they can be managed as a unit.

5.       Resource management between pluggable databases in a container

Multitenancy brings standardization which reduces the operating expense. Cost often increases with variations in size and traffic on databases. Customers benefit when the databases can scale out and remain elastic. Both operational expense and capital expense reduces.

It is also possible to host distinct application backends into the same database and this approach is called schema-based consolidation. This reduces the number of databases and increases the consolidation density. There are a few caveats to this approach which include 1. Name collision hampers schema-based consolidation, 2. It brings weak security 3. Per application backend point-in-time recovery is prohibitively difficult. 4. Resource management between application backends is difficult, 5. Patching for a single application backend is disallowed and 6. Cloning a single application backend is not permitted.

The difference between dedicated database architecture and this multitenant architecture is that the latter brings true within-database virtualization. This is implemented by partitioning the data dictionary horizontally. Within database virtualization removes the drawbacks of schema-based consolidation. The physical separation between the container and the databases brings pluggability. This unplug/plug action set brings a new paradigm for patching. The sharing model and the background processes is the same for conventional, pluggable databases and application backends. The container or root is the new tenet of the multitenant architecture. Just as operating systems orchestrate provisioning tasks, SQL statements executed against the root implement them for pluggable databases.

In this case, multitenancy becomes a deployment choice with neither the application backend nor the client code required to change.

Reference: Multitenancy: https://1drv.ms/w/s!Ashlm-Nw-wnWhLMfc6pdJbQZ6XiPWA?e=fBoKcN     

 

 Databases as a Multitenant Provider 

 

The previous article described the multitenancy in databases using schema-based consolidation and within-database virtualization by partitioning the data dictionary horizontally. This article focuses on elastic pools, sharding patterns, row-level security and key management.

Elastic pools share compute resources between several databases on the same server. This helps to achieve performance elasticity of each database. The sharing of provisioned resources across databases reduced their unit costs. There are built-in protections against noisy neighbor problems.

Resource management in dense elastic pools is achieved by implementing resource governance.

Within a database there can be multiple resource pools and workload groups, with resource limits set at both pool and group levels. User workloads and internal workloads are classified into separate resource pools and workload groups. User workload on the primary and readable secondary replicas into a designated pool and partitioned workload groups for various internal workloads. Other pools and workload groups may be reserved for various internal workloads.

In addition, job objects may be used for process level resource governance and File Server Resource Manager may be reserved for storage quota management.

Resource governance is hierarchical in nature. From top to bottom, limits can be enforced at various levels using their level appropriate mechanisms starting with the operating systems, then the resource pools and the workload groups. Data I/O governance limits both the read and the write physical I/O against data files of a database. IOPS limits are set for each service level to minimize the noisy neighbor effect. This approach allows customers to use dense elastic pools to achieve adequate performance and major cost savings. The only shortcoming with this approach is that dense competition gives rise to significant resource contention which can impact internal processes. One of the following three mitigation actions can be chosen by the customers. First, the query workload can be tuned to reduce resource consumption. Second, the pool density can be reduced by moving some databases to another pool and 3. the pool can be scaled up to get more resources.

The Sharding pattern enables to scale the workloads across multiple databases. Tools provided by the databases support the management of shard maps which track the tenants assigned to each shard. They also initiate and track queries and management operations on multiple shards by using elastic jobs.

These jobs can be periodically executed against one or many databases to run queries and perform maintenance tasks. The scripts must be defined, maintained, and persisted across a group of databases

Row-level security is useful for enforcing tenant level isolation in sharded tables. Group memberships or execution contexts are used to control access to the rows in a database table. It simplifies the design and coding of security in the application and implements restrictions on data row access. This access restriction logic is based out of the database tier rather than one spanning the application tier. This system is made more reliable and robust by reducing the surface area of the security system.

End-to-end encryption of data at rest and in transit is achieved through encryption keys and separation of databases for each tenant and always enabling the always encrypted feature.

Storage and data approaches for multitenancy must consider scale, performance predictability, data isolation, complexity of implementation, complexity of management and operations, costs, patterns and anti-patterns and best practices.

 

 

 Sample program to add claim to token in delegated auth use case:

using System.IO;

using IdentityClaim = Microsoft.IdentityModel.Claims.Claim;

using IdentityClaimTypes = Microsoft.IdentityModel.Claims.ClaimTypes;

using IdentityClaimsPrincipal = Microsoft.IdentityModel.Claims.ClaimsPrincipal;

using ClaimsIdentityCollection = Microsoft.IdentityModel.Claims.ClaimsIdentityCollection;

            IClaimsIdentity claimsIdentity = new ClaimsIdentity(Thread.CurrentPrincipal.Identity);

            var claimValue = string.Format("claim://{0}@{1}", TargetResourceRole.PrivilegedDeploymentOperator, "sample-resource-folder-test");

            var identityClaim = new IdentityClaim(IdentityClaimTypes.Role, claimValue);

            claimsIdentity.Claims.Add(identityClaim);

            ClaimsIdentityCollection claimsIdentityCollection = new ClaimsIdentityCollection(new List<IClaimsIdentity>() { claimsIdentity });

            var newIcp = IdentityClaimsPrincipal.CreateFromIdentities(claimsIdentityCollection);

            Thread.CurrentPrincipal = newIcp;

The  above example uses the Microsoft.IdentityModel namespace to describe the elevation of privilege to run some code.

Now for the delegated auth use case:

           string homeSecurityTokenService = ConfigurationManager.GetSetting("HomeSecurityTokenService");

           string SecurityTokenServiceRealm = ConfigurationManager.GetSetting("SecurityTokenServiceRealm");

           string serviceName = ConfigurationManager.GetSetting("ServiceName");

           var serverHomeSecurityTokenService = new ServerHomeSecurityTokenService(

                    new Uri(SecurityTokenServiceRealm),

                    homeSecurityTokenService,

                    null);

           var serviceIdentity = new ServiceIdentity(

                serviceDnsHostName: targetDnsName,

                serviceNames: new string[] { serviceName });

           WebSecurityTokenAuthenticator authenticator = new WebSecurityTokenAuthenticator(serverHomeSecurityTokenService, serviceIdentity);

           ClaimsIdentityCollection collection = authenticator.Authenticate(authorizationHeader, resourceName);

           var claimValue = string.Format("claim://{0}@{1}", TargetResourceRole.PrivilegedDeploymentOperator, payload.Properties.Folder);

           collection.Add(new ClaimsIdentity(new List<Claim>() { new Claim(ClaimTypes.Role, claimValue) }));

var authContext = new Microsoft.IdentityModel.Clients.ActiveDirectory.AuthenticationContext(

                        tokenIssuanceUrl, true);

StringBuilder sb = new StringBuilder();

collection.ForEach(x => x.Claims.ForEach(c => sb.Append(c.Value + ",")));

var claims = sb.ToString().Trim(',');

var authenticationResult = 

authContext.AcquireTokenAsync(resourceName, clientCredential.ClientId, new Uri("https://DstsInternalNativeClient"), new PlatformParameters(PromptBehavior.Auto), userIdentifier, extraQueryParameters, claims, synchronizationContext);

var newDelegatedToken = authResult.AccessToken;

This is a continuation of a series of articles on hosting solutions and services on Azure public cloud with the most recent discussion on Multitenancy here. The previous articles introduced virtual SAN followed by the operational efficiencies and this follows up on the design points and the integrated systems.

HCI products can also be combined into an integrated offering with the opportunity to provide a solution to customers.  They are usually built on one or more proven building blocks for the Software-defined data center. The vSphere and vSan provide two such building blocks and the integrated appliance is preconfigured and pretested. It delivers features for resiliency, quality of service, and centralized management functionality, enabling faster, better, and simpler management of consolidated workloads, virtual desktops, business critical applications and remote office infrastructure.

HCI capabilities can also be offered with a mix and match of storage offerings from different vendors for the purposes of replication, backup and cloud tiering, at no additional cost. These appliances can also integrate with cloud management platform and end-user computing solution. Such appliances can also be promoted to visibility in the management plane. Integrated systems can also combine compute, storage and networking products with certified partner hardware into HCI appliances.

The advantages of a HyperConverged Infrastructure include the following:

-          It can be used to cut acquisition cost, maintenance cost and operational expenses. Both CapEx and OpEx can reduce

-          From acquisition to commission, the HCI requires fewer steps and fewer people than conventional techniques

-          It can enable dynamic responsiveness especially to fluctuations in traffic. The heavier the workload, the more resources are required. The changes are transparent and automatic due to Storage based policy management.

-          It improves precision and granularity because storage services are consumed on a pay-as-you-go basis. The performance, capacity and protection are provided just as much as needed.

-          Consumption can also be made flexible with the partnership between admins. For example, storage admins can provide large datastores for VMs to be created and the virtual infrastructure admins can set policies for individual VMS.

-          Both scale-up and scale out are supported by the HCI system for capacity and performance on a grow-as-you-go basis

-          Newer applications with dynamic resourcing requirements can be quickly accomodated. The popularity of containers and cloud applications is maintained by HCI.

-          There is consistent performance for every application from business critical applications to cloud native applications.

-          There is high availability from these systems even when failures occur.

-          HCI provide a building block towards the private cloud with its modular architecture.

Finally, HCI participates in making the design modular for the foundation of a private cloud.

 

 

This is a continuation of a series of articles on hosting solutions and services on Azure public cloud with the most recent discussion on Multitenancy here. The previous articles introduced virtual SAN followed by the operational efficiencies and this follows up on the design points and the use of Virtual SAN ready nodes.

Virtual SAN provides the ability to take operations down to the virtual machine level.  This fine-grain control is required from the application delivery perspective that matter most to enterprise IT users. Users are concerned about their applications and not about the infrastructure. HCI and vSAN recognize this requirement and are aligned with it. All storage services and the VM hosts that they reside on can be adjusted.

Storage systems have been traditionally defined to be application agnostic and providing multiple layers of abstraction over the raw bytes on the disk. The user facing organizational units differ from the storage facing ones and there is usually a table or a directory to map one with the other. When storage services are configured by application requirements rather than storage constraints, it gives greater control to the users.

More efficient operations are enabled with the use of storage policies to drive automation. The policies can be set for an application’s requirement on capacity, performance, availability, redundancy, and such others. The management control plane can automate the VM placement by finding the datastores that have capacity to provision them. Together the automation and policy-based management simplify storage management. It helps to quickly deliver value to the customers and those who use IT services.

It doesn’t mean application admin and storage admin can be one and the same. The former view the storage as a service without being slowed down by the service-fulfillment bottlenecks and might even expect a pay-as-you-go billing. The latter are primarily interested in automation and operational efficiencies. Similarly, virtual infrastructure administrators and storage administrators participate in a symbiotic relationship. Although they might be involved in a tug-of-war, software defined storage elminiates the reasons for it mostly with the help of storage policy based management. The storage admin is responsible for the upfront setup of the storage capacity and data services which are published as the so-called virtual datastore.  The Virtual Infrastructure admin uses the virtual datastore as a menu of storage items to serve the needs of the Virtual Machines.

No operational model is complete without unified management. IT operations have been dogged by swivel chair operation where the operator must jump from one interface to another to complete all aspects of a workflow. An HCI has the potential to provide a single pane of glass management instead of managing compute, storage and networking individually. Not al HCI provide this and some even come up with a mashup of underlying technology management but  seasoned HCI will  generally have a unified management. When it comes to HCI, system monitoring is critical to aid management. Although custom dashboards and historical trends might be available from metrics providers, a built-in monitoring system for an HCI goes hand in hand with its management portal. End-to-end monitoring and the whole picture of the software and its environment not helps with proactive monitoring but also troubleshooting and reducing costs

This is a continuation of a series of articles on hosting solutions and services on Azure public cloud with the most recent discussion on Multitenancy here. The previous articles introduced virtual SAN but this one delves into the operational efficiencies.

vSAN brings additional efficiencies in the form of deduplication, compression, and erasure coding. Processors and SSD provide the performance to run these data reduction technologies. These features improve the storage utilization rate which means less physical storage is required to store the same amount of data.

The delivery of maximum value and flexibility occurs only when the data center is completely software defined and it should not be tied to a specific hardware platform. This can be done in two ways:

First, allow the flexibility via choices in the commodity hardware used for the hypervisor and vSAN in terms of the x86 servers and their vendor.

Second, allow a fast-track of the HCI through turnkey appliances.

In addition to flexibility, a vSAN must be designed for nondisruptive and seamless scaling. Usually this is not a problem when the server additions do not affect the initial lot but it does get impacted if the hypervisor and vSAN must be reconfigured over the adjusted base. Recent improvements in cluster technologies have provided easier scalability options via addition of nodes without impacting the memberships of existing nodes in the ensemble. In such a case, the cluster provides the infrastructure while the datacenter becomes more like a cloud service provider. It must be stressed that a vSAN must be allowed to both scale up and scale out otherwise infrastructure management via platforms like Kubernetes are poised to take on more of the infrastructure management routines from the storage technologies. Most businesses investing in vSAN are keen to pursue a “grow-as-you-go” model.

vSAN’s focus is on storage not infrastructure.  It is more disk oriented than servers on which the storage is virtualized. It must be configurable as an all-flash or hybrid storage. In the hybrid mode, it pools HDDs and SSDs to create a distributed shared datastore even though it might internally prefer to use Flash as a read cache/write buffer to boost performance and HDD for data persistence. While flash prices are declining rapidly allowing more possibilities, organizations cannot retire their existing inventory. Hybrid approach is often a necessity even when workloads can be segregated to take advantage of new versus existing. 

No operational model is complete without unified management. IT operations have been dogged by swivel chair operation where the operator must jump from one interface to another to complete all aspects of a workflow. An HCI has the potential to provide a single pane of glass management instead of managing compute, storage and networking individually. Not al HCI provide this and some even come up with a mashup of underlying technology management but  seasoned HCI will  generally have a unified management. When it comes to HCI, system monitoring is critical to aid management. Although custom dashboards and historical trends might be available from metrics providers, a built-in monitoring system for an HCI goes hand in hand with its management portal. End-to-end monitoring and the whole picture of the software and its environment not helps with proactive monitoring but also troubleshooting and reducing costs.

 

This is a continuation of a series of articles on hosting solutions and services on Azure public cloud with the most recent discussion on Multitenancy here. The previous articles introduced multitenancy via hyperconvergence specifically with examples of on-premises technologies. This article continues with a specific example of virtual SAN.

Storage industry and databases have a rich history and tradition of providing multitenancy. As the hardware and software evolved and transformed the storage industry, hyperconvergence and multitenancy also changed. While there are many examples and products that have their own tales to tell as the exhibits in a museum, this article studies virtual SAN and the databases it powered.

Traditional databases particularly the eCommerce catalogs of enterprises required large Storage Area Networking (SAN) because the databases were designed to persist everything to disk from the database objects, prepared plans, materialized views and others. When the disks of a few gigabytes don’t suffice, Storage Area Networking offered the possibility of near limitless storage for that perspective. vSAN took it a notch higher with its ability to virtualize SAN over several devices.

A market leader in vSAN, for instance, pools together server-attached storage (SSDs, HDDs and other flash devices). It creates a shared data store with advanced data services designed for virtual environments. This datastore is highly resilient with no single point of failure. It is optimized for the latest flash technologies. Spread over many VMs with these disks, the virtual SAN can expand to large storage capabilities tolerating failures and providing a single shared datastore.

This vSAN provider integrates the virtual SAN by building it into the kernel of the hypervisor. Hyperconverged solutions use the hypervisor to support and deliver storage functions and storage networking in software eliminating the need for dedicated storage hardware. Since it is embedded in the kernel, it can deliver the highest levels of performance without taxing the CPU with additional overhead. The in-kernel architecture also simplifies management and eliminates risk associated with extra components and points of integration. In this way, it differs from the many virtualized storage appliances that run separately on top of the hypervisor.

Since storage is a key contributor to performance and efficiency, the load passed on by the hypervisor to virtual SAN storage must be dealt with adequately. In this regard, the vSAN is matured over the Flash storage for nearly a decade. The software that used to be implemented on a disk array, moves onto the hosts. A hyperconverged storage is built from the grounds up to integrate and leverage all the functionality of the hypervisor without operational overhead or any reduction of core functionality. The virtualization layer provides features such as high availability and the live migration of running virtual machines from one physical server to another with zero downtime, continuous service availability and complete transaction integrity. This creates a dynamic, automated, and self-optimizing data center.

 

#algorithms

Determine cycles in a graph

Bellman-Ford(G, w, s)
Initialize-single-source(G,s)
for i = 1 to number of vertices -1
    for each edge (u,v) belongs to edges E in G
          relax(u,v,w)
for each edge(u,v) belongs to edges E in G
     if (v.d  > u.d  + w(u,v))
         return False
return True

Friends Pairing problem:

Given n friends, each one can remain single or can be paired up with some other friend. Each friend can be paired only once so ordering is irrelevant

The total number of ways in which the friends can be paired is given by;

Int GetPairs(int n)

{

If (n <=2) return n;

Return GetPairs(n-1) + GetPairs(n-2)*(n-1);

}

This is a continuation of a series of articles on hosting solutions and services on Azure public cloud with the most recent discussion on Multitenancy here. The previous articles introduced multitenancy via hyperconvergence.  This article explores it some more.

Hyperconverged infrastructure helps multitenancy by providing high consolidation density. When there are more pluggable tenants on a single platform, there are some benefits but there also some barriers. There are ways to overcome the barriers and operating problems. One of the way to do that is by leveraging software-defined resources so that the setup and tear-down are configurable, easy and fast. Even patching and upgrade can be rapid by virtue of the consistency across tenants. Finally, multiple instances can be managed as one.  The multitenant architecture thereby provides a single pane of glass for management across resources and their containers.

Till now we have seen several examples of Microsoft technologies-based multitenancy considerations but for hyperconvergence, let us take a look at technologies that specifically cover Infrastructure-as-a-service. The VMWare storage architecture, for example, delivers HCI via one of two options: First, bolting storage software onto a hypervisor, and building storage into the hypervisor. The bolt-on approach runs third-party storage software in virtual machines that sit on top of a hypervisor. This is an easier approach, but it comes with the following limitations: 1. Excessive resource usage, 2. Lower performance and longer latencies and 3. Hybrid and multiple environments with limited integration. The other approach is the built-in were the storage software is in kernel or built directly into the hypervisor. The convergence does not happen on a hypervisor using a virtual appliance but instead happens inside the hypervisor.

The advantages of this approach include 1. reduced resource usage, 2. Better performance and lower latencies and 3. Tight integration enabling end-to-end management from a single tool and a simplified operational model. The advantages of a built-in hyperconverged storage are 1)  there is no need to dedicate certain virtual central processing units or a virtual storage appliance on a per-host basis, 2. CPU resources  which are used only when they are needed and CPUs  don’t need to be reserved for the worst case scenario and 3. CPU cycles go through one stack comprising of just the hypervisor instead of both they hypervisor and the guest operating system.

VMWare does this by providing compute, storage, networking and management on a single integrated layer of software that runs on industry-standard Intel based hardware. This helps to radically simplify the data center. Fully configured x86 servers have VMWare virtual SAN, vSphere and vCenter installed which then provide a single layer on which VMs can be hosted.

Some of the considerations towards the appropriateness of a HCI for business needs include: licensing and support, use of embedded storage or virtual storage appliance, combination and scalability with hybrid resources, native backup and disaster recovery capabilities and integration with cloud computing. 

 

This is a continuation of a series of articles on hosting solutions and services on Azure public cloud with the most recent discussion on Multitenancy here. The previous articles introduced serverless batch processing with durable functions in Azure Container Instances. This article introduces multitenancy via hyperconvergence.  

Hyperconverged infrastructure integrates compute, storage, networking, virtualization and automation into a single system that is delivered via an appliance or software that can be installed on an existing hardware.

The advantages of a hyperconverged over conventional infrastructure include the following:

1.      It increases agility for IT systems to be brought online and scaled out to support dynamic businesses. When new resources are needed, a hyperconverged infrastructure can transition from power on to provisioning readiness within a short time.

2.      It reduces upfront capital and operational costs because there are fewer components that are required in conventional infrastructures. There are no more specialists needed for each discipline as there were with conventional datacenters and which made integration harder. The holistic and integrated approach in hyperconverged saves costs.

3.      This system is prebuilt and workload optimized which reduces complexity. The blocks of architecture can be easily assembled by “snapping” them together and this permits scaling seamlessly.

4.      By providing one platform, it helps to transition applications and resources to the cloud. The deployment of applications to a hyperconverged infrastructure paves the way for deploying to cloud as cloud native applications and reduces a number of concerns from the applications.

5.      This system is performant and resilient and it is quick and affordable way to modernize IT infrastructure with it. This helps businesses operate securely, and efficiently.

Virtualization is a tenet of most hyperconverged systems.  The hypervisor included for virtualization enables multiple operating systems to be hosted and enable nesting. Clusters are deployed to distribute operational functions across nodes and enable scalability with the addition of new nodes.

Hyperconverged infrastructure is sometimes referred to as a datacenter in a box because the initial cabling and minimal networking configuration. A single vendor can now provide servers, storage and the hypervisor making it easier to support, patch, upgrade, and manage the resources. This reduces costs, time to deploy and training requirements for personnel.

Since the storage is directly embedded, many inefficiencies from legacy protocols are avoided. It also improves efficiency and performance. Manageability improves with a native hypervisor.

Some of the considerations towards the appropriateness of a HCI for business needs include: licensing and support, use of embedded storage or virtual storage appliance, combination and scalability with hybrid resources, native backup and disaster recovery capabilities and integration with cloud computing.

 

This is a continuation of a series of articles on hosting solutions and services on Azure public cloud with the most recent discussion on Multitenancy here. The previous articles introduced serverless batch processing with durable functions in Azure Container Instances. This article mentions some of the tenancy considerations in Azure Active Directory.

Azure Active Directory organizes objects like users and applications into groups called tenants. These are security and operational management units from the Azure Active Directory’s point of view. Organizations can set policies on the users pertaining to a tenant and on the applications that the organization owns. Developers can choose to configure applications to be either single-tenant or multi-tenant during the app registration process in the Azure Portal. Single tenants applications are bound to the home tenant.Multi-tenant applications are available to users across tenants. Azure Active Directory uses the audience property to set configure single-tenancy or multi-tenancy.  In the single-tenant mode, all the accounts are in this directory only where the users and guest accounts in the directory can use the application  or API. This is the preferred choice for an audience internal to an organization. In the multi-tenant mode such as for schools and businesses using Microsoft 365, all users and guests can use the application or API even if the accounts are in any Azure AD directory. It is also possible to target an audience that wants to use both work and personal accounts with the multitenant

Setting and this is the preferred choice for the widest possible audience involving Microsoft accounts.

When applications are added to Azure AD, there are two representations: 1. Application Objects – which are the proper definitions of applications, and 2. Service principals which can be considered as representative of applications but there can be multiple representatives for the same application object. Applications can be managed in the Azure Portal through the App registrations experience.

Application objects can be created using application registrations in the Azure portal, creating new applications using Visual Studio and configured to use Azure AD authentication, adding one from the application store, or using Graph API or PowerShell to register or via developer centers. Service principals can be created when users sign into a third-party application integrated with Azure AD or when users sign in to Microsoft online services like Microsoft 365, or by adding an application from the application store, or when an application uses the Azure AD Application Proxy, or if it is configured to be part of Single Sign-On, or programmatically created via the Microsoft Graph API or PowerShell.

 

The best-practices for multi-tenant applications depend on the number of different policies that the IT administrators can set in their tenants. Some practices are applicable generally for multi-tenant applications which include testing the application in a tenant that has conditional access policies configured, following the principle of least user access to ensure that the application only requests permissions it actually needs and providing appropriate names and descriptions for permissions available from the application.

 

Reference: Multitenancy: https://1drv.ms/w/s!Ashlm-Nw-wnWhLMfc6pdJbQZ6XiPWA?e=fBoKcN