The previous articles on Multitenant Applications talked about organization of data particularly keys and values. This article talks about branding.

Branding is perhaps most noticeable when users sign-in to the multitenant solution.  When the sign-in process can include the company logo and customized experiences based on browser language, it creates a powerful branding for the organization over the multitenant solution that is re-used across the organizations.

The technique behind such flexibility in customization involves an inventory of all user interface properties, such that their customizations can be looked up and applied to compose a unique and holistic branded page to the users. This inventory consists of the text, its culture and locale, references to the images to be used, stylesheets and other assets and where to locate them on the rendered page. Even dark mode or light mode can be set as default and not rely on users’ preferences. The inventory makes it easy to add customizations for new tenants and organize their variations over the defaults.

The tenant provider composes the page by referring to the tenant ID obtained from the domain name and request headers. Tenants might want to bring their own domain names. This might be important for business purposes. It might also be for technical purposes such as they supply their own TLS certificates which bear subject names.

The name resolution to an IP address depends on whether there is a single instance or many instances of the multitenant application. For example, a CNAME for the custom domain of a tenant might have a value pointing to a multi-part subdomain of the multitenant application solution provider. Since this provider might want to set up proper routing to multiple instances, they might have a CNAME record for subdomains of their individual instance to route to that instance. They will also have an A name record for that specific instance to point to the IP address of the provider’s domain name. This chain of records resolves the requests for the custom domain to the IP address of the instance within the multiple instances deployed by the provider.

Host header resolution is also significant. All the web components need to be aware of how to handle the requests that arrive with the provider’s domain name in their host request header. Each tenant’s domain name might be a subdomain or a custom domain and this adds operational overhead to the onboarding of tenants. Host headers can also be rewritten by say the Azure FrontDoor so that the web server receives a single Host header. The example of Azure FrontDoor also propagates the original value of the host header in a X-Forwarded-Host header so the multitenant application can properly resolve the tenant.

Custom branding appears after the users’ sign-in. Some of the prerequisites for custom branding includes licenses for Azure Active Directory Premium 1 and 2, Office 365 for office applications, and Microsoft 365 for keeping the user signed in. When users sign into the organization’s tenant-specific applications, such as https://outlook.com/woodgrove.com, or when passing a domain variable such as https://passwordreset.microsoftonline.com/?whr=woodgrove.com All branding elements are optional. Default settings will remain, if left unchanged. Images have different image and file-size requirements. Language, background image, banner logo, username hint and sign-in page text can all be part of the branding strategy.

If the branding must appear before sign-in, technologies like pagelets can be used so that the server has the chance to discern and render the page without relying on out-of-box default pages.

By retrieving the proper assets from the inventory of customizations, a multitenant solution can compose the user interface page with the proper branding experience for users.

 

The previous articles on Multitenant Applications talked about recovery and replication. This article talks about organization of data particularly keys and values.

Configuration data is stored as key-values. They are a simple and flexible representation of application settings. Keys serve as identifiers to retrieve corresponding values. A best practice involves organizing keys in hierarchical namespaces by using a character delimiter. A convention is not mandated for multiple tenants, but it helps. Keys regardless of their format must be treated as a whole. When parsing is avoided, it is easier to not break any usages. Data used within application frameworks might dictate specific naming schemes for key-values. A combined size limit of 10KB usually applies on a key-value.

Key namespaces must be easier to read with proper use of delimiters to denote hierarchical information. They must also be easier to manage. A key-name hierarchy must represent logical groups of configuration data. They should be easy to query using pattern matching.

Key-values can optionally have a label attribute. Labels are used to differentiate key-values with the same key. No labels are associated initially, and key-values can be referenced without a label.  A common use of labels is to denote environments for the same key. Labels can also be used to create versions. It is possible to jump forward or fall back between keys using versions. Values depend on content-type with Unicode as the most popular form.  MIME types are also applicable for feature flags, Key Vault references, and JSON key-values.

Key-values can be soft-deleted which means that they can be recovered. Soft delete will act as a safeguard to scenarios including the case when a deleted app configuration store could be recovered in the retention time period and the case when an app configuration store is permanently deleted. A soft-deleted store will be retained for a short time known as the retention period. When it elapses, the store is deleted permanently. Key-values can also be purged before the retention period expires. Permission to read and purge deleted stores are granted to owner and contributor roles by default.

Json content-type is preferable over other formats for key-values because it provides simpler data management, enhanced data export, and native support with app configuration provider. When configuration data changes, Event Grid can be used to receive data change notifications. These events can trigger web hooks, Azure functions, Azure storage queues, or any other event handler. Typically, a resource group and a message endpoint are created to subscribe to the topic.

The previous articles on Multitenant Applications talked about recovery and replication. This article talks about organization of data particularly keys and values.

Configuration data is stored as key-values. They are a simple and flexible representation of application settings. Keys serve as identifiers to retrieve corresponding values. A best practice involves organizing keys in hierarchical namespaces by using a character delimiter. A convention is not mandated for multiple tenants, but it helps. Keys regardless of their format must be treated as a whole. When parsing is avoided, it is easier to not break any usages. Data used within application frameworks might dictate specific naming schemes for key-values. A combined size limit of 10KB usually applies on a key-value.

Key namespaces must be easier to read with proper use of delimiters to denote hierarchical information. They must also be easier to manage. A key-name hierarchy must represent logical groups of configuration data. They should be easy to query using pattern matching.

Key-values can optionally have a label attribute. Labels are used to differentiate key-values with the same key. No labels are associated initially, and key-values can be referenced without a label.  A common use of labels is to denote environments for the same key. Labels can also be used to create versions. It is possible to jump forward or fall back between keys using versions. Values depend on content-type with Unicode as the most popular form.  MIME types are also applicable for feature flags, Key Vault references, and JSON key-values.

Key-values can be soft-deleted which means that they can be recovered. Soft delete will act as a safeguard to scenarios including the case when a deleted app configuration store could be recovered in the retention time period and the case when an app configuration store is permanently deleted. A soft-deleted store will be retained for a short time known as the retention period. When it elapses, the store is deleted permanently. Key-values can also be purged before the retention period expires. Permission to read and purge deleted stores are granted to owner and contributor roles by default.

Json content-type is preferable over other formats for key-values because it provides simpler data management, enhanced data export, and native support with app configuration provider. When configuration data changes, Event Grid can be used to receive data change notifications. These events can trigger web hooks, Azure functions, Azure storage queues, or any other event handler. Typically, a resource group and a message endpoint are created to subscribe to the topic.

 

Some Terminologies for geo-replication:

Automatic asynchronous replication – a geo-secondary is created for an existing database on any logical server other than the server where the primary database is located. When it is created, it is populated with the data of the primary database by a process known as seeding. Updates to the database are automatically and asynchronously replicated. Asynchronous replication means that the transactions are committed on the primary database before they are replicated.

Readable geo-secondary replicas – An application can access a geo-secondary replica to execute read-only queries using the same or different security principals. The geo-secondary replicas are chargeable after replication and failover but if they remain read-only geo-secondary to the primary, they are free of charge.

A planned geo-failover switches the roles of the primary and geo-secondary databases after completing full data synchronization. A planned failover does not result in data loss. Since the replication is done based on well-known log shipping techniques, it duration depends on the size of the log at the origin. This kind of failover is applicable to performing disaster recovery drills, relocating the database to a different region, returning the database to the primary region after the outage has been mitigated. 

On the other hand, unplanned geo-failover immediately switches the geo-secondary to the primary role without any synchronization with the primary. Any transactions committed on the primary but not yet replicated to the secondary are lost. Only when the primary is not available, should the geo-failover be unplanned. 

There can be at most four geo-secondaries for a single primary. Multiple geo-secondaries is tactical redundancy. Additional secondaries can also be used to scale out read-only workloads. 

There can be at most four geo-secondaries for a single primary. Multiple geo-secondaries are tactical redundancy. Additional secondaries can also be used to scale out read-only workloads. If there’s only one secondary and it fails, the application is exposed to higher risk until a new secondary is created. 

Each geo-secondary can be a single database or a database in an elastic pool. The elastic pool choice for each geo-secondary database is separate and does not depend on the configuration of any other replica. Each elastic pool is contained within a single logical server. Database names must be unique in a pool so multiple geo secondaries cannot share the same pool. 

 

A geo-secondary that has finished initial seeding can be failed over on demand by the user. If the primary is unavailable, only the unplanned geo-failover can be used. The geo-secondary becomes the new primary. Once the outage is mitigated, the system makes the recovered primary a geo-secondary. Linking of all secondaries to the new primary is automatically done and replication relationships are reconfigured. After the outage that caused the geo-failover is mitigated, it may be desirable to return the primary to its original region.

 

Preparing for a geo-failover involves validating that the authentication and network access for the secondary server are properly configured. The backup retention policy on the secondary database matches that of the primary. This setting is not part of the database, and it is not replicated from the primary. The default configuration of a geo-secondary has a default Point-in-time Recovery (PITR) retention period of 7 days. 

Recovery and Replication of data in Multitenant Applications:   

This is a continuation of the detailed article on recovery options. In next section, we discuss replication options.   

  

The example used here is a full disaster recovery scenario for a multitenant SaaS application implemented with the database per tenant model. The concepts introduced here are geo-restore and geo-replication.   

  

GeoReplication helps to create a continuously synchronized readable secondary database for a primary database. It is preferable although not required to have the secondary database in a different region. Since this kind of secondary database is merely readable, it is called a georeplica. This option serves to perform quick recovery of individual databases in case of a regional disaster or a large-scale outage. Once georeplication is setup, a geo-failover helps maintain continuity of business.   

  

There can be at most four geo-secondaries for a single primary. Multiple geo-secondaries are tactical redundancy. Additional secondaries can also be used to scale out read-only workloads. If there’s only one secondary and it fails, the application is exposed to higher risk until a new secondary is created.  

Each geo-secondary can be a single database or a database in an elastic pool. The elastic pool choice for each geo-secondary database is separate and does not depend on the configuration of any other replica. Each elastic pool is contained within a single logical server. Database names must be unique in a pool so multiple geo secondaries cannot share the same pool.  

  

A geo-secondary that has finished initial seeding can be failed over on demand by the user. If the primary is unavailable, only the unplanned geo-failover can be used. The geo-secondary becomes the new primary. Once the outage is mitigated, the system makes the recovered primary a geo-secondary. Linking of all secondaries to the new primary is automatically done and replication relationships are reconfigured. After the outage that caused the geo-failover is mitigated, it may be desirable to return the primary to its original region.  

Preparing for a geo-failover involves validating that the authentication and network access for the secondary server are properly configured. The backup retention policy on the secondary database matches that of the primary. This setting is not part of the database, and it is not replicated from the primary. The default configuration of a geo-secondary has a default PITR retention period of 7 days.  

 

In an on-premises Availability Group setup, the secondary database is not charged if read queries are not offloaded to it and SA (software assurance) agreement is in place. But secondary databases of geo-replication or failover groups will always be charged. 

 

Azure Hybrid benefits are such that secondary databases of geo-replication or failover groups will still be charged. This is even though secondary databases of geo-replication or failover groups will be in read-only mode. 

 

Failover groups initiate automatic failover after 60 minutes grace period. Forced failover can be initiated anytime if customer initiates it themselves or sets up their own automation. To monitor the replication lag between the primary database and the geo-secondary, there is a dynamic management view available from Azure SQL managed instance called the sys.dm_geo_replication_link_status. 

 

Recovery and Replication of data in Multitenant Applications:

 

This is a continuation of the detailed article on recovery options. In next section, we discuss replication options.

 

The example used here is a full disaster recovery scenario for a multitenant SaaS application implemented with the database per tenant model. The concepts introduced here are geo-restore and geo-replication.

 

GeoReplication helps to create a continuously synchronized readable secondary database for a primary database. It is preferable although not required to have the secondary database in a different region. Since this kind of secondary database is merely readable, it is called a georeplica. This option serves to perform quick recovery of individual databases in case of a regional disaster or a large-scale outage. Once georeplication is setup, a geo-failover helps maintain continuity of business.

 

There can be at most four geo-secondaries for a single primary. Multiple geo-secondaries are tactical redundancy. Additional secondaries can also be used to scale out read-only workloads. If there’s only one secondary and it fails, the application is exposed to higher risk until a new secondary is created.

Each geo-secondary can be a single database or a database in an elastic pool. The elastic pool choice for each geo-secondary database is separate and does not depend on the configuration of any other replica. Each elastic pool is contained within a single logical server. Database names must be unique in a pool so multiple geo secondaries cannot share the same pool.

 

A geo-secondary that has finished initial seeding can be failed over on demand by the user. If the primary is unavailable, only the unplanned geo-failover can be used. The geo-secondary becomes the new primary. Once the outage is mitigated, the system makes the recovered primary a geo-secondary. Linking of all secondaries to the new primary is automatically done and replication relationships are reconfigured. After the outage that caused the geo-failover is mitigated, it may be desirable to return the primary to its original region.

Preparing for a geo-failover involves validating that the authentication and network access for the secondary server are properly configured. The backup retention policy on the secondary database matches that of the primary. This setting is not part of the database, and it is not replicated from the primary. The default configuration of a geo-secondary has a default PITR retention period of 7 days.

 

 

This article discusses using the checklist for architecting and building multitenant solutions. Administrators will find that this list is familiar to them. 

 

The checklist is structured around business and technical considerations as well as the five pillars of the Azure well-architected framework.  These pillars include 1) Reliability, 2) Security, 3) Cost Optimization, 4) Operational Excellence, and 5) Performance efficiency.  The elements that support these pillars are Azure well-architected review, azure advisor, documentation, patterns-support-and-service offers, reference architectures and design principles. Out of these, cost optimization is one of the primary benefits of using the right tool for the right solution. It helps to analyze the spend over time as well as the effects of scale out and scale up. The Azure Advisor can help improve reusability, on-demand scaling, reduced data duplication, among many others. Performance is usually based on external factors and is very close to customer satisfaction. Continuous telemetry and reactiveness are essential to tuned up performance. The shared environment controls for management and monitoring create alerts, dashboards, and notifications specific to the performance of the workload. Performance considerations include storage and compute abstractions, dynamic scaling, partitioning, storage pruning, enhanced drivers, and multilayer cache.

Operational excellence comes with security and reliability. Security and data management must be built right into the system at layers for every application and workload. The data management and analytics scenario focus on establishing a foundation for security. Although workload specific solutions might be required, the foundation for security is built with the Azure landing zones and managed independently from the workload. Confidentiality and integrity of data including privilege management, data privacy and appropriate controls must be ensured. Network isolation and end-to-end encryption must be implemented. SSO, MFA, conditional access and managed service identities are involved to secure authentication. Separation of concerns between azure control plane and data plane as well as RBAC access control must be used.

The checklist for business considerations include 1. understanding what kind of solution is being created such as business-to-business, business-to-consumer, or enterprise software 2. Defining the tenants in terms of number and growth plans, 3. Defining the pricing model and ensuring it aligns with the tenants’ consumption of Azure resources. 4. Understanding whether we need to separate the tenants into different tiers and based on the customer’s requirements, deciding on the tenancy model. Finally, promoting the multitenant solution in the commercial marketplace.

The technical considerations emphasize design and service-level objectives, as well as the scale of the solution. It also suggests applying Chaos engineering to test the reliability of the solution. The security considerations involve Zero Trust and least privilege principles.