Saturday, June 26, 2021

 Zone Down failure simulation: 

Introduction:  A public cloud enables geo-redundancy for resources by providing many geographical regions where resources can be allocated. A region comprises several availability zones where resources allocated redundantly across the zones. If the resources fail in one zone, they are switched out with those from another zone. A zone may comprise several datacenters each of which may be stadium-sized centers that provision compute, storage and networking. When the user requests a resource from the public cloud, it usually has 99.99% availability. When the resources are provisioned for zone-redundancy, their availability increases to 99.999%. The verification for resources to failover to an alternate zone when one goes down is key to measuring that improvement in availability. This has been a manual exercise so far. This article attempts to explore the options to automate the testing from a resource perspective. 

Description: 

Though they sound like Availability sets, the AZs comprise datacenters with independent power, cooling, and networking and the availability sets are logical groupings of virtual machines. AZ is a combination of both a fault domain as well as an update domain, so changes do not occur at the same time. Services that support availability zones fall under two categories: zonal services – where a resource is pinned to a specific zone and a zone-redundant service for a platform that replicates automatically across zones. Business continuity is established with a combination of zones and azure region pairs. 

The availability zones can be queried from SDK, and they are mere numbers within a location. For example, az VM list-skus --location eastus2 --output table will list VM SKUs based on region and zones. The zones are identified by numbers such as 1, 2, 3 and these do not mean anything other than that the zones are distinct. The numbers don’t change for the lifetime of the zone, but they don’t have any direct correlation to physical zone representations. 

There are ways in which individual zone-resilient services can allow zone redundancy to be configured. 

When the services allow in-place migration of resources from zonal to zone-redundancy or changing the number of zones for the resource that the service provisions, the simulation of the zone down behavior is as straightforward as asking the service to reconfigure the resource by specifying exactly what zones to have. For example, it could start with [“1”, “2”, “3”] and to simulate a zone down the failure of “3”, it could be reprovisioned with [“1”, “2”] This in-place migration is not expected to cause any downtime for the resource because “1” and “2” continue to remain as part of the configuration. Also, the re-provisioning can be revolving around the zones requiring only the source and target zone pair and since there are three zones, that resource can always be accessed from one zone or the other.  

Conclusion: Zone down can be simulated when there is adequate support from the services that provide the resource. 

Reference the earlier discussion on this topic: https://1drv.ms/w/s!Ashlm-Nw-wnWzhemFZTD0rT35pTS?e=kTGWox


Friday, June 25, 2021

 Learnings from Deployment technologies: 

Introduction: The following article summarizes some learnings from deployment technologies that serve to take an organization’s software redistributable, package it and deploy it so that the software may run in different environments and as separate instances. They evolve from time to time and become narrowed to the environments to which they serve. For example, earlier there was WixSharp to build MS for installing and uninstalling applications or tarball on Linux for the deployment of binaries as an archive. Now, we have a lot more involved technologies that deploy to both on-premises and the cloud. Some of the salient learnings from such technologies are included here and this article continues from the previous discussion here. 

Description: 

An introduction to WixSharp might be in order. It is a language for writing Microsoft installers (MSI) that generates Wix extension files that describe the set of actions to be taken on a target computer for installing, upgrading, or rolling back a software.  The artifacts are compiled with an executable called candle and therefore the artifacts have a rhyming file extension as Wix. WixSharp makes it easy to author the logic for listing all the dependencies of your application for deployment. It converts the C# code to wxs file which in turn is compiled to build the MSI. There are a wide variety of samples in the WixSharp toolkit. Some of them require very few lines to be written for a wide variety of deployment time actions. The appeal in using such libraries is to be able to get the task done sooner with few lines of code. The earlier way of writing and editing the WXS file was error-prone and tedious. This is probably one of the most important learnings. Any language or framework that allows precise and succinct code or declarations is going to be a lot more popular than verbose ones. 

The second learning is about the preference for declarative syntax over logic. It is very tempting to encapsulate all the deployment logic in one place as a procedure. But this tends to become monolithic and takes away all the benefits of out-of-band testing and validation of artifacts. It is also involving developer attention as versions change. On the other hand, the declarative format expands the number of artifacts into self-contained declarations that can be independently validated.  

The third learning is about the reduction of custom logic. Having several and involved custom logic for organizations defeats the purpose of a general-purpose infrastructure that can bring automation, consistency, and framework-based improvements to deployment. Prevention of custom logic also prevents hacks and makes the deployments more mainstream and less vulnerable to conflicts and issues. The use of general-purpose logic will help with enhancements that serve new and multiple teams as opposed to a single customer. 

 

 

Thursday, June 24, 2021

 

This is a continuation of the article originally written here.

This diagram depicts the organization of deployment-as-a-service stack.

Some of the features for the above platform include:

·        best practice in deployment for all tenant's
·        automatic migration for all cloud dependencies
·        Virtualization of deployment technologies and migrations from one set of artifacts to another
·        Asynchronous and background processing with continuous monitoring
·        Programmability options to work with various clients.
·        Analysis and Reporting dashboard
·        Ability to scope down deployments from cloud to on-premises.
·        Curated collection of recipes in automation
·        App-Store integration for allowing clients to opt into deployment stack via published and downloadable applications.
·        Support for browser-based request-response processing
·        Isolated and protected data for all tenants
·        Globally accessible and remote invocable deployment automations
·        Scalable number of client connections allowed for same repository of artifacts.
·        Extensions and customizations for all tenants.
·        Full transparency in the form of logs and events
·        Continuous availability of Deployment stack
·        Ability to provide service-levels for clients.
·        Providing multiple language support including internationalization and globalization
·        Opt-in modernization of existing deployment dependencies
·        One-stop-shop for all deployment activities in the public cloud

Some of the challenges that would be encountered in this regard would include the following:

1) The scope and the environment for a deployment might be different between technologies and mapping must be formed to enable declarations against one form of technology to be repurposed for another.

2) There are many external data stores where configurations may be kept.  File-system or source control is not the only source. Collecting and collocating all the configuration poses a significant task.

3) Artifacts might vary in syntax and semantics and a one-to-one correspondence might not exist between two technologies. This would require some canonicalization, reconciliation and prior translations to be setup.

4) Even if the same technology is used, the artifacts might have different scope and levels of change which may vary widely across deployments. Rolling one deployment to another environment or topology might require additional steps.

5) The recipes must be portable across infrastructure to allow them to vary independently. The more hardcoded literals are used in a script, the tougher it becomes to move or re-purpose the script for another angle.

6) There are several deployments where the environment variable and transient data are used a lot. These must be eliminated in favor of declarative syntax.

Conclusion:  This article tried to pose deployment options for a cloud software maker to be one that is repetitive and requires continued investments over time. In such a case, outsourcing the deployment logic to a deployment-as-a-service offering provides opportunities to save costs and develop best practices.

 

Wednesday, June 23, 2021

 Problem statement: Public cloud computing infrastructure hosts customer workloads as well as the ever-increasing portfolio of services offered by the cloud to its customers. These services from publishers both external and internal to the cloud require deployments over the public cloud. They write and maintain this logic and bear the cost of keeping up with the improvements and features available for this deployment logic. This article investigates the implementation platform for a global multi-tenant deployment-as-a-service offering from the public cloud. 

Solution:  

Multi-tenancy and software-as-a-service model is possible only with a cloud computing infrastructure.  The deployment logic for a service for a cloud differs significantly from that for a desktop. A cloud expects more conformance than a desktop or enterprise deployment justifying the need for a managed program. As Cloud service developers struggle to keep up with the speed of software development for cloud-savvy clients, they face deployment as a necessary evil that draws their effort from their mission. Even when organizations pay the cost up upfront in the first version released with a dedicated staff, they realize that the cloud is evolving at a pace that rivals their own release timeframes. Some may be able to keep up with the investments year after year but for most, this is better outsourced so that they spend less time on rewriting with newer deployment technologies or embracing the enhancements features to the cloud. 

Cloud service developers have an incentive to join this program. They need not be declarative about the resources they use. They just need to define the policies. This is a significant shift in the paradigm that has cost them tremendously in all their deployments so far. Cloud resources are not only scalable and limitless, but their efficient usage is also often neglected by service developers who often use the expedient solution or over-allocate the resources.  A managed program for deployment across these internal and external software makers not only passes on savings to the customers but also helps the cloud migrate quickly to better forms of development with little or no disruption to their customers. 

Deployment logic is quite complicated involving considerations for install, upgrade, rollback, and cleanup of control resources as well as the storage, migration, protection, and replication of data. Fortunately, the bulk of the deployment logic involves cloud resources and a managed program from the cloud is best positioned to onboard the service to the cloud. Concerns addressed and considerations made in the cloud can now be offered in the form of a billable service that will articulate the savings passed on to the customer.  


Tuesday, June 22, 2021

 Recipes and their relevance to automation. 

The world of automation is dependent on commands and sequences that achieve a certain outcome or move to the desired state for a resource. The infrastructure may be elastic, scalable, self-healing, and state-reconciling but it’s the collections of recipes that make one automation system more popular than another. Let us take a closer look at these recipes. Logic and recipes are somewhat different. Logic is defined as flows and represented by flow charts. These are suitable for programming languages. Recipes are more suited for scripting languages and cookie-cutter tasks. An automation engineer will demand consistency, easy identification, troubleshooting, and resolution. She will have little patience for debugging complex multi-branch workflows and focus instead on dividing and conquering a flat list of steps to find the buggy step. The scripts and programs have at least one thing in common. Both are curated over time so they become even more sought after as they get better at not only what they do but also what they can do. The joy of automation is the savings in time and cost when it is repeated without human intervention. Thousands of repetitions can be kicked off in no time and the process will not only assign individual identifiers but also correctly maintain the mapping between the results and their respective set of input variables. One such example is the commissioning of computing systems where the stack involves multiple layers. Automations have interesting by-products in the form of intermediary results, states, and configurations. They execute all the tasks local to a host, unlike services and functions that may even be serverless. Automation can easily be monitored for a predictable start, progress, and stop. A controller leverages this aspect for scheduling their executions which are also called jobs. Automations have certain characteristics just like many other massive software products. They are usually written once and run many times over and on different hosts. They are portable. They embrace shell-based invocations. They become a library of low-level commands and high-level scripts, or recipes, and their organization is determined by their reusability and the requirements of those that use them.  Some hierarchy might be introduced for nomenclature, but the recipes remain a single-level collection even if they have groupings represented by prefixes and suffixes. This collection is often called a cookbook or run book and includes shell-based invocations. The language of automation is PowerShell on windows and Bash on Linux. Remote processing tools such as curl and data transformation to JSON have popularized the scripts and made them even more powerful as they delegate the intensive processing to the right resource. Another aspect of automation that is quite popular is pipelining. The data output from a previous stage becomes the input to the next stage and the operators allow for transformation and analysis between stages of the pipelining process. Automations have a ubiquitous affinity to state and configuration persistence in the form of a relational store, a high-performance message broker such as RabbitMQ, ZeroMQ, or MSMQ, a query, and analysis stack as well as a reporting and visualization stack. Even if their representations vary, automation stands out for their recipes.


Monday, June 21, 2021

This is a continuation of the earlier article that introduced ServiceBus replication:
using System;
using System.Collections.Generic;
using System.Threading.Tasks;
using Microsoft.Extensions.Configuration;
using Microsoft.ServiceBus;
using Azure.Messaging.ServiceBus;
using Microsoft.ServiceBus.Messaging;

namespace SBQueueSender
{
    /*
     * Program to complete structure and content replication of SB Entities from source namespace to destination namespace
     */
    class Program
    {
        private static string connectionString = "";
        private static string secondaryConnectionString = "";
        private static string sourceTopicOrQueueName = "";
        private static string sourceSubscriptionName = "";
        private static string destinationTopicName = "";
        private static string destinationSubscriptionName = "";
        private static string replicateStructure = "";
        private static string replicateContent = "";

        static async Task Main(string[] args)
        {
            IConfigurationBuilder builder = new ConfigurationBuilder().AddJsonFile("appsettings.json");
            IConfigurationRoot config = builder.Build();
            connectionString = config["primaryConnectionString"];
            secondaryConnectionString = config["secondaryConnectionString"];
            sourceTopicOrQueueName = config["sourceTopicName"];
            sourceSubscriptionName = config["sourceSubscriptionName"];
            destinationTopicName = config["destinationTopicName"];
            destinationSubscriptionName = config["destinationSubscriptionName"];
            replicateStructure = config["replicateStructure"];
            replicateContent = config["replicateContent"];

            if (string.IsNullOrWhiteSpace(connectionString) ||
                string.IsNullOrWhiteSpace(secondaryConnectionString) ||
                string.IsNullOrWhiteSpace(sourceTopicOrQueueName) ||
                string.IsNullOrWhiteSpace(sourceSubscriptionName) ||
                string.IsNullOrWhiteSpace(destinationTopicName) ||
                string.IsNullOrWhiteSpace(destinationSubscriptionName) ||
                string.IsNullOrWhiteSpace(replicateStructure) ||
                string.IsNullOrWhiteSpace(replicateContent))
            {
                Console.WriteLine("Please enter appropriate values in appsettings.json file. Exiting...");
                await Task.Delay(2000);
            }
            else
            {
                NamespaceManager namespaceManager = NamespaceManager.CreateFromConnectionString(connectionString);
                if (Boolean.TrueString.CompareTo(replicateStructure) == 0)
                {
                    await ReplicateStructure(namespaceManager, secondaryConnectionString);
                }

                if (Boolean.TrueString.CompareTo(replicateContent) == 0)
                {
                    await ReplicateContent(namespaceManager, secondaryConnectionString);
                }
            }
        }

        /// <summary>
        /// Replicates the structure of the SB namespace.
        /// </summary>
        /// <param name="namespaceManager"></param>
        /// <param name="destinationConnectionString"></param>
        /// <returns>A <see cref="Task"/> representing the result of the asynchronous operation.</returns>
        public static async Task ReplicateStructure(NamespaceManager namespaceManager, string destinationConnectionString)
        {
            var targetNamespaceManager = NamespaceManager.CreateFromConnectionString(destinationConnectionString);
            foreach (var queue in namespaceManager.GetQueues())
            {
                try
                {
                    Console.WriteLine(string.Format("Creating queue {0} ", queue.Path));
                    targetNamespaceManager.CreateQueue(queue);
                }
                catch (MessagingEntityAlreadyExistsException)
                {
                    Console.WriteLine(string.Format("Queue {0} already exists.", queue.Path));
                }
            }

            foreach (var topic in namespaceManager.GetTopics())
            {
                try
                {
                    Console.WriteLine(string.Format("Creating topic {0} ", topic.Path));
                    targetNamespaceManager.CreateTopic(topic);
                }
                catch (MessagingEntityAlreadyExistsException)
                {
                    Console.WriteLine(string.Format("Topic {0}  already exits...", topic.Path));
                }

                foreach (var subscription in namespaceManager.GetSubscriptions(topic.Path))
                {
                    try
                    {
                        Console.WriteLine(string.Format("Creating subscription {0} ", subscription.Name));
                        targetNamespaceManager.CreateSubscription(subscription);
                    }
                    catch (MessagingEntityAlreadyExistsException)
                    {
                        Console.WriteLine(string.Format("Subscription {0}  already exits...", subscription.Name));
                    }
                }
            }
        }

        /// <summary>
        /// Replicates the contents of the SB entities
        /// </summary>
        /// <param name="namespaceManager"></param>
        /// <param name="destinationConnectionString"></param>
        /// <returns>A <see cref="Task"/> representing the result of the asynchronous operation.</returns>
        public static async Task ReplicateContent(NamespaceManager namespaceManager, string destinationConnectionString)
        {
            await using var client = new ServiceBusClient(connectionString);
            foreach (var sbQueue in namespaceManager.GetQueues("messageCount Gt 0"))
            {
                sourceTopicOrQueueName = sbQueue.Path;
                ServiceBusReceiver receiver = client.CreateReceiver(sbQueue.Path);
                ServiceBusReceivedMessage receivedMessage = await receiver.ReceiveMessageAsync(new TimeSpan(0,0,30));
                await SendMessageToTopicAsync(destinationConnectionString, sbQueue.Path, receivedMessage);
            }

            foreach (var sbTopic in namespaceManager.GetTopics("messageCount Gt 0"))
            {
                sourceTopicOrQueueName = sbTopic.Path;
                ServiceBusReceiver receiver = client.CreateReceiver(sbTopic.Path);
                foreach (var sbSub in namespaceManager.GetSubscriptions(sbTopic.Path))
                {
                    sourceSubscriptionName = sbSub.Name;
                    Console.WriteLine("Listening on topic: {0} with subscription: {1}", sourceTopicOrQueueName, sourceSubscriptionName);
                    await ReceiveMessagesFromSubscriptionAsync(namespaceManager, sbTopic.Path, connectionString, sbTopic.Path, sbSub.Name);
                }
            }
        }

        private static async Task ReceiveMessagesFromSubscriptionAsync(NamespaceManager namespaceManager, string sbTopicPath, string connectionString, string sourceTopicName, string sourceSubscriptionName)
        {
            try
            {
                await using (ServiceBusClient client = new ServiceBusClient(connectionString))
                {
                    // create a processor that we can use to process the messages
                    ServiceBusProcessor processor = client.CreateProcessor(sourceTopicName, sourceSubscriptionName, new ServiceBusProcessorOptions()
                    {
                        MaxConcurrentCalls = 1,
                        AutoCompleteMessages = false,
                    });

                    // add handler to process messages
                    processor.ProcessMessageAsync += MessageHandler;

                    // add handler to process any errors
                    processor.ProcessErrorAsync += ErrorHandler;

                    // start processing 
                    await processor.StartProcessingAsync();

                    Console.WriteLine("press any key to stop ...");
                    Console.ReadKey();

                    // stop processing 
                    Console.WriteLine("\nStopping the receiver...");
                    await processor.StopProcessingAsync();
                    Console.WriteLine("Stopped receiving messages");
                }
            }
            catch (Exception e)
            {
                Console.WriteLine("Error: ", e);
            }
        }

        private static async Task MessageHandler(ProcessMessageEventArgs args)
        {
            string id = args.Message.MessageId;
            Console.WriteLine($"Received: {id} from subscription: {sourceSubscriptionName}");

            await SendMessageToTopicAsync(secondaryConnectionString, sourceTopicOrQueueName, args.Message);
            Console.WriteLine($"Sent: {id} from subscription: {sourceSubscriptionName}");

            // complete the message. messages is deleted from the queue.
            await args.CompleteMessageAsync(args.Message);
        }

        private static Task ErrorHandler(ProcessErrorEventArgs args)
        {
            Console.WriteLine(args.Exception.ToString());
            return Task.CompletedTask;
        }

        private static async Task SendMessageToTopicAsync(string connectionString, string destinationTopicName, ServiceBusReceivedMessage serviceBusReceivedMessage)
        {
            if (serviceBusReceivedMessage != null)
            {
                try
                {
                    // create a Service Bus client
                    await using (ServiceBusClient client = new ServiceBusClient(connectionString))
                    {
                        // create a sender for the topic
                        ServiceBusSender sender = client.CreateSender(destinationTopicName);
                        await sender.SendMessageAsync(new ServiceBusMessage(serviceBusReceivedMessage));
                        Console.WriteLine($"Sent a single message to the topic: {destinationTopicName}");
                    }
                }
                catch (Exception e)
                {
                    Console.WriteLine("Error: ", e);
                }
            }
            else
            {
                Console.WriteLine("received null message.");
            }
        }

        private static async Task SendTestMessageToTopicAsync(string body)
        {
            // create a Service Bus client 
            try
            {
                await using (ServiceBusClient client = new ServiceBusClient(connectionString))
                {
                    // create a sender for the topic
                    ServiceBusSender sender = client.CreateSender(sourceTopicOrQueueName);
                    await sender.SendMessageAsync(new ServiceBusMessage(body));
                    Console.WriteLine($"Sent a single message to the topic: {sourceTopicOrQueueName}");
                }
            }
            catch (Exception e)
            {
                Console.WriteLine("Error: ", e);
            }
        }
    }
}

Sunday, June 20, 2021

This is a continuation of the earlier article that introduced ServiceBus replication:
 Reference:
// Sample program for data migration
using System;
using System.Collections.Generic;
using System.Threading.Tasks;
using Microsoft.Extensions.Configuration;
using Microsoft.ServiceBus;
using Azure.Messaging.ServiceBus;
 
namespace SBQueueSender
{
    class Program
    {
        private static string connectionString = "";
        private static string secondaryConnectionString = "";
        private static string sourceTopicOrQueueName = "";
        private static string sourceSubscriptionName = "";
        private static string destinationTopicOrQueueName = "";
        private static string destinationSubscriptionName = "";
 
        static async Task Main(string[] args)
        {
            IConfigurationBuilder builder = new ConfigurationBuilder().AddJsonFile("appsettings.json");
            IConfigurationRoot config = builder.Build();
            connectionString = config["primaryConnectionString"];
            secondaryConnectionString = config["secondaryConnectionString"];
            sourceTopicOrQueueName = config["sourceTopicOrQueueName"];
            sourceSubscriptionName = config["sourceSubscriptionName"];
            destinationTopicOrQueueName = config["destinationTopicOrQueueName"];
            destinationSubscriptionName = config["destinationSubscriptionName"];
 
            if (string.IsNullOrWhiteSpace(connectionString) ||
                string.IsNullOrWhiteSpace(secondaryConnectionString) ||
                string.IsNullOrWhiteSpace(sourceTopicOrQueueName) ||
                string.IsNullOrWhiteSpace(sourceSubscriptionName) ||
                string.IsNullOrWhiteSpace(destinationTopicOrQueueName) ||
                string.IsNullOrWhiteSpace(destinationSubscriptionName))
            {
                Console.WriteLine("Please enter appropriate values in appsettings.json file. Exiting...");
                await Task.Delay(2000);
            }
            else
            {
                NamespaceManager namespaceManager = NamespaceManager.CreateFromConnectionString(connectionString);
                await using var client = new ServiceBusClient(connectionString);
                foreach (var sbQueue in namespaceManager.GetQueues("messageCount Gt 0"))
                {
                    sourceTopicOrQueueName = sbQueue.Path;
                    ServiceBusReceiver receiver = client.CreateReceiver(sbQueue.Path);
                    ServiceBusReceivedMessage receivedMessage = await receiver.ReceiveMessageAsync();
                    await SendMessageToTopicAsync(secondaryConnectionString, sbQueue.Path, receivedMessage);
                }
 
                foreach (var sbTopic in namespaceManager.GetTopics("messageCount Gt 0"))
                {
                    sourceTopicOrQueueName = sbTopic.Path;
                    ServiceBusReceiver receiver = client.CreateReceiver(sbTopic.Path);
                    foreach (var sbSub in namespaceManager.GetSubscriptions(sbTopic.Path))
                    {
                        sourceSubscriptionName = sbSub.Name;
                        Console.WriteLine("Listening on topic: {0} with subscription: {1}", sourceTopicOrQueueName, sourceSubscriptionName);
                        await ReceiveMessagesFromSubscriptionAsync(connectionString, sbTopic.Path, sbSub.Name);
                    }
                }
            }
        }
 
        private static async Task ReceiveMessagesFromSubscriptionAsync(string connectionString, string sourceTopicName, string sourceSubscriptionName)
        {
            await using (ServiceBusClient client = new ServiceBusClient(connectionString))
            {
                // create a processor that we can use to process the messages
                ServiceBusProcessor processor = client.CreateProcessor(sourceTopicName, sourceSubscriptionName, new ServiceBusProcessorOptions());
 
                // add handler to process messages
                processor.ProcessMessageAsync += MessageHandler;
 
                // add handler to process any errors
                processor.ProcessErrorAsync += ErrorHandler;
 
                // start processing 
                await processor.StartProcessingAsync();
 
                while (processor.IsProcessing)
                {
                    System.Threading.Thread.Sleep(30000);
                }
 
                Console.WriteLine("Wait for a minute and then press any key for confirmation to end the processing");
                Console.ReadKey();
 
                // stop processing 
                Console.WriteLine("\nStopping the receiver...");
                await processor.StopProcessingAsync();
                Console.WriteLine("Stopped receiving messages");
            }
        }
 
        private static async Task MessageHandler(ProcessMessageEventArgs args)
        {
            string id = args.Message.MessageId;
            Console.WriteLine($"Received: {id} from subscription: {sourceSubscriptionName}");
 
            await SendMessageToTopicAsync(secondaryConnectionString, sourceTopicOrQueueName, args.Message);
            Console.WriteLine($"Sent: {id} from subscription: {sourceSubscriptionName}");
 
            // complete the message. messages is deleted from the queue. 
            await args.CompleteMessageAsync(args.Message);
        }
 
        private static Task ErrorHandler(ProcessErrorEventArgs args)
        {
            Console.WriteLine(args.Exception.ToString());
            return Task.CompletedTask;
        }
 
        private static async Task SendMessageToTopicAsync(string connectionString, string destinationTopicOrQueueName, ServiceBusReceivedMessage serviceBusReceivedMessage)
        {
            // create a Service Bus client 
            await using (ServiceBusClient client = new ServiceBusClient(connectionString))
            {
                // create a sender for the topic
                ServiceBusSender sender = client.CreateSender(destinationTopicOrQueueName);
                await sender.SendMessageAsync(new ServiceBusMessage(serviceBusReceivedMessage));
                Console.WriteLine($"Sent a single message to the topic: {destinationTopicOrQueueName}");
            }
        }
 
        private static async Task SendTestMessageToTopicAsync(string body)
        {
            // create a Service Bus client 
            await using (ServiceBusClient client = new ServiceBusClient(connectionString))
            {
                // create a sender for the topic
                ServiceBusSender sender = client.CreateSender(sourceTopicOrQueueName);
                await sender.SendMessageAsync(new ServiceBusMessage(body));
                Console.WriteLine($"Sent a single message to the topic: {sourceTopicOrQueueName}");
            }
        }
    }
}