Azure Synapse Link for Cosmos DB ...

Introduction: This article is a continuation of the series of articles starting with the description of SignalR service which was followed by a discussion on Azure Gateway service, Azure Private Link, and Azure Private Endpoint and the benefit of diverting traffic to the Azure Backbone network. Then we started reviewing a more public internet-facing service such as the Bing API. and the benefits it provided when used together with Azure Cognitive Services. We then discussed infrastructure API such as Provider API, ARM resources, and Azure Pipeline and followed it up with a brief overview of the Azure services support for Kubernetes Control Plane via the OSBA and Azure operator. Then we followed it with an example of Azure integration service for Host Integration Server (HIS). We started discussing the Azure FHIR service next. We were reviewing its search capabilities, followed by its regulatory compliance and security policies. In this article, we discuss Azure Synapse link for Cosmos DB.

Description: 

Analytics use case for any storage is usually understated but heavily used because the data, no matter its size, is useful only if there is usage. The Azure Synapse Link for Azure Cosmos DB creates a tight seamless integration between Azure Cosmos DB and Azure Synapse Analytics which is the de facto standard for enterprise analytics service. Azure Synapse quickens insights across data warehouses and big data systems. It brings together the best of SQL technologies used in data warehousing, Spark technologies used for Big Data, Pipelines for data integration and ETL/ELT for transformations and deep integration with Power BI, Azure ML and CosmosDB. What makes Synapse popular is its support for SQL queries which is offered both in serverless and dedicated resource models with predictable performance and costs. It has built-in streaming capabilities to write cloud data to tables and it also helps to query data in stores like Azure Data Lake Storage and Azure CosmosDB without having to run import tasks. Its integration with Apache Spark eliminates the need to manage clusters and has fast startup and aggressive autoscaling.

Azure Synapse Link for Azure Cosmos DB is a cloud-native hybrid transactional and analytical processing capability that supports near real-time analytics over operational data in Azure Cosmos DB. It achieves this without impacting the performance of online transactional processing using CosmosDB. This is different from the link between the Azure Cosmos DB’s internal transactional and analytical stores which is set-up out-of-box for auto-sync purposes.  The cloud-native HTAP connects the whole of CosmosDB with the Spark/SQL supportive Azure Synapse Analytics. The benefits are huge when it comes to eliminating the import/export or ETL operations necessary to take data out of multiple operational data sources and eliminating the use of traditional warehouses completely for a faster, streamlined and highly scaleable analytics experience.  It is, however, not recommended for cases where a traditional data warehouse requirements such as high concurrency, workload management and persistence of aggregates is required across multiple data sources.