Tuesday, May 23, 2023

Creating and curating Kusto databases for integration with ServiceNow incidents tables:

 Azure as a public cloud aggregates functionalities and traffic from distributed on-premises and private datacenters with the benefits of managed resources, their lifecycle management, elasticity, scalability, and efficient operations management. As it attracts on-premises applications and data, owning teams are overwhelmed by many organizational units such as tenants, subscriptions, resource groups, and resources such as dashboards, alert groups, identities, roles, and permissions. An inventory of these resources usually finds their place in the Azure Data Explorer in the form of one or more Kusto databases. Together with Microsoft Graph and the Kusto Query language as the universal language of querying across resource and data explorers, the cloud engineers gravitate to these query editors for getting meaningful virtualized information across diverse datasets. One such dataset that continues to remain external is the incidents table from ITSM products.

Azure resources are as important to IT service management as any other on-premises resources and enterprise applications. With one example of an ITSM product as ServiceNow that provides robust ITSM capabilities, there are often talks about integration points between the two. ServiceNow enables its users to automate incident and issue management. Additionally, it provides users with access to real-life performance analysis and change management. The integration points between the two products comprise connectors for log analytics and azure monitor, linking action groups and incident creation or update, webhooks and azure AD authentication or runbooks and scripts-based automations, azure devops and service management ticketing.

Many of the integration points establish connections from the cloud to the ITSM such that the intelligent queries written in KQL in the cloud serve to trigger actions for service management. The reverse direction of propagating incidents information and history to databases on Kusto clusters is considered upstream but occur especially when the deprecation of ITSM in favor of the native cloud tools and practices is more than technological and cultural shifts.

Azure Data Factory facilitates data transfers between diverse source and destination. The CopyActivity and LookupActivity in the data factory are sufficient to copy with incremental progress and at scale. The rich information captured by the ITSM brings to Azure unparalleled opportunity for analyzing resources and pain points along with a deeper understanding of the trends, predictions, and quality of service for customer usages.

Kusto is popular both with Azure monitor as well as Azure data explorer. It is a read only request to process data and return results in plain text. If uses a data flow model that is remarkably like the slice and dice operators in the shell commands.IT can work with structured data with the help of tables, rows, and columns but it is not restricted to schema-based entities. It can be applied to unstructured data such as telemetry data. It consists of a sequence of statements delimited by semicolon operator and has at least one tabular query operator. The name of a table is sufficient to stream the rows to a pipeline operator that separates the filtering into its own stage with the help of a SQL like where clause. Sequences of where clauses can be chained to result in a more refined set of resulting rows. It can be as short as a tabular query operator, a data source, and a transformation. Any use of new tables, rows and columns requires the use of control commands that are differentiated from Kusto queries because they begin with a dot character. The separation of these control commands helps with security of the overall data analysis routines. Administrators will have less hesitation for Kusto queries to run on their data. Control commands also help to manage entities or discover their metadata. A sample control command is a “.show” command that shows all the tables in the current database.

This article explores the integrations between the public cloud and ITSM with the benefits of bringing the best of both worlds.

 

 

 

No comments:

Post a Comment