Monday, March 2, 2026

 An AI-generated monthly insights reports for Terraform GitHub repository can be realized by building a small automated pipeline, that puts all GitHub issues for the past month, embeds them into vectors, clusters and analyzes them, feeds the structured data into an LLM, produces a leadership friendly markdown report, and publishes it automatically via a Teams message. These are explained in detail below:

1. Data ingestion: A scheduled github action that runs monthly and fetches all issues created or updated in the last 30 days, their comments, labels, module references, and severity or impact indicators. This produces a json dataset like:

[

  {

    "id": 1234,

    "title": "Databricks workspace recreation on VNet change",

    "body": "Changing the VNet CIDR causes full workspace recreation...",

    "labels": ["bug", "module/databricks-workspace"],

    "comments": ["We hit this again last week..."],

    "created_at": "2026-02-01",

    "updated_at": "2026-02-05"

  }

]

2. And use Azure OpenAI embeddings (text-embedding-3-large) to convert each issue into a vector. The store has issue_id, embedding, module (parsed from labels or text), text (title + body + comments) and these can be stored in Pincone or a dedicated Azure AI Search (vector index)

For a simple implementation, pgvector is enough.

3. We can use unsupervised clustering to detect running themes: K-means, HDBScan, and agglomerative clustering. This lets you identify recurring problems, common root causes, hotspots in databricks deployments, and modules with repeated issues.

Sample output:

Cluster 0: Databricks workspace recreation issues (7 issues)

Cluster 1: Private endpoint misconfiguration (4 issues)

Cluster 2: Missing tags / policy violations (5 issues)

Cluster 3: Module version drift (3 issues)

4. This structured data is then fed into an LLM with a prompt like:

You are an expert Terraform and Azure Databricks architect.

Summarize the following issue clusters into a leadership-friendly monthly report.

Include:

- Top recurring problems

- Modules with the most issues

- Common root causes

- Suggested improvements to Terraform modules

- Hotspots in Databricks workspace deployments

- A short executive summary

- A recommended action plan for the next month

Data:

<insert JSON clusters + issue summaries>

And the LLM produced a polished Markdown report.

5. Sample output: for what’s presented to the leadership:

# Monthly Terraform Insights Report — February 2026

## Executive Summary

This month saw 19 issues across 7 Terraform modules. The majority were related to Databricks workspace networking, private endpoints, and tag compliance. Workspace recreation remains the most disruptive pattern.

## Top Recurring Problems

- Databricks workspace recreation due to VNet CIDR changes (7 issues)

- Private endpoint misconfiguration (4 issues)

- Missing required tags (5 issues)

- Module version drift (3 issues)

## Modules with the Most Issues

- module/databricks-workspace (9 issues)

- module/private-endpoints (4 issues)

- module/networking (3 issues)

## Common Root Causes

- Inconsistent module usage patterns

- Lack of lifecycle rules preventing accidental recreation

- Missing validation rules in modules

- Insufficient documentation around networking constraints

## Suggested Improvements

- Add `prevent_destroy` lifecycle blocks to workspace modules

- Introduce schema validation for required tags

- Add automated tests for private endpoint creation

- Publish module usage examples for networking patterns

## Hotspots in Databricks Deployments

- Workspace recreation triggered by minor networking changes

- Cluster policy misalignment with workspace settings

- Missing Unity Catalog configuration in new workspaces

## Action Plan for Next Month

- Refactor workspace module to isolate networking dependencies

- Add tag validation to all modules

- Create a “safe update” guide for Databricks workspaces

- Introduce CI checks for module version drift

6. That’s all!


No comments:

Post a Comment