An AI-generated monthly insights reports for Terraform GitHub repository can be realized by building a small automated pipeline, that puts all GitHub issues for the past month, embeds them into vectors, clusters and analyzes them, feeds the structured data into an LLM, produces a leadership friendly markdown report, and publishes it automatically via a Teams message. These are explained in detail below:
1. Data ingestion: A scheduled github action that runs monthly and fetches all issues created or updated in the last 30 days, their comments, labels, module references, and severity or impact indicators. This produces a json dataset like:
[
{
"id": 1234,
"title": "Databricks workspace recreation on VNet change",
"body": "Changing the VNet CIDR causes full workspace recreation...",
"labels": ["bug", "module/databricks-workspace"],
"comments": ["We hit this again last week..."],
"created_at": "2026-02-01",
"updated_at": "2026-02-05"
}
]
2. And use Azure OpenAI embeddings (text-embedding-3-large) to convert each issue into a vector. The store has issue_id, embedding, module (parsed from labels or text), text (title + body + comments) and these can be stored in Pincone or a dedicated Azure AI Search (vector index)
For a simple implementation, pgvector is enough.
3. We can use unsupervised clustering to detect running themes: K-means, HDBScan, and agglomerative clustering. This lets you identify recurring problems, common root causes, hotspots in databricks deployments, and modules with repeated issues.
Sample output:
Cluster 0: Databricks workspace recreation issues (7 issues)
Cluster 1: Private endpoint misconfiguration (4 issues)
Cluster 2: Missing tags / policy violations (5 issues)
Cluster 3: Module version drift (3 issues)
4. This structured data is then fed into an LLM with a prompt like:
You are an expert Terraform and Azure Databricks architect.
Summarize the following issue clusters into a leadership-friendly monthly report.
Include:
- Top recurring problems
- Modules with the most issues
- Common root causes
- Suggested improvements to Terraform modules
- Hotspots in Databricks workspace deployments
- A short executive summary
- A recommended action plan for the next month
Data:
<insert JSON clusters + issue summaries>
And the LLM produced a polished Markdown report.
5. Sample output: for what’s presented to the leadership:
# Monthly Terraform Insights Report — February 2026
## Executive Summary
This month saw 19 issues across 7 Terraform modules. The majority were related to Databricks workspace networking, private endpoints, and tag compliance. Workspace recreation remains the most disruptive pattern.
## Top Recurring Problems
- Databricks workspace recreation due to VNet CIDR changes (7 issues)
- Private endpoint misconfiguration (4 issues)
- Missing required tags (5 issues)
- Module version drift (3 issues)
## Modules with the Most Issues
- module/databricks-workspace (9 issues)
- module/private-endpoints (4 issues)
- module/networking (3 issues)
## Common Root Causes
- Inconsistent module usage patterns
- Lack of lifecycle rules preventing accidental recreation
- Missing validation rules in modules
- Insufficient documentation around networking constraints
## Suggested Improvements
- Add `prevent_destroy` lifecycle blocks to workspace modules
- Introduce schema validation for required tags
- Add automated tests for private endpoint creation
- Publish module usage examples for networking patterns
## Hotspots in Databricks Deployments
- Workspace recreation triggered by minor networking changes
- Cluster policy misalignment with workspace settings
- Missing Unity Catalog configuration in new workspaces
## Action Plan for Next Month
- Refactor workspace module to isolate networking dependencies
- Add tag validation to all modules
- Create a “safe update” guide for Databricks workspaces
- Introduce CI checks for module version drift
6. That’s all!
No comments:
Post a Comment