Skip to main content

7 posts tagged with "cost-management"

View All Tags

Azure tagging strategy, part 3: cost per data product on Databricks and Fabric

· 12 min read
Shpend Kelmendi
Software Engineer & Architect

The invoice says "Azure Databricks: CHF X". Which product caused it? Azure tags alone can't answer that, because Databricks and Microsoft Fabric each add their own tag system. This is part 3 of 3. It shows which tags reach which billing data, how we get one CHF figure per data product, and where Fabric tags fit. It uses the data-product tag and the Terraform module from part 2.

TL;DR
  • Databricks and Fabric bring a second tag system. Databricks compute tags reach billing data. Fabric tags don't reach Azure billing; they help with discovery and feed your own cost allocation.
  • The concept is the same for both: export the Azure cost data, load it into tables, attribute the cost lines to products, report. Export and ingestion are already described in my Databricks cost control series.
  • Look at your own cost export first. If the cost lines carry data-product, Cost Management answers. If not, you need usage data from the platform: DBUs (Databricks Units) for Databricks, capacity unit consumption for Fabric.
  • Databricks: add VM cost from Cost Management and the Databricks charges split by DBU share, per workspace. The shares always add up to the invoice, so check the assumptions against independent data.
  • Fabric: the capacity is the smallest unit in the Azure bill. Separate capacities per cost center are the simplest split.
  • Report shared cost as its own line. Allocate it only when someone needs full cost per product.

AI Governance: Budget Control with Unity Gateway and Terraform

· 5 min read
Shpend Kelmendi
Software Engineer & Architect

In Part 4 of my Databricks cost control series, we configured Genie budget controls manually because Terraform support was missing. With Databricks Terraform provider v1.128.0 and Unity Gateway, we can revisit that technical debt and apply budget controls to more AI usage paths.

Controlling Databricks Costs on Azure, Part 4: Budget Control & Spend Blocking

· 6 min read
Shpend Kelmendi
Software Engineer & Architect

This is the final part of the series. Part 1 compared the alternatives and explained our combined approach. Part 2 built the cost dashboard, and Part 3 added proactive SQL alerts. One piece is still missing: enforcement. Dashboards show and alerts warn. Neither of them stops anything. That is the job of Budget Control.

In this post, you'll learn:

  • What Budget Control (Public Preview) can and cannot do today
  • How we set up budgets per user and user group, and how we derived role-based limits
  • How spend blocking works (currently Genie only) and how our override process looks
  • How we manage the ClickOps setup as documented technical debt until Terraform support catches up