Skip to main content

Azure tagging strategy, part 3: cost per data product on Databricks and Fabric

· 12 min read
Shpend Kelmendi
Software Engineer & Architect

The invoice says "Azure Databricks: CHF X". Which product caused it? Azure tags alone can't answer that, because Databricks and Microsoft Fabric each add their own tag system. This is part 3 of 3. It shows which tags reach which billing data, how we get one CHF figure per data product, and where Fabric tags fit. It uses the data-product tag and the Terraform module from part 2.

TL;DR
  • Databricks and Fabric bring a second tag system. Databricks compute tags reach billing data. Fabric tags don't reach Azure billing; they help with discovery and feed your own cost allocation.
  • The concept is the same for both: export the Azure cost data, load it into tables, attribute the cost lines to products, report. Export and ingestion are already described in my Databricks cost control series.
  • Look at your own cost export first. If the cost lines carry data-product, Cost Management answers. If not, you need usage data from the platform: DBUs (Databricks Units) for Databricks, capacity unit consumption for Fabric.
  • Databricks: add VM cost from Cost Management and the Databricks charges split by DBU share, per workspace. The shares always add up to the invoice, so check the assumptions against independent data.
  • Fabric: the capacity is the smallest unit in the Azure bill. Separate capacities per cost center are the simplest split.
  • Report shared cost as its own line. Allocate it only when someone needs full cost per product.
Series: Azure tagging strategy for a data platform

The common concept: cost per product from an export​

Azure bills resources, not data products. A cost line only knows its resource, so a product is only visible if a tag on that resource says so. For Databricks and Fabric the answer has the same four steps, and most of it is already described in my Databricks cost control series.

  1. Export. Cost Management writes the cost data of a scope, for example a subscription, to a storage account on a schedule. Part 2 of the Databricks cost control series sets this up with Terraform and explains which export type to choose. Databricks and Fabric capacity lines end up in the same dataset.
  2. Ingestion. Load the files into tables. The same post does this with Auto Loader into Unity Catalog. Tags arrive as JSON in the Tags column.
  3. Attribution. Open your export, filter on the meters of the platform and check which lines carry data-product. Documentation tells you what should happen. Only your own data shows what does.
    • Lines carry the tag: the export already gives invoiced CHF per product. There is nothing to allocate.
    • Lines don't carry it: you need usage data from the platform and a share per product. The platform sections below show which data that is.
    • Everything without a product is shared cost.
  4. Report. Where to show the result depends on who reads it.

Shared costs​

Not every cost belongs to one product. I sort shared cost into three groups:

GroupExamplesHow to handle it
Belongs to one productStorage accounts, Key Vaults or Event Hubs deployed per productTag with data-product through the module. Nothing to allocate.
Shared, but measurable per productJob compute in a shared Databricks workspace, workspaces on a shared Fabric capacityPlatform usage data, see the sections below.
Shared platformWorkspace networking, metastore storage, monitoring, platform jobsReport as its own shared line.

Start with the separate line. This is called showback: you show the cost without charging it. The FinOps Framework notes that managing shared costs can be complex and that many organizations start without allocating them.

If management wants the full cost per product, Cost Management offers cost allocation rules. A rule moves the cost of a source (subscription, resource group or tag) to targets. You split it evenly, by percentage, or in proportion to the cost of the targets. Three limits apply:

  • Rules work on cost, not on usage. They can't use usage data such as a DBU share or capacity unit consumption.
  • Allocated amounts no longer match the invoice. Filter the rules out when you reconcile.
  • A rule can take up to 24 hours to apply.

Check that your agreement type supports cost allocation rules before you plan around them.

Where to show the result​

The AI/BI dashboard from part 2 of the cost control series already shows Azure cost next to the Databricks system tables. Its gold layer groups infrastructure cost by resource group and meter category. To show cost per product, add the data-product tag from the Tags column as a column.

The right place for the result depends on who reads it:

  • Product teams and the platform team: that dashboard for Databricks, because VM and DBU cost sit in one dataset. For Fabric, see the ready-made option in the Fabric section.
  • Finance and management: Power BI. Microsoft recommends exporting cost data to a storage account and reporting from there. The FinOps toolkit ships Power BI reports for it.
  • Budgets and alerts: Quick Azure Cost Management: Budget and Notifications shows how to create Azure budgets with Bicep, the Azure CLI and the portal. A budget per product needs a filter on the data-product tag, so check the filter options for your scope. For unusual Databricks spend, see SQL alerts in part 3 and budget control and spend blocking in part 4 of the series.

Where Databricks and Fabric differ​

Both platforms have two tag worlds: Azure resource tags on the control plane and platform-native tags. Mixing them up is the most common source of "why don't my tags show up in the cost report?". The cost side is where the platforms differ:

DatabricksFabric
Smallest cost unit in the Azure billVMs in the managed resource group (classic compute) and the Databricks meters per workspaceThe capacity
Usage data below that unitsystem.billing.usage: DBUs with custom_tagsCapacity Metrics app: capacity unit seconds per workspace and item
Typical way to split costDBU share per workspace, with job and cluster tagsOne capacity per cost center, or your own allocation based on capacity unit consumption

Databricks​

  • Workspace tags are combined with cluster or pool tags and the default tags, then applied to the VMs in the managed resource group. Your workspace's ITSM tags show up again on every cluster VM, so in a shared workspace they name the platform, never a product.
  • x_ prefix on collision: a custom tag with the same key as a default tag is propagated as x_<key>. So avoid custom keys like Vendor, Creator, ClusterName, JobId, RunName.
  • The 50-tag limit applies to the VM. If the aggregated tags exceed it, Databricks ignores x_ tags first, then others, in alphabetical order. A generous workspace tag set leaves less room for cluster tags.
  • Pools are special: clusters created from a pool only get workspace + pool tags on their VMs. A job's data-product tag doesn't reach the VMs of a pooled cluster, so VM cost of a pool shared by several products can't be split by tag. The DBU usage records still carry the cluster tags. Use one pool per product, or job compute without a pool, for workloads whose VM cost you want to attribute.
  • Other limits: stricter allowed characters, and workspace tag changes take up to an hour and reach existing compute only after a restart. See Use tags to attribute and track usage.
warning

Conflicting tags in compute policies don't get the x_ treatment. They make cluster creation fail with an invalid settings error.

Our recommendation for a shared workspace with many products:

LevelTag typePurpose
WorkspaceAzure tags from the central module with shared = true (itsm-*, env, managed-by, no data-product)ITSM routing to the platform team; flows to all VMs
Job / clusterDatabricks custom tags: data-product (from the module output), layer (bronze, silver, gold)Cost per data product and layer, same key and value as on Azure resources
Compute policyFixed custom tagsNobody can launch untagged classic compute
ServerlessServerless usage policy per product, with the same data-product tagCost per data product for serverless usage. The tags reach system.billing.usage; check whether they reach your Cost Management export

Job tags propagate to the job cluster. The value comes from the tag module, not from a hand-typed variable:

databricks-job.tf
resource "databricks_job" "silver" {
name = "sales-forecast-silver-${var.identifier}"
# ...
tags = {
"data-product" = module.tags.data_product
layer = "silver"
}
}

From DBUs to CHF​

How DBUs and prices add up to a Databricks charge is explained in Controlling Databricks Costs on Azure. The bill has two parts, and each part can be visible in a different place:

Part of the billWhere the money isWhere the product is
Infrastructure of classic compute (VMs in the managed resource group)Cost Management, in CHFdata-product on the VM tags
Databricks charges (DBUs, classic and serverless)Cost Management, in CHF, on the Databricks meterssystem.billing.usage, as DBUs; possibly also on the Cost Management lines (see below)

For Databricks, the documentation says tags propagate to cost analysis reports, but it names the VMs. Whether the DBU lines carry the tag in your tenant is the question from step 3 above. Tags arrive in the export's Tags column.

Serverless: check it separately. It runs without VMs in your managed resource group. Its tags always reach system.billing.usage, but they may not reach the export, so the share method is the fallback.

For the parts the export can't answer, we add the two parts per product, per workspace and per month. With DEV, INT and PRD that means three separate allocations:

  1. Infrastructure: in Cost Management, filter on the workspace's managed resource group and group by data-product. That's invoiced CHF per product.
  2. Databricks charges: take the invoiced CHF on the Databricks meters of that workspace resource for the same month. Multiply it by each product's share for the same workspace_id. The numeric workspace ID is the adb-<id> part of the workspace URL, so each Azure resource maps to exactly one ID.
  3. Sum both parts per workspace, then across workspaces if you want one number per product. What stays untagged (shared) is platform cost. Report it as its own line; don't spread it silently.

Using a share instead of list_usd takes care of currency and negotiated discounts, if two assumptions hold. The shares always add up to 1, so the total proves nothing. Check the assumptions against independent data:

  • Same usage, same period. Compare SUM(dbus) per workspace and month with the DBU quantity on that workspace's Databricks meters in the export. A gap of more than a few percent means the periods don't line up (for example, day boundaries in different time zones), or usage is missing on one side.
  • Uniform discount. Run the query grouped by sku_name. Divide each SKU group's invoiced CHF by its list_usd. If the rates differ by more than a few percent, do step 2 per SKU group.

Neither check tells you whether the tags are right. Watch the shared line: if it grows from month to month, untagged compute grows with it.

Governed tags and domains​

Unity Catalog has a second tag system for governance. Governed tags come with a tag policy that enforces how a tag can be used. They apply to Unity Catalog objects such as tables and catalogs, and to workspace objects such as dashboards and notebooks. Domains group data products and assets by governed tags, at account level.

This is the Databricks counterpart of Fabric tags and domains: discovery and governance. The documentation I read describes them for classification and access control and doesn't mention cost data. The compute tags above are the ones that reach cost data. I haven't tested whether governed tags show up in system.billing.usage or in the Cost Management export. The documentation names preview requirements for domains, so check their status before you build on them.

Fabric​

Azure tags on the capacity​

A Fabric capacity (Microsoft.Fabric/capacities) is an ARM resource, and its tags reach Cost Management like any other. But the capacity is the smallest unit of cost Azure knows about: all workspaces on it share one invoice line. If finance needs a clean split, separate capacities per cost center are the simplest answer.

Fabric tags in the tenant​

Status

Fabric tags launched in public preview. The current overview (last updated March 2026) no longer carries a preview label. Limits and APIs in this area still change, so check the linked pages before you build on them.

Fabric admins define tenant-level tags, Fabric or domain admins domain-level tags (max. 10,000 per tenant, names up to 40 characters). Workspace admins apply them to workspaces, users with Write or Contributor permissions to items. They power search, OneLake catalog filters and lineage views. The scanner API returns tag IDs per item and workspace.

Microsoft calls workspace tags useful for cost attribution, and they are, but as an input. They label which workspace belongs to which product. They don't split the bill.

Not in Azure Cost Management

Fabric tags don't reach Azure billing data. To get cost by Fabric tag, read the tags through the scanner API and use them in the attribution step.

Splitting cost below the capacity​

If one capacity serves several products, you need usage data from inside Fabric. The Capacity Metrics app shows consumption in capacity unit seconds (CU(s)) per workspace and per item. You must be a capacity admin to install and view it. The compute page covers about the last two weeks. The item history page covers 30 days and was in preview when I read about it. For a monthly allocation, you have to store the data yourself.

An allocation would work like the DBU share: map each workspace to a product (workspace tags read through the scanner API are one way), calculate each product's share of CU(s) within a capacity, and apply it to the capacity's invoiced CHF. I haven't built or tested this. Decide first how to treat unused capacity. It is cost too, and a pure CU share spreads it over all products.

Ready-made option: Fabric Cost Analysis​

Fabric Cost Analysis (FCA) is a solution accelerator in the fabric-toolbox repository, and the Fabric counterpart of the dashboard from part 2. It is built in Fabric. Pipelines and Notebooks extract Azure cost data in FOCUS format into a Lakehouse, and a Direct Lake semantic model feeds Power BI reports. It shows Fabric costs only, for example per capacity, per category and resource, and for reservations and quotas.

Two things to know. The README states that it is not an official Microsoft product and comes without official support. It also doesn't mention tags or an allocation per workspace, so a split per data product would be your own extension.

Comparison table​

AspectAzure tags (generic)Databricks custom tagsDatabricks governed tagsFabric capacity (Azure tags)Fabric tags
Formatkey = valuekey = valuekey with allowed valueskey = valuelabel only
Defined byAnyone with write rightsWorkspace users and admins, policiesTag policyAnyone with write rights on capacityFabric or domain admins
Applied toAzure resourcesWorkspace, cluster, job, pool, SQL warehouseUnity Catalog and workspace objectsCapacityItems, workspaces
Max per object50Shares the 50 on the VMNot checked5010 (item and workspace counted separately)
Visible in Azure Cost ManagementYesYes, via VM tags (classic compute)Not documentedYesNo
Visible in platform billingn/asystem.billing.usage (DBUs; list price in USD via list_prices)Not documentedCapacity Metrics app (no tags)Scanner API only
EnforcementAzure Policy, IaCCompute policiesTag policiesAzure Policy, IaCAdmin-curated list
Main purposeCost, ownership, ITSMCost per workload or productDiscovery (domains), classification, access controlCost per capacityDiscovery, governance; input for own cost allocation

Conclusion​

Start from the question, not the tag. The next time someone proposes a new tag, ask them which report, policy or process will read it. If they can't name one, the answer is no. If they can, that reader already tells you the key, the allowed values and where to enforce it.

Further Readings​