Skip to main content

Azure Terraform

You'll deploy a Databricks workspace and catalog on Azure using Terraform in ~20 min.

Prereqs: Azure subscription, Terraform CLI, Databricks account console

What you'll walk away with​

A Databricks workspace with VNet injection on Azure plus Unity Catalog resources, all deployed from Terraform. Pick one template from Terraform resources below.

The Workspace + Catalog template creates a new VNet. The Workspace template takes a new or existing VNet. The Private Link template adds backend private endpoints for control plane and DBFS traffic.

Prerequisites​

  • An Azure subscription with Contributor at subscription scope (not resource-group only, because Databricks creates a managed resource group).
  • A Databricks account with account-admin privileges.
  • Terraform CLI installed locally.
  • Azure CLI authenticated (az login).

YouTube walkthrough​

The video walks through the Workspace + Catalog template (azure-vnet-injection-uc). The flow is the same for all templates: copy tf/terraform.tfvars.example, set your variables, then run terraform init, plan, and apply per the repo README.

Terraform resources​

Open the repository for your scenario and follow its README.md. In each repo, run commands from the tf/ directory: copy terraform.tfvars.example to terraform.tfvars, set your values, then run terraform init && terraform apply.

Repeat for development, staging, and production. Use workspace_name, resource_group_name, or your naming prefix to tell environments apart (for example dev, staging, prod).

If you are new to Databricks, start with Workspace + Catalog (first row).

Terraform templateWhat it createsURL
💎 Workspace + Catalog
  • Resource groups (workspace + VNet)
  • VNet, public/private subnets, NSG, NAT Gateway
  • Databricks workspace (VNet injection, No Public IP)
  • Unity Catalog metastore (new or existing)
  • Storage credential, external location, user-defined catalog
  • Single-node UC cluster (Personal Compute policy)
azure-vnet-injection-uc
Workspace
  • VNet (new or existing)
  • Subnets, NSG, NAT Gateway (when creating a new VNet)
  • Databricks workspace (VNet injection)
  • Unity Catalog metastore (optional, new or existing)
azure-vnet-injection
Private Link
  • Resource group (new or existing)
  • VNet, subnets, NSG, NAT Gateway
  • Databricks workspace (VNet injection, public access enabled)
  • Private DNS zones, private endpoints (control plane + DBFS)
  • Network Connectivity Config for serverless Private Link
  • Unity Catalog metastore assignment (optional)
azure-privatelink-classic
warning

When you use an existing metastore with Workspace + Catalog, admin_user must have CREATE EXTERNAL LOCATION on that metastore.

tip

Pin the Databricks provider to Azure CLI auth before apply: export DATABRICKS_AUTH_TYPE=azure-cli

Verify​

  1. Log in to the Databricks account console.
  2. Open Workspaces and confirm the new workspace shows Running.
  3. Open the workspace, go to Catalog, and confirm the new catalog appears (for the Workspace + Catalog template).

Where people trip​

PERMISSION_DENIED: User is not an owner of Metastore while creating catalog

The identity running Terraform lacks metastore-level permissions. Fix it one of two ways:

  • Option 1: Add the user or service principal to the metastore admins group.
  • Option 2: Grant catalog creation on the metastore:
GRANT CREATE CATALOG ON METASTORE TO `principal_name`;
Terraform apply fails with permission errors

Confirm az login (or your service principal) has Contributor access on the target subscription. Run az account show to check the active subscription.

VNet injection fails with subnet errors

Subnets must be delegated to Microsoft.Databricks/workspaces and must not overlap existing address spaces. Make sure subnet_public_cidr and subnet_private_cidr fit inside the VNet cidr. vnet_resource_group_name must differ from resource_group_name.

Next​