GCP Hybrid

You'll deploy a hybrid Databricks workspace with BYOVPC on GCP using Terraform in ~20 min.

Prereqs: GCP project, Terraform CLI, Databricks account console

The big picture

In this section, you will find materials to help you deploy a Hybrid workspace on GCP. A hybrid workspace can use serverless compute, which Databricks manages for you, and it also lets you run compute resources (classic clusters) inside your own network: a customer-managed VPC in your Google Cloud project. This gives your team more control over networking and security, and covers workloads that serverless doesn't support yet. You can set up Unity Catalog resources alongside the workspace, so your data is ready to govern from day one.

This page covers three ways to deploy a hybrid workspace:

• Manual Deployment: click through the cloud and Databricks account consoles by following the YouTube tutorial.
• Automated Deployment
  • Platform Deployment Kit: a guided web UI that exports ready-to-run Terraform.
  • AI Platform Kit: agent skills your AI coding agent uses to generate and apply the Terraform.

Recommended for beginners

If you are new to Databricks, start with the Platform Deployment Kit. It walks you through each decision (networking, storage, Unity Catalog) in a guided UI, so you don't need to know Terraform or how to prompt a coding agent up front. The output is deterministic: the same inputs always produce the same Terraform, which you can review before you apply and rerun for development, staging, and production so every workspace matches. Use the AI Platform Kit when you need customizations the UI doesn't cover, and keep manual deployment for learning how the pieces fit together.

Manual Deployment - YouTube Tutorial

warning

The video shows the manual UI steps. Clicking through the console is easy to miss a setting on, and the next workspace will drift from this one. Use the Automated Deployment options on this page instead so every workspace matches.

Prerequisites

• A GCP project with a Google service account that has roles/Owner on the project (or the tighter roles in the template README).
• A Databricks account with account-admin privileges (GCP Marketplace subscription).
• Terraform CLI v1.3+ installed locally.
• Google Cloud SDK authenticated. The templates use GSA impersonation and GOOGLE_OAUTH_ACCESS_TOKEN (see each repo README and service-account-impersonation.md).
• The GSA added to the Databricks account console as an account user with the account-admin role.

Automated Deployment

The Platform Kit is a suite of tools designed to help you deploy your workspaces faster. It features a deterministic app, for teams who want to define their deployment through a guided UI experience, and a set of LLM skills, for teams who want to use an AI coding agent to help with the deployment.

Platform Deployment Kit

The Platform Deployment Kit is a guided web UI that helps you define your environment and automatically exports the Terraform code to run your deployment. Use it when you want a deterministic, click-through path instead of prompting an agent.

Access it here: https://databricks-solutions.github.io/platform-deployment-kit/#/

AI Platform Kit

The AI Platform Kit is a set of agent skills that your AI coding agent uses to define your workspace settings, create customizations, and deploy with Terraform.

Paste this into your coding agent to provision the hybrid GCP workspace with AI Platform Kit.

Prompt

Install and use the Databricks AI Platform Kit (https://github.com/databricks-solutions/ai-platform-kit) if it is not already installed.

macOS/Linux: bash <(curl -sL https://raw.githubusercontent.com/databricks-solutions/ai-platform-kit/main/install.sh) Windows PowerShell: irm https://raw.githubusercontent.com/databricks-solutions/ai-platform-kit/main/install.ps1 | iex

Then follow the kit skills (platform-provisioning, unity-catalog-setup, identity-governance, workspace-config, deployment-verification as needed) to provision a classic Databricks workspace on GCP with a customer-managed VPC (BYOVPC / Standalone new VPC) and Unity Catalog or metastore assignment as supported.

Requirements for this deploy:

• Cloud: GCP
• Architecture: classic workspace with customer-managed VPC (Standalone / new VPC preferred unless I say otherwise)
• Include Unity Catalog or metastore assignment when my account supports it
• Environments and naming: ask me for project ID, region, workspace name, network settings, and any compliance constraints before generating Terraform

Workflow:

1. Confirm cloud and Databricks auth (gcloud auth, service account / impersonation as needed, Databricks CLI / account admin access). Enable compute.googleapis.com if it is not already enabled.
2. Ask me for any missing value. Do not guess project IDs, regions, CIDRs, or account IDs.
3. Generate Terraform from the kit skills for my requirements. Do not clone a fixed technical-services-solutions template as the source of truth.
4. Run terraform init and terraform plan. Explain the plan (VPC/subnet, router/NAT, MWS network and workspace, UC/metastore pieces) before any apply.
5. Run terraform apply only after I explicitly approve the plan. Never apply or destroy without approval.
6. After workspace and UC are up, run the kit's mandatory three-path verification: classic cluster, serverless SQL warehouse, and serverless notebook job.

If something fails, diagnose with the kit gotchas guidance and re-plan before retrying.

Terraform resources

Open the repository for your scenario and follow its README.md. Run Terraform from the repository root (not a tf/ subdirectory): copy terraform.tfvars.example to terraform.tfvars, set your values, then run terraform init && terraform apply.

Repeat for development, staging, and production. Use databricks_workspace_name (and separate GCP projects or subnets) to tell environments apart.

If you are new to Databricks and do not already have a VPC, start with Standalone (first row).

Terraform template	What it creates	URL
💎 Standalone (new VPC)	VPC, subnet (private_ip_google_access) Cloud Router and Cloud NAT Databricks MWS network and workspace Workspace admin user in admins group GCS buckets for workspace storage (Databricks-managed)	gcp-byovpc-standalone
Shared VPC (existing network)	Databricks MWS network referencing existing VPC/subnet Databricks workspace Workspace admin user in admins group Optional Unity Catalog metastore assignment (metastore_id)	gcp-byovpc-shared-vpc

warning

For Shared VPC when the VPC lives in a host project, grant the GSA roles/compute.networkUser and roles/compute.networkViewer on the host project. Enable Shared VPC and attach the service project per the template README.

tip

Enable the Compute API before apply: gcloud services enable compute.googleapis.com --project <PROJECT_ID>. For least-privilege auth, see service-account-impersonation.md in each template repo.

Validate your Deployment

Validate · 6 steps

1

Log in to the Databricks account console.

2

Open Workspaces and confirm the new workspace shows Running.

3

Run terraform output -raw workspace_url and confirm you reach the workspace landing page.

4

In the GCP console, confirm the VPC/subnet (Standalone) or that the existing network is referenced (Shared VPC).

5

If you set metastore_id, open Catalog in the workspace and confirm the metastore is attached.

6

Create a simple general purpose single node cluster and confirm it starts. This proves clusters can spin up after the deployment.

Where people trip

Terraform apply fails with API not enabled

Enable the required APIs on the target project:

gcloud services enable compute.googleapis.com --project <PROJECT_ID>

Permission errors during apply

Confirm the GSA has roles/Owner on the service project (Standalone) and the required network roles on the host project (Shared VPC). Refresh the access token:

export GOOGLE_OAUTH_ACCESS_TOKEN=$(gcloud auth print-access-token)

VPC/Subnet not found (Shared VPC)

Check vpc_name, subnet_name, vpc_network_project_id, and google_region. The subnet must exist in the region you deploy the workspace to.

terraform destroy fails because the network is still in use

Databricks may leave firewall rules on the VPC. Destroy in order per the Standalone README: workspace first, then NAT/router, then the rest. Delete orphaned firewall rules with gcloud compute firewall-rules delete if needed.

Regulated Deployments and Security Reference Architecture

For regulated environments, deploy a hardened workspace with Private Link and the Security Reference Architecture (SRA) defaults, so traffic stays off the public internet. Use the SRA Terraform templates:

Resource	Link
SRA documentation site	databricks.github.io/terraform-databricks-sra
GitHub repo (GCP)	databricks/terraform-databricks-sra/gcp

Next

• Do next: Add Users
• Learn why: Unity Catalog foundations
• Reference: Databricks Terraform provider