The Lakeflow Framework¶
The Lakeflow Framework (LFF)
Metadata-driven pipelines for Databricks Lakeflow SDP
Configure once. Reuse everywhere. LFF transforms declarative metadata into production-ready Lakeflow Spark Declarative Pipelines (SDP) using reusable patterns and Databricks Declarative Automation Bundles (DABs).
One framework. Any operating model.
Built to fit the way your organization delivers data. Whether your teams operate through a centralized platform, a hybrid model, or federated domain ownership, LFF adapts to your operating model. Use consistent implementation patterns to build medallion pipelines, domain-owned data products, or architectures tailored to your business.
New to LFF? Start at Get Started — product overview, Quick Start, or straight into the docs.
Architecture
Operating models, Framework and Pipeline Bundles, data flow specs, and DABs on Lakeflow SDP.
Explore architectureSamples
Feature samples, pattern samples, and the end-to-end TPCH reference warehouse.
Browse feature & pattern samplesBuild
Bundle structure, pipeline bundle steps, data flow spec reference, and medallion patterns.
Author pipeline bundles & specsDeploy
Deploy the Framework Bundle and Pipeline Bundles — flat DAB deploy, pip wheel, local CLI, or CI/CD.
Deploy framework & pipelinesFeatures
Metadata-driven specs, configuration, Python extensions, data quality, sources and targets, and platform features.
Browse features by categoryAgent Skills
Agent Skills for Cursor, Claude Code, Genie Code, and more — generate production-ready data flow spec pipeline bundles from natural language.
Build specs with agent skills