logo
Lakeflow Framework
Advanced Composition
Initializing search
    lakeflow_framework
    • Home
    • Get Started
    • Architecture
    • Samples
    • Build
    • Deploy
    • Features
    • Contributors
    lakeflow_framework
    • Home
    • Get Started
      • What is the Lakeflow Framework?
      • Quick Start
    • Architecture
    • Samples
    • Build
      • Bundle Scope and Structure
      • Building a Pipeline Bundle
      • Data Flow Spec Reference
        • Creating a Standard Data Flow Spec Reference
        • Creating a Flows Data Flow Spec Reference
        • Creating a Materialized View Data Flow Spec Reference
      • Data Flows & Pipeline Patterns
        • Base Patterns
          • Pattern - Basic 1:1
          • Pattern - Gold Materialized Views
        • Advanced composition
          • Pattern - Multi-Source Streaming
          • Pattern - Stream-Static - Basic
          • Pattern - Stream-Static - Streaming Data Warehouse
          • Pattern - CDC Stream from Snapshots
        • Scaling and decomposing pipelines
        • Mix and Match Patterns
    • Deploy
      • Before you deploy
      • Deploy the Framework
        • Deployment options
        • Deploy from local machine
        • Installing as a wheel
      • Deploy Pipeline Bundles
      • Setting up CI/CD
    • Features
      • Metadata-Driven Development
        • Data Flow Specification Format
        • Schema Management
        • Substitutions
        • Templates
        • Validation
      • Authoring and Tooling
        • Auto Complete / IntelliSense
        • UI Integration
      • Configuration Management
        • Framework configuration
        • Mandatory Table Properties
        • Spark Configuration
        • Operational Metadata
        • Secrets Management
        • Logging
        • Builder Parallelization
      • Supported Sources and Targets
        • Supported Source Types
        • Supported Target Types
        • SQL Source
      • Key Databricks Features
        • Change Data Capture (CDC)
        • Change Data Feed (CDF)
        • Multi-Source Streaming (Flows)
        • Liquid Clustering
        • Materialized Views
        • Schema-related Databricks Features
        • Soft Deletes
        • Target Catalog and Schema
      • Python Development and Extensibility
        • Python Code, Libraries & Init Scripts
        • Python Function Transforms
        • Python Dependency Management
        • Python Source
      • Data Quality
        • Data Quality - Expectations
        • Data Quality - Quarantine
      • Environments and Versioning
        • Logical Environments
        • Versioning - DataFlow Specs
        • Versioning - Framework
      • Migrations
        • Table Migration
      • Features A–Z
    • Contributors
      • Set up your environment
      • Branching, versioning & releases
      • Contribution workflow
      • Import conventions
      • Write & build docs
      • Contributing to contrib

    Advanced Composition¶

    Patterns for multi-source streaming, stream-static joins, and CDC-from-snapshot sources.

    See Data Flow and Pipeline Patterns for the pattern catalog, Scaling and Decomposing Pipelines, and Mix and Match Patterns.

    • Pattern - Multi-Source Streaming
    • Pattern - Stream-Static - Basic
    • Pattern - Stream-Static - Streaming Data Warehouse
    • Pattern - CDC Stream from Snapshots
    © Copyright 2026, Databricks.
    Created using Sphinx 9.0.4. and Sphinx-Immaterial