Skip to main content

7. Automation & Orchestration

info
  • Learn how to orchestrate end-to-end Databricks projects using Lakeflow Jobs.

Journey checklist​

  • Identify target cloud tenant(s).
  • Infra setup.
  • Data Governance Strategy.
  • Access your data.
  • Build the first pipeline.
  • Automation and orchestration.
  • Query and explore.
  • Databricks AI/BI

Get Started​

In your data+AI project, every aspect related to:

  • Orchestration.
  • Tasks, DAGs and dependencies.
  • Schedule of execution and triggers.
  • Retries and notifications (success, failures).

Should be setup as one or multiple Lakeflow Jobs (AKA Databricks Jobs).

Create the first job​

  • UI – Orchestrate workflows using the Databricks UI
  • DABs – Orchestrate workflows using Databricks Asset Bundles
  • MCP skills – Orchestrate workflows using MCP skills