Skip to main content

Sync RollUps to Unity Catalog

RollUp Tables live in Lakebase. To expose them to Spark jobs, BI dashboards, and ML pipelines, sync them to a Unity Catalog Managed Table via Lakebase CDF.

Enable sync​

SELECT lakets.enable_sync('metrics_hourly');

This call:

  1. Creates an unpartitioned shadow table lakets_cdf._shadow_rollup_metrics_hourly with REPLICA IDENTITY FULL.
  2. Installs a mirror trigger that forwards every INSERT / UPDATE / DELETE from the RollUp Table to the shadow.
  3. Lakebase CDF syncs the shadow to the Unity Catalog Managed Table lb__shadow_rollup_metrics_hourly_history.
Why the shadow layer?

Lakebase CDF cannot sync partitioned tables and fails on schemas that contain them. Synced tables are therefore isolated in the unpartitioned lakets_cdf schema. This layer is a workaround that can be removed once Lakebase lifts the partitioned-table limitation.

Disable sync​

SELECT lakets.disable_sync('metrics_hourly');

Drops the shadow table and removes the mirror trigger. The UC destination table is not dropped.

Notes​

  • RollUp sync assumes incremental refresh; the UC destination is an append-only change feed.
  • enable_sync / disable_sync are idempotent — safe to call multiple times.
  • To sync a ChronoTable (not a RollUp), use the same functions — see Lakebase CDF setup.

Reference​

Full function signatures: Lakebase CDF reference.