Skip to main content

Warehouses and Lakehouses

This section explains how to connect Bauplan‑managed Iceberg tables in your object storage to other platforms your teams already use, such as Databricks Unity Catalog, AWS Glue and Athena.

Bauplan writes tables in Apache Iceberg format and exposes a standards-compliant Iceberg REST catalog. External engines discover tables through that catalog and read files directly from your bucket with their own cloud credentials.

Integration options

Outbound connectors unsupported in bauplan v0.3

An outbound connector allows a Bauplan pipeline to pull external tables (from Snowflake or BigQuery) into the pipeline. Beginning with the v0.3.0 CLI/SDK release (bauplan~=0.3.0), Bauplan's new pipeline DSL does not support outbound connectors.

  • External catalog: plug-and-play, auto-discovery, recommended whenever supported.
  • External tables: manual, per-table setup, fallback for engines without catalog support.

Bauplan ensures that whichever route you take, your tables remain in open formats on your own storage, accessible to the rest of your stack.

Most modern engines now support the Iceberg REST catalog. This lets them attach directly to Bauplan and see all namespaces and tables, with no manual refresh required.

  • Bauplan catalog endpoint: https://api.use1.aprod.bauplanlabs.com/iceberg
  • Auth: Bearer <token> with a read-only Bauplan user
  • Storage: Your object store (usually Amazon S3). Engines authenticate with their own IAM/service accounts.
  • Engines supported: Snowflake Open Catalog, Databricks Unity Catalog, BigQuery, Trino, Athena, Spark, DuckDB, and others with native Iceberg connectors.

Option 2: External tables (per-table)

If your engine does not yet support the REST catalog, you can still integrate by registering each table directly from its metadata location.

  • Use Bauplan's API (client.get_table(...)) to retrieve the Iceberg metadata path.
  • Register that path manually in your engine as an external table.
  • You must refresh tables manually when schema or snapshots change.

This approach works everywhere, but requires more operational overhead than catalog integration.