Pipelines

In Bauplan, pipelines are implicitly defined by chaining models together through their declared inputs and outputs. Pipelines take the form of Directed Acyclic Graphs (DAGs).

When you define a model, you only declare which inputs come from which models. Bauplan analyzes the dependency graph based on model definitions and automatically infers the correct execution order.

To declare model dependencies (chain models into a DAG), annotate a function parameter with a Bauplan model, using Annotated and bauplan.Model:

from typing import Annotated
import bauplan

class StepOutput(bauplan.TableSchema):
    """The columns each step passes to the next."""

    col: bauplan.Int64 | None

@bauplan.model()
@bauplan.python('3.11')
def step_1(
    data: Annotated[pyarrow.Table, bauplan.Model('input_table')],
) -> Annotated[pyarrow.Table, StepOutput]:
    return data

@bauplan.model()
@bauplan.python('3.11')
def step_2(
    data: Annotated[pyarrow.Table, bauplan.Model('step_1')],
) -> Annotated[pyarrow.Table, StepOutput]:
    return data

@bauplan.model()
@bauplan.python('3.11')
def step_3(
    data: Annotated[pyarrow.Table, bauplan.Model('step_2')],
) -> Annotated[pyarrow.Table, StepOutput]:
    return data

Pipeline constraints

Models can take multiple tabular inputs but must return a single tabular output. Currently, pyarrow.Table is the only supported return type.

✅ This is a valid Bauplan DAG, every model has 1 output.

┌────────────┐
│  Model 1   │──────────────┐
└────────────┘              │
                            ▼
┌────────────┐      ┌────────────┐      ┌────────────┐
│  Model 2   │─────►│  Model 3   │─────►│  Model 4   │
└────────────┘      └────────────┘      └────────────┘
                            ▲
┌────────────┐              │
│ Iceberg    │──────────────┘
│  Table     │
└────────────┘

❌ This is NOT a valid Bauplan DAG, Model 2 returns 2 tables.

                                        ┌────────────┐
                                    ┌──►│ Iceberg    │
                                    │   │  Table A   │
                                    │   └────────────┘
┌────────────┐      ┌────────────┐  │
│  Model 1   │─────►│  Model 2   │──┤
└────────────┘      └────────────┘  │
                                    │   ┌────────────┐
                                    └──►│ Iceberg    │
                                        │  Table B   │
                                        └────────────┘

Write two models instead. Both can read the same upstream model.

Best practices

To keep your project organized and make it easy to run, test, and inspect individual pipelines, we recommend the following structure. See Projects for more details.

my_project/
└── pipelines/
    ├── sales_reporting/
    │   ├── models.py
    │   └── bauplan_project.yml
    ├── customer_segmentation/
    │   ├── models.py
    │   └── bauplan_project.yml
    └── churn_prediction/
        ├── models.py
        └── bauplan_project.yml

• Group models that form a logical pipeline into a single file: models.py.
• Place that file inside a folder named after the pipeline (sales_reporting, churn_prediction, etc.).
• Each pipeline folder contains its own bauplan_project.yml file, which defines environment settings and dependencies for that pipeline only.
• Separate function bodies into external modules and call them within the Bauplan models. This keeps business logic code neatly separated from the DAG and environment declaration, making code refactoring easier and future-proof.

This layout:

• Makes each pipeline self-contained and easy to run with bauplan run from inside the folder.
• Keeps models modular and promotes reuse.
• Helps enforce consistent dependency tracking and ref resolution.