Skip to main content

Quick start

Prerequisites

  • Python 3.11 or higher.
  • Install bauplan.
Tip

Rather get started with an AI agent?▾

Create a branch

Create a new data branch and switch to it:

bauplan checkout -b <YOUR_USERNAME>.quickstart

Explore the data

Examine public datasets pre-loaded into the Bauplan sandbox; table ls will show you the tables in your current branch (newly created branches inherit all tables from the parent branch).

bauplan table ls

You should see a list of tables. Let's explore the schema of the table we're going to work with:

bauplan table get titanic

You'll see the table's columns and their types - this is the input our pipeline will transform.

Scaffold a project

Run bauplan init to generate a ready-to-run project:

bauplan init

This creates three files:

  • bauplan_project.yaml - project metadata (id and name), this is also the place where secrets and parameters are stored.
  • models.py - a pipeline with one model (survival_rate_by_age) that reads from the titanic table, plus one expectation test (test_age) that validates the output
  • pyproject.toml - Python dependencies, which in this case is just bauplan itself.

Run the pipeline

The generated project defines a small pipeline: it reads from the titanic table, computes survival rates by age, and validates the output with an expectation test. Let's run it:

bauplan run

Looking at models.py, you'll find two functions:

  • survival_rate_by_age: a model decorated with @bauplan.model(). It reads the Age and Survived columns from the titanic table, groups passengers by age, and computes the average survival rate per age group.
  • test_age: an expectation decorated with @bauplan.expectation(). Expectations are Bauplan's built-in way to run data quality checks. This one verifies that the Age column in the output has no duplicate values.

Materialize the output as a table

If you check the output of bauplan run, you'll see that the survival_rate_by_age model executed successfully, and the expectation test passed. However, if you look at your tables again:

bauplan table ls --name survival_rate_by_age

You'll see that no new table was created for the model's output. By default, materialization_strategy parameter in @bauplan.model() is set to NONE, which results in memory without writing anything to the catalog. To persist the output as a table, make this change to models.py:

@bauplan.model(materialization_strategy='REPLACE')

This tells Bauplan to write (and fully replace on each run) the model's output as an Iceberg table in your branch. Run the pipeline again:

bauplan run

You can confirm the table is persisted by running the following command (only 10 rows will show by default):

bauplan query "SELECT * FROM survival_rate_by_age"

Learn more about materialization strategies and models here.

You should see two columns: Age and Survived, showing the average survival rate for each age group. Congratulations - you've just built and run your first Bauplan pipeline!