Skip to main content

Commits and refs

Bauplan versions everything: code, data, and execution environments.

  • Code lives in Git or similar tools, and Bauplan also captures it immutably at execution time.
  • Data changes through the catalog: evolving table snapshots track all writes, replacements, and deletions, where each data change creates a new commit.
  • Execution environments use code-based definitions and versioning as well - Python versions, packages, parameters, and run metadata.

What is a commit?

A commit is a record of a change in the data lake.

Every mutation results in a commit, whether it comes from running a pipeline, importing data, or modifying a table.

Commits form the linear history of branches and represent the fundamental units of change tracking in your data. Each commit contains:

  • A unique commit_hash that identifies the operation.
  • A parent_hash, which references the previous state of the lake.
  • In the case of a merge operation, a second parent hash, parent_hash[1], that represents the merged branch.
  • Additional metadata such as job-id if created by a pipeline run, and project-id, user-id, task-id, etc.

Example: Viewing commit history

Here is a simple example of output from the commit API:

bauplan commit --limit 1

commit 4307792b4e2a325e4e42e3e0b595cf79c68016be6023314a5ac345c85833a9bc
Author: John Doe <john.doe@bauplanlabs.com>
Author Date: 2025-05-13T18:31:40.133920Z
Commit Date: 2025-05-13T18:31:40.139839199Z
Properties:
    bpln_organization_id = org_2oAi2K36inuMyeRy3YfCOcR6yUi
    bpln_task_id = 80ddee4b-5568-4b69-8f0b-b573766450a7
    bpln_job_id = 7d3c81d7-a779-4754-8c72-2311cc905da7
    bpln_project_id = 7a1878f8-d736-4079-bc1d-c23910948153
    bpln_username = jdoe
    bpln_user_id = user_2oAw0PTViaoSfvKX55Yp9Xu5HMV
Parent Hashes:
    300266b7781b59093649742c254f639d2f9085f629f9f1eaa9a0d732838b0e72
    268bbd29de8113a0faf5478b85972fdd755c9503569124d21d72f32fef562810

    Run job_id=7d3c81d7-a779-4754-8c72-2311cc905da7

What's a ref?

A Ref is an immutable, addressable handle to a specific commit. Unlike commits, which are the record of a change, a Ref is what you use in the API to reference a version of the lake.

Bauplan APIs like query, run, scan, and revert_table accept ref as a parameter to operate on that exact version of the lake. Refs let you perform point-in-time operations like:

  • Querying the state of a table at a specific version
  • Reproducing a pipeline run
  • Comparing development environments

Ref syntax

  • A branch name, like main, is a shortcut that points to the latest commit on the branch.
  • main@abc123 points to the commit abc123 on the branch main.
  • @abc123 is a detached ref pointing directly to commit abc123 without going through a branch or tag.

Refs are read-only: only branches can be used for write operations, because branches are mutable while refs point to commits which are not.

bauplan query "SELECT * FROM my_table" --ref ciro.mybranch-run-20250517181234-ca3ad62f-a2c7-4ef4-b87a-53d12909fde
import bauplan

client = bauplan.Client()
client.query("SELECT * FROM my_table", ref="ciro.mybranch-run-20250517181234-ca3ad62f-a2c7-4ef4-b87a-53d12909fde")