bauplan.store module

This module contains methods to interact with Bauplan key-value object store. In particular, it provides methods to save and load Python objects from the store: for example, you main save a scikit model after training and load it later to make predictions, or you may pass a parameter outside of the usual Arrow tables between nodes.

Note that some basic checks are performed to ensure that the object can be serialized and deserialized.

import bauplan

# train a model
tree = DecisionTreeClassifier().fit(X, y)
# save the model in the store
bauplan.store.save_obj("tree", tree)

# later in the DAG, retrieve the model with the same key
tree = bauplan.load_obj("tree")
# use the Python object as intended, e.g. make predictions
y_hat = tree.predict(X_test)
bauplan.store.load_obj(key: str) → Any

Return the Python object previously stored at the given key.

Parameters:

key (str) – the key associated with the object to retrieve.

Returns:

the Python object stored at the given key, deserialized from the store.

Raises:

UserObjectKeyNotExistsError: If no object with key was stored as part of this DAG run

Raises:

MismatchedPythonVersionsError – If the Python version executing this DAG node is different from the Python version of a DAG node that save this object

bauplan.store.save_obj(key: str, obj: Any) → None

Store a Python object with a key, to be later retrieved in other parts of the DAG.

Parameters:

  • key (Any) – the key associated with the object to store.

  • obj – the Python object to store, must be serializable using pickling

Raises:
Returns:

None