cursus.pipeline_catalog.shared_dags

Shared DAG Definitions for Pipeline Catalog

JSON-based DAG store. Each DAG is a .dag.json file containing nodes, edges, and metadata. The catalog_index.json provides a queryable index of all DAGs.

Usage:

from cursus.api.dag import import_dag_from_json dag = import_dag_from_json(“path/to/some.dag.json”)

# Or use the catalog: from cursus.pipeline_catalog.shared_dags import load_shared_dag, get_all_shared_dags dag = load_shared_dag(“bedrock_pytorch_incremental_edx”)

load_shared_dag(dag_id)

Load a shared DAG by ID from the JSON catalog.

Parameters:

dag_id (str) – DAG identifier (e.g., “bedrock_pytorch_incremental_edx”)

Returns:

PipelineDAG ready for compilation

Return type:

PipelineDAG

get_all_shared_dags()

Get metadata for all available shared DAGs from the catalog index.

Returns:

Dict mapping DAG id to metadata dict

Return type:

Dict[str, Dict[str, Any]]

get_catalog_index()

Load the catalog index.

class DAGMetadata(**kwargs)

Bases: object

Legacy metadata class. Use catalog_index.json instead.

Modules

dummy	Dummy Shared DAG Definitions
lightgbm
pytorch	PyTorch Shared DAG Definitions
singleton	Singleton DAG Definitions
xgboost	XGBoost Shared DAG Definitions