Pipeline Catalog

Cursus ships 44 pre-built shared pipeline DAGs across 8 frameworks (bedrock, dummy, generic, lightgbm, lightgbmmt, pytorch, xgboost, xgboost_mt). Discover and build them with the data-driven router:

from cursus.pipeline_catalog import recommend_dag, load_shared_dag
from cursus import PipelineDAGCompiler

# recommend_dag returns a ranked list of matches (dicts with 'id', 'score', ...)
recommendations = recommend_dag(framework="xgboost", task_type="end_to_end")
dag = load_shared_dag(recommendations[0]["id"])

pipeline, report = PipelineDAGCompiler(config_path="config.json").compile_with_report(dag)

All DAGs

DAG id	framework	task type	complexity	nodes	description
bedrock_batch_data_processing	bedrock		standard	3	Shared DAG definition for Bedrock Batch Data Processing Pipeline
bedrock_batch_simple_training	bedrock		standard	4	Shared DAG definition for Bedrock Batch-Enhanced Simple Training Pipeline
bedrock_batch_sopa	bedrock		standard	1	SOPA (Standard Operating Procedure Automation) instruction tuning singleton pipeline. Single-node DAG for fine-tuning a Bedrock LLM on domain-specific SOP instructions.
bedrock_simple_training	bedrock		standard	4	Shared DAG definition for Bedrock-Enhanced Simple Training Pipeline
dummy_e2e_basic	dummy	end_to_end	simple	4	Basic end-to-end pipeline with dummy training, packaging, payload preparation, and registration
dummy_inference_with_wiki	dummy	inference	standard	7	Dummy training pipeline with inference, metrics computation, and wiki generation
cradle_data_loading_singleton	generic	data_loading	simple	1	Singleton pipeline for CradleDataLoading training data step
lightgbm_complete_e2e	lightgbm	end_to_end	comprehensive	10	Complete LightGBM end-to-end pipeline with training, calibration, packaging, registration, and evaluation
lightgbm_complete_e2e_with_percentile_calibration_and_testing	lightgbm	end_to_end	comprehensive	15	Complete LightGBM end-to-end pipeline with training, percentile model calibration path, testing path (no calibration), packaging, registration, inference, metrics computation, and wiki generation
lightgbmmt_complete_e2e	lightgbmmt	multi_task_end_to_end	comprehensive	10	Complete LightGBMMT multi-task end-to-end pipeline with training, calibration, packaging, registration, and evaluation
lightgbmmt_ssl_training	lightgbmmt	multi_task_semi_supervised_learning	advanced	16	LightGBMMT multi-task Semi-Supervised Learning pipeline with pretraining, pseudo-labeling, active sampling, merge, and fine-tuning
lightgbmmt_temporal_split_e2e	lightgbmmt	multi_task_temporal_split_end_to_end	comprehensive	11	LightGBMMT multi-task end-to-end pipeline with TEMPORAL-SPLIT preprocessing (time-based train split), dual evaluation (testing + calibration paths), model calibration, packaging, and registration.
lightgbmmt_with_label_ruleset_e2e	lightgbmmt	multi_task_end_to_end_with_label_ruleset	comprehensive	13	LightGBMMT multi-task end-to-end pipeline with label ruleset generation/execution for transparent rule-based label transformation, training, calibration, packaging, and registration
bedrock_batch_pytorch_e2e	pytorch	end_to_end_with_batch_llm	comprehensive	13	Bedrock Batch-enhanced PyTorch end-to-end pipeline with cost-efficient LLM-based data processing, training, calibration, packaging, registration, and evaluation
bedrock_batch_pytorch_with_label_ruleset_e2e	pytorch	end_to_end_with_batch_llm_and_label_ruleset	comprehensive	16	Bedrock Batch-enhanced PyTorch end-to-end pipeline with label ruleset generation/execution for transparent rule-based label transformation, training, calibration, packaging, and registration
bedrock_pytorch_e2e	pytorch	end_to_end_with_realtime_llm	comprehensive	13	Bedrock Real-time-enhanced PyTorch end-to-end pipeline with LLM-based data processing, training, calibration, packaging, registration, and evaluation
bedrock_pytorch_incremental_edx	pytorch	incremental_training_with_llm_scoring	comprehensive	16	Incremental PyTorch training with Bedrock scoring, EDX anti-join, and EDX upload
bedrock_pytorch_with_data_upload	pytorch	inference_upload_then_train_from_edx	comprehensive	8	Two-pipeline pattern: LLM inference uploads to Andes, training consumes from EDX
bedrock_pytorch_with_label_ruleset_e2e	pytorch	end_to_end_with_realtime_llm_and_label_ruleset	comprehensive	16	Bedrock Real-time-enhanced PyTorch end-to-end pipeline with label ruleset generation/execution for transparent rule-based label transformation, training, calibration, packaging, and registration
complete_e2e	pytorch		standard	10	Shared DAG definition for PyTorch Complete End-to-End Pipeline
complete_e2e_dummy	pytorch		standard	10	Shared DAG definition for PyTorch Complete End-to-End Pipeline with Dummy Data Loading
pytorch_hybrid_xgboost_e2e	pytorch		comprehensive	13	Two-stage hybrid pipeline: PyTorch encoder (Stage-1) produces embeddings, then XGBoost (Stage-2) stacks on [tabular | embedding]. Calibration path threads through both stages.
pytorch_stratified_percentile_e2e	pytorch		comprehensive	11	PyTorch end-to-end pipeline with StratifiedSampling for class-balanced training and PercentileCalibration for rank-ordered scoring
pytorch_tokenizer_percentile_e2e	pytorch		comprehensive	11	PyTorch end-to-end pipeline with custom TokenizerTraining step and PercentileCalibration. Trains a domain-specific tokenizer before model training.
pytorch_tsa_complete_e2e	pytorch		comprehensive	16	TSA (Temporal Self-Attention) complete end-to-end pipeline with TSA-specific preprocessing, separate testing and calibration paths, and model calibration.
simple_training	pytorch		standard	3	Shared DAG definition for Simple Training Pipeline
standard_e2e	pytorch		standard	9	Shared DAG definition for PyTorch Standard End-to-End Pipeline
training	pytorch		standard	6	Shared DAG definition for PyTorch training pipeline.
xgboost_complete_e2e	xgboost	end_to_end	comprehensive	10	Complete XGBoost end-to-end pipeline with training, calibration, packaging, registration, and evaluation
xgboost_complete_e2e_dummy	xgboost	end_to_end	comprehensive	10	Complete XGBoost end-to-end pipeline with dummy data loading, training, calibration, packaging, registration, and evaluation
xgboost_complete_e2e_with_calibration_and_testing	xgboost	end_to_end	comprehensive	15	Complete XGBoost end-to-end pipeline with training, calibration path, testing path (no calibration), packaging, registration, inference, metrics computation, and wiki generation
xgboost_complete_e2e_with_percentile_calibration_and_testing	xgboost	end_to_end	comprehensive	15	Complete XGBoost end-to-end pipeline with training, percentile model calibration path, testing path (no calibration), packaging, registration, inference, metrics computation, and wiki generation
xgboost_complete_e2e_with_wiki	xgboost	end_to_end	comprehensive	13	Complete XGBoost end-to-end pipeline with training, calibration, packaging, registration, inference, metrics computation, and wiki generation
xgboost_percentile_e2e	xgboost		standard	10	XGBoost end-to-end pipeline with PercentileModelCalibration (rank-ordered scoring). Minimal calibration path without separate testing or wiki generation.
xgboost_simple	xgboost	training	simple	5	Simple XGBoost training pipeline with data loading and preprocessing
xgboost_ssl_training	xgboost	semi_supervised_learning	advanced	16	XGBoost Semi-Supervised Learning pipeline with pretraining, pseudo-labeling, active sampling, merge, and fine-tuning
xgboost_training_with_calibration	xgboost	training	standard	6	XGBoost training pipeline with model calibration
xgboost_training_with_calibration_fs	xgboost	training	advanced	8	XGBoost training pipeline with feature selection and model calibration
xgboost_training_with_evaluation	xgboost	training	standard	6	XGBoost training pipeline with model evaluation
xgboost_training_with_evaluation_dummy	xgboost	training	standard	6	XGBoost training pipeline with model evaluation using dummy data loading
xgboost_training_with_evaluation_stratified	xgboost	training	standard	7	XGBoost training pipeline with stratified sampling and model evaluation
xgboost_training_with_feature_selection	xgboost	training	advanced	12	XGBoost training pipeline with feature selection and model evaluation
xgboost_training_with_preprocessing	xgboost	training	advanced	10	XGBoost training pipeline with advanced preprocessing (missing value imputation and risk table mapping) and model evaluation
xgboost_mt_temporal_split_e2e	xgboost_mt	multi_task_temporal_split_end_to_end	comprehensive	11	XGBoost multi-task end-to-end pipeline with TEMPORAL-SPLIT preprocessing (time-based train split), dual evaluation (testing + calibration paths), model calibration, packaging, and registration.