cursus.core.assembler.pipeline_template_base

Base class for pipeline templates.

This module provides a base class for all pipeline templates, ensuring consistent structure, proper component lifecycle management, and best practices across different pipeline templates.

class PipelineTemplateBase(config_path, sagemaker_session=None, role=None, registry_manager=None, dependency_resolver=None, pipeline_parameters=None, step_catalog=None)

Bases: ABC

Base class for all pipeline templates.

This class provides a consistent structure and common functionality for all pipeline templates, enforcing best practices and ensuring proper component lifecycle management.

The template follows these steps to build a pipeline: 1. Load configurations from file 2. Initialize component dependencies (registry_manager, dependency_resolver) 3. Create the DAG, config_map, and step_builder_map 4. Use PipelineAssembler to assemble the pipeline

This provides a standardized approach for creating pipeline templates, reducing code duplication and enforcing best practices.

CONFIG_CLASSES: Dict[str, Type[BasePipelineConfig]] = {}

pipeline_metadata: Dict[str, Any]

set_pipeline_parameters(parameters=None)

Set pipeline parameters for this template.

This method allows DAGCompiler to inject custom parameters that will be used instead of the default parameters defined in subclasses.

Parameters:

parameters (List[ParameterString] | None) – List of pipeline parameters to use

generate_pipeline()

Generate the SageMaker Pipeline.

This method coordinates the pipeline generation process: 1. Create the DAG and config_map (the assembler self-discovers builders) 2. Create the PipelineAssembler 3. Generate the pipeline 4. Store pipeline metadata

Returns:

SageMaker Pipeline

Return type:

Pipeline

classmethod create_with_components(config_path, context_name=None, **kwargs)

Create template with managed dependency components.

This factory method creates a template with properly configured dependency resolution components from the factory module.

Parameters:

• config_path (str) – Path to configuration file

• context_name (str | None) – Optional context name for registry isolation

• **kwargs (Any) – Additional arguments to pass to constructor

Returns:

Template instance with managed components

Return type:

PipelineTemplateBase

classmethod build_with_context(config_path, **kwargs)

Build pipeline with scoped dependency resolution context.

This method creates a template with a dependency resolution context that ensures proper cleanup of resources after pipeline generation.

Parameters:

• config_path (str) – Path to configuration file

• **kwargs (Any) – Additional arguments to pass to constructor

Returns:

Generated pipeline

Return type:

Pipeline

classmethod build_in_thread(config_path, **kwargs)

Build pipeline using thread-local component instances.

This method creates a template with thread-local component instances, ensuring thread safety in multi-threaded environments.

Parameters:

• config_path (str) – Path to configuration file

• **kwargs (Any) – Additional arguments to pass to constructor

Returns:

Generated pipeline

Return type:

Pipeline

analyze_pipeline_structure()

Analyze and print the complete pipeline structure.

Delegates to the PipelineAssembler’s analyze_pipeline_structure method. Must be called after generate_pipeline().

Raises:

AttributeError – If called before generate_pipeline()