cursus.core.base.config_base

Base Pipeline Configuration with Self-Contained Derivation Logic

This module implements the base configuration class for pipeline steps using a self-contained design where each configuration class is responsible for its own field derivations through private fields and read-only properties.

class BasePipelineConfig(*, author, bucket, role, region, service_name, pipeline_version, model_class='xgboost', current_date=<factory>, framework_version='2.1.0', py_version='py310', source_dir=None, enable_caching=False, use_secure_pypi=False, max_runtime_seconds=172800, project_root_folder, **extra_data)

Bases: BaseModel, ABC

Base configuration with shared pipeline attributes and self-contained derivation logic.

author: str

bucket: str

role: str

region: str

service_name: str

pipeline_version: str

model_class: str

current_date: str

framework_version: str

py_version: str

source_dir: str | None

enable_caching: bool

use_secure_pypi: bool

max_runtime_seconds: int

project_root_folder: str

model_config: ClassVar[ConfigDict] = {'arbitrary_types_allowed': True, 'extra': 'allow', 'protected_namespaces': (), 'validate_assignment': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

property aws_region: str

Get AWS region based on region code.

property pipeline_name: str

Get pipeline name derived from author, service_name, model_class, and region.

property pipeline_description: str

Get pipeline description derived from service_name, model_class, and region.

property pipeline_s3_loc: str

Get S3 location for pipeline artifacts.

property effective_source_dir: str | None

Get effective source directory with hybrid resolution.

This base implementation works with just source_dir (which can be None). Processing configs override this to handle both processing_source_dir and source_dir.

Resolution Priority: 1. Hybrid resolution of source_dir 2. Legacy value (source_dir) 3. None if source_dir is not provided

model_dump(**kwargs)

Override model_dump to include derived properties.

initialize_derived_fields()

Initialize all derived fields once after validation.

property step_catalog: Any | None

Lazy-loaded step catalog instance for optimized component discovery.

Returns:

StepCatalog instance or None if initialization fails

get_script_contract()

Get script contract for this configuration using optimized step catalog discovery.

This optimized implementation uses the step catalog system for efficient contract discovery with O(1) lookups and workspace awareness, falling back to legacy methods for backward compatibility.

Returns:

Script contract instance or None if not available

Return type:

Any | None

property script_contract: Any | None

Property accessor for script contract.

Returns:

Script contract instance or None if not available

get_script_path(default_path=None)

Get script path for this configuration.

This method provides a default implementation that returns None, since not all step types require scripts (e.g., Model creation steps don’t need scripts).

Derived classes that need script paths should override this method with their specific requirements: - Processing steps: Combine entry_point with source_dir - Training steps: Use contract entry_point or combine with source_dir - Model steps: May not need scripts at all

Parameters:

default_path (str | None) – Default script path to use if not found via other methods

Returns:

Script path, default_path, or None if not applicable for this step type

Return type:

str | None

get_environment_variables(declared_env_vars=None)

Resolve the env-var VALUES for the names the step interface DECLARES (FZ 31e1d3g).

The interface (.step.yaml env_vars) declares WHICH env vars a step uses; this config supplies the VALUES. The single-source rule: the interface drives the key set, config resolves each one — by _env_overrides() first (computed/aliased), else convention (NAME -> self.name, type-formatted). Names that resolve to nothing are omitted, so a declared-but-absent optional falls back to its interface default in the builder.

Subclasses with a bespoke collector may override this entirely (and ignore declared_env_vars) to return their full env dict — that path is preserved for back-compat.

get_job_arguments()

Build the script’s CLI arguments — config is the single source (FZ 31e1d3h).

Base default: no CLI arguments (None). The builder’s _get_job_arguments calls this; a step-config that passes arguments to its script OVERRIDES this method. The common --job_type case has a ready helper, _job_type_arg() — a config opts in with return self._job_type_arg(). None (not []) is the “no args” contract the SDK ProcessingStep / processor.run expect.

resolve_hybrid_path(relative_path)

Resolve a path using the hybrid path resolution system.

This method uses the hybrid path resolution system to find files across different deployment scenarios (Lambda/MODS bundled, development monorepo, pip-installed separated).

Parameters:

relative_path (str) – Relative path from project root to target

Returns:

Resolved absolute path if found, None otherwise

Return type:

str | None

property resolved_source_dir: str | None

Get resolved source directory using hybrid resolution.

Returns None if source_dir is not provided, since it’s optional in base class. Processing, training, and model step configs should ensure source_dir is provided.

classmethod get_step_name(config_class_name)

Get the step name for a configuration class using existing registry functions.

classmethod get_config_class_name(step_name)

Get the configuration class name from a step name using existing registry functions.

classmethod from_base_config(base_config, **kwargs)

Create a new configuration instance from a base configuration. This is a virtual method that all derived classes can use to inherit from a parent config.

Parameters:

• base_config (BasePipelineConfig) – Parent BasePipelineConfig instance

• **kwargs (Any) – Additional arguments specific to the derived class

Returns:

A new instance of the derived class initialized with parent fields and additional kwargs

Return type:

BasePipelineConfig

categorize_fields()

Categorize all fields into three tiers: 1. Tier 1: Essential User Inputs - public fields with no defaults (required) 2. Tier 2: System Inputs - public fields with defaults (optional) 3. Tier 3: Derived Fields - properties that access private attributes

Returns:

Dict with keys ‘essential’, ‘system’, and ‘derived’ mapping to lists of field names

Return type:

Dict[str, List[str]]

print_config()

Print complete configuration information organized by tiers. This method automatically categorizes fields by examining their characteristics: - Tier 1: Essential User Inputs (public fields without defaults) - Tier 2: System Inputs (public fields with defaults) - Tier 3: Derived Fields (properties that provide access to private fields)

get_public_init_fields()

Get a dictionary of public fields suitable for initializing a child config. Only includes fields that should be passed to child class constructors. Both essential user inputs and system inputs with defaults or user-overridden values are included to ensure all user customizations are properly propagated to derived classes.

Returns:

Dictionary of field names to values for child initialization

Return type:

Dict[str, Any]

model_post_init(context, /)

This function is meant to behave like a BaseModel method to initialize private attributes.

It takes context as an argument since that’s what pydantic-core passes when calling it.

Parameters:

• self (BaseModel) – The BaseModel instance.

• context (Any) – The context.