cursus.steps.configs.config_edx_uploading_step

EdxUploading Step Configuration.

Configuration for uploading S3 data to EDX via EdxDataLoader. Inherits ProcessingStepConfigBase for standard processing step fields.

Supports two input modes: 1. Direct ARN: provide edx_arn, components parsed automatically 2. Component-based: provide edx_provider, edx_subject, edx_dataset, edx_manifest_key

class EdxUploadingConfig(*, author, bucket, role, region, service_name, pipeline_version, model_class='xgboost', current_date=<factory>, framework_version='2.1.0', py_version='py310', source_dir=None, enable_caching=False, use_secure_pypi=False, max_runtime_seconds=172800, project_root_folder, processing_instance_count=1, processing_volume_size=500, processing_instance_type_large='ml.m5.4xlarge', processing_instance_type_small='ml.m5.2xlarge', use_large_processing_instance=False, skip_volume_kms=None, processing_source_dir=None, processing_entry_point='edx_uploading.py', processing_script_arguments=None, processing_framework_version='1.2-1', edx_arn=None, edx_provider=None, edx_subject=None, edx_dataset=None, edx_manifest_key=None, job_type=None, edx_manifest_key_parts=None, edx_output_columns=None, **extra_data)

Bases: ProcessingStepConfigBase

Configuration for the EdxUploading step (S3 → EDX via EdxDataLoader).

Supports two input modes (same pattern as EdxDataSourceConfig in CradleDataLoading): 1. Direct ARN input: provide edx_arn, components are parsed from it 2. Component-based input: provide edx_provider, edx_subject, edx_dataset, edx_manifest_key

edx_arn: str | None

edx_provider: str | None

edx_subject: str | None

edx_dataset: str | None

edx_manifest_key: str | None

job_type: str | None

processing_entry_point: str

edx_manifest_key_parts: Dict[str, str] | None

edx_output_columns: List[str] | None

validate_input_mode()

Ensure either edx_arn OR all 4 components are provided. Parse ARN if given.

property edx_arn_base: str

Construct EDX manifest ARN base from provider/subject/dataset.

get_environment_variables(declared_env_vars=None)

EDX-upload env vars (the single env source; moved here from the builder, FZ 31e1d3g).

Bespoke values (constant regional-endpoint flag, computed edx_arn_base, JSON-encoded manifest-key-parts / output-columns), so this returns the full env dict; declared_env_vars is accepted for the builder’s names-driven contract but ignored.

model_config: ClassVar[ConfigDict] = {'arbitrary_types_allowed': True, 'extra': 'allow', 'protected_namespaces': (), 'validate_assignment': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_post_init(context, /)

This function is meant to behave like a BaseModel method to initialize private attributes.

It takes context as an argument since that’s what pydantic-core passes when calling it.

Parameters:

• self (BaseModel) – The BaseModel instance.

• context (Any) – The context.