Step Interface Catalog¶

Cursus ships 54 declarative step interfaces (steps/interfaces/*.step.yaml). Each step is one YAML file that unifies the script contract (input/output paths, env vars, CLI job arguments) with the spec (typed dependencies and outputs used for DAG dependency resolution). Load one at runtime with load_step_interface("<StepType>").

Summary¶

Step	Node type	SageMaker step	Deps	Outputs
`ActiveSampleSelection`	internal	Processing	1	2
`BatchTransform`	internal	Transform	2	1
`BedrockBatchProcessing`	internal	Processing	3	2
`BedrockProcessing`	internal	Processing	3	2
`BedrockPromptTemplateGeneration`	internal	Processing	1	3
`CradleDataLoading`	source	CradleDataLoading	0	3
`CurrencyConversion`	internal	Processing	1	1
`DataUploading`	sink	Processing	1	0
`DummyDataLoading`	internal	Processing	1	3
`DummyTraining`	internal	Processing	2	1
`EdxUploading`	sink	Processing	1	0
`FeatureSelection`	internal	Processing	2	2
`LabelRulesetExecution`	internal	Processing	2	2
`LabelRulesetGeneration`	internal	Processing	1	2
`LightGBMModelEval`	internal	Processing	2	2
`LightGBMModelInference`	internal	Processing	2	1
`LightGBMTraining`	internal	Training	3	2
`LightGBMMTModelEval`	internal	Processing	2	3
`LightGBMMTModelInference`	internal	Processing	2	1
`LightGBMMTTraining`	internal	Training	3	2
`MissingValueImputation`	internal	Processing	2	2
`ModelCalibration`	internal	Processing	1	3
`ModelMetricsComputation`	internal	Processing	1	2
`ModelWikiGenerator`	internal	Processing	2	1
`Package`	internal	Processing	3	1
`Payload`	internal	Processing	2	1
`PercentileModelCalibration`	internal	Processing	2	3
`PiperMetricGeneration`	internal	Processing	1	1
`PseudoLabelMerge`	internal	Processing	2	1
`PyTorchModel`	internal	CreateModel	1	1
`PyTorchModelEval`	internal	Processing	2	2
`PyTorchModelInference`	internal	Processing	2	1
`PyTorchTraining`	internal	Training	3	2
`RedshiftDataLoading`	source	RedshiftDataLoading	0	1
`Registration`	sink	MimsModelRegistrationProcessing	2	0
`RiskTableMapping`	internal	Processing	3	2
`SOPAInstructionTuning`	internal	Training	4	2
`StratifiedSampling`	internal	Processing	1	1
`TabularPreprocessing`	internal	Processing	3	1
`TemporalFeatureEngineering`	internal	Processing	1	1
`TemporalSequenceNormalization`	internal	Processing	2	1
`TemporalSplitPreprocessing`	internal	Processing	2	2
`TokenizerTraining`	internal	Processing	1	1
`TSAModelCalibration`	internal	Processing	2	3
`TSAModelEval`	internal	Processing	2	2
`TSAPreprocessing`	internal	Processing	3	1
`TSATabularPreprocessing`	internal	Processing	3	2
`TSATraining`	internal	Training	2	2
`XGBoostModel`	internal	CreateModel	1	1
`XGBoostModelEval`	internal	Processing	2	2
`XGBoostModelInference`	internal	Processing	2	1
`XgboostMtModelEval`	internal	Processing	2	2
`XgboostMtTraining`	internal	Training	3	2
`XGBoostTraining`	internal	Training	3	2

ActiveSampleSelection¶

Active sample selection script. Intelligently selects high-value samples from model predictions for SSL or Active Learning workflows.

node type internal · SageMaker Processing · entry point active_sample_selection.py

Dependencies (inputs)

logical name	required	data type	compatible sources
`evaluation_data`	yes		XGBoostModelInference, LightGBMModelInference, PyTorchModelInference, XGBoostModelEval, LightGBMModelEval, PyTorchModelEval, BedrockBatchProcessing, BedrockProcessing, LabelRulesetExecution

Outputs

logical name	data type	property path
`selected_samples`	S3Uri	`properties.ProcessingOutputConfig.Outputs['selected_samples'].S3Output.S3Uri`
`selection_metadata`	S3Uri	`properties.ProcessingOutputConfig.Outputs['selection_metadata'].S3Output.S3Uri`

Environment variables — optional: CONFIDENCE_THRESHOLD, UNCERTAINTY_MODE, MAX_SAMPLES, SELECTION_STRATEGY, USE_CASE, ID_FIELD, BATCH_SIZE, OUTPUT_FORMAT, K_PER_CLASS, METRIC, RANDOM_SEED, SCORE_FIELD, SCORE_FIELD_PREFIX

Job arguments — --job_type

BatchTransform¶

SageMaker Batch Transform step. Uses a registered model to run batch inference on preprocessed data. No script — managed by SageMaker TransformStep.

node type internal · SageMaker Transform · config BatchTransformStepConfig

Dependencies (inputs)

logical name	required	data type	compatible sources
`model_name`	yes		PyTorchModel, XGBoostModel
`processed_data`	yes		TabularPreprocessing

Outputs

logical name	data type	property path
`transform_output`	S3Uri	`properties.TransformOutput.S3OutputPath`

BedrockBatchProcessing¶

Bedrock batch processing script with batch inference and automatic fallback to real-time. Integrates with prompt templates and validation schemas from BedrockPromptTemplateGeneration.

node type internal · SageMaker Processing · entry point bedrock_batch_processing.py

Dependencies (inputs)

logical name	required	compatible sources
`input_data`	yes	DummyDataLoading, CradleDataLoading, TabularPreprocessing, TemporalSequenceNormalization, TemporalFeatureEngineering, StratifiedSampling, MissingValueImputation, FeatureSelection, CurrencyConversion
`prompt_templates`	yes	BedrockPromptTemplateGeneration
`validation_schema`	yes	BedrockPromptTemplateGeneration

Outputs

logical name	data type	property path
`processed_data`	S3Uri	`properties.ProcessingOutputConfig.Outputs['processed_data'].S3Output.S3Uri`
`analysis_summary`	S3Uri	`properties.ProcessingOutputConfig.Outputs['analysis_summary'].S3Output.S3Uri`

Environment variables — required: BEDROCK_PRIMARY_MODEL_ID; optional: BEDROCK_FALLBACK_MODEL_ID, BEDROCK_MAX_TOKENS, BEDROCK_BATCH_SIZE, BEDROCK_CONCURRENCY_MODE, BEDROCK_INFERENCE_PROFILE_ARN, BEDROCK_INFERENCE_PROFILE_REQUIRED_MODELS, BEDROCK_TEMPERATURE, BEDROCK_TOP_P, BEDROCK_MAX_RETRIES, BEDROCK_OUTPUT_COLUMN_PREFIX, BEDROCK_SKIP_ERROR_RECORDS, BEDROCK_MAX_CONCURRENT_WORKERS, BEDROCK_RATE_LIMIT_PER_SECOND, BEDROCK_BATCH_MODE, BEDROCK_BATCH_THRESHOLD, BEDROCK_BATCH_ROLE_ARN, BEDROCK_BATCH_TIMEOUT_HOURS, BEDROCK_MAX_RECORDS_PER_JOB, BEDROCK_MAX_CONCURRENT_BATCH_JOBS, BEDROCK_MAX_INPUT_FIELD_LENGTH, BEDROCK_TRUNCATION_ENABLED, BEDROCK_LOG_TRUNCATIONS

Job arguments — --job_type, --batch-size, --max-retries

BedrockProcessing¶

Bedrock processing script with invoke_model, structured output, and Converse API modes. Supports self-contained mode via env var templates. Circuit breaker and adaptive rate limiting.

node type internal · SageMaker Processing · entry point bedrock_processing.py

Dependencies (inputs)

logical name	required	compatible sources
`input_data`	yes	DummyDataLoading, CradleDataLoading, TabularPreprocessing, TemporalSequenceNormalization, TemporalFeatureEngineering, StratifiedSampling, MissingValueImputation, FeatureSelection, CurrencyConversion
`prompt_templates`	no	BedrockPromptTemplateGeneration
`validation_schema`	no	BedrockPromptTemplateGeneration

Outputs

logical name	data type	property path
`processed_data`	S3Uri	`properties.ProcessingOutputConfig.Outputs['processed_data'].S3Output.S3Uri`
`analysis_summary`	S3Uri	`properties.ProcessingOutputConfig.Outputs['analysis_summary'].S3Output.S3Uri`

Environment variables — required: BEDROCK_PRIMARY_MODEL_ID; optional: BEDROCK_FALLBACK_MODEL_ID, BEDROCK_INFERENCE_PROFILE_ARN, BEDROCK_MAX_TOKENS, BEDROCK_TEMPERATURE, BEDROCK_BATCH_SIZE, BEDROCK_CONCURRENCY_MODE, BEDROCK_USE_STRUCTURED_OUTPUT, BEDROCK_USE_CONVERSE_API, BEDROCK_INFERENCE_PROFILE_REQUIRED_MODELS, BEDROCK_TOP_P, BEDROCK_MAX_RETRIES, BEDROCK_OUTPUT_COLUMN_PREFIX, BEDROCK_SKIP_ERROR_RECORDS, BEDROCK_MAX_CONCURRENT_WORKERS, BEDROCK_RATE_LIMIT_PER_SECOND, BEDROCK_MAX_INPUT_FIELD_LENGTH, BEDROCK_TRUNCATION_ENABLED, BEDROCK_LOG_TRUNCATIONS, BEDROCK_USER_PROMPT_TEMPLATE, BEDROCK_INPUT_PLACEHOLDERS, BEDROCK_SYSTEM_PROMPT, BEDROCK_VALIDATION_SCHEMA, BEDROCK_ADAPTIVE_RATE_LIMITING

Job arguments — --job_type, --batch-size, --max-retries

BedrockPromptTemplateGeneration¶

Bedrock prompt template generation script. Creates structured prompt templates for classification tasks using the 5-component architecture pattern.

node type internal · SageMaker Processing · entry point bedrock_prompt_template_generation.py

Dependencies (inputs)

logical name	required	data type	compatible sources
`prompt_configs`	no		PromptConfiguration, ProcessingStep

Outputs

logical name	data type	property path
`prompt_templates`	S3Uri	`properties.ProcessingOutputConfig.Outputs['prompt_templates'].S3Output.S3Uri`
`template_metadata`	S3Uri	`properties.ProcessingOutputConfig.Outputs['template_metadata'].S3Output.S3Uri`
`validation_schema`	S3Uri	`properties.ProcessingOutputConfig.Outputs['validation_schema'].S3Output.S3Uri`

Environment variables — optional: TEMPLATE_TASK_TYPE, TEMPLATE_STYLE, VALIDATION_LEVEL, INPUT_PLACEHOLDERS, INCLUDE_EXAMPLES, GENERATE_VALIDATION_SCHEMA, TEMPLATE_VERSION

Job arguments — --include-examples, --generate-validation-schema, --template-version

CradleDataLoading¶

Cradle data loading script that reads config, writes output signature/metadata, creates and executes a Cradle data load job, and waits for completion. Data is loaded directly to S3 by the Cradle service.

node type source · SageMaker CradleDataLoading · entry point scripts.py

Outputs

logical name	data type	property path
`DATA`	S3Uri	`properties.ProcessingOutputConfig.Outputs['DATA'].S3Output.S3Uri`
`METADATA`	S3Uri	`properties.ProcessingOutputConfig.Outputs['METADATA'].S3Output.S3Uri`
`SIGNATURE`	S3Uri	`properties.ProcessingOutputConfig.Outputs['SIGNATURE'].S3Output.S3Uri`

Environment variables — optional: OUTPUT_PATH

CurrencyConversion¶

Currency conversion script. Converts monetary values across currencies based on marketplace information and exchange rates.

node type internal · SageMaker Processing · entry point currency_conversion.py

Dependencies (inputs)

logical name	required	data type	compatible sources
`input_data`	yes		TabularPreprocessing, ProcessingStep, CradleDataLoading, MissingValueImputation, RiskTableMapping, StratifiedSampling, FeatureSelection

Outputs

logical name	data type	property path
`processed_data`	S3Uri	`properties.ProcessingOutputConfig.Outputs['processed_data'].S3Output.S3Uri`

Environment variables — optional: CURRENCY_CODE_FIELD, MARKETPLACE_ID_FIELD, DEFAULT_CURRENCY, N_WORKERS, CURRENCY_CONVERSION_VARS, CURRENCY_CONVERSION_DICT

Job arguments — --job_type

DataUploading¶

SDK delegation step. Uploads S3 data to BDT (EDX/Andes). SINK node — no outputs. SDK DataUploadProcessor handles arguments internally.

node type sink · SageMaker Processing · entry point scripts.py

Dependencies (inputs)

logical name	required	data type	compatible sources
`input_data`	yes		TabularPreprocessing, StratifiedSampling, XGBoostTraining, CradleDataLoading, Processing

DummyDataLoading¶

Dummy data loading script. Drop-in replacement for CradleDataLoading that processes user-provided data instead of calling Cradle services.

node type internal · SageMaker Processing · entry point dummy_data_loading.py

Dependencies (inputs)

logical name	required	data type	compatible sources
`INPUT_DATA`	yes		DataUploadStep, S3DataStep, LocalDataStep

Outputs

logical name	data type	property path
`DATA`	S3Uri	`properties.ProcessingOutputConfig.Outputs['DATA'].S3Output.S3Uri`
`METADATA`	S3Uri	`properties.ProcessingOutputConfig.Outputs['METADATA'].S3Output.S3Uri`
`SIGNATURE`	S3Uri	`properties.ProcessingOutputConfig.Outputs['SIGNATURE'].S3Output.S3Uri`

Environment variables — optional: WRITE_DATA_SHARDS, SHARD_SIZE, OUTPUT_FORMAT, MAX_WORKERS, BATCH_SIZE, OPTIMIZE_MEMORY, STREAMING_BATCH_SIZE, ENABLE_TRUE_STREAMING, METADATA_FORMAT

DummyTraining¶

Dummy training step with flexible input modes. Adds hyperparameters.json to model.tar.gz for downstream packaging. Supports INTERNAL mode (accepts inputs) or SOURCE mode (reads from source directory).

node type internal · SageMaker Processing · entry point dummy_training.py

Dependencies (inputs)

logical name	required	data type	compatible sources
`hyperparameters_s3_uri`	no		HyperparameterPrep, ProcessingStep
`model_artifacts_input`	no		PyTorchTraining, XGBoostTraining, LightGBMTraining, ProcessingStep

Outputs

logical name	data type	property path
`model_output`	S3Uri	`properties.ProcessingOutputConfig.Outputs['model_output'].S3Output.S3Uri`

EdxUploading¶

EDX upload script. Uploads S3 data to EDX via EdxDataLoader. SINK node — data exits pipeline to EDX. No SageMaker outputs.

node type sink · SageMaker Processing · entry point edx_uploading.py

Dependencies (inputs)

logical name	required	data type	compatible sources
`input_data`	yes		CradleDataLoading, RedshiftDataLoading, TabularPreprocessing, StratifiedSampling, BedrockProcessing, ProcessingStep

Environment variables — required: EDX_DATASET_ARN, EDX_MANIFEST_KEY; optional: EDX_MANIFEST_KEY_PARTS, EDX_OUTPUT_COLUMNS

FeatureSelection¶

Feature selection script. Applies statistical and ML-based feature selection methods for dimensionality reduction. Training mode fits selectors; inference applies pre-computed.

node type internal · SageMaker Processing · entry point feature_selection.py

Dependencies (inputs)

logical name	required	data type	compatible sources
`input_data`	yes		TabularPreprocessing, StratifiedSampling, RiskTableMapping, MissingValueImputation, ProcessingStep
`model_artifacts_input`	no		FeatureSelection_Training, FeatureSelection, ProcessingStep

Outputs

logical name	data type	property path
`processed_data`	S3Uri	`properties.ProcessingOutputConfig.Outputs['processed_data'].S3Output.S3Uri`
`model_artifacts_output`	S3Uri	`properties.ProcessingOutputConfig.Outputs['model_artifacts_output'].S3Output.S3Uri`

Environment variables — required: LABEL_FIELD; optional: FEATURE_SELECTION_METHODS, N_FEATURES_TO_SELECT, CORRELATION_THRESHOLD, VARIANCE_THRESHOLD, RANDOM_STATE, COMBINATION_STRATEGY

Job arguments — --job_type

LabelRulesetExecution¶

Label ruleset execution script. Applies validated rulesets to processed data to generate classification labels using priority-based rule evaluation.

node type internal · SageMaker Processing · entry point label_ruleset_execution.py

Dependencies (inputs)

logical name	required	data type	compatible sources
`validated_ruleset`	yes		LabelRulesetGeneration
`input_data`	yes		TabularPreprocessing, BedrockProcessing, BedrockBatchProcessing, TemporalSequenceNormalization, TemporalFeatureEngineering, StratifiedSampling, MissingValueImputation, FeatureSelection, CurrencyConversion, RiskTableMapping

Outputs

logical name	data type	property path
`processed_data`	S3Uri	`properties.ProcessingOutputConfig.Outputs['processed_data'].S3Output.S3Uri`
`execution_report`	S3Uri	`properties.ProcessingOutputConfig.Outputs['execution_report'].S3Output.S3Uri`

Environment variables — optional: FAIL_ON_MISSING_FIELDS, ENABLE_RULE_MATCH_TRACKING, ENABLE_PROGRESS_LOGGING, PREFERRED_INPUT_FORMAT

Job arguments — --job-type

LabelRulesetGeneration¶

Label ruleset generation script. Validates and optimizes user-defined classification rules for transparent, maintainable rule-based classification.

node type internal · SageMaker Processing · entry point label_ruleset_generation.py

Dependencies (inputs)

logical name	required	data type	compatible sources
`ruleset_configs`	no		RulesetConfiguration, ProcessingStep

Outputs

logical name	data type	property path
`validated_ruleset`	S3Uri	`properties.ProcessingOutputConfig.Outputs['validated_ruleset'].S3Output.S3Uri`
`validation_report`	S3Uri	`properties.ProcessingOutputConfig.Outputs['validation_report'].S3Output.S3Uri`

Environment variables — optional: ENABLE_FIELD_VALIDATION, ENABLE_LABEL_VALIDATION, ENABLE_LOGIC_VALIDATION, ENABLE_RULE_OPTIMIZATION

LightGBMModelEval¶

LightGBM model evaluation script. Loads trained model, processes evaluation data, generates performance metrics and visualizations.

node type internal · SageMaker Processing · entry point lightgbm_model_eval.py

Dependencies (inputs)

logical name	required	data type	compatible sources
`model_input`	yes		LightGBMTraining, XGBoostTraining, PyTorchTraining, DummyTraining, LightGBMModel, XGBoostModel, PyTorchModel
`processed_data`	yes		TabularPreprocessing, CradleDataLoading, RiskTableMapping, CurrencyConversion

Outputs

logical name	data type	property path
`eval_output`	S3Uri	`properties.ProcessingOutputConfig.Outputs['eval_output'].S3Output.S3Uri`
`metrics_output`	S3Uri	`properties.ProcessingOutputConfig.Outputs['metrics_output'].S3Output.S3Uri`

Environment variables — optional: COMPARISON_MODE, PREVIOUS_SCORE_FIELD, STATISTICAL_TESTS, ID_FIELD, LABEL_FIELD, COMPARISON_METRICS, COMPARISON_PLOTS

Job arguments — --job_type

LightGBMModelInference¶

LightGBM model inference script. Loads trained model, preprocesses evaluation data, generates predictions without metrics computation.

node type internal · SageMaker Processing · entry point lightgbm_model_inference.py

Dependencies (inputs)

logical name	required	data type	compatible sources
`model_input`	yes		LightGBMTraining, XGBoostTraining, PyTorchTraining, DummyTraining, LightGBMModel, XGBoostModel, PyTorchModel
`processed_data`	yes		TabularPreprocessing, CradleDataLoading, RiskTableMapping, CurrencyConversion

Outputs

logical name	data type	property path
`eval_output`	S3Uri	`properties.ProcessingOutputConfig.Outputs['eval_output'].S3Output.S3Uri`

Environment variables — optional: OUTPUT_FORMAT, JSON_ORIENT, ID_FIELD, LABEL_FIELD

Job arguments — --job_type

LightGBMTraining¶

LightGBM training for tabular classification with risk table mapping and numerical imputation. Supports native categorical features, binary/multiclass, class weights, pre-computed artifacts, and comprehensive evaluation metrics.

node type internal · SageMaker Training · entry point lightgbm_training.py

Dependencies (inputs)

logical name	required	compatible sources
`input_path`	yes	TabularPreprocessing, BedrockProcessing, StratifiedSampling, RiskTableMapping, MissingValueImputation, ProcessingStep, DataLoad
`hyperparameters_s3_uri`	no	HyperparameterPrep, ProcessingStep
`model_artifacts_input`	no	LightGBMTraining, MissingValueImputation, RiskTableMapping, FeatureSelection

Outputs

logical name	data type	property path
`model_output`	S3Uri	`properties.ModelArtifacts.S3ModelArtifacts`
`evaluation_output`	S3Uri	`properties.OutputDataConfig.S3OutputPath`

Environment variables — optional: USE_SECURE_PYPI, USE_PRECOMPUTED_IMPUTATION, USE_PRECOMPUTED_RISK_TABLES, USE_PRECOMPUTED_FEATURES, USE_NATIVE_CATEGORICAL, REGION, CA_REPOSITORY_ARN

Job arguments — --job_type

LightGBMMTModelEval¶

LightGBMMT multi-task model evaluation. Generates per-task and aggregate metrics with visualizations.

node type internal · SageMaker Processing · entry point lightgbmmt_model_eval.py

Dependencies (inputs)

logical name	required	data type	compatible sources
`model_input`	yes		LightGBMMTTraining, LightGBMTraining, LightGBMMTModel, LightGBMModel, DummyTraining
`processed_data`	yes		TabularPreprocessing, CradleDataLoading, RiskTableMapping, CurrencyConversion, LabelRulesetExecution, BedrockBatchProcessing, BedrockProcessing, TemporalSplitPreprocessing

Outputs

logical name	data type	property path
`eval_output`	S3Uri	`properties.ProcessingOutputConfig.Outputs['eval_output'].S3Output.S3Uri`
`metrics_output`	S3Uri	`properties.ProcessingOutputConfig.Outputs['metrics_output'].S3Output.S3Uri`
`plots_output`	S3Uri	`properties.ProcessingOutputConfig.Outputs['plots_output'].S3Output.S3Uri`

Environment variables — required: TASK_LABEL_NAMES; optional: GENERATE_PLOTS, COMPARISON_MODE, STATISTICAL_TESTS, ID_FIELD, PREVIOUS_SCORE_FIELDS, COMPARISON_METRICS, COMPARISON_PLOTS, DOLLAR_COLUMN

Job arguments — --job_type

LightGBMMTModelInference¶

LightGBMMT multi-task model inference. Generates per-task predictions without evaluation, metrics, or plots.

node type internal · SageMaker Processing · entry point lightgbmmt_model_inference.py

Dependencies (inputs)

logical name	required	data type	compatible sources
`model_input`	yes		LightGBMMTTraining, LightGBMTraining, LightGBMMTModel, LightGBMModel, XGBoostTraining, PyTorchTraining, DummyTraining, XGBoostModel, PyTorchModel
`processed_data`	yes		TabularPreprocessing, CradleDataLoading, RiskTableMapping, CurrencyConversion, LabelRulesetExecution, BedrockBatchProcessing, BedrockProcessing

Outputs

logical name	data type	property path
`eval_output`	S3Uri	`properties.ProcessingOutputConfig.Outputs['eval_output'].S3Output.S3Uri`

Environment variables — optional: OUTPUT_FORMAT, JSON_ORIENT, ID_FIELD, TASK_LABEL_NAMES

Job arguments — --job_type

LightGBMMTTraining¶

LightGBMMT multi-task training for multi-label tabular classification with adaptive task weighting and knowledge distillation. Supports shared tree structures, JS-divergence weight adaptation, and per-task evaluation metrics.

node type internal · SageMaker Training · entry point lightgbmmt_training.py

Dependencies (inputs)

logical name	required	compatible sources
`input_path`	yes	TabularPreprocessing, StratifiedSampling, ProcessingStep, DataLoad, TemporalSplitPreprocessing
`hyperparameters_s3_uri`	no	HyperparameterPrep, ProcessingStep
`model_artifacts_input`	no	LightGBMMTTraining, MissingValueImputation, RiskTableMapping, FeatureSelection

Outputs

logical name	data type	property path
`model_output`	S3Uri	`properties.ModelArtifacts.S3ModelArtifacts`
`evaluation_output`	S3Uri	`properties.OutputDataConfig.S3OutputPath`

Job arguments — --job_type

MissingValueImputation¶

Missing value imputation script. Handles missing values using statistical methods (mean, median, mode, constant). Training mode fits imputers; inference applies pre-fitted. Per-column strategies can also be supplied dynamically via COLUMN_STRATEGY_<column_name> environment variables (discovered at runtime, not pre-declared).

node type internal · SageMaker Processing · entry point missing_value_imputation.py

Dependencies (inputs)

logical name	required	data type	compatible sources
`input_data`	yes		TabularPreprocessing, StratifiedSampling, RiskTableMapping, ProcessingStep
`model_artifacts_input`	no		MissingValueImputation_Training, ProcessingStep

Outputs

logical name	data type	property path
`processed_data`	S3Uri	`properties.ProcessingOutputConfig.Outputs['processed_data'].S3Output.S3Uri`
`model_artifacts_output`	S3Uri	`properties.ProcessingOutputConfig.Outputs['model_artifacts_output'].S3Output.S3Uri`

Environment variables — optional: DEFAULT_NUMERICAL_STRATEGY, DEFAULT_CATEGORICAL_STRATEGY, LABEL_FIELD, DEFAULT_TEXT_STRATEGY, NUMERICAL_CONSTANT_VALUE, CATEGORICAL_CONSTANT_VALUE, TEXT_CONSTANT_VALUE, CATEGORICAL_PRESERVE_DTYPE, AUTO_DETECT_CATEGORICAL, CATEGORICAL_UNIQUE_RATIO_THRESHOLD, VALIDATE_FILL_VALUES, EXCLUDE_COLUMNS, ENABLE_TRUE_STREAMING, MAX_WORKERS

Job arguments — --job_type

ModelCalibration¶

Model calibration step that calibrates raw prediction scores to true probabilities. Supports GAM, isotonic, and Platt methods. Handles binary, multi-class, and multi-task scenarios with per-task calibrators and aggregate metrics.

node type internal · SageMaker Processing · entry point model_calibration.py

Dependencies (inputs)

logical name	required	data type	compatible sources
`evaluation_data`	yes		XGBoostTraining, XGBoostModelEval, XGBoostModelInference, LightGBMTraining, LightGBMModelEval, LightGBMModelInference, LightGBMMTTraining, LightGBMMTModelEval, PyTorchTraining, PyTorchModelEval, PyTorchModelInference, ModelEvaluation, TrainingEvaluation, CrossValidation, XgboostMtModelEval

Outputs

logical name	data type	property path
`calibration_output`	S3Uri	`properties.ProcessingOutputConfig.Outputs['calibration_output'].S3Output.S3Uri`
`metrics_output`	S3Uri	`properties.ProcessingOutputConfig.Outputs['metrics_output'].S3Output.S3Uri`
`calibrated_data`	S3Uri	`properties.ProcessingOutputConfig.Outputs['calibrated_data'].S3Output.S3Uri`

Environment variables — optional: SCORE_FIELD, SCORE_FIELDS, TASK_LABEL_NAMES, MONOTONIC_CONSTRAINT, GAM_SPLINES, ERROR_THRESHOLD, NUM_CLASSES, SCORE_FIELD_PREFIX, MULTICLASS_CATEGORIES, CALIBRATION_SAMPLE_POINTS, CALIBRATION_METHOD, IS_BINARY, LABEL_FIELD

Job arguments — --job_type

ModelMetricsComputation¶

Model metrics computation script. Loads prediction data, computes comprehensive performance metrics, generates visualizations and detailed reports.

node type internal · SageMaker Processing · entry point model_metrics_computation.py

Dependencies (inputs)

logical name	required	data type	compatible sources
`eval_output`	yes		XGBoostModelInference, XGBoostModelEval, LightGBMMTModelInference, LightGBMModelInference, PyTorchModelInference, TabularPreprocessing, CradleDataLoading, RiskTableMapping, CurrencyConversion

Outputs

logical name	data type	property path
`metrics_output`	S3Uri	`properties.ProcessingOutputConfig.Outputs['metrics_output'].S3Output.S3Uri`
`plots_output`	S3Uri	`properties.ProcessingOutputConfig.Outputs['plots_output'].S3Output.S3Uri`

Environment variables — optional: AMOUNT_FIELD, GENERATE_PLOTS, ID_FIELD, LABEL_FIELD, INPUT_FORMAT, SCORE_FIELDS, SCORE_FIELD, TASK_LABEL_NAMES, PREVIOUS_SCORE_FIELDS, COMPUTE_DOLLAR_RECALL, COMPUTE_COUNT_RECALL, DOLLAR_RECALL_FPR, COUNT_RECALL_CUTOFF, COMPARISON_MODE, PREVIOUS_SCORE_FIELD, COMPARISON_METRICS, STATISTICAL_TESTS, COMPARISON_PLOTS

Job arguments — --job_type

ModelWikiGenerator¶

Model wiki generator script. Loads metrics and visualizations, generates comprehensive multi-format model documentation (Wiki, HTML, Markdown).

node type internal · SageMaker Processing · entry point model_wiki_generator.py

Dependencies (inputs)

logical name	required	data type	compatible sources
`metrics_output`	yes		ModelMetricsComputation, XGBoostModelEval, XGBoostModelInference, PyTorchModelInference
`plots_output`	no		ModelMetricsComputation, XGBoostModelEval, XGBoostModelInference, PyTorchModelInference

Outputs

logical name	data type	property path
`wiki_output`	S3Uri	`properties.ProcessingOutputConfig.Outputs['wiki_output'].S3Output.S3Uri`

Environment variables — required: MODEL_NAME; optional: MODEL_USE_CASE, MODEL_VERSION, PIPELINE_NAME, AUTHOR, TEAM_ALIAS, CONTACT_EMAIL, CTI_CLASSIFICATION, REGION, OUTPUT_FORMATS, INCLUDE_TECHNICAL_DETAILS, MODEL_DESCRIPTION, MODEL_PURPOSE

Package¶

MIMS packaging script that extracts model artifacts, includes calibration model if available, copies inference scripts, and creates a packaged model.tar.gz for deployment.

node type internal · SageMaker Processing · entry point package.py

Dependencies (inputs)

logical name	required	compatible sources
`model_input`	yes	XGBoostTraining, TrainingStep, ModelStep, PyTorchTraining, XgboostMtTraining, LightGBMMTTraining
`inference_scripts_input`	no	ProcessingStep, ScriptStep
`calibration_model`	no	ModelCalibration, PercentileModelCalibration

Outputs

logical name	data type	property path
`packaged_model`	S3Uri	`properties.ProcessingOutputConfig.Outputs['packaged_model'].S3Output.S3Uri`

Environment variables — optional: WORKING_DIRECTORY

Payload¶

MIMS payload generation script that extracts hyperparameters from model artifacts, detects model type (tabular/bimodal/trimodal), generates sample payloads with text field support, and archives payload files for deployment. Per-field overrides can also be supplied dynamically via SPECIAL_FIELD_<field_name> environment variables (discovered at runtime, not pre-declared).

node type internal · SageMaker Processing · entry point payload.py

Dependencies (inputs)

logical name	required	data type	compatible sources
`model_input`	yes		XGBoostTraining, LightGBMTraining, LightGBMMTTraining, PyTorchTraining, DummyTraining, TrainingStep, ModelStep, XgboostMtTraining
`custom_payload_input`	no		ProcessingStep, S3Source, UserProvided

Outputs

logical name	data type	property path
`payload_sample`	S3Uri	`properties.ProcessingOutputConfig.Outputs['payload_sample'].S3Output.S3Uri`

Environment variables — optional: CONTENT_TYPES, DEFAULT_NUMERIC_VALUE, DEFAULT_TEXT_VALUE, FIELD_DEFAULTS, WORKING_DIRECTORY

PercentileModelCalibration¶

Percentile model calibration that converts raw model scores to calibrated percentile values using ROC curve analysis. Supports single-task and multi-task calibration with configurable calibration dictionary.

node type internal · SageMaker Processing · entry point percentile_model_calibration.py

Dependencies (inputs)

logical name	required	data type	compatible sources
`evaluation_data`	yes		XGBoostTraining, XGBoostModelEval, XGBoostModelInference, LightGBMTraining, LightGBMModelEval, LightGBMModelInference, LightGBMMTTraining, LightGBMMTModelEval, PyTorchTraining, PyTorchModelEval, PyTorchModelInference, ModelEvaluation, TrainingEvaluation, CrossValidation, ModelCalibration
`calibration_config`	no		ConfigurationStep, DataPreprocessing, FeatureEngineering, ModelConfiguration

Outputs

logical name	data type	property path
`calibration_output`	S3Uri	`properties.ProcessingOutputConfig.Outputs['calibration_output'].S3Output.S3Uri`
`metrics_output`	S3Uri	`properties.ProcessingOutputConfig.Outputs['metrics_output'].S3Output.S3Uri`
`calibrated_data`	S3Uri	`properties.ProcessingOutputConfig.Outputs['calibrated_data'].S3Output.S3Uri`

Environment variables — optional: SCORE_FIELD, SCORE_FIELDS, N_BINS, ACCURACY

Job arguments — --job_type

PiperMetricGeneration¶

PIPER metric generation script. Loads prediction data, recomputes ROC and PR curves, and emits PIPER .metric JSON files with paired 2-column data CSVs written flat to the output root for PIPER rendering.

node type internal · SageMaker Processing · entry point piper_metric_generation.py

Dependencies (inputs)

logical name	required	data type	compatible sources
`eval_output`	yes		XGBoostModelInference, XGBoostModelEval, LightGBMMTModelInference, LightGBMModelInference, PyTorchModelInference, TabularPreprocessing, CradleDataLoading, RiskTableMapping, CurrencyConversion

Outputs

logical name	data type	property path
`metric_output`	S3Uri	`properties.ProcessingOutputConfig.Outputs['metric_output'].S3Output.S3Uri`

Environment variables — optional: ID_FIELD, LABEL_FIELD, SCORE_FIELD, SCORE_FIELDS, TASK_LABEL_NAMES, AMOUNT_FIELD, INPUT_FORMAT, COMPUTE_DOLLAR_RECALL, COMPUTE_COUNT_RECALL, DOLLAR_RECALL_FPR, COUNT_RECALL_CUTOFF, GENERATE_PLOTS, COMPARISON_MODE, PREVIOUS_SCORE_FIELD, PREVIOUS_SCORE_FIELDS, COMPARISON_METRICS, STATISTICAL_TESTS, COMPARISON_PLOTS, VARIANT_MODEL_ID, CONTROL_MODEL_ID, PIPELINE_NAME, DATASET_TYPE, METRICS_TO_RENDER

Job arguments — --job_type

PseudoLabelMerge¶

Pseudo label merge script. Intelligently merges labeled base data with pseudo-labeled or augmented samples for SSL and Active Learning workflows.

node type internal · SageMaker Processing · entry point pseudo_label_merge.py

Dependencies (inputs)

logical name	required	data type	compatible sources
`base_data`	yes		TabularPreprocessing, RiskTableMapping, MissingValueImputation, FeatureSelection, StratifiedSampling, TemporalSequenceNormalization, TemporalFeatureEngineering, LabelRulesetExecution
`augmentation_data`	yes		ActiveSampleSelection, XGBoostModelInference, LightGBMModelInference, PyTorchModelInference, XGBoostModelEval, LightGBMModelEval, PyTorchModelEval, BedrockBatchProcessing, BedrockProcessing, LabelRulesetExecution

Outputs

logical name	data type	property path
`merged_data`	S3Uri	`properties.ProcessingOutputConfig.Outputs['merged_data'].S3Output.S3Uri`

Environment variables — required: LABEL_FIELD; optional: ADD_PROVENANCE, OUTPUT_FORMAT, USE_AUTO_SPLIT_RATIOS, TRAIN_RATIO, TEST_VAL_RATIO, PSEUDO_LABEL_COLUMN, ID_FIELD, PRESERVE_CONFIDENCE, STRATIFY, RANDOM_SEED

Job arguments — --job_type

PyTorchModel¶

SageMaker Model creation step for PyTorch. No script — managed by SageMaker ModelStep. Creates a deployable model from training artifacts.

node type internal · SageMaker CreateModel · config PyTorchModelStepConfig

Dependencies (inputs)

logical name	required	data type	compatible sources
`model_data`	yes		PyTorchTraining, ProcessingStep, ModelArtifactsStep, Package

Outputs

logical name	data type	property path
`model_name`	S3Uri	`properties.ModelName`

PyTorchModelEval¶

PyTorch model evaluation script. Loads trained model, processes evaluation data, generates performance metrics and visualizations.

node type internal · SageMaker Processing · entry point pytorch_model_eval.py

Dependencies (inputs)

logical name	required	data type	compatible sources
`model_input`	yes		PyTorchTraining, XGBoostTraining, DummyTraining, PyTorchModel, XGBoostModel
`processed_data`	yes		TabularPreprocessing, CradleDataLoading, RiskTableMapping, CurrencyConversion, BedrockProcessing

Outputs

logical name	data type	property path
`eval_output`	S3Uri	`properties.ProcessingOutputConfig.Outputs['eval_output'].S3Output.S3Uri`
`metrics_output`	S3Uri	`properties.ProcessingOutputConfig.Outputs['metrics_output'].S3Output.S3Uri`

Environment variables — required: ID_FIELD, LABEL_FIELD; optional: COMPARISON_MODE, PREVIOUS_SCORE_FIELD, STATISTICAL_TESTS, COMPARISON_METRICS, COMPARISON_PLOTS, DEVICE, ACCELERATOR, BATCH_SIZE, NUM_WORKERS, ENABLE_TRUE_STREAMING, NUM_WORKERS_PER_RANK, PREFETCH_FACTOR, USE_PERSISTENT_WORKERS

Job arguments — --job_type

PyTorchModelInference¶

PyTorch model inference script. Loads trained model, preprocesses evaluation data, generates predictions without metrics computation.

node type internal · SageMaker Processing · entry point pytorch_model_inference.py

Dependencies (inputs)

logical name	required	data type	compatible sources
`model_input`	yes		PyTorchTraining, XGBoostTraining, DummyTraining, PyTorchModel, XGBoostModel
`processed_data`	yes		TabularPreprocessing, CradleDataLoading, RiskTableMapping, CurrencyConversion, BedrockProcessing

Outputs

logical name	data type	property path
`eval_output`	S3Uri	`properties.ProcessingOutputConfig.Outputs['eval_output'].S3Output.S3Uri`

Environment variables — optional: ID_FIELD, DEVICE, LABEL_FIELD, EMBEDDING_MODE

Job arguments — --job_type

PyTorchTraining¶

PyTorch Lightning training for multimodal (text+tabular) models. Supports BERT, CNN, LSTM, multimodal variants. Handles binary/multiclass classification with early stopping, checkpointing, ONNX export, and streaming mode for memory-efficient loading.

node type internal · SageMaker Training · entry point pytorch_training.py

Dependencies (inputs)

logical name	required	compatible sources
`input_path`	yes	TabularPreprocessing, BedrockProcessing, StratifiedSampling, RiskTableMapping, MissingValueImputation, LabelRulesetExecution, ProcessingStep, DataLoad
`hyperparameters_s3_uri`	no	HyperparameterPrep, ProcessingStep
`model_artifacts_input`	no	PyTorchTraining, TokenizerTraining, MissingValueImputation, RiskTableMapping, FeatureSelection

Outputs

logical name	data type	property path
`model_output`	S3Uri	`properties.ModelArtifacts.S3ModelArtifacts`
`evaluation_output`	S3Uri	`properties.OutputDataConfig.S3OutputPath`

Environment variables — optional: USE_PRECOMPUTED_IMPUTATION, USE_PRECOMPUTED_RISK_TABLES, ENABLE_TRUE_STREAMING, NUM_WORKERS_PER_RANK, PREFETCH_FACTOR, USE_PERSISTENT_WORKERS, REGION

Job arguments — --job_type

RedshiftDataLoading¶

Redshift data loading script. Source node that executes SQL against Redshift and writes results as CSV to S3. Optionally uploads to EDX as side effect.

node type source · SageMaker RedshiftDataLoading · entry point redshift_data_loading.py

Outputs

logical name	data type	property path
`output`	S3Uri	`properties.ProcessingOutputConfig.Outputs['output'].S3Output.S3Uri`

Registration¶

MIMS model registration script that uploads model artifacts and payload samples, registers the model with MIMS service, tracks workflow execution ID, and cleans up temporary resources. No output files produced - registration is a side effect.

node type sink · SageMaker MimsModelRegistrationProcessing · entry point script.py

Dependencies (inputs)

logical name	required	data type	compatible sources
`PackagedModel`	yes		PackagingStep, Package, ProcessingStep
`GeneratedPayloadSamples`	yes		Payload, PayloadTestStep, PayloadStep, ProcessingStep

Environment variables — optional: PERFORMANCE_METADATA_PATH

RiskTableMapping¶

Risk table mapping script. Creates risk tables for categorical features and handles missing value imputation for numeric features. Training mode creates tables; inference applies.

node type internal · SageMaker Processing · entry point risk_table_mapping.py

Dependencies (inputs)

logical name	required	compatible sources
`input_data`	yes	TabularPreprocessing, MissingValueImputation, ProcessingStep
`hyperparameters_s3_uri`	no	HyperparameterPrep, ProcessingStep, ConfigurationStep
`model_artifacts_input`	no	RiskTableMapping_Training, ProcessingStep

Outputs

logical name	data type	property path
`processed_data`	S3Uri	`properties.ProcessingOutputConfig.Outputs['processed_data'].S3Output.S3Uri`
`model_artifacts_output`	S3Uri	`properties.ProcessingOutputConfig.Outputs['model_artifacts_output'].S3Output.S3Uri`

Environment variables — optional: SMOOTH_FACTOR, COUNT_THRESHOLD, MAX_UNIQUE_THRESHOLD, ENABLE_TRUE_STREAMING, MAX_WORKERS

Job arguments — --job_type

SOPAInstructionTuning¶

SOPA Stage 2 instruction fine-tuning script for AFN Return MDR model that fine-tunes a BLIP2-based model (Q-Former + Phi-3 LLM) for tabular-to-text instruction following. Loads a pre-trained Phi-3 LLM (frozen), a Stage 0 tabular autoencoder (frozen encoder), and a Stage 1 Q-Former (trainable), then trains a projection layer (llm_proj) and the Q-Former to align tabular embeddings with the LLM input space. Supports three tasks (return_risk, customer_risk, refund_decision) and saves best-only and final model checkpoints (stage2_{task}best.pth, stage2{task}_final.pth). All configuration is provided via argparse arguments; no environment variables are required.

node type internal · SageMaker Training · entry point SOPA_instruction_tuning.py

Dependencies (inputs)

logical name	required	data type	compatible sources
`data_path`	no	S3Uri	TabularPreprocessing, ProcessingStep, DataLoad
`llm_model_path`	no	S3Uri	ProcessingStep, DataLoad
`tabular_encoder_path`	no	S3Uri	ProcessingStep, DataLoad
`stage1_qformer_path`	no	S3Uri	ProcessingStep, DataLoad

Outputs

logical name	data type	property path
`model_output`	S3Uri	`properties.ModelArtifacts.S3ModelArtifacts`
`checkpoints_output`	S3Uri	`properties.OutputDataConfig.S3OutputPath`

StratifiedSampling¶

Stratified sampling with four allocation strategies (balanced, proportional_min, optimal, external_proportional). Handles class imbalance correction, causal analysis, and variance optimization with per-split diagnostics.

node type internal · SageMaker Processing · entry point stratified_sampling.py

Dependencies (inputs)

logical name	required	data type	compatible sources
`input_data`	yes		TabularPreprocessing, ProcessingStep

Outputs

logical name	data type	property path
`processed_data`	S3Uri	`properties.ProcessingOutputConfig.Outputs['processed_data'].S3Output.S3Uri`

Environment variables — required: STRATA_COLUMN; optional: SAMPLING_STRATEGY, TARGET_SAMPLE_SIZE, MIN_SAMPLES_PER_STRATUM, VARIANCE_COLUMN, RANDOM_STATE, SAMPLING_MULTIPLIER, ALLOW_REPLACEMENT, REFERENCE_COUNTS_JSON, SAMPLING_FILTER_COLUMN, SAMPLING_FILTER_VALUE

Job arguments — --job_type

TabularPreprocessing¶

Tabular preprocessing script that combines data shards, loads column signature, cleans/processes label field, splits data into train/test/val, and outputs in configurable format (CSV/TSV/Parquet). Supports streaming mode for large datasets.

node type internal · SageMaker Processing · entry point tabular_preprocessing.py

Dependencies (inputs)

logical name	required	compatible sources
`DATA`	yes	CradleDataLoading, DummyDataLoading, RedshiftDataLoading, DataLoad, ProcessingStep, BedrockProcessing, StratifiedSampling
`DATA_SECONDARY`	no	CradleDataLoading, DummyDataLoading, RedshiftDataLoading, DataLoad, ProcessingStep, BedrockProcessing, StratifiedSampling
`SIGNATURE`	no	CradleDataLoading, DummyDataLoading

Outputs

logical name	data type	property path
`processed_data`	S3Uri	`properties.ProcessingOutputConfig.Outputs['processed_data'].S3Output.S3Uri`

Environment variables — optional: LABEL_FIELD, OUTPUT_FORMAT, MAX_WORKERS, BATCH_SIZE, OPTIMIZE_MEMORY, STREAMING_BATCH_SIZE, ENABLE_TRUE_STREAMING, TRAIN_RATIO, TEST_VAL_RATIO

Job arguments — --job_type

TemporalFeatureEngineering¶

Temporal feature engineering script. Extracts comprehensive temporal features from normalized sequence data combining generic temporal features with time window aggregations.

node type internal · SageMaker Processing · entry point temporal_feature_engineering.py

Dependencies (inputs)

logical name	required	data type	compatible sources
`normalized_sequences`	yes		TemporalSequenceNormalization, ProcessingStep

Outputs

logical name	data type	property path
`temporal_feature_tensors`	S3Uri	`properties.ProcessingOutputConfig.Outputs['temporal_feature_tensors'].S3Output.S3Uri`

Environment variables — optional: FEATURE_TYPES, CATEGORICAL_FIELDS, WINDOW_SIZES, AGGREGATION_FUNCTIONS, SEQUENCE_GROUPING_FIELD, TIMESTAMP_FIELD, VALUE_FIELDS, LAG_FEATURES, EXPONENTIAL_SMOOTHING_ALPHA, TIME_UNIT, INPUT_FORMAT, OUTPUT_FORMAT, ENABLE_VALIDATION, MISSING_VALUE_THRESHOLD, CORRELATION_THRESHOLD, VARIANCE_THRESHOLD, OUTLIER_DETECTION

Job arguments — --job_type

TemporalSequenceNormalization¶

Temporal sequence normalization script. Handles temporal sequence data loading, validation, normalization, and padding/truncation for ML models.

node type internal · SageMaker Processing · entry point temporal_sequence_normalization.py

Dependencies (inputs)

logical name	required	data type	compatible sources
`DATA`	yes		CradleDataLoading, DummyDataLoading, DataLoad, ProcessingStep, TabularPreprocessing
`SIGNATURE`	no		CradleDataLoading, DummyDataLoading

Outputs

logical name	data type	property path
`normalized_sequences`	S3Uri	`properties.ProcessingOutputConfig.Outputs['normalized_sequences'].S3Output.S3Uri`

Environment variables — optional: SEQUENCE_LENGTH, PADDING_STRATEGY, SEQUENCE_GROUPING_FIELD, TEMPORAL_FIELD, SEQUENCE_SEPARATOR, RECORD_ID_FIELD, MISSING_INDICATORS, TIME_DELTA_MAX_SECONDS, TRUNCATION_STRATEGY, ENABLE_MULTI_SEQUENCE, SECONDARY_ENTITY_FIELD, SEQUENCE_NAMING_PATTERN, VALIDATION_STRATEGY, OUTPUT_FORMAT, INCLUDE_ATTENTION_MASKS

Job arguments — --job_type

TemporalSplitPreprocessing¶

Temporal split preprocessing script. Handles data loading, temporal splitting, customer-level splitting, and main task label generation.

node type internal · SageMaker Processing · entry point temporal_split_preprocessing.py

Dependencies (inputs)

logical name	required	data type	compatible sources
`DATA`	yes		CradleDataLoading, DummyDataLoading, DataLoad, ProcessingStep
`SIGNATURE`	no		CradleDataLoading, DummyDataLoading

Outputs

logical name	data type	property path
`training_data`	S3Uri	`properties.ProcessingOutputConfig.Outputs['training_data'].S3Output.S3Uri`
`oot_data`	S3Uri	`properties.ProcessingOutputConfig.Outputs['oot_data'].S3Output.S3Uri`

Environment variables — required: DATE_COLUMN, GROUP_ID_COLUMN, SPLIT_DATE; optional: TRAIN_RATIO, RANDOM_SEED, OUTPUT_FORMAT, LABEL_FIELD, JOB_TYPE, MAX_WORKERS, BATCH_SIZE, ENABLE_TRUE_STREAMING, TARGETS, MAIN_TASK_INDEX

Job arguments — --job_type

TokenizerTraining¶

Tokenizer training script. Trains custom BPE tokenizer optimized for customer name data with automatic vocabulary size tuning.

node type internal · SageMaker Processing · entry point tokenizer_training.py

Dependencies (inputs)

logical name	required	data type	compatible sources
`input_data`	yes		TabularPreprocessing, ProcessingStep

Outputs

logical name	data type	property path
`model_artifacts_output`	S3Uri	`properties.ProcessingOutputConfig.Outputs['model_artifacts_output'].S3Output.S3Uri`

Environment variables — optional: TARGET_COMPRESSION, MIN_FREQUENCY, MAX_VOCAB_SIZE, TEXT_FIELD

Job arguments — --job_type

TSAModelCalibration¶

Contract for TSA model calibration processing step. The TSA model calibration step implements monotone B-spline calibration specifically for Temporal Self-Attention fraud detection models. It converts raw model prediction scores into well-calibrated probabilities using constrained optimization, which is essential for risk-based decision-making and reliable fraud detection thresholds. This calibration method is based on the generic_rfuge.r approach and uses: - B-spline basis functions with adaptive knot placement - Monotonicity constraints to ensure score ordering preservation - Iterative reweighted least squares (IRLS) with quadratic programming - Emphasis on high-score regions (90th-100th percentile) for fraud detection Input Structure: - /opt/ml/processing/input/eval_data: Evaluation dataset with ground truth labels and model predictions * Supports multiple formats: CSV, TSV, Parquet * Can handle nested tarballs from SageMaker training job outputs * Expected columns: label field (ground truth) and score field (raw predictions) Output Structure: - /opt/ml/processing/output/calibration: Calibration model artifacts * calibration_model.pkl: Pickled B-spline calibrator (for backward compatibility) * tsa_bspline_calibrator.json: JSON format calibrator (for inspection) * tsa_calibration_summary.json: Summary of calibration results - /opt/ml/processing/output/metrics: Calibration quality metrics and visualizations * tsa_calibration_metrics.json: Comprehensive metrics (ECE, MCE, Brier score, AUC) * tsa_reliability_diagram.png: Visual comparison of uncalibrated vs calibrated - /opt/ml/processing/output/calibrated_data: Dataset with calibrated probabilities * Original format preserved (CSV/TSV/Parquet) * New column: calibrated_{SCORE_FIELD} * All original columns retained Command-Line Arguments: - job-type: Determines data loading strategy * “training”: Uses nested tarball extraction for training job outputs * “calibration”/”validation”/”testing”: Uses standard data loading Environment Variables (Required): - CALIBRATION_METHOD: Calibration method to use (currently only “bspline” supported) - LABEL_FIELD: Name of the ground truth label column (e.g., “is_abusive_mdr”) - SCORE_FIELD: Name of the raw prediction score column (e.g., “prob_class_1”) Environment Variables (Optional - B-spline Configuration): - BSPLINE_DEGREE: Degree of B-spline basis functions (default: 3 for cubic splines) - ADAPTIVE_KNOTS: Whether to use adaptive knot placement based on data size (default: True) - BASE_KNOTS: Fixed number of knots to use (overrides adaptive if set) Environment Variables (Optional - Quality Thresholds): - MIN_RECORDS: Minimum number of records required for calibration (default: 1000) - MIN_FRAUD: Minimum number of fraud cases required (default: 10) - MAX_COEF_THRESHOLD: Maximum acceptable coefficient magnitude (default: 1e12) - MIN_UNIQUE_VALUES: Minimum unique calibrated predictions required (default: 10) Environment Variables (Optional - Optimization Parameters): - LAMBDA_SMOOTH: Smoothness penalty for P-spline regularization (default: 1e-10) - MAX_ITER: Maximum iterations for IRLS optimization (default: 1000) - TOLERANCE: Convergence tolerance for coefficient updates (default: 1e-6) Infrastructure: - USE_SECURE_PYPI: Whether to use secure CodeArtifact PyPI for package installation (default: false) Key Features: - Monotone B-spline calibration preserves score ordering - Adaptive knot placement with emphasis on high-score regions - Format preservation for input/output data (CSV/TSV/Parquet) - Nested tarball support for SageMaker training job outputs - Comprehensive metrics: ECE, MCE, Brier score, AUC - Visual reliability diagrams for calibration quality assessment - Quality validation with automatic status determination Calibration Quality Metrics: - Expected Calibration Error (ECE): Average calibration error across bins - Maximum Calibration Error (MCE): Worst-case calibration error - Brier Score: Mean squared difference between predictions and outcomes - AUC-ROC: Area under receiver operating characteristic curve - Model MSE: Mean squared error of fitted B-spline - Coefficient magnitude: Maximum absolute coefficient value - Unique predictions: Number of distinct calibrated probabilities Success Criteria: - Convergence: IRLS optimization converges within MAX_ITER iterations - No NaN coefficients: All B-spline coefficients are finite - Sufficient unique values: At least MIN_UNIQUE_VALUES distinct predictions - MSE improvement: Model MSE better than baseline (mean prediction) - Coefficient stability: Maximum coefficient below MAX_COEF_THRESHOLD Supported Job Types: - training: Extracts data from nested tarballs (output.tar.gz -> val.tar.gz/test.tar.gz) - calibration: Standard data loading for dedicated calibration datasets - validation: Standard data loading for validation datasets - testing: Standard data loading for test datasets Performance Optimizations: The script includes transparent I/O optimizations that automatically improve performance without requiring any configuration changes: - PyArrow-based Parquet I/O: * Uses PyArrow engine for Parquet files when available (30-50% faster loading) * Writes Parquet with Snappy compression (40-60% smaller files, 20-30% faster) * Automatic fallback to default pandas engine if PyArrow unavailable * No configuration required - optimizations are transparent - Enhanced Logging: * Tracks file sizes, row counts, and column counts during I/O operations * Reports detected file formats and compression ratios * Provides visibility into I/O performance - Format Preservation: * Input format automatically detected (CSV, TSV, or Parquet) * Output saved in same format as input for consistency * Parquet format recommended for best performance (2x faster, 60% smaller) Expected Performance: - Small datasets (<100K rows): 20-40% faster I/O - Medium datasets (1M rows): 30-50% faster I/O, 40-60% smaller files - Large datasets (>10M rows): 40-60% faster I/O, significant memory savings The optimizations primarily benefit data I/O operations. Calibration optimization itself (B-spline fitting) remains CPU-bound and represents the main processing time. Overall speedup for typical calibration workflows: ~30-40% faster end-to-end.

node type internal · SageMaker Processing · entry point tsa_model_calibration.py

Dependencies (inputs)

logical name	required	data type	compatible sources
`preprocessor_input`	yes	S3Uri	TabularPreprocessing, TSAPreprocessing, TSATabularPreprocessing, DataPreprocessing
`evaluation_data`	yes	S3Uri	TSATraining, TSAModelEval, PyTorchTraining, PyTorchModelEval, XGBoostTraining, XGBoostModelEval, LightGBMTraining, LightGBMModelEval, ModelEvaluation, TrainingEvaluation

Outputs

logical name	data type	property path
`calibration_output`	S3Uri	`properties.ProcessingOutputConfig.Outputs['calibration_output'].S3Output.S3Uri`
`metrics_output`	S3Uri	`properties.ProcessingOutputConfig.Outputs['metrics_output'].S3Output.S3Uri`
`calibrated_data`	S3Uri	`properties.ProcessingOutputConfig.Outputs['calibrated_data'].S3Output.S3Uri`

Environment variables — required: CALIBRATION_METHOD, LABEL_FIELD, SCORE_FIELD; optional: BSPLINE_DEGREE, ADAPTIVE_KNOTS, BASE_KNOTS, MIN_RECORDS, MIN_FRAUD, LAMBDA_SMOOTH, MAX_ITER, TOLERANCE, MAX_COEF_THRESHOLD, MIN_UNIQUE_VALUES, USE_SECURE_PYPI

TSAModelEval¶

TSA model evaluation script that: 1. Loads trained TSA PyTorch model and configuration from model directory 2. Loads and processes evaluation data from numpy arrays (sequences + static features) 3. Generates predictions for dual-task learning objectives 4. Computes comprehensive performance metrics with business impact analysis 5. Creates visualizations comparing task performance 6. Saves predictions and metrics to separate output directories Input Structure: - /opt/ml/processing/input/model: Model artifacts directory containing: - model.pth or model.pt: Trained PyTorch model state dict - hyperparameters.json: Complete model configuration and hyperparameters - Contains model architecture parameters (n_cat_features, n_num_features, etc.) - /opt/ml/processing/input/eval_data: Evaluation data directory containing numpy arrays: - {prefix}X_num{version}.npy: Static numerical features - {prefix}cid_X_seq_num{version}.npy: Customer ID numerical sequences - {prefix}cid_X_seq_cat{version}.npy: Customer ID categorical sequences - {prefix}Y{version}.npy: Multi-column labels and amounts * Format: [task1_label, task2_label, …, amount] * Last column: transaction amounts for dollar-weighted metrics * Label columns: binary labels (0/1) for each task - Prefix can be empty or “cid_” (script handles both) Standard Output Structure: - /opt/ml/processing/output/eval: Model predictions (numpy arrays) - scores1.npy: Task 1 prediction scores (probabilities) - labels1.npy: Task 1 true labels - scores2.npy: Task 2 prediction scores (probabilities) - labels2.npy: Task 2 true labels - amounts.npy: Transaction amounts for business metrics - /opt/ml/processing/output/metrics: Performance metrics and visualizations - {test_name}_metrics.json: Comprehensive performance metrics - {test_name}_report.txt: Human-readable metrics summary - {test_name}_evaluation.png: Visualization comparing both tasks - _SUCCESS: Success marker file (created on successful completion) Metrics Computed (Dual-Task Performance): Task 1 Metrics (Primary Task): - Binary classification metrics: * auc1: Area under ROC curve * precision1: Precision score * recall1: Recall score * dollar_recall1: Dollar-weighted recall using transaction amounts Task 2 Metrics (Secondary Task): - Binary classification metrics: * auc2: Area under ROC curve * precision2: Precision score * recall2: Recall score * dollar_recall2: Dollar-weighted recall using transaction amounts Aggregate Metrics: - auc_avg: Average AUC across both tasks - loss: Average evaluation loss Visualization Components: - AUC Comparison: Histogram comparing Task 1 vs Task 2 AUC scores - Recall Comparison: Bar chart comparing recall metrics - Precision Comparison: Bar chart comparing precision metrics - Dollar Recall Comparison: Bar chart comparing business impact metrics Required Environment Variables: None (all have defaults) Optional Environment Variables: - DATA_VERSION: Version suffix for numpy array files (default: “v0”) * Locates files like X_num_v0.npy, Y_v0.npy - ID_FIELD: Name of ID field for output formatting (default: “id”) - LABEL_FIELD: Name of label field for output formatting (default: “label”) - USE_SECURE_PYPI: Use secure CodeArtifact PyPI during setup (default: “false”) - LOCAL_RANK: Distributed training rank for multi-GPU evaluation (default: “-1”) * Set to 0+ for distributed evaluation * -1 for single-GPU or CPU evaluation - ENABLE_EVAL_STREAMING: Enable two-pass streaming evaluation mode (default: “false”) * Set to “true” for streaming mode with 30-40% speedup and 50% memory savings * Uses memory-mapped files for incremental prediction storage * Pass 1: Stream predictions to disk during inference * Pass 2: Load predictions from disk for metrics computation * Benefits: Faster evaluation, lower memory, handles 2-3x larger datasets * Maintains 100% metric accuracy (identical results to non-streaming) - ENABLE_AMP: Enable mixed precision (AMP) for GPU inference (default: “true”) * Automatic on CUDA devices, provides 2-3x speedup * Uses torch.cuda.amp.autocast() for faster inference * No accuracy loss - predictions remain identical - NUM_WORKERS: Number of parallel data loading workers (default: “4”) * Set to 0 to disable parallel loading (single-threaded) * 4 workers recommended for production (30-50% faster I/O) * 2 workers for testing/debugging - ENABLE_CPU_OPTIMIZATION: Enable CPU-specific optimizations (default: “true”) * Provides 2-5x faster evaluation on CPU-only systems * Automatically detects CPU and applies optimizations: · Intel MKL threading (20-30% speedup): Optimizes thread count and enables MKL-DNN · TorchScript JIT compilation (30-50% speedup): Compiles model to optimized machine code · Optimized batch sizing (10-20% speedup): Reduces batch size to 64 for better CPU cache utilization · CPU-specific data loading (5-10% speedup): Disables pin_memory for faster CPU data transfer * Set to “false” to disable for baseline comparison or troubleshooting * Has no effect on GPU systems (GPU optimizations used instead) * Graceful fallback if any optimization fails - EVAL_BATCH_SIZE: Override batch_size for evaluation (default: auto-detect or use hyperparameters.json) * Independent from training batch_size - allows larger batches for faster evaluation * Auto-detects optimal size based on instance type if not specified: · ml.p3.16xlarge (8x V100): 512 per GPU, effective 4096 with DDP · ml.p3.8xlarge (4x V100): 512 per GPU, effective 2048 with DDP · ml.p3.2xlarge (1x V100): 256-512 · ml.g5.16xlarge (1x A10G): 256-512 · ml.g4dn.xlarge (1x T4): 128-256 · CPU instances: 64-128 (cache-friendly) * Falls back to hyperparameters.json batch_size if not set (typically 2 for training) * Larger batches dramatically reduce overhead: 512 vs 2 = 256x fewer iterations * Performance impact: 18-36x faster evaluation for large datasets (1M+ samples) * Set explicitly to override auto-detection (e.g., “512”, “256”, “128”) * Leave empty (“”) to use auto-detection or hyperparameters.json fallback Arguments: - –job_type: Type of evaluation job to perform (e.g., “evaluation”) Model Architecture Support: - OrderFeatureAttentionClassifier: Temporal attention model for sequence analysis - Dual-task learning with shared representations - Supports categorical and numerical sequence features - Static feature integration for enhanced predictions Data Loading Details: - Memory-mapped numpy arrays (mmap_mode=”r+”) for efficient large dataset handling - Flexible file naming: supports both “X_num” and “cid_X_num” prefixes - Label extraction: First N-1 columns are task labels, last column is amounts - Empty string amounts converted to 0.0 - Batch processing with configurable batch_size from hyperparameters Distributed Evaluation Support: - Multi-GPU evaluation via DistributedDataParallel (DDP) - Automatic result aggregation across processes - Synchronized batch normalization for consistency - Results saved only from rank 0 to avoid conflicts Performance Considerations: - GPU acceleration when CUDA available - Memory-mapped arrays for handling large datasets - Efficient batch processing with DataLoader - Progress logging every 100 batches - Streaming mode optimization (ENABLE_EVAL_STREAMING=true): * Two-pass evaluation: inference → disk → metrics * 30-40% faster evaluation with 50% lower memory * Handles 2-3x larger evaluation datasets * Maintains 100% metric accuracy * Creates temporary streaming_temp/ directory in eval_output * Recommended for large datasets (>1M samples) or memory-constrained environments - Mixed Precision (AMP) optimization (ENABLE_AMP=true, default enabled): * Automatic mixed precision for 2-3x faster GPU inference * Enabled by default on CUDA devices * Uses torch.cuda.amp.autocast() during forward pass * Zero accuracy loss - identical predictions to FP32 * Can be disabled by setting ENABLE_AMP=false - Parallel data loading (NUM_WORKERS=4, default): * 4 parallel workers for faster data I/O (30-50% speedup) * Includes pinned memory for faster GPU transfer * Persistent workers to avoid startup overhead * Prefetching for better pipeline utilization * Set NUM_WORKERS=0 to disable for debugging Error Handling: - Validates model file existence (.pth or .pt) - Validates evaluation data directory existence - Comprehensive error logging with stack traces - Creates _FAILURE marker on errors for pipeline monitoring - Exits with code 1 on failure, 0 on success Integration Notes: - Output structure matches PyTorch model eval contract - Predictions saved to eval_output (separate from metrics) - Metrics and visualizations saved to metrics_output - Success/failure markers enable downstream monitoring - Compatible with Cursus pipeline orchestration The script uses the same contract signature as PyTorch model eval: - main(input_paths, output_paths, environ_vars, job_args) - Loads config and model internally (not passed as parameters) - Returns None (void function) - All logging via CloudWatch-compatible logger

node type internal · SageMaker Processing · entry point tsa_model_eval.py

Dependencies (inputs)

logical name	required	data type	compatible sources
`model_input`	yes	S3Uri	TSATraining, PyTorchTraining, DummyTraining, PyTorchModel
`processed_data`	yes	S3Uri	TSAPreprocessing, TabularPreprocessing, ProcessingStep

Outputs

logical name	data type	property path
`eval_output`	S3Uri	`properties.ProcessingOutputConfig.Outputs['eval_output'].S3Output.S3Uri`
`metrics_output`	S3Uri	`properties.ProcessingOutputConfig.Outputs['metrics_output'].S3Output.S3Uri`

Environment variables — optional: DATA_VERSION, ID_FIELD, LABEL_FIELD, USE_SECURE_PYPI, LOCAL_RANK, ENABLE_EVAL_STREAMING, ENABLE_AMP, NUM_WORKERS, ENABLE_CPU_OPTIMIZATION, EVAL_BATCH_SIZE

TSAPreprocessing¶

TSA preprocessing script that performs CID sequence processing for fraud detection. It loads model artifacts (preprocessor, categorical mappings, default values, Python modules), loads and combines data from tabular preprocessing output, processes Customer ID (CID) sequences, applies feature transformations, scaling, and categorical encoding, handles time windowing and downsampling for different dataset types, and outputs numpy arrays for TSA model training. Inputs are artifacts (model artifacts read from /opt/ml/processing/input/code/artifacts), preprocessor (optional training-fitted scaling parameters, used via PREPROCESSOR_PATH when provided), and processed_data (tabular preprocessing output with train/test/val splits). Output is tsa_processed_data (5 numpy arrays per dataset - CID categorical sequences, CID numerical sequences, static features, labels, amounts). Supports streaming mode (ENABLE_TSA_STREAMING) for memory-efficient processing of large datasets.

node type internal · SageMaker Processing · entry point tsa_preprocessing.py

Dependencies (inputs)

logical name	required	data type	compatible sources
`artifacts`	no	S3Uri	CradleDataLoading, DummyDataLoading, DataLoad, ProcessingStep, TabularPreprocessing
`preprocessor`	no	S3Uri	TSATabularPreprocessing, ProcessingStep
`processed_data`	yes	S3Uri	TabularPreprocessing, TSATabularPreprocessing, CradleDataLoading, DummyDataLoading, DataLoad, ProcessingStep

Outputs

logical name	data type	property path
`tsa_processed_data`	S3Uri	`properties.ProcessingOutputConfig.Outputs['tsa_processed_data'].S3Output.S3Uri`

Environment variables — optional: REGION, TAG, TAG2, TARGET_POSITIVE_RATE, TIME_WINDOW_TRAIN, TIME_WINDOW_CALIB, TIME_WINDOW_VALI, AMOUNT_FIELD, PREPROCESSOR_PATH, TRAINING_PREPROCESSOR_PATH, ENABLE_TSA_STREAMING, TSA_STREAMING_BATCH_SIZE, VALIDATION_SPLIT_RATIO

TSATabularPreprocessing¶

TSA tabular preprocessing script that combines data shards, loads column signature, applies TSA-domain feature engineering (label encoding, ID field handling, date-based feature extraction), splits data into train/test/val, serialises the fitted sklearn preprocessor pipeline to preprocessor.pkl, and outputs both processed CSV and the preprocessor artifact. Supports streaming mode for large datasets.

node type internal · SageMaker Processing · entry point tsa_tabular_preprocessing.py

Dependencies (inputs)

logical name	required	compatible sources
`DATA`	yes	CradleDataLoading, DummyDataLoading, RedshiftDataLoading, DataLoad, ProcessingStep, BedrockProcessing, StratifiedSampling
`DATA_SECONDARY`	no	CradleDataLoading, DummyDataLoading, RedshiftDataLoading, DataLoad, ProcessingStep, BedrockProcessing, StratifiedSampling
`SIGNATURE`	no	CradleDataLoading, DummyDataLoading

Outputs

logical name	data type	property path
`tsa_processed_data`	S3Uri	`properties.ProcessingOutputConfig.Outputs['tsa_processed_data'].S3Output.S3Uri`
`preprocessor`	S3Uri	`properties.ProcessingOutputConfig.Outputs['preprocessor'].S3Output.S3Uri`

Environment variables — optional: TSA_LABEL_FIELD, TSA_ID_FIELDS, TSA_DATE_FIELD, OUTPUT_FORMAT, MAX_WORKERS, BATCH_SIZE, OPTIMIZE_MEMORY, STREAMING_BATCH_SIZE, ENABLE_TRUE_STREAMING, TRAIN_RATIO, TEST_VAL_RATIO

Job arguments — --job_type, --label-field, --id-fields, --date-field, --preprocessor-output-path

TSATraining¶

TSA (Temporal Self-Attention) training script for AFN Return Kickout model that: 1. Loads pre-processed training data from TSA preprocessing output (4 numpy arrays) 2. Builds temporal attention-based neural network model (OrderFeatureAttentionClassifier) 3. Supports distributed training with PyTorch DDP (DistributedDataParallel) 4. Trains model with configurable hyperparameters including focal loss support 5. Implements OneCycleLR learning rate scheduling 6. Saves training checkpoints periodically 7. Generates training loss plots for monitoring 8. Saves trained model with all artifacts following standard pattern (model.tar.gz) 9. Supports region-specific hyperparameters (NA, EU, FE) via REGION environment variable

node type internal · SageMaker Training · entry point tsa_training.py

Dependencies (inputs)

logical name	required	data type	compatible sources
`input_path`	yes	S3Uri	TSAPreprocessing, TemporalSequenceNormalization, TemporalFeatureEngineering, TabularPreprocessing, ProcessingStep, DataLoad
`hyperparameters_s3_uri`	no	S3Uri	HyperparameterPrep, ProcessingStep

Outputs

logical name	data type	property path
`model_output`	S3Uri	`properties.ModelArtifacts.S3ModelArtifacts`
`evaluation_output`	S3Uri	`properties.OutputDataConfig.S3OutputPath`

Environment variables — optional: LOCAL_RANK, REGION, USE_SECURE_PYPI, USE_AMP, GRADIENT_ACCUMULATION_STEPS, MAX_GRAD_NORM, CHECKPOINT_FREQ, USE_DISTRIBUTED_TRAINING, MAX_RUNTIME_SECONDS

XGBoostModel¶

SageMaker Model creation step for XGBoost. No script — managed by SageMaker ModelStep. Creates a deployable model from training artifacts.

node type internal · SageMaker CreateModel · config XGBoostModelStepConfig

Dependencies (inputs)

logical name	required	data type	compatible sources
`model_data`	yes		XGBoostTraining, ProcessingStep, ModelArtifactsStep, Package

Outputs

logical name	data type	property path
`model_name`	S3Uri	`properties.ModelName`

XGBoostModelEval¶

XGBoost model evaluation script. Loads trained model, processes evaluation data, generates performance metrics and visualizations.

node type internal · SageMaker Processing · entry point xgboost_model_eval.py

Dependencies (inputs)

logical name	required	data type	compatible sources
`model_input`	yes		XGBoostTraining, PyTorchTraining, DummyTraining, XGBoostModel, PyTorchModel
`processed_data`	yes		TabularPreprocessing, CradleDataLoading, RiskTableMapping, CurrencyConversion

Outputs

logical name	data type	property path
`eval_output`	S3Uri	`properties.ProcessingOutputConfig.Outputs['eval_output'].S3Output.S3Uri`
`metrics_output`	S3Uri	`properties.ProcessingOutputConfig.Outputs['metrics_output'].S3Output.S3Uri`

Environment variables — optional: COMPARISON_MODE, PREVIOUS_SCORE_FIELD, STATISTICAL_TESTS, ID_FIELD, LABEL_FIELD, COMPARISON_METRICS, COMPARISON_PLOTS

Job arguments — --job_type

XGBoostModelInference¶

XGBoost model inference script. Loads trained model, preprocesses evaluation data, generates predictions without metrics computation.

node type internal · SageMaker Processing · entry point xgboost_model_inference.py

Dependencies (inputs)

logical name	required	data type	compatible sources
`model_input`	yes		XGBoostTraining, PyTorchTraining, DummyTraining, XGBoostModel, PyTorchModel, PyTorchModelInference
`processed_data`	yes		TabularPreprocessing, CradleDataLoading, RiskTableMapping, CurrencyConversion

Outputs

logical name	data type	property path
`eval_output`	S3Uri	`properties.ProcessingOutputConfig.Outputs['eval_output'].S3Output.S3Uri`

Environment variables — optional: OUTPUT_FORMAT, JSON_ORIENT, ID_FIELD, LABEL_FIELD

Job arguments — --job_type

XgboostMtModelEval¶

XgboostMt multi-task model evaluation. Generates per-task and aggregate metrics with visualizations.

node type internal · SageMaker Processing · entry point xgboost_mt_model_eval.py

Dependencies (inputs)

logical name	required	data type	compatible sources
`model_input`	yes		XgboostMtTraining, XGBoostTraining, XgboostMtModel, XGBoostModel, DummyTraining
`processed_data`	yes		TabularPreprocessing, CradleDataLoading, RiskTableMapping, CurrencyConversion, LabelRulesetExecution, BedrockBatchProcessing, BedrockProcessing, TemporalSplitPreprocessing

Outputs

logical name	data type	property path
`eval_output`	S3Uri	`properties.ProcessingOutputConfig.Outputs['eval_output'].S3Output.S3Uri`
`metrics_output`	S3Uri	`properties.ProcessingOutputConfig.Outputs['metrics_output'].S3Output.S3Uri`

Environment variables — required: ID_FIELD, TASK_LABEL_NAMES; optional: GENERATE_PLOTS, COMPARISON_MODE, STATISTICAL_TESTS

XgboostMtTraining¶

XgboostMt multi-task training for multi-label tabular classification with adaptive task weighting and knowledge distillation. Supports shared tree structures, JS-divergence weight adaptation, and per-task evaluation metrics.

node type internal · SageMaker Training · entry point xgboost_mt_training.py

Dependencies (inputs)

logical name	required	compatible sources
`input_path`	yes	TabularPreprocessing, StratifiedSampling, ProcessingStep, DataLoad, TemporalSplitPreprocessing
`hyperparameters_s3_uri`	no	HyperparameterPrep, ProcessingStep
`model_artifacts_input`	no	XgboostMtTraining, MissingValueImputation, RiskTableMapping, FeatureSelection

Outputs

logical name	data type	property path
`model_output`	S3Uri	`properties.ModelArtifacts.S3ModelArtifacts`
`evaluation_output`	S3Uri	`properties.OutputDataConfig.S3OutputPath`

Environment variables — optional: USE_SECURE_PYPI, USE_PRECOMPUTED_IMPUTATION, USE_PRECOMPUTED_RISK_TABLES, USE_PRECOMPUTED_FEATURES, REGION

XGBoostTraining¶

XGBoost training for tabular classification with risk table mapping and numerical imputation. Supports binary/multiclass, class weights, pre-computed artifacts, and comprehensive evaluation metrics.

node type internal · SageMaker Training · entry point xgboost_training.py

Dependencies (inputs)

logical name	required	compatible sources
`input_path`	yes	TabularPreprocessing, BedrockProcessing, StratifiedSampling, RiskTableMapping, MissingValueImputation, LabelRulesetExecution, ProcessingStep, DataLoad, PyTorchModelInference
`hyperparameters_s3_uri`	no	HyperparameterPrep, ProcessingStep
`model_artifacts_input`	no	XGBoostTraining, MissingValueImputation, RiskTableMapping, FeatureSelection

Outputs

logical name	data type	property path
`model_output`	S3Uri	`properties.ModelArtifacts.S3ModelArtifacts`
`evaluation_output`	S3Uri	`properties.OutputDataConfig.S3OutputPath`

Environment variables — optional: USE_SECURE_PYPI, USE_PRECOMPUTED_IMPUTATION, USE_PRECOMPUTED_RISK_TABLES, USE_PRECOMPUTED_FEATURES, REGION

Job arguments — --job_type