cursus.steps.scripts.currency_conversion

Currency Conversion Processing Script

This script handles currency conversion for tabular data using exchange rates. It supports both training mode (all splits) and inference mode (single split). Follows the same pattern as feature_selection.py and missing_value_imputation.py for consistency.

load_split_data(job_type, input_dir)

Load data according to job_type with automatic format detection.

For ‘training’: Loads data from train, test, and val subdirectories For others: Loads single job_type split

Returns:

Dictionary with DataFrames and detected format stored in ‘_format’ key

Return type:

Dict[str, DataFrame]

save_output_data(job_type, output_dir, data_dict)

Save processed data according to job_type, preserving input format.

For ‘training’: Saves data to train, test, and val subdirectories For others: Saves to single job_type directory

get_currency_code(row, currency_code_field, marketplace_id_field, conversion_dict, default_currency)

Get currency code for a given row based on available fields.

Parameters:

• row (Series) – Data row

• currency_code_field (str | None) – Name of column containing currency codes directly

• marketplace_id_field (str | None) – Name of column containing marketplace IDs

• conversion_dict (Dict[str, Any]) – Dictionary with currency conversion mappings

• default_currency (str) – Default currency code to use when lookup fails

Returns:

Currency code for the row

Return type:

str

currency_conversion_single_variable(args)

Convert single variable’s currency values.

parallel_currency_conversion(df, exchange_rate_series, currency_conversion_vars, n_workers=50)

Perform parallel currency conversion on multiple variables.

process_currency_conversion(df, currency_code_field, marketplace_id_field, currency_conversion_vars, currency_conversion_dict, default_currency='USD', n_workers=50)

Process currency conversion for a DataFrame.

process_data(data_dict, job_type, currency_config)

Core data processing logic for currency conversion.

Parameters:

• data_dict (Dict[str, DataFrame]) – Dictionary of dataframes keyed by split name

• job_type (str) – Type of job (training, validation, testing, calibration)

• currency_config (Dict[str, Any]) – Currency conversion configuration dictionary

Returns:

Dictionary of converted dataframes

Return type:

Dict[str, DataFrame]

internal_main(job_type, input_dir, output_dir, currency_config, load_data_func=<function load_split_data>, save_data_func=<function save_output_data>)

Main logic for currency conversion, handling both training and inference modes.

Parameters:

• job_type (str) – Type of job (training, validation, testing, calibration)

• input_dir (str) – Input directory for data

• output_dir (str) – Output directory for processed data

• currency_config (Dict[str, Any]) – Currency conversion configuration dictionary

• load_data_func – Function to load data (for dependency injection in tests)

• save_data_func – Function to save data (for dependency injection in tests)

Returns:

Dictionary of converted dataframes

Return type:

Dict[str, DataFrame]

main(input_paths, output_paths, environ_vars, job_args=None)

Standardized main entry point for currency conversion script.

Parameters:

• input_paths (Dict[str, str]) – Dictionary of input paths with logical names - “processed_data”: Input data directory (from previous preprocessing step)

• output_paths (Dict[str, str]) – Dictionary of output paths with logical names - “processed_data”: Output directory for converted data

• environ_vars (Dict[str, str]) – Dictionary of environment variables

• job_args (Namespace | None) – Command line arguments containing job_type

Returns:

Dictionary of converted dataframes

Return type:

Dict[str, DataFrame]