cursus.steps.hyperparams.hyperparameters_transformer2risk

class Transformer2RiskHyperparameters(*, full_field_list, cat_field_list, tab_field_list, id_name, label_name, multiclass_categories, categorical_features_to_encode=<factory>, model_class='transformer2risk', device=-1, header=0, lr=3e-05, batch_size=2, eval_batch_size_multiplier=2.0, max_epochs=3, metric_choices=['f1_score', 'auroc'], optimizer='SGD', class_weights=None, text_name, text_source_fields=None, max_sen_len=100, fixed_tokenizer_length=True, text_input_ids_key='input_ids', text_attention_mask_key='attention_mask', text_processing_steps=[], embedding_size=128, dropout_rate=0.2, hidden_size=256, n_embed=4000, n_blocks=8, n_heads=8, lr_decay=0.05, momentum=0.9, weight_decay=0.0, adam_epsilon=1e-08, warmup_steps=300, run_scheduler=True, val_check_interval=0.25, gradient_clip_val=1.0, fp16=False, use_gradient_checkpointing=False, early_stop_metric='val_loss', early_stop_patience=3, load_ckpt=False, smooth_factor=0.0, count_threshold=0, text_field_overwrite=False, chunk_trancate=True, max_total_chunks=3, **extra_data)

Bases: ModelHyperparameters

Hyperparameters for Transformer2Risk bimodal fraud detection model.

This class extends the base ModelHyperparameters with Transformer-specific architecture parameters needed for the Transformer2Risk model which combines: - Transformer encoder with self-attention for text sequence encoding - MLP for tabular feature encoding - Bimodal fusion for fraud prediction

Key architectural differences from LSTM2Risk: - Uses self-attention mechanism instead of recurrent connections - Larger embedding dimensions (128 vs 16) for richer representations - Fixed-length sequences with positional embeddings (vs variable-length LSTM) - Multi-head attention for parallel attention to different aspects

Inherits all base fields including: - Data field management (full_field_list, cat_field_list, tab_field_list) - Training parameters (lr, batch_size, max_epochs, optimizer) - Classification parameters (multiclass_categories, class_weights) - Derived properties (input_tab_dim, num_classes, is_binary)

Example Usage: ```python hyperparam = Transformer2RiskHyperparameters(

# Essential fields (Tier 1) - required full_field_list=[“name”, “email”, “age”, “income”, “label”], cat_field_list=[“name”, “email”], tab_field_list=[“age”, “income”], id_name=”customer_id”, label_name=”label”, multiclass_categories=[0, 1],

# Transformer-specific fields (Tier 2) - optional, using defaults embedding_size=128, hidden_size=256, n_embed=4000, n_blocks=8, n_heads=8, block_size=100, dropout_rate=0.2,

# Can also override base fields lr=3e-5, batch_size=32, max_epochs=5

)

# Access derived properties print(f”Input tabular dimension: {hyperparam.input_tab_dim}”) print(f”Number of classes: {hyperparam.num_classes}”) print(f”Is binary classification: {hyperparam.is_binary}”)

# Serialize for SageMaker config = hyperparam.serialize_config() ```

text_name: str

text_source_fields: List[str] | None

model_class: str

max_sen_len: int

fixed_tokenizer_length: bool

text_input_ids_key: str

text_attention_mask_key: str

text_processing_steps: List[str]

embedding_size: int

dropout_rate: float

hidden_size: int

n_embed: int

n_blocks: int

n_heads: int

lr_decay: float

momentum: float

weight_decay: float

adam_epsilon: float

warmup_steps: int

run_scheduler: bool

val_check_interval: float

gradient_clip_val: float

fp16: bool

use_gradient_checkpointing: bool

early_stop_metric: str

early_stop_patience: int

load_ckpt: bool

smooth_factor: float

count_threshold: int

text_field_overwrite: bool

chunk_trancate: bool

max_total_chunks: int

get_public_init_fields()

Override get_public_init_fields to include bimodal-specific derived fields. Gets a dictionary of public fields suitable for initializing a child config.

validate_transformer_hyperparameters()

Validate transformer-specific constraints.

model_config: ClassVar[ConfigDict] = {'arbitrary_types_allowed': True, 'extra': 'allow', 'protected_namespaces': (), 'validate_assignment': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_post_init(context, /)

This function is meant to behave like a BaseModel method to initialize private attributes.

It takes context as an argument since that’s what pydantic-core passes when calling it.

Parameters:

• self (BaseModel) – The BaseModel instance.

• context (Any) – The context.

full_field_list: List[str]

cat_field_list: List[str]

tab_field_list: List[str]

id_name: str

label_name: str

multiclass_categories: List[int | str]

categorical_features_to_encode: List[str]

device: int

header: int

lr: float

batch_size: int

eval_batch_size_multiplier: float

max_epochs: int

metric_choices: List[str]

optimizer: str

class_weights: List[float] | None