cursus.api.dag.pipeline_dag_serializer¶

Pipeline DAG Serialization Module.

This module provides utilities for serializing and deserializing PipelineDAG objects to enable easy knowledge transfer and storage of pipeline topologies.

class PipelineDAGWriter(dag, metadata=None)[source]¶

Bases: object

Writer for serializing PipelineDAG to various formats.

Supports: - JSON format with metadata - Pretty printing for readability - Validation before writing - Metadata tracking (creation time, etc.)

to_dict()[source]¶

Convert PipelineDAG to dictionary representation.

Returns:: Dictionary with complete DAG representation including metadata
Return type:: Dict[str, Any]

to_json(pretty=True, indent=2)[source]¶

Convert PipelineDAG to JSON string.

Parameters:

pretty (bool) – If True, format JSON with indentation
indent (int) – Number of spaces for indentation (if pretty=True)

Returns:

JSON string representation of the DAG

Return type:

str

write_to_file(filepath, pretty=True, indent=2, validate=True)[source]¶

Write PipelineDAG to a JSON file.

Parameters:

filepath (str | Path) – Path to output file
pretty (bool) – If True, format JSON with indentation
indent (int) – Number of spaces for indentation
validate (bool) – If True, validate DAG before writing

Raises:

ValueError – If DAG validation fails
IOError – If file cannot be written

class PipelineDAGReader[source]¶

Bases: object

Reader for deserializing PipelineDAG from various formats.

Supports: - JSON format - Validation during reading - Metadata extraction

classmethod from_dict(data, validate=True)[source]¶

Create PipelineDAG from dictionary representation.

Parameters:

data (Dict[str, Any]) – Dictionary containing DAG data
validate (bool) – If True, validate data before creating DAG

Returns:

PipelineDAG instance

Raises:

ValueError – If data is invalid or version is unsupported

Return type:

PipelineDAG

classmethod from_json(json_str, validate=True)[source]¶

Create PipelineDAG from JSON string.

Parameters:

json_str (str) – JSON string representation of DAG
validate (bool) – If True, validate data before creating DAG

Returns:

PipelineDAG instance

Raises:

ValueError – If JSON is invalid or data is malformed

Return type:

PipelineDAG

classmethod read_from_file(filepath, validate=True)[source]¶

Read PipelineDAG from a JSON file.

Parameters:

filepath (str | Path) – Path to input file
validate (bool) – If True, validate data before creating DAG

Returns:

PipelineDAG instance

Raises:

FileNotFoundError – If file doesn’t exist
ValueError – If file content is invalid

Return type:

PipelineDAG

classmethod extract_metadata(filepath)[source]¶

Extract metadata from a DAG file without loading the full DAG.

Parameters:: filepath (str | Path) – Path to input file
Returns:: Dictionary containing metadata
Return type:: Dict[str, Any]

export_dag_to_json(dag, filepath, metadata=None, pretty=True)[source]¶

Convenience function to export a PipelineDAG to JSON file.

Parameters:

dag (PipelineDAG) – PipelineDAG instance to export
filepath (str | Path) – Output file path
metadata (Dict[str, Any] | None) – Optional metadata to include
pretty (bool) – If True, format JSON with indentation

import_dag_from_json(filepath)[source]¶

Convenience function to import a PipelineDAG from JSON file.

Parameters:: filepath (str | Path) – Input file path
Returns:: PipelineDAG instance
Return type:: PipelineDAG