cursus.core.utils.project_discovery

Discover and summarize Cursus pipeline projects under a root directory.

A “pipeline project” here is a deployable Cursus pipeline directory — e.g. the projects/* folders under the AmazonCursus root (atoz_xgboost, rnr_pytorch_bedrock, …) or the per-pipeline folders under a consumer repo such as BuyerAbuseModsTemplate. Such a project is recognized by a configuration directory (pipeline_config or pipeline_configs) holding one or more config JSON files, typically alongside a dockers/scripts directory and a pipeline-definition .py module.

This is intentionally a read-only inspection utility built on the two existing primitives — cursus.core.utils path discovery (to locate a named project) and the config JSON each project already ships — so it answers “what pipeline projects exist under here, and what does each contain?” without the multi-developer-workspace authoring machinery. It does not import or require cursus.workspace.

Public API:

discover_pipeline_projects(root=None, names=None) -> List[ProjectInfo] summarize_project(project_dir) -> Optional[ProjectInfo]

Each project’s config JSON is read for its metadata.config_types (node -> config class) and configuration.specific (per-node settings), which is how a Cursus config file records the pipeline’s nodes. No SageMaker/engine objects are constructed, so this is cheap and safe to run anywhere.

class ConfigSummary(file, node_count=0, config_types=<factory>, nodes=<factory>, created_at=None, error=None)

Bases: object

Summary of one config JSON file inside a project.

file: str

node_count: int = 0

config_types: Dict[str, str]

nodes: List[str]

created_at: str | None = None

error: str | None = None

to_dict()

class ProjectInfo(name, path, config_dir=None, config_files=<factory>, has_dockers=False, has_scripts=False, pipeline_modules=<factory>)

Bases: object

Summary of one discovered pipeline project.

name: str

path: str

config_dir: str | None = None

config_files: List[ConfigSummary]

has_dockers: bool = False

has_scripts: bool = False

pipeline_modules: List[str]

property config_file_count: int

property distinct_config_types: List[str]

Union of config class names referenced across all of the project’s configs.

to_dict()

summarize_project(project_dir)

Summarize a single pipeline-project directory.

Returns a ProjectInfo, or None if project_dir is not a recognizable Cursus pipeline project (no pipeline_config/pipeline_configs directory).

discover_pipeline_projects(root=None, names=None)

Discover Cursus pipeline projects under root (or locate specific ones by name).

Parameters:

• root (str | Path | None) – Directory to scan for project subdirectories. When None and names is given, each name is located via cursus.core.utils.find_project_folder_generic() (cross-deployment search). When both are None, returns an empty list (nothing to scan).

• names (List[str] | None) – Optional explicit list of project folder names. If given, only these are returned (located under root if provided, else via generic discovery).

Returns:

A list of ProjectInfo, one per recognized pipeline project, sorted by name. Directories that are not pipeline projects (no config dir) are skipped.

Return type:

List[ProjectInfo]