cursus.mods.exe_doc

Execution Document Generation Module.

This module provides standalone execution document generation capabilities that are independent from the pipeline generation system.

class ExecutionDocumentGenerator(config_path, sagemaker_session=None, role=None, config_resolver=None, project_root=None, anchor_file=None)

Bases: object

Standalone execution document generator.

Takes a PipelineDAG and configuration data as input, generates execution documents by collecting and processing step configurations independently from the pipeline generation system.

fill_execution_document(dag, execution_document)

Fill in the execution document with pipeline metadata.

This method uses an optimized approach: 1. First identify which steps need execution document processing 2. Filter steps by helper type 3. Only call helper-specific methods if relevant steps exist

Parameters:

• dag (PipelineDAG) – PipelineDAG defining the pipeline structure

• execution_document (Dict[str, Any]) – Execution document to fill

Returns:

Updated execution document

Raises:

ExecutionDocumentGenerationError – If generation fails

Return type:

Dict[str, Any]

class ExecutionDocumentHelper

Bases: ABC

Base class for execution document helpers.

abstractmethod can_handle_step(step_name, config)

Check if this helper can handle the given step.

Parameters:

• step_name (str) – Name of the step

• config (Any) – Configuration object for the step

Returns:

True if this helper can process the step, False otherwise

Return type:

bool

abstractmethod extract_step_config(step_name, config)

Extract step configuration for execution document.

Parameters:

• step_name (str) – Name of the step

• config (Any) – Configuration object for the step

Returns:

Dictionary containing the step configuration for execution document

Return type:

Dict[str, Any]

class CradleDataLoadingHelper

Bases: ExecutionDocumentHelper

Helper for extracting execution document configurations from Cradle data loading steps.

This helper ports the logic from CradleDataLoadingStepBuilder._build_request() and get_request_dict() methods to generate execution document configurations.

can_handle_step(step_name, config)

Check if this helper can handle the given step configuration.

Parameters:

• step_name (str) – Name of the step

• config – Step configuration object

Returns:

True if this helper can handle the configuration, False otherwise

Return type:

bool

extract_step_config(step_name, config)

Extract execution document configuration from Cradle data loading step config.

Parameters:

• step_name (str) – Name of the step

• config – Cradle data loading configuration object

Returns:

Dictionary containing the execution document configuration

Raises:

ExecutionDocumentGenerationError – If configuration extraction fails

Return type:

Dict[str, Any]

get_execution_step_name(step_name, config)

Get execution document step name following step builder naming convention.

Transforms step names from DAG format to execution document format: - “CradleDataLoading_training” -> “CradleDataLoading-Training” - “CradleDataLoading_calibration” -> “CradleDataLoading-Calibration”

This follows the same logic as CradleDataLoadingStepBuilder._get_step_name(): 1. Extract base name by removing job_type suffix 2. Add hyphen separator and capitalize job_type

Parameters:

• step_name (str) – Original step name from DAG (e.g., “CradleDataLoading_training”)

• config – Configuration object containing job_type

Returns:

Execution document step name (e.g., “CradleDataLoading-Training”)

Return type:

str

class RegistrationHelper

Bases: ExecutionDocumentHelper

Helper for extracting execution document configurations from registration steps.

This helper ports the logic from DynamicPipelineTemplate._fill_registration_configurations() and _create_execution_doc_config() methods to generate execution document configurations.

can_handle_step(step_name, config)

Check if this helper can handle the given step configuration.

Parameters:

• step_name (str) – Name of the step

• config – Step configuration object

Returns:

True if this helper can handle the configuration, False otherwise

Return type:

bool

create_execution_doc_config_with_related_configs(registration_config, payload_config=None, package_config=None)

Create execution document configuration with related payload and package configs.

This method extends the basic execution document configuration with load testing information from payload and package configurations.

Parameters:

• registration_config – Registration configuration object

• payload_config – Optional payload configuration object

• package_config – Optional package configuration object

Returns:

Dictionary with complete execution document configuration

Return type:

Dict[str, Any]

extract_step_config(step_name, config, all_configs=None)

Extract execution document configuration from registration step config.

Parameters:

• step_name (str) – Name of the step

• config – Registration configuration object

• all_configs (Dict[str, Any] | None) – Optional dictionary of all available configurations for related config lookup

Returns:

Dictionary containing the execution document configuration

Raises:

ExecutionDocumentGenerationError – If configuration extraction fails

Return type:

Dict[str, Any]

find_registration_step_patterns(step_names, region='')

Find step name patterns that likely correspond to registration steps.

This method is ported from DynamicPipelineTemplate._fill_registration_configurations().

Parameters:

• step_names (List[str]) – List of step names to search through

• region (str) – Optional region string to create region-specific patterns

Returns:

List of step name patterns for registration steps

Return type:

List[str]

get_execution_step_name(step_name, config)

Get execution document step name following step builder naming convention.

Transforms step names from DAG format to execution document format: - “Registration” -> “Registration-NA” (adds region suffix)

This follows the same logic as RegistrationStepBuilder.create_step(): step_name = self._get_step_name() + “-” + self.config.region

Parameters:

• step_name (str) – Original step name from DAG (e.g., “Registration”)

• config – Configuration object containing region

Returns:

Execution document step name (e.g., “Registration-NA”)

Return type:

str

validate_registration_config(config)

Validate that the registration config has all required fields.

Parameters:

config – Registration configuration object

Returns:

True if all required fields are present, False otherwise

Return type:

bool

Modules

base	Base classes and interfaces for execution document generation.
cradle_helper	Cradle Data Loading Helper for execution document generation.
data_uploading_helper	Data Uploading Helper for execution document generation.
generator	Main execution document generator class.
redshift_data_loading_helper	Redshift Data Loading Helper for execution document generation.
registration_helper	Registration Helper for execution document generation.
utils	Utility functions for execution document generation.