cursus.validation.alignment.utils¶
Utilities and Models Module
This module contains utility functions, data models, and configuration classes for the alignment validation system. It provides common functionality used across all validation components.
Components: - alignment_utils.py: Core utility functions and helper classes - core_models.py: Core data models and type definitions - level3_validation_config.py: Level 3 validation configuration and modes - script_analysis_models.py: Models for script analysis results - utils.py: General utility functions and helpers
Utility Features: - Common data structures and enums - Validation configuration management - Helper functions for alignment operations - Type definitions and model classes - Configuration validation and defaults
- class ValidationLevel(*values)[source]¶
Bases:
EnumThe 3 validation BOUNDARIES — what construction can’t self-check (FZ 31e1d3h / Phase D5).
The unified
.step.yamlmade Contract<->Spec alignment a construction-time Pydantic invariant (StepInterface._sync_and_align), so the former Level-2 (CONTRACT_SPEC=2) validated a tautology and has been removed. The three surviving levels are renamed in SEMANTICS to the 3 boundaries — SCRIPT_CONTRACT is the Script<->Interface fidelity boundary (B1), SPEC_DEPENDENCY is the cross-step DAG-resolvability boundary (B2, which now also owns the SageMaker property-path check folded in from the old L2), and BUILDER_CONFIG is the registry<->handler<->config binding boundary (B3). The member NAMES + integer VALUES are kept (1/3/4, non-contiguous) soValidationLevel(1/3/4)coercion and every existing reference keep working; onlyCONTRACT_SPEC(value 2) is gone.- SCRIPT_CONTRACT = 1¶
- SPEC_DEPENDENCY = 3¶
- BUILDER_CONFIG = 4¶
- class ValidationStatus(*values)[source]¶
Bases:
EnumStatus of validation operations.
- PASSED = 'PASSED'¶
- FAILED = 'FAILED'¶
- EXCLUDED = 'EXCLUDED'¶
- ERROR = 'ERROR'¶
- ISSUES_FOUND = 'ISSUES_FOUND'¶
- PASSED_WITH_WARNINGS = 'PASSED_WITH_WARNINGS'¶
- NO_VALIDATOR = 'NO_VALIDATOR'¶
- COMPLETED = 'COMPLETED'¶
- class IssueLevel(*values)[source]¶
Bases:
EnumSeverity levels for validation issues.
- ERROR = 'ERROR'¶
- WARNING = 'WARNING'¶
- INFO = 'INFO'¶
- class RuleType(*values)[source]¶
Bases:
EnumTypes of validation rules.
- UNIVERSAL = 'universal'¶
- STEP_SPECIFIC = 'step_specific'¶
- METHOD_INTERFACE = 'method_interface'¶
- CONFIGURATION = 'configuration'¶
- class ValidationIssue(level, message, method_name=None, rule_type=None, details=None, step_name=None, file_path=None, line_number=None)[source]¶
Bases:
objectRepresents a validation issue found during alignment checking.
- level: IssueLevel¶
- class ValidationResult(status, step_name, validation_level=None, issues=None, metadata=None, error_message=None)[source]¶
Bases:
objectRepresents the result of a validation operation.
- issues: List[ValidationIssue] = None¶
- validation_level: ValidationLevel | None = None¶
- status: ValidationStatus¶
- class MethodValidationInfo(method_name, is_required, expected_signature=None, return_type=None, purpose=None, validation_rules=None)[source]¶
Bases:
objectInformation about method validation requirements.
- class StepValidationContext(step_name, step_type, builder_class_name=None, workspace_dirs=None, validation_config=None)[source]¶
Bases:
objectContext information for step validation.
- create_validation_issue(level, message, method_name=None, rule_type=None, details=None, **kwargs)[source]¶
Create a validation issue with proper type conversion.
- create_validation_result(status, step_name, validation_level=None, issues=None, **kwargs)[source]¶
Create a validation result with proper type conversion.
- FlexibleFileResolver¶
alias of
FlexibleFileResolverAdapter
- get_sagemaker_step_type(step_name, workspace_id=None)[source]¶
Get SageMaker step type with workspace context.
- get_canonical_name_from_file_name(file_name, workspace_id=None)[source]¶
Enhanced file name resolution with workspace context awareness.
- Parameters:
- Returns:
Canonical step name (e.g., “XGBoostModelEval”, “DummyTraining”)
- Raises:
ValueError – If file name cannot be mapped to a canonical name
- Return type:
- class StepCatalog(workspace_dirs=None)[source]¶
Bases:
objectUnified step catalog addressing all validated user stories (US1-US5).
This single class consolidates the functionality of 16+ discovery systems while maintaining simple, efficient O(1) lookups through dictionary-based indexing.
PHASE 1 ENHANCEMENT: Now includes StepBuilderRegistry functionality: - Config-to-builder resolution - Legacy alias support - Pipeline construction interface - Enhanced registry integration
- LEGACY_ALIASES = {'MIMSPackaging': 'Package', 'MIMSPayload': 'Payload', 'ModelRegistration': 'Registration', 'PytorchModel': 'PyTorchModel', 'PytorchTraining': 'PyTorchTraining'}¶
- build_complete_config_classes(project_id=None)[source]¶
Build complete mapping integrating manual registration with auto-discovery.
This addresses the TODO in the existing build_complete_config_classes() function.
- create_unified_specification(contract_name)[source]¶
Create unified specification from multiple variants using smart selection.
This method delegates to SpecAutoDiscovery for smart specification handling, integrating SmartSpecificationSelector functionality: - Multi-variant specification discovery - Union of dependencies and outputs from all variants - Smart validation logic with detailed feedback - Primary specification selection (training > generic > first available)
- discover_config_classes(project_id=None)[source]¶
Auto-discover configuration classes from core and workspace directories.
- discover_contracts_with_scripts()[source]¶
DISCOVERY: Find all steps that have both contract and script components.
- discover_cross_workspace_components(workspace_ids=None)[source]¶
DISCOVERY: Find components across multiple workspaces.
- discover_script_files(project_id=None)[source]¶
Discover script files from package and workspaces with prioritization.
This method enables script discovery for interactive runtime testing by finding scripts referenced in config and contract entry points.
- discover_scripts_from_dag(dag)[source]¶
Discover scripts referenced in a DAG with intelligent node-to-script mapping.
This method enables DAG-guided script discovery for interactive runtime testing by mapping DAG nodes to actual script files using step catalog intelligence.
- extract_parent_values_for_inheritance(target_config_class_name, completed_configs)[source]¶
Extract parent values for inheritance from completed configs.
This method enables Smart Default Value Inheritance by extracting field values from the immediate parent config for pre-populating child config forms. It implements cascading inheritance to eliminate redundant user input.
- Parameters:
- Returns:
Dictionary of field values from immediate parent config
- Return type:
Example
- completed_configs = {
“BasePipelineConfig”: base_config_instance, “ProcessingStepConfigBase”: processing_config_instance
}
- parent_values = step_catalog.extract_parent_values_for_inheritance(
“TabularPreprocessingConfig”, completed_configs
) # Returns: ALL field values from ProcessingStepConfigBase # (which includes cascaded values from BasePipelineConfig) # Result: {“author”: “lukexie”, “bucket”: “my-bucket”, # “processing_instance_type”: “ml.m5.2xlarge”, …}
- find_contracts_by_entry_point(entry_point)[source]¶
Find contracts that reference a specific script entry point.
- find_specs_by_contract(contract_name)[source]¶
Find all specifications that reference a specific contract.
This method enables contract-specification alignment validation by finding specifications that are associated with a given contract name.
- get_all_builders()[source]¶
Get all available builders with canonical names.
This method provides comprehensive builder discovery for dynamic testing without requiring hard-coded maintenance.
- get_builder_class_path(step_name)[source]¶
Get builder class path for a step using BuilderAutoDiscovery component.
- get_builder_for_config(config, node_name=None)[source]¶
Map a config instance to a builder PROVIDER (the thing the assembler calls with its 5-kwarg signature to get a StepBuilderBase).
Returns a
BuilderProvider(FZ 31e1d3g1 Phase 1): TODAY this is the per-step builder CLASS (a class is already such a callable), so behavior is unchanged. The annotation is provider-based so the classless Design-B end-state can return a non-class factory here without breaking the contract — callers must invoke the result, never assume it is a class.
- get_builder_for_step_type(step_type)[source]¶
Get the builder PROVIDER for a step type, with legacy alias support.
Returns a
BuilderProvider(FZ 31e1d3g1 Phase 1) — currently the builder class; seeget_builder_for_configfor the dual-mode contract.
- get_builders_by_step_type(step_type)[source]¶
Get builders filtered by SageMaker step type.
This method enables step-type-specific testing by filtering builders based on their registered SageMaker step type.
- get_immediate_parent_config_class(config_class_name)[source]¶
Get the immediate parent config class name for inheritance.
This method enables Smart Default Value Inheritance by identifying which parent config class a specific config should inherit from. It implements cascading inheritance where TabularPreprocessingConfig inherits from ProcessingStepConfigBase (not BasePipelineConfig directly) to get all cascaded values.
- Parameters:
config_class_name (str) – Target config class name (e.g., “TabularPreprocessingConfig”)
- Returns:
Immediate parent class name (e.g., “ProcessingStepConfigBase”) or None if not found
- Return type:
str | None
Example
parent = step_catalog.get_immediate_parent_config_class(“TabularPreprocessingConfig”) # Returns: “ProcessingStepConfigBase” (not “BasePipelineConfig”)
parent = step_catalog.get_immediate_parent_config_class(“CradleDataLoadConfig”) # Returns: “BasePipelineConfig” (direct inheritance)
- get_script_discovery_stats()[source]¶
Get script discovery statistics.
This method provides discovery statistics for interactive runtime testing to help users understand the script discovery process.
- get_script_info(script_name)[source]¶
Get information about a script in dictionary format.
This method provides script information for interactive runtime testing in a user-friendly dictionary format.
- get_sdk_output_location(built_step, output_type=None)[source]¶
Read the output S3 location(s) off a BUILT SDK-delegation step (FZ 31e1d3i).
A generic, DUCK-TYPED accessor for the MODSPredefinedProcessingStep family: any built SDK step that exposes
get_output_locations(the SAIS contract) is supported — CradleDataLoadingStep returns a{DATA, SIGNATURE, METADATA}dict (or one entry whenoutput_typeis given), DataUploadingStep returnsNone(a SINK — output goes to Andes/EDX), and a step lacking the method (e.g. RedshiftDataLoadingStep) returnsNone. This REPLACES the per-stepget_output_location/get_step_outputshelpers the cradle builder used to carry (so that builder becomes a pure shell). It reads a method off an ALREADY-CONSTRUCTED step instance, so it introduces NO offline SAIS import — the SAIS class was necessarily importable to build the step.- Parameters:
- Returns:
The step’s output location(s) as the SDK reports them, or
Noneif the step does not exposeget_output_locations.- Return type:
- get_spec_job_type_variants(base_step_name)[source]¶
Get all job type variants for a base step name from specifications.
This method discovers different job type variants (training, validation, testing, etc.) for a given base step name by examining specification file naming patterns.
- get_step_builder_suggestions(config_class_name)[source]¶
Get suggestions for step builders based on config class name.
- get_step_info(step_name, job_type=None)[source]¶
Get complete information about a step, optionally with job_type variant.
- get_step_interface(step_name, job_type=None)[source]¶
Load a step’s unified
.step.yamlinterface — the single canonical accessor.This is THE one load function for interface data (FZ 31e1d3g3 follow-up): both
load_contract_class(→iface.contract) andload_spec_class(→iface) are thin views over it, so the job_type-suffix fallback below is written once and every consumer inherits it.On the first (exact) load miss, if
job_typewas not supplied, we resolve the node’s base step name ROBUSTLY (naming.resolve_base_step_nameagainst the registry — validates the base against actual step names, not a hardcoded suffix list) and retry with the trailing segment as the job_type (ModelCalibration_calibration→ModelCalibration+ job_typecalibration;CradleDataLoading_munged→CradleDataLoading+munged). Because the base is validated against the registry, ANY suffix resolves (job_type is open now) while a base likeXGBoostModelis never mis-stripped (noXGBooststep exists to strip to).
- has_builder_provider(step_name)[source]¶
Whether a step has a buildable builder, WITHOUT importing/instantiating it (FZ 31e1d3g3 D4).
This is a ROUTABILITY check answered from declarative data — the registry’s
sagemaker_step_type(+ the.step.yamlpatterns.step_assembly) must resolve to a realPatternHandler. It does NOT callload_builder_class(which would import the builder module — and fail for an SDK-bound builder offline even though the step is perfectly routable). So it stays True for a fileless-but-routable step (the factory-shell end-state) and for the SDK steps offline, and False only for genuinely-absent or non-routable rows (Base/Lambda/unknown). Used by the DAG resolver’s component-availability check, which must not encode the deleted-builder_*.py-file assumption.
- is_step_type_supported(step_type)[source]¶
Check if step type is supported (including legacy aliases).
- list_available_scripts()[source]¶
List all available script names.
This method enables script enumeration for interactive runtime testing by listing all discovered script names.
- list_available_steps(workspace_id=None, job_type=None)[source]¶
List all available concrete pipeline steps with deduplication.
Excludes base configuration steps (‘Base’, ‘Processing’) and applies canonical name deduplication following standardization rules.
- list_steps_with_scripts(workspace_id=None, job_type=None)[source]¶
List all steps that have script components.
This method filters available steps to only return those that have script file components, which is useful for alignment validation and script-based testing frameworks.
- list_steps_with_specs(workspace_id=None, job_type=None)[source]¶
List all steps that have specification components.
This method filters available steps to only return those that have specification file components, which is useful for validation frameworks that need to work specifically with steps that have specifications.
- load_all_specifications()[source]¶
Load all specification instances from both package and workspace directories.
This method provides comprehensive specification loading for validation frameworks and dependency analysis tools. It discovers and loads all available specifications, serializing them to dictionary format for easy consumption.
- load_builder_class(step_name)[source]¶
Load builder class for a step with job type variant fallback support.
- load_contract_class(step_name)[source]¶
Load the contract for a step — a VIEW onto its
.step.yamlinterface.
- load_script_info(script_name)[source]¶
Load script information for a specific script with workspace-aware discovery.
This method enables script information retrieval for interactive runtime testing with workspace prioritization support.
- load_spec_class(step_name)[source]¶
Load the specification for a step — the
.step.yamlinterface itself.The StepInterface is a drop-in for the legacy StepSpecification (exposes
dependencies/outputs/…), so it IS the spec view.
- resolve_pipeline_node(node_name)[source]¶
Resolve PipelineDAG node name to StepInfo (handles job_type variants).
- search_steps(query, job_type=None)[source]¶
Search steps by name with basic fuzzy matching.
- Parameters:
- Returns:
List of search results sorted by relevance
- Return type:
- serialize_contract(contract_instance)[source]¶
Convert contract instance to dictionary format.
This method provides standardized serialization of ScriptContract objects for use in script-contract alignment validation.
- serialize_spec(spec_instance)[source]¶
Convert specification instance to dictionary format.
This method provides standardized serialization of StepSpecification objects for use in validation and alignment testing.
- validate_builder_availability(step_types)[source]¶
Validate that builders are available for step types.
- validate_contract_script_mapping()[source]¶
Validate contract-script relationships across the system.
- validate_logical_names_smart(contract, contract_name)[source]¶
Smart validation using multi-variant specification logic.
This method delegates to SpecAutoDiscovery for smart validation, implementing the core Smart Specification Selection validation: - Contract input is valid if it exists in ANY variant - Contract must cover intersection of REQUIRED dependencies - Provides detailed feedback about which variants need what
- class Level3ValidationConfig(mode=ValidationMode.RELAXED)[source]¶
Bases:
objectConfiguration for Level 3 validation thresholds and behavior.
- classmethod create_custom_config(pass_threshold, warning_threshold, error_threshold)[source]¶
Create a custom validation configuration with specific thresholds.
- Parameters:
- Returns:
Custom Level3ValidationConfig instance
- Return type:
- determine_severity_from_score(score, is_required)[source]¶
Determine issue severity based on compatibility score and requirement.
- class ValidationMode(*values)[source]¶
Bases:
EnumValidation modes with different threshold requirements.
- STRICT = 'strict'¶
- RELAXED = 'relaxed'¶
- PERMISSIVE = 'permissive'¶
- extract_logical_name_from_path(path)[source]¶
Extract logical name from a SageMaker path.
For paths like ‘/opt/ml/processing/input/data’, extracts ‘data’.
- format_alignment_issue(issue)[source]¶
Format a validation issue for display.
- Parameters:
issue (ValidationIssue) – The validation issue to format
- Returns:
Formatted string representation
- Return type:
- group_issues_by_severity(issues)[source]¶
Group validation issues by severity level.
- Parameters:
issues (List[ValidationIssue]) – List of validation issues
- Returns:
Dictionary mapping severity levels to lists of issues
- Return type:
- get_highest_severity(issues)[source]¶
Get the highest severity level among a list of issues.
- Parameters:
issues (List[ValidationIssue]) – List of validation issues
- Returns:
Highest severity level or None if no issues
- Return type:
IssueLevel | None
- validate_environment_setup()[source]¶
Validate that the environment is properly set up for alignment validation.
- get_validation_summary_stats(issues)[source]¶
Get summary statistics for a list of validation issues.
- Parameters:
issues (List[ValidationIssue]) – List of validation issues
- Returns:
Dictionary with summary statistics
- Return type:
Modules
Utility functions for alignment validation. |
|
Validation Models |