cursus.steps.configs.config_graph_feature_processing_step¶

Graph Feature Processing Configuration with Self-Contained Derivation Logic.

Configuration for the GraphFeatureProcessing Processing step — turns per-seed subgraph pickles + labelled seeds into the GraphStorm GConstruct input tree (per-type node/edge parquets, reverse edges, node-ID-keyed multi-task masks, gconstruct_config.json). Runs the custom GraphStorm image (BYO container).

Three-tier field design: 1. Essential User Inputs (Tier 1) - Required fields users must provide. 2. System Fields (Tier 2) - Defaults that can be overridden. 3. Derived Fields (Tier 3) - Private, exposed via read-only properties.

The heavy feature-engineering modules (prepare_graphstorm_format.py + helpers) live in the BYO image’s code bundle, NOT this package; the structured knobs that do not fit env vars (per-type feature dicts + the multi-task target list) are assembled here into a config.yaml payload the container reads via –config.

class TargetNodeSpec(*, node_type, id_col, label_cols)[source]¶

Bases: BaseModel

One multi-task target-node spec — a train/val/test split + idx parquet set is generated per label column on this node type.

node_type: str¶

id_col: str¶

label_cols: List[str]¶

model_config: ClassVar[ConfigDict] = {}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class SplitCfg(*, enabled=False, num_chunks=50, split_type='both')[source]¶

Bases: BaseModel

Optional post-hoc split_for_gconstruct config (re-split parquets for parallel gconstruct).

enabled: bool¶

num_chunks: int¶

split_type: str¶

model_config: ClassVar[ConfigDict] = {}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class GraphFeatureProcessingConfig(*, author, bucket, role, region, service_name, pipeline_version, model_class='xgboost', current_date=<factory>, framework_version='2.1.0', py_version='py310', image_uri=None, subnets=None, security_group_ids=None, enable_network_isolation=None, source_dir=None, enable_caching=False, use_secure_pypi=False, max_runtime_seconds=86400, project_root_folder, processing_instance_count=1, processing_volume_size=500, processing_instance_type_large='ml.m5.4xlarge', processing_instance_type_small='ml.m5.2xlarge', use_large_processing_instance=False, skip_volume_kms=None, processing_source_dir=None, processing_entry_point='graph_feature_processing.py', processing_script_arguments=None, processing_framework_version='1.2-1', target_node_types, subgraph_source, num_chunks=2, chunk_size=50000, reverse_edge=True, train_frac=0.8, val_frac=0.1, query_type='relation', numerical_feat_dict=<factory>, interval_edge_types=<factory>, unwanted_properties=<factory>, etype_with_feat=<factory>, bert_model='bert-base-uncased', max_seq_length=16, co_train_lm=False, max_load_workers=8, max_workers=30, enable_purchase_collapse=False, split_files=None, gconstruct_config_name='gconstruct_config.json', config_yaml_name='graph_feature_config.yaml', **extra_data)[source]¶

Bases: ProcessingStepConfigBase

Configuration for the GraphFeatureProcessing step with three-tier field categorization. Inherits from ProcessingStepConfigBase. image_uri is inherited from BasePipelineConfig (the BYO GraphStorm ECR image) and required-in-practice (compute.kind is byo_container).

target_node_types: List[TargetNodeSpec]¶

subgraph_source: str¶

processing_entry_point: str¶

num_chunks: int¶

chunk_size: int¶

reverse_edge: bool¶

train_frac: float¶

val_frac: float¶

query_type: str¶

numerical_feat_dict: Dict[str, List[str]]¶

interval_edge_types: List[str]¶

unwanted_properties: List[str]¶

etype_with_feat: List[str]¶

bert_model: str¶

max_seq_length: int¶

co_train_lm: bool¶

max_load_workers: int¶

max_workers: int¶

enable_purchase_collapse: bool¶

split_files: SplitCfg | None¶

gconstruct_config_name: str¶

processing_volume_size: int¶

max_runtime_seconds: int¶

config_yaml_name: str¶

model_config: ClassVar[ConfigDict] = {'arbitrary_types_allowed': True, 'extra': 'allow', 'protected_namespaces': (), 'validate_assignment': True}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

property config_yaml_payload: Dict[str, Any]¶: The nested config.yaml the container reads via –config (structured knobs that do not fit env vars: per-type feature dicts + the multi-task target list).

property config_yaml_container_path: str¶

where the materialized config.yaml is mounted in the container (the builder stages it into the source dir, which mounts under /opt/ml/processing/input/code).

Type:: The –config arg

model_post_init(context, /)¶

This function is meant to behave like a BaseModel method to initialize private attributes.

It takes context as an argument since that’s what pydantic-core passes when calling it.

Parameters:

self (BaseModel) – The BaseModel instance.
context (Any) – The context.