cursus.steps.scripts.percentile_model_calibration

Percentile Model Calibration Script for SageMaker Processing.

This script performs percentile score mapping calibration to convert raw model scores to calibrated percentile values using ROC curve analysis. It replicates the functionality of percentile_score_mapping.py but follows the cursus framework patterns with environment variable configuration and standardized I/O channels.

load_dataframe_with_format(file_path)[source]

Load DataFrame and detect its format.

Parameters:

file_path (str) – Path to the file

Returns:

Tuple of (DataFrame, format_string)

Return type:

Tuple[DataFrame, str]

save_dataframe_with_format(df, output_path, format_str)[source]

Save DataFrame in specified format.

Parameters:

  • df (DataFrame) – DataFrame to save

  • output_path (str) – Base output path (without extension)

  • format_str (str) – Format to save in (‘csv’, ‘tsv’, or ‘parquet’)

Returns:

Path to saved file

Return type:

str

parse_score_fields(environ_vars)[source]

Parse score fields from environment variables with backward compatibility.

Priority: 1. SCORE_FIELDS (multi-task) - comma-separated list 2. SCORE_FIELD (single-task) - single field fallback

Parameters:

environ_vars (Dict[str, str]) – Dictionary of environment variables

Returns:

List of score field names to calibrate

Return type:

List[str]

Examples

>>> parse_score_fields({"SCORE_FIELDS": "task_0_prob,task_1_prob"})
["task_0_prob", "task_1_prob"]

>>> parse_score_fields({"SCORE_FIELD": "prob_class_1"})
["prob_class_1"]
validate_score_fields(df, score_fields)[source]

Validate that score fields exist in dataframe.

Parameters:

  • df (DataFrame) – Input dataframe

  • score_fields (List[str]) – List of score field names to validate

Returns:

Tuple of (valid_fields, missing_fields)

Return type:

Tuple[List[str], List[str]]

get_calibrated_score_map(df, score_field, calibration_dictionary, weight_field=None)[source]

Calculate the calibrated score map based on the input data frame and calibration dictionary.

Parameters:

  • df (pd.DataFrame) – The input data frame containing the score and optional weight fields.

  • score_field (str) – The name of the column in the data frame that contains the score values.

  • calibration_dictionary (Dict[float, float]) – A dictionary mapping from calibrated scores to the corresponding percentiles. If None, no calibration is applied. Defaults to None.

  • weight_field (Optional[str], optional) – The name of the column in the data frame that contains the weight values. If None, no weights are used. Defaults to None.

Returns:

A list of tuples, each containing a pair of (score, calibrated_score).

The list is sorted by the score values.

Return type:

List[Tuple[float, float]]

Note

  • The function adds an ‘all’ column to the input data frame, set to 1 for all rows, to assist in the calculation of the ROC curve.

  • The calibrated scores are calculated by finding the appropriate position in the ROC curve for each percentile defined in the calibration dictionary, and interpolating between the neighboring points on the ROC curve to find the exact score corresponding to the calibrated percentile.

  • The function returns a list of (score, calibrated_score) pairs, including (0, 0) and (1, 1) to define the mapping at the boundaries.

find_first_data_file(data_dir)[source]

Find the most appropriate data file in directory, handling multiple XGBoost output files.

Parameters:

data_dir (str) – Directory to search for data files

Returns:

Path to the most appropriate data file found

Return type:

str

Raises:

FileNotFoundError – If no supported data file is found

load_calibration_dictionary(input_paths)[source]

Load calibration dictionary from input path or use built-in default.

Parameters:

input_paths (Dict[str, str]) – Dictionary of input paths with logical names

Returns:

Calibration dictionary mapping scores to percentiles

Return type:

Dict[float, float]

main(input_paths, output_paths, environ_vars, job_args=None)[source]

Main entry point for the percentile model calibration script.

Parameters:

  • input_paths (dict) – Dictionary of input paths with logical names

  • output_paths (dict) – Dictionary of output paths with logical names

  • environ_vars (dict) – Dictionary of environment variables

  • job_args (Namespace) – Command line arguments (optional)

Returns:

Dictionary with metrics and results

Return type:

dict