Splitting#

class dataforge_ml.splitting._splitter.DataSplitter(df, target=None, random_seed=None, config=None)[source]#

Bases: object

Splits a Polars DataFrame into train/test or cross-validation folds.

Parameters:

  • df (pl.DataFrame) – Source data. Must be non-empty.

  • target (str, optional) – Name of the target column. Required for stratified splits.

  • random_seed (int, optional) – Seed forwarded to sklearn splitters for reproducibility.

  • config (SplitConfig, optional) – Splitting thresholds. Defaults to SplitConfig() (library defaults).

group_kfold(k, group_column, stratify=<object object>)[source]#

Return k group-disjoint cross-validation folds.

Every row belonging to a given group_column value stays together in one partition, so no group value spans a train/validation boundary in any fold. Group-disjointness is a hard constraint that is never traded away. When a target is available and stratify resolves true, target class ratios are balanced across folds as a best-effort layer on top of the group constraint (only the target is balanced, never a wider signal matrix); the balance is approximate, not exact, because whole groups cannot be broken to equalise ratios.

Parameters:

  • k (int) – Number of folds.

  • group_column (str) – Name of the column identifying the group each row belongs to. Must exist in the DataFrame.

  • stratify (bool, optional) – Whether to balance the target class distribution across folds. Defaults to True when a target was provided, False otherwise.

Returns:

Exactly k folds with zero-based fold_index.

Return type:

list[FoldResult]

Raises:

ValueError – If group_column is not a column of the DataFrame, or if stratify is True (explicitly or by default) but no target column was provided.

group_split(test_size, group_column)[source]#

Return a single group-disjoint train/test split.

Every row belonging to a given group_column value lands entirely on one side of the split, so no group value appears in both partitions. Group-disjointness is a hard guarantee: the grouping is the user’s domain knowledge and is never inferred from the data.

Parameters:

  • test_size (float) – Fraction of groups to reserve for the test set (0 < test_size < 1).

  • group_column (str) – Name of the column identifying the group each row belongs to. Must exist in the DataFrame.

Return type:

SplitResult

Raises:

ValueError – If group_column is not a column of the DataFrame.

holdout_cv(test_size, k, stratify=<object object>)[source]#

Reserve a hold-out test set, then cross-validate the remainder.

First reserves a test partition of test_size, then runs k-fold cross-validation over the remaining rows. The held-out test rows are disjoint from every fold, so the final evaluation is untouched by the cross-validation. This is the hold-out-then-CV protocol; it is deliberately not nested CV — the library runs no models, so there is no inner hyperparameter loop.

Parameters:

  • test_size (float) – Fraction of rows to reserve for the test set (0 < test_size < 1).

  • k (int) – Number of cross-validation folds over the training remainder.

  • stratify (bool, optional) – Whether to stratify the hold-out and the inner folds on the target column. Defaults to True when a target was provided, False otherwise.

Returns:

The held-out test partition and the k folds over the remainder.

Return type:

HoldoutCVResult

Raises:

ValueError – If stratify is True (explicitly or by default) but no target column was provided.

kfold(k, stratify=<object object>)[source]#

Return a list of k cross-validation folds.

Parameters:

  • k (int) – Number of folds.

  • stratify (bool, optional) – Whether to stratify on the target column. Defaults to True when a target was provided, False otherwise.

Returns:

Exactly k folds with zero-based fold_index.

Return type:

list[FoldResult]

profile_stratified_kfold(profile, k)[source]#

Return k cross-validation folds stratified across all at-risk signals derived from the Phase 1 profile.

Falls back to unstratified KFold when the profile yields no usable signals. When exactly one signal survives the viability and redundancy gates, that lone signal is stratified on directly (iterstrat needs two or more label columns), rather than being discarded.

In every fold, rows belonging to a target class or rare categorical value too sparse to satisfy the viability floor are deterministically routed into that fold’s training partition (ADR-0048), so the label is always learned at fit time and never surfaces unseen at transform time. This routing is applied silently in every path, including the KFold fallback.

Parameters:

  • profile (StructuralProfileResult) – Output of StructuralProfiler.profile() run on the same DataFrame.

  • k (int) – Number of folds.

Returns:

Exactly k folds with zero-based fold_index.

Return type:

list[FoldResult]

profile_stratified_split(profile, test_size)[source]#

Return a train/test split stratified across all at-risk signals derived from the Phase 1 profile (missingness, extremes, rare categories, target).

Falls back to an unstratified random split when the profile yields no usable signals. When exactly one signal survives the viability and redundancy gates, that lone signal is stratified on directly (iterstrat needs two or more label columns), rather than being discarded.

After the base split is decided, rows belonging to a target class or rare categorical value too sparse to satisfy the viability floor are deterministically routed into the training partition (ADR-0048), so the label is learned at fit time and never surfaces unseen at transform time. This routing is applied silently in every path, including the random fallback.

Parameters:

  • profile (StructuralProfileResult) – Output of StructuralProfiler.profile() run on the same DataFrame.

  • test_size (float) – Fraction of rows to reserve for the test set (0 < test_size < 1).

Return type:

SplitResult

random_split(test_size, stratify=<object object>)[source]#

Return a single randomised train/test split.

Parameters:

  • test_size (float) – Fraction of rows to reserve for the test set (0 < test_size < 1).

  • stratify (bool, optional) – Whether to stratify on the target column. Defaults to True when a target was provided, False otherwise.

Return type:

SplitResult

repeated_kfold(k, n_repeats, stratify=<object object>)[source]#

Return k * n_repeats cross-validation folds from repeated k-fold.

Runs the fold assignment n_repeats times with a different seed each time to lower the variance of the cross-validated estimate. Each returned fold carries its originating repeat via repeat_index; the validation partitions within a single repeat cover every row exactly once.

Parameters:

  • k (int) – Number of folds per repeat.

  • n_repeats (int) – Number of times the k-fold assignment is repeated.

  • stratify (bool, optional) – Whether to stratify on the target column. Defaults to True when a target was provided, False otherwise.

Returns:

Exactly k * n_repeats folds. fold_index runs 0..k-1 within each repeat and repeat_index runs 0..n_repeats-1.

Return type:

list[FoldResult]

Raises:

ValueError – If stratify is True (explicitly or by default) but no target column was provided.

time_split(time_column, test_size=None, cutoff=None)[source]#

Return a chronological train/test split with no temporal leakage.

The DataFrame is sorted ascending by time_column before splitting. cutoff takes priority over test_size when both are supplied.

Parameters:

  • time_column (str) – Column to sort by. Must exist in the DataFrame.

  • test_size (float, optional) – Fraction of rows (from the end of the sorted series) to use as the test set. floor(len(df) * test_size) rows are taken.

  • cutoff (scalar, optional) – Threshold value. Rows where time_column >= cutoff go to test; all earlier rows go to train.

Return type:

SplitResult

class dataforge_ml.splitting._config.SplitConfig(max_stratification_signals=50, rows_per_signal=10, boolean_minority_threshold=0.05, redundancy_correlation_threshold=0.95)[source]#

Bases: object

Threshold configuration for the splitting sub-processor.

All fields default to the library’s original hard-coded constants so that constructing SplitConfig() produces identical behaviour to the pre-config implementation.

Parameters:

  • max_stratification_signals (int) – Upper bound on the number of binary stratification signals retained by build_label_matrix. The effective cap (ADR-0047 gate 4) is min(max_stratification_signals, n_rows / rows_per_signal); when more signals survive the viability and redundancy gates than the cap allows, signals are retained by importance rank (target first, missingness last) rather than by rarity, to bound the multilabel-stratification cost.

  • rows_per_signal (int) – Rows-per-signal budget (ADR-0047 gate 4). The data can support at most n_rows / rows_per_signal simultaneous stratification constraints, so this term co-bounds the cap alongside max_stratification_signals. The default of 10 is a standard per-constraint rule of thumb; a wide-but-short dataset is reduced to fewer columns, and one too small to support even the target degrades to a random split.

  • boolean_minority_threshold (float) – Minority-class ratio below which a boolean column contributes a stratification signal. A column whose true_ratio or false_ratio falls below this value is treated as imbalanced and receives a signal.

  • redundancy_correlation_threshold (float) – Absolute-correlation strength at or above which two stratification signals are treated as redundant (ADR-0047 gate 3) and collapsed to the higher-priority one. Correlation magnitude catches both near-identical (+1) and mirror-image (−1) pairs. The default is conservative so that only near-perfect duplicates are removed, never merely-similar signals.

boolean_minority_threshold: float = 0.05#
classmethod from_dict(data)[source]#

Construct a SplitConfig from a plain dictionary.

Parameters:

data (dict) – Mapping produced by to_dict(). Missing keys fall back to field defaults.

Returns:

Reconstructed config instance.

Return type:

SplitConfig

max_stratification_signals: int = 50#
redundancy_correlation_threshold: float = 0.95#
rows_per_signal: int = 10#
to_dict()[source]#

Serialise the config to a plain dictionary.

Returns:

All field values keyed by field name.

Return type:

dict

class dataforge_ml.splitting._config.SplitResult(train, test, train_size, test_size, train_ratio, test_ratio)[source]#

Bases: object

A single train/test partition of a dataset.

Parameters:

  • train (DataFrame)

  • test (DataFrame)

  • train_size (int)

  • test_size (int)

  • train_ratio (float)

  • test_ratio (float)

train#

Training partition.

Type:

pl.DataFrame

test#

Test/hold-out partition.

Type:

pl.DataFrame

train_size#

Number of rows in the training partition.

Type:

int

test_size#

Number of rows in the test partition.

Type:

int

train_ratio#

Fraction of total rows assigned to training (0.0–1.0).

Type:

float

test_ratio#

Fraction of total rows assigned to testing (0.0–1.0).

Type:

float

class dataforge_ml.splitting._config.FoldResult(train, val, fold_index, train_size, val_size, repeat_index=0)[source]#

Bases: object

One cross-validation fold: its train/validation partitions and position.

Parameters:

  • train (DataFrame)

  • val (DataFrame)

  • fold_index (int)

  • train_size (int)

  • val_size (int)

  • repeat_index (int)

train#

Training partition for this fold.

Type:

pl.DataFrame

val#

Validation partition for this fold.

Type:

pl.DataFrame

fold_index#

Zero-based index of this fold within the CV run.

Type:

int

train_size#

Number of rows in the training partition.

Type:

int

val_size#

Number of rows in the validation partition.

Type:

int

repeat_index#

Zero-based index of the repeat this fold belongs to. Defaults to 0 for single-pass schemes (kfold, profile_stratified_kfold); only repeated_kfold sets it to the originating repeat number.

Type:

int

class dataforge_ml.splitting._config.HoldoutCVResult(test, folds)[source]#

Bases: object

A held-out test partition plus cross-validation folds over the remainder.

Parameters:
test#

Held-out test partition, disjoint from every fold in folds.

Type:

pl.DataFrame

folds#

Cross-validation folds over the training remainder (all rows not in test).

Type:

list[FoldResult]