Pipeline

class dataforge_ml.config.SemanticType(*values)

Bases: StrEnum

The ML-level interpretation assigned to a column by the type detector.

Used throughout the pipeline to route columns to the correct sub-processors and to determine which statistical operations apply. See CONTEXT.md §SemanticType for the full type taxonomy and the Text vs Categorical distinction.

Boolean = 'boolean'

Categorical = 'categorical'

Datetime = 'datetime'

Identifier = 'identifier'

Numeric = 'numeric'

Text = 'text'

class dataforge_ml.config.Modality(*values)

Bases: StrEnum

The data modality the pipeline operates on.

Currently only Tabular is supported. Reserved for future expansion to additional modalities (time-series, image, etc.).

Tabular = 'tabular'

class dataforge_ml.config.PipelinePhase(*values)

Bases: StrEnum

The six sequential phases of the DataForgeML feature engineering pipeline.

Phase Orchestrators call PipelineConfig.resolve_active_columns with one of these values to obtain the column set for that phase after Hard and Soft Exclusions are applied.

Encoding = 'encoding'

Imputation = 'imputation'

Normalization = 'normalization'

OutlierDetection = 'outlier_detection'

Profiling = 'profiling'

Scaling = 'scaling'

class dataforge_ml.config.PipelineConfig(profiling=<factory>, imputation=<factory>, split=<factory>, random_seed=None)

Bases: object

Master configuration for the full 6-phase feature engineering pipeline.

Parameters:

• profiling (ProfileConfig) – Phase 1-specific parameters (correlation, chunking, memory threshold).

• imputation (ImputationConfig) – Phase 2-specific parameters (strategy thresholds, size guards).

• split (SplitConfig) – Splitting thresholds (stratification signal cap, boolean minority bar).

• random_seed (int, optional) – Single seed for all stochastic pipeline operations, including GMM Sampling during bimodal imputation. None produces non-deterministic output.

add_exclusion(column)

Add columns to the hard exclusion set, deduplicating automatically.

Columns already present in the exclusion list and duplicate entries within the input are silently ignored. Calling with an empty list is a no-op.

Parameters:

column (str or list[str]) – Column name(s) to register as hard exclusions. Deduplication is handled here; callers do not need to pre-deduplicate.

Return type:

None

add_phase_exclusion(phase, column)

Add columns to the soft exclusion set for a specific phase.

Parameters:

• phase (PipelinePhase or str) – The phase for which to exclude the column(s).

• column (str or list[str]) – Column name(s) to register as soft exclusions for this phase. Deduplication is handled automatically.

Return type:

None

property column_overrides: MappingProxyType

Explicit semantic type assignments respected by all downstream phases.

Returns:

A read-only view mapping columns to their explicitly assigned SemanticType.

Return type:

MappingProxyType[str, SemanticType]

property exclude_columns: tuple[str, ...]

Hard exclusions — columns dropped globally from every phase.

Returns:

A snapshot tuple of columns registered as hard exclusions.

Return type:

tuple[str, …]

classmethod from_dict(data)

Reconstruct a PipelineConfig from a plain dictionary.

Parameters:

data (dict) – Dictionary as produced by to_dict().

Returns:

Fully populated configuration instance with all nested sub-configs restored.

Return type:

PipelineConfig

classmethod from_json(json_str)

Reconstruct a PipelineConfig from a JSON string.

Parameters:

json_str (str) – JSON string as produced by to_json().

Returns:

Fully populated configuration instance.

Return type:

PipelineConfig

imputation: ImputationConfig

property numeric_kind_overrides: MappingProxyType[str, NumericKind]

Explicit NumericKind assignments for individual columns, applied after auto-detection in Phase 1.

Only valid for columns whose final SemanticType is Numeric; raises at orchestrator time otherwise.

Returns:

A read-only view mapping columns to their explicitly assigned NumericKind.

Return type:

MappingProxyType[str, NumericKind]

property phase_exclusions: MappingProxyType

Soft exclusions — columns bypassed for a specific phase but retained in the dataset.

Returns:

A read-only view mapping each phase to a tuple of excluded columns.

Return type:

MappingProxyType[PipelinePhase, tuple[str, …]]

profiling: ProfileConfig

random_seed: int | None = None

resolve_active_columns(phase, available_columns)

Return the columns the given phase should operate on.

Hard Exclusions are applied first, then phase-specific Soft Exclusions. Columns absent from available_columns are silently ignored in both exclusion lists.

Parameters:

• phase (PipelinePhase) – The pipeline phase requesting the active column set.

• available_columns (list[str]) – The full list of columns currently present in the DataFrame.

Returns:

Columns from available_columns that are not excluded by either Hard or Soft Exclusion rules for the given phase, preserving the original order.

Return type:

list[str]

set_column_type(column, semantic_type)

Explicitly set the semantic type for one or more columns, overriding auto-detection.

Parameters:

• column (str or list[str]) – Name of the column(s) to override.

• semantic_type (str or SemanticType) – The desired semantic type. Accepts enum values or their string equivalents (e.g. "numeric", "categorical").

Raises:

ValueError – When semantic_type is a string that does not match any SemanticType value.

Return type:

None

set_numeric_kind(column, kind)

Explicitly set the NumericKind for one or more columns.

Parameters:

• column (str or list[str]) – Name of the column(s) to override.

• kind (str or NumericKind) – The desired numeric kind. Accepts enum values or their string equivalents ("continuous", "bounded_discrete").

Raises:

ValueError – When kind is a string that does not match any NumericKind value.

Return type:

None

split: SplitConfig

to_dict()

Serialise the pipeline configuration to a plain dictionary.

Returns:

All fields serialised to JSON-compatible types; nested configs are recursively serialised via their own to_dict methods.

Return type:

dict

to_json(indent=2)

Serialise the pipeline configuration to a JSON string.

Parameters:

indent (int) – Number of spaces used for JSON indentation.

Returns:

JSON representation of to_dict().

Return type:

str