conformal_fit()

Fit a GAM with conformal calibration.

Usage

conformal_fit(
    formula,
    data,
    *,
    method="split",
    family=None,
    level=0.95,
    cal_fraction=0.25,
    n_folds=10,
    fit_method="REML",
    select=False,
    seed=None,
)

Fits a GAM and calibrates it so that ConformalPredictor.predict() returns prediction intervals with a distribution-free, finite-sample marginal coverage guarantee, using one of three conformal methods:

Split conformal: the data is randomly split into a training set (fit the GAM) and a calibration set (compute absolute residuals). The interval half-width is the ceil((n_cal+1) * level) / n_cal empirical quantile of the calibration residuals, giving intervals of constant width values +/- quantile. Simple and fast, but “wastes” data on the calibration split and can be less efficient than the alternatives.
CV+: the data is split into n_folds folds; each fold is used to compute out-of-fold residuals from a model trained on the rest. At prediction time, all fold models’ predictions are combined with all fold residuals via a min/max construction (Barber et al. 2021), giving tighter, per-observation intervals without a dedicated calibration split.
Jackknife+: the leave-one-out analogue of CV+, using n individual leave-one-out refits. Provides the tightest intervals of the three but is the most computationally expensive since it requires n refits.

Parameters

formula: str: GAM formula string.
data: InputData: Column-oriented data dict.
method: str = "split": Conformal method: "split" (default), "cv+", or "jackknife+".
family: Family | None = None: Response distribution family. Defaults to Gaussian().
level: float = 0.95: Nominal coverage probability (default 0.95).
cal_fraction: float = 0.25: Fraction of data held out for calibration in the split method (default 0.25). Ignored for "cv+" and "jackknife+".
n_folds: int = 10: Number of folds for the "cv+" method (default 10). Ignored for "split" and "jackknife+".
fit_method: str = "REML": Smoothing parameter selection method for the GAM (default "REML").
select: bool = False: If True, enable double-penalty variable selection.
seed: int | None = None: Random seed for data splitting.

Notes

Split conformal computes the calibration quantile as

\hat q = \left\lceil (n_{\text{cal}} + 1) \cdot \text{level} \right\rceil \big/ n_{\text{cal}} \quad \text{quantile of} \quad \{|y_i - \hat\mu(x_i)| : i \in \text{calibration set}\},

which, under exchangeability of calibration and test points, guarantees P(y \in [\hat\mu(x) - \hat q, \hat\mu(x) + \hat q]) \ge \text{level} marginally over new draws. CV+ and jackknife+ replace this single quantile with, for each test point, the appropriate quantile of the n (or n_folds) values {fold/LOO prediction +/- that fold's residual}, trading extra computation for tighter, locally-adapted intervals while retaining the same finite-sample coverage guarantee.

Returns

ConformalPredictor: A calibrated predictor that can produce intervals on new data.

Examples

import numpy as np
from whittaker.conformal import conformal_fit, conformal_coverage

rng = np.random.default_rng(0)
n = 500
x = rng.uniform(0, 1, n)
y = np.sin(2 * np.pi * x) + rng.normal(scale=0.3, size=n)

predictor = conformal_fit("y ~ s(x)", {"x": x, "y": y}, method="split", level=0.9, seed=0)
result = predictor.predict({"x": x[:5]})
print(result.lower, result.upper)
print(conformal_coverage(predictor, {"x": x, "y": y}, response="y"))

[-1.33066482  0.41215593 -0.28890212 -0.39394914 -1.5030175 ] [-0.25196998  1.49085076  0.78979271  0.6847457  -0.42432267]
0.948