Build Conformal Prediction Intervals

Construct distribution-free prediction intervals with guaranteed coverage using split conformal prediction.

Conformal prediction is a framework for constructing prediction intervals with a finite-sample coverage guarantee. Unlike parametric intervals, which require assumptions about the error distribution, conformal intervals are valid under any exchangeable data-generating process. The split conformal method reserves a calibration set during fitting, uses it to measure residual quantiles, and inflates the interval accordingly (a procedure that is simple, fast, and theoretically grounded).

Fit

Load the mcycle dataset and fit a conformal predictor with the split method at a nominal 95% coverage level. The seed argument ensures reproducibility of the train/calibration split.

import numpy as np
import whittaker as wk

# Fit a split conformal predictor at 95% coverage
data = wk.load_dataset("mcycle")
predictor = wk.conformal_fit(
    "accel ~ s(times, k=15)",
    data=data,
    method="split",
    level=0.95,
    seed=23,
)

Predict

Generate predictions on an evenly spaced grid across the observed time range. The result is a ConformalResult with point predictions and interval bounds.

# Predict on a fine grid and inspect point estimates
new_data = {"times": np.linspace(data["times"].min(), data["times"].max(), 200)}
result = predictor.predict(new_data)
result.values[:5]

array([3.70250183, 4.29428186, 4.88249012, 5.46090784, 6.0212509 ])

Inspect the lower and upper interval bounds separately.

result.lower[:5]

array([-40.20727696, -39.61549693, -39.02728866, -38.44887094,
       -37.88852789])

result.upper[:5]

array([47.61228061, 48.20406064, 48.7922689 , 49.37068663, 49.93102968])

Verify Coverage

Compute empirical coverage on the training data. This measures the fraction of observed responses that fall within their respective prediction intervals.

wk.conformal_coverage(predictor, data, response="accel")

0.9398496240601504

Interpret

The empirical coverage should meet or exceed the nominal 0.95 level. This is not a coincidence or a tuning outcome but rather it is a finite-sample guarantee that holds as long as the training and test data are exchangeable. Unlike Bayesian credible intervals, which express posterior uncertainty under a model, conformal intervals make no assumption about how the errors are distributed. The cost of this generality is that intervals are uniform in width across the covariate space. Methods such as cv+ or jackknife+ can produce locally adaptive widths at the expense of additional computation.