calibration_incidence()

Aalen-Johansen calibration error for cause-specific cumulative incidence predictions.

Usage

calibration_incidence(
    surv,
    incidence_prob,
    times,
    *,
    data=None,
    cause,
)

At each evaluation time, compares the mean predicted CIF across all subjects against the marginal Aalen-Johansen CIF estimate (the observed reference). Returns the absolute calibration error at each time. A well-calibrated model has errors close to zero.

This implements a marginal calibration check: for a given cause, the average predicted CIF should match the nonparametric Aalen-Johansen estimate at every time point.

Interpretation:

0.0: Perfect marginal calibration (mean predictions match the AJ reference exactly).
Positive values indicate miscalibration. Large errors at early times suggest the model over- or under-predicts the initial incidence. Large errors at late times suggest drift in the tail.

Parameters

surv: Surv | Outcome | str: A multi-state Surv response (from gw.Surv() with a categorical event). An Outcome or a formula response such as 'Surv(time, status == 2)' is also accepted, with its columns read from data.
incidence_prob: Any: Predicted cumulative incidence probabilities for the cause of interest, shape (n_subjects, n_times). Each entry is a predicted probability that cause cause has occurred by the corresponding time.
times: Any: Evaluation times where calibration is assessed. 1-D array-like. Must have length equal to the second dimension of incidence_prob.
cause: int: The cause of interest, as an integer event code from the Surv response.
data: Any = None: A data frame (pandas, Polars, PyArrow, DuckDB, a lazy frame, …) holding the columns named by the response and incidence_prob. When surv is an Outcome or a formula, rows with a missing value in any column used are dropped first, along with the matching rows of any arrays passed alongside.

Returns

ndarray: Absolute calibration error at each time, shape (len(times),). Lower is better.

Details

The calibration error at time t for cause k is:

\mathrm{CalErr}_k(t) = \left|\bar{\hat{F}}_k(t) - \hat{F}_k^{\mathrm{AJ}}(t)\right|

where \bar{\hat{F}}_k(t) = \frac{1}{n}\sum_{i=1}^n \hat{F}_k(t \mid \mathbf{x}_i) is the mean predicted CIF and \hat{F}_k^{\mathrm{AJ}}(t) is the marginal Aalen-Johansen CIF. The AJ estimator is fit internally from the supplied surv response.

Examples

Name the competing endpoints in the mgus2 dataset and check calibration of a marginal baseline model (which should be perfectly calibrated by construction):

import greenwood as gw
import numpy as np

# The first of progression (PCM) or death, each with its own time and status columns
mgus2 = gw.load_dataset("mgus2", backend="polars")
first = gw.Outcome.first_event(
    endpoints={"pcm": ("ptime", "pstat"), "death": ("futime", "death")}
)

# Fit Aalen-Johansen for the marginal CIF
aj = gw.AalenJohansen().fit(first, data=mgus2)
cif_df = aj.to_frame(format="polars")
pcm_cif = cif_df.filter(cif_df["cause"] == "pcm")

# Naive model: same marginal CIF for every subject
times = np.array([60, 120, 240])
marginal = np.array(
    [float(np.interp(t, pcm_cif["time"].to_numpy(), pcm_cif["estimate"].to_numpy()))
     for t in times]
)
probs = np.tile(marginal, (mgus2.height, 1))

# Calibration error should be near zero for the marginal model
cal_err = gw.calibration_incidence(
    first, incidence_prob=probs, times=times, cause=1, data=mgus2
)
cal_err

array([6.93889390e-18, 1.38777878e-17, 1.38777878e-17])