accuracy_in_time()

Time-dependent classification accuracy for competing-risks predictions.

Usage

accuracy_in_time(
    surv,
    incidence_probs,
    times,
    *,
    data=None,
)

At each evaluation time, predicts each subject’s most likely outcome (survival or one of the competing events) by taking the argmax of predicted cumulative incidence functions, then compares against the observed outcome. Subjects censored before the evaluation time are excluded.

This metric answers: “At a given time horizon, does the model correctly identify which event a subject will experience?” It complements the concordance index, which measures relative ranking rather than absolute classification.

Interpretation:

1.0: Every uncensored subject’s most likely predicted event matches the observed event.
Higher is better.
Early times tend to have high accuracy because most subjects have survived and the model predicts survival. The metric becomes more informative at later times, when the model must discriminate between competing events.

Parameters

surv: Surv | Outcome | str: A multi-state Surv response (from Surv() with a categorical event). An Outcome or a formula response such as 'Surv(time, status == 2)' is also accepted, with its columns read from data.
incidence_probs: Any: Predicted cumulative incidence probabilities, shape (n_subjects, n_causes, n_times). Axis 1 indexes causes in the same order as the states in the Surv response (cause 1 first, cause 2 second, etc.), including declared states with no observed events in the evaluation data. The survival probability (no event) is computed internally as 1 - sum(CIFs) and used as class 0 in the argmax.
times: Any: Evaluation times. 1-D array-like. Must have length equal to the third dimension of incidence_probs=.
data: Any = None: A data frame (pandas, Polars, PyArrow, DuckDB, a lazy frame, …) holding the columns named by the response and incidence_probs. When surv is an Outcome or a formula, rows with a missing value in any column used are dropped first, along with the matching rows of any arrays passed alongside.

Returns

ndarray: Classification accuracy at each time, shape (len(times),). Values are in [0, 1].

Details

At each time horizon \zeta:

Subjects censored before \zeta (status = 0 and T_i \le \zeta) are excluded.
The predicted class is \hat{y}_i = \arg\max_{k \in \{0,\dots,K\}} \hat{F}_k(\zeta \mid \mathbf{x}_i), where \hat{F}_0 = \hat{S} is the predicted survival probability.
The observed class is y_{i,\zeta} = \delta_i \cdot \mathbb{1}(T_i \le \zeta), which is 0 (survived) if the subject has not yet experienced any event by \zeta, and \delta_i if an event occurred.
Accuracy is the proportion of uncensored subjects whose predicted class matches the observed class.

Examples

Name the competing endpoints in the mgus2 dataset and evaluate the accuracy of a marginal Aalen-Johansen model:

import greenwood as gw
import numpy as np

# The first of progression (PCM) or death, each with its own time and status columns
mgus2 = gw.load_dataset("mgus2", backend="polars")
first = gw.Outcome.first_event(
    endpoints={"pcm": ("ptime", "pstat"), "death": ("futime", "death")}
)

# Fit Aalen-Johansen for marginal CIFs
aj = gw.AalenJohansen().fit(first, data=mgus2)
cif_df = aj.to_frame(format="polars")
pcm_cif = cif_df.filter(cif_df["cause"] == "pcm")
death_cif = cif_df.filter(cif_df["cause"] == "death")

# Build (n_subjects, n_causes, n_times) prediction array
times = np.array([60, 120, 240])
pcm_vals = np.array(
    [float(np.interp(t, pcm_cif["time"].to_numpy(), pcm_cif["estimate"].to_numpy()))
     for t in times]
)
death_vals = np.array(
    [float(np.interp(t, death_cif["time"].to_numpy(), death_cif["estimate"].to_numpy()))
     for t in times]
)
probs = np.stack(
    [np.tile(pcm_vals, (mgus2.height, 1)), np.tile(death_vals, (mgus2.height, 1))], axis=1
)

acc = gw.accuracy_in_time(first, incidence_probs=probs, times=times, data=mgus2)
acc

array([0.63884786, 0.58021613, 0.83711747])