integrated_brier_score()

Integrated (time-averaged) Brier score across multiple time points.

Usage

integrated_brier_score(
    surv,
    survival_prob,
    times,
)

Summarizes Brier scores computed at multiple time horizons into a single summary metric via trapezoidal integration. This provides an overall calibration measure that doesn’t emphasize any particular time point.

Use this to: Reduce multiple Brier scores (one per time point) to a single number for model comparison. A single IBS score makes it easier to compare two models or report a single “calibration quality” metric.

Interpretation: Same scale as Brier score (0 = perfect, 1 = worst). Values of 0.15-0.25 are typical for reasonable survival models; values > 0.30 suggest poor calibration.

Parameters

surv: Surv: A right-censored Surv response.
survival_prob: Any: Predicted survival probabilities, shape (n_subjects, n_times).
times: Any: Evaluation times (must be at least 2 to define an interval). The IBS is computed as the area under the Brier-score curve from times[0] to times[-1], normalized by the time span.

Returns

float: Integrated Brier score (time-averaged). Lower is better.

Details

Computation: The integrated Brier score is

\[ IBS = \frac{1}{t_{\max} - t_{\min}} \int BS(t) \, dt \]

Using trapezoidal rule to approximate the integral. This ensures the summary score balances contributions from all times without emphasizing early or late horizons.

Time scale sensitivity: IBS weights contribution proportionally to time intervals. If you have many time points clustered early (e.g., 10 times in [0, 100] and 1 time at [1000]), early times dominate. Use evenly-spaced time points for balanced assessment.

Examples

Fit a Cox model and compute its integrated Brier score over a range of times:

import greenwood as gw

lung = gw.load_dataset("lung", backend="polars")
y = gw.Surv.right(lung["time"], event=(lung["status"] == 2))
cox = gw.CoxPH().fit(y, lung[["age", "sex"]])

times = [180, 365, 540]
surv_pred = cox.predict(lung[["age", "sex"]], type="survival", times=times, format="pandas")
probs = surv_pred.iloc[:, 1:].to_numpy().T
ibs = gw.integrated_brier_score(y, probs, times)
ibs

0.212620468119578

Compare two models via their integrated Brier scores. Lower is better:

# cox2 = CoxPH().fit(y, lung[["age", "sex", "ph.ecog"]])  # More covariates
# surv_pred2 = cox2.predict(...)
# ibs2 = gw.integrated_brier_score(y, probs2, times)
# print(f"Model 1 IBS: {ibs:.3f}")
# print(f"Model 2 IBS: {ibs2:.3f}")
# print(f"Better model: {'Model 2' if ibs2 < ibs else 'Model 1'}")

Compute integrated Brier score over a wide range of times to get an overall calibration assessment:

times_wide = list(range(100, 700, 50))
surv_pred_wide = cox.predict(
    lung[["age", "sex"]], type="survival", times=times_wide, format="pandas"
)
probs_wide = surv_pred_wide.iloc[:, 1:].to_numpy().T
ibs_wide = gw.integrated_brier_score(y, probs_wide, times_wide)
print(f"IBS over {len(times_wide)} time points: {ibs_wide:.3f}")

IBS over 12 time points: 0.200