Evaluate a survival model’s out-of-sample performance using k-fold cross-validation.
cross_validate(
model,
surv,
covariates,
*,
data=None,
k=5,
metric="concordance",
times=None,
stratified=True,
seed=None
)
Provides an honest, unbiased estimate of model performance by splitting data into folds, fitting on training folds, and evaluating on held-out test folds. This avoids overfitting bias that occurs when fitting and scoring on the same data.
Why cross-validate? Fitting and scoring on the training data gives overly optimistic performance estimates. A model may fit the training data well due to overfitting, not true predictive ability. Cross-validation repeatedly fits on different training splits and evaluates on held-out data, simulating performance on new subjects.
Metrics:
"concordance" (default): Harrell’s C-statistic on the test fold. Higher is better (0.5 = random, 1.0 = perfect). Requires CoxPH, CoxNet, or AFT model."brier": Integrated IPCW Brier score over specified times. Lower is better (0 = perfect calibration, 1 = worst). Requires explicit times= parameter.
Parameters
model: Any-
An unfitted estimator instance (e.g., CoxPH(), CoxNet(), AFT("weibull")). A fresh copy is fit on each training fold, leaving the passed object unchanged. Supported: CoxPH, CoxNet, AFT (for concordance) and any of those (for Brier).
surv: Surv-
A Surv response (time-to-event data). Can be right-censored or counting-process. Weights in the response are carried through the cross-validation.
covariates: Any-
Covariates/predictors for the model. Can be:
- A 2-D array or pandas/Polars DataFrame with one row per subject
- A formula string (as in CoxPH.fit()), evaluated against
data
data: Any = None-
If covariates is a formula string, the data frame to evaluate it against.
k: int = 5-
Number of folds (default 5). Each fold serves as test data once; subjects are split randomly and evenly across folds. Typical choices: 5 or 10.
metric: str = "concordance"-
Performance metric for evaluation:
"concordance" (default): Harrell’s C-statistic. Requires CoxPH, CoxNet, or AFT."brier": Integrated inverse-probability-of-censoring-weighted (IPCW) Brier score. Requires times= with at least 2 time points.
times: Any = None-
For metric="brier", evaluation time points (1-D array-like, length \ge 2). The Brier score is computed at each time, then integrated (time-averaged). Example: times=[365, 730, 1095] for 1, 2, 3-year predictions.
stratified: bool = True-
If True (default), use stratified k-fold ensuring balanced event/censoring representation across folds. This prevents singular matrix errors and biased CV estimates on imbalanced survival data (rare events). If False, use simple random k-fold shuffling.
seed: int | None = None-
Random seed for fold shuffling, ensures reproducibility. If
None, results may vary between runs. Use a fixed seed for consistent comparisons.
Returns
dict-
Dictionary with keys:
"metric": Metric name used ("concordance" or "brier")."k": Number of folds."scores": List of per-fold scores (one per fold)."mean": Mean score across folds (primary summary)."std": Standard deviation of scores (variability estimate).
For concordance, higher mean is better. For Brier, lower mean is better.
Details
How folds work: By default (stratified=True), subjects are grouped by event status (censored vs. event, or multiple event types), then randomly shuffled within each stratum and split into k roughly equal-sized groups. This ensures each fold has approximately the same proportion of events and censored observations as the overall dataset. This is crucial for imbalanced data (e.g., rare events) to prevent singular matrix errors and ensures unbiased cross-validation estimates.
If stratified=False, subjects are simply shuffled and split randomly, which may lead to folds with very different event rates and can destabilize model fitting on sparse data.
Completeness: Subjects with missing covariates are dropped before folding. This ensures all folds use the same cleaned data, avoiding alignment issues.
AFT model note: For AFT, concordance uses the negated linear predictor (since in AFT, larger lp means longer survival, opposite to Cox). This is handled automatically.
Reproducibility: Set seed= to ensure the same folds are used across runs. This is important for comparing different models or reporting consistent results.
Examples
Evaluate a Cox model with 5-fold cross-validation using concordance:
import greenwood as gw
# Load data and build a right-censored response
lung = gw.load_dataset("lung", backend="polars")
y = gw.Surv.right(lung["time"], event=(lung["status"] == 2))
# Run 5-fold cross-validation with concordance
result = gw.cross_validate(
gw.CoxPH(), y, lung[["age", "sex"]], k=5, metric="concordance", seed=1
)
result
{'metric': 'concordance',
'k': 5,
'scores': [0.5459697732997482,
0.6741645244215938,
0.5275779376498801,
0.696520618556701,
0.5741469816272966],
'mean': 0.603675967111044,
'std': 0.07678157638643948}
Access individual components. The mean concordance across folds:
# Mean concordance across folds
result["mean"]
Per-fold scores (variability check):
# Per-fold concordance scores
result["scores"]
[0.5459697732997482,
0.6741645244215938,
0.5275779376498801,
0.696520618556701,
0.5741469816272966]
Standard deviation (estimate of generalization uncertainty):
# Standard deviation of fold scores
result["std"]
Use Brier score (calibration) instead of concordance (discrimination):
# Evaluate calibration with the integrated Brier score
result_brier = gw.cross_validate(
gw.CoxPH(), y, lung[["age", "sex"]], k=5,
metric="brier", times=[180, 365, 540], seed=1
)
result_brier
{'metric': 'brier',
'k': 5,
'scores': [0.21362074884792628,
0.23838812973571863,
0.23271598116657624,
0.19349410160275995,
0.21970890492256906],
'mean': 0.21958557325511002,
'std': 0.017623159308294382}
Compare two models via cross-validation. Model with higher mean concordance (or lower mean Brier) generalizes better:
# Compare a simple vs. complex model (uncomment to run)
# simple_model = gw.CoxPH()
# complex_model = gw.CoxPH()
# simple_cv = gw.cross_validate(simple_model, y, lung[["age"]], seed=1)
# complex_cv = gw.cross_validate(complex_model, y, lung[["age", "sex", "ph.ecog"]], seed=1)
# print(f"Simple model C-index: {simple_cv['mean']:.3f} ± {simple_cv['std']:.3f}")
# print(f"Complex model C-index: {complex_cv['mean']:.3f} ± {complex_cv['std']:.3f}")