CVResult

Result of cross_validate().

Usage

Source

CVResult(
    cv_score,
    cv_scores,
    cv_se,
    n_folds,
)

Holds the per-fold and aggregate out-of-sample loss values produced by k-fold cross-validation of a ~whittaker.gam.GAM specification, along with the number of folds used to obtain them. Use cv_score as a single summary number for model comparison, and cv_scores together with cv_se to gauge how much that summary varies across folds.

Parameters

cv_score: float

Mean out-of-sample loss across all folds, i.e. numpy.mean(cv_scores). This is on the scale of whichever metric was requested from cross_validate() — mean deviance per test observation for metric="deviance", or mean squared error on the response scale for metric="mse". Lower values indicate better out-of-sample predictive performance; use this value to compare competing formulas, families, or fitting methods evaluated on the same data and folds.

cv_scores: numpy.ndarray

Per-fold out-of-sample loss values, shape (n_folds,). Element i is the loss computed by fitting the GAM on every fold except i and scoring it on fold i. Inspect this array directly to check whether the CV estimate is driven by a small number of unusual folds.

cv_se: float

Standard error of the mean CV score across folds, computed as the sample standard deviation of cv_scores (with Bessel’s correction, ddof=1) divided by sqrt(n_folds). Provides a rough measure of the uncertainty in cv_score due to the particular random fold assignment; useful for judging whether a difference in cv_score between two models is likely to be meaningful.

n_folds: int

Number of folds actually requested when this result was produced. Matches the n_folds argument passed to cross_validate().