ShrinkageCRS

Shrinkage Cubic Regression Spline.

Usage

Source

ShrinkageCRS(k=10)

Equivalent to mgcv’s bs="cs" basis. CRS penalizes only the curvature of a natural cubic spline, leaving its 2-dimensional null space of constant and linear functions completely free — a large smoothing parameter flattens the fit to a line, but never removes it from the model. ShrinkageCRS adds a second penalty, built directly from the eigenstructure of the ordinary CRS penalty, that specifically targets this null space. Following the double-penalty approach of Marra & Wood (2011), the two penalties are given independent smoothing parameters during fitting, so that both the wiggly and the linear/constant parts of the term can be shrunk simultaneously, letting the term drop out of the model entirely. Prefer ShrinkageCRS over plain CRS for univariate terms whose presence in the model is uncertain and where automatic selection via GCV or REML is preferred over an explicit inclusion/exclusion test.

The basis functions are identical to CRS (bs="cr"); only the penalty structure differs.

Parameters

k: int = 10
Number of basis functions (equal to the number of knots). Must be at least 3. The default is 10. See CRS for guidance on choosing k.

Notes

ShrinkageCRS reuses the exact basis construction of CRS: k knots at evenly-spaced quantiles of the training data, with the design matrix built from natural-cubic-spline basis functions (see CRS for the full construction). What changes is the penalty. The ordinary CRS penalty is

\mathbf{S}_{\text{wiggle}} = \mathbf{Q} \mathbf{R}^{-1} \mathbf{Q}^\top,

which is positive semi-definite with rank k - 2 and a 2-dimensional null space spanned by the constant and linear functions evaluated at the knots. ShrinkageCRS eigendecomposes S_wiggle, identifies the eigenvectors U_null whose eigenvalues are numerically zero (below 1e-10 times the largest eigenvalue), and builds a second penalty matrix from their outer product:

\mathbf{S}_{\text{null}} = \mathbf{U}_{\text{null}} \mathbf{U}_{\text{null}}^\top.

S_null is positive semi-definite with rank 2, non-zero exactly on the subspace that S_wiggle leaves unpenalized. During fitting each matrix is scaled by its own smoothing parameter and the two contributions are summed:

\lambda_{\text{wiggle}} \, \boldsymbol{\beta}^\top \mathbf{S}_{\text{wiggle}} \boldsymbol{\beta} + \lambda_{\text{null}} \, \boldsymbol{\beta}^\top \mathbf{S}_{\text{null}} \boldsymbol{\beta}.

Because S_wiggle + S_null is strictly positive definite, the combined penalty has no null space, so null_space_dimension() returns 0: none of the k basis functions escapes penalization once both smoothing parameters are positive.

Examples

import numpy as np
from whittaker.smooths import ShrinkageCRS

rng = np.random.default_rng(0)
x = rng.uniform(0, 1, 100)

basis = ShrinkageCRS(k=10).fit(x)
B = basis.basis_matrix(x)
S_wiggle, S_null = basis.penalty_matrices()
B.shape, S_wiggle.shape, S_null.shape
((100, 10), (10, 10), (10, 10))

Methods

Name Description
null_space_dimension() Return 0: the combined double penalty leaves no unpenalized null space.
penalty_matrices() Return the two penalty matrices used for shrinkage.

null_space_dimension()

Return 0: the combined double penalty leaves no unpenalized null space.

Usage

Source

null_space_dimension()

Returns

int
Always 0 for ShrinkageCRS, since S_wiggle + S_null is positive definite.

penalty_matrices()

Return the two penalty matrices used for shrinkage.

Usage

Source

penalty_matrices()

Returns

list[NDArray]
A two-element list [S_wiggle, S_null], each of shape (k, k). S_wiggle is the ordinary CRS wiggliness penalty Q R^{-1} Q'. S_null is a projection penalty built from the eigenvectors of S_wiggle with (numerically) zero eigenvalue, targeting the constant and linear null space. Each matrix is intended to be paired with its own independently-estimated smoothing parameter during fitting.