Model Count Data with Poisson

Fit a GAM when the response is a non-negative integer count.

Use the Poisson family whenever your response is a non-negative integer count (e.g., fish caught per survey, page views per hour, defects per batch). The log link means that each covariate has a multiplicative effect on the expected count, so the coefficients are interpretable as log-rate ratios. The smooth term s(temperature) captures any non-linear relationship without you having to guess the functional form.

Load data

Load the built-in fish dataset as a DataFrame and inspect its columns.

import whittaker as wk

data = wk.load_dataset("fish", as_frame=True)

# column names
data.columns.tolist()

['temperature', 'depth', 'count']

The fish dataset has three columns: temperature (water temperature in degrees Celsius), depth (sampling depth in meters), and count (fish observed per transect).

data.head()

	temperature	depth	count
0	13.232243	20.213893	7.0
1	13.969823	20.798765	14.0
2	24.284515	36.965959	3.0
3	9.838319	30.989895	2.0
4	20.002011	45.968733	4.0

Each row is one transect observation. Temperature has a curved relationship with count that a linear term would miss.

Fit

Fit a Poisson GAM with a smooth term for temperature and a linear term for depth.

model = wk.GAM(
    "count ~ s(temperature) + depth",
    family=wk.Poisson(),
).fit(data)

model.summary()

GAM fit summary
============================================================
Formula:    count ~ s(temperature) + depth
Family:     Poisson(link='log')
Observations: 300
Coefficients: 11

Parametric coefficients:
  Term                       Estimate    Std.Err    z value    p-value
  ------------------------ ---------- ---------- ---------- ----------
  (Intercept)                  2.1998     0.0479     45.901    < 1e-16
  depth                       -0.0249     0.0018    -13.916    < 1e-16

Approximate significance of smooth terms:
  Term                        EDF Ref.df     Chi.sq    p-value
  ------------------------ ------ ------ ---------- ----------
  s(temperature)             4.24      5    294.821    < 1e-16

Total EDF:  6.24
Deviance:   321.0387
Null dev:   843.4578
Dev. expl:  61.9%
GCV score:  1.116056
Scale est:  1.000000
AIC:        1304.77
BIC:        1327.88

The summary reports the effective degrees of freedom (EDF) for s(temperature). An EDF near 1 indicates a near-linear smooth. A higher EDF signals a more wiggly curve. The depth term enters as a parametric linear predictor because depth effects are often close to linear once temperature is accounted for.

Partial effects

Plotting the partial effects lets you see each smooth on its own scale.

wk.partial_effects(model)

Each panel shows the contribution of one term to the linear predictor, holding all other terms at their mean. The shaded band is a 95% pointwise confidence interval. A flat band crossing zero means the term is not contributing meaningfully.

On the response scale

Predictions are returned on the count scale automatically (there’s no need to apply exp() yourself). Sweep temperature across its observed range at a fixed depth to trace the fitted count curve.

import numpy as np

# Build grid over temperature at fixed depth
new_data = {
    "temperature": np.linspace(8, 28, 50),
    "depth": np.full(50, 10.0),
}

# Predict counts on the response scale
preds = model.predict(new_data).values

# first five predicted counts
preds[:5]

array([2.11545194, 2.38988486, 2.69989897, 3.04998022, 3.44464172])

Sweeping temperature from 8 to 28 at a fixed depth of 10 m gives a count curve that tracks the non-linear partial effect you saw in the plot above.

Overdispersion

If the residual deviance is much larger than the residual degrees of freedom, the Poisson assumption of equal mean and variance may be too restrictive. In that case swap the family for wk.NegativeBinomial(), which adds a dispersion parameter and absorbs the extra variation without changing the formula.