ContextWindow

Manages fitting content into a model’s token budget.

Usage

Source

ContextWindow()

Combines token estimation with truncation strategies to ensure prompts and conversation messages stay within a model’s context window.

Parameters

max_tokens: int | None = None

Explicit token budget. If provided, overrides any model profile lookup.

model: str | ModelProfile | None = None

A model key (e.g., "ollama:llama3.2:latest") or a ~talk_box.models.ModelProfile instance. The profile’s context_window is used as the budget.

reserve_output: int = 1024

Tokens to reserve for the model’s response. Subtracted from the budget before fitting input content. Defaults to 1024.

strategy: str | FitStrategy = FitStrategy.TRUNCATE_OLDEST

The default strategy for fitting content. Can be overridden per call.

token_counter: Callable[[str], int] | None = None
Optional custom token counting function. Defaults to estimate_tokens.

Examples

import talk_box as tb

# From a model profile
ctx = tb.ContextWindow(model="ollama:llama3.2:latest")

# With explicit budget
ctx = tb.ContextWindow(max_tokens=8192)

# With custom settings
ctx = tb.ContextWindow(
    max_tokens=32_768,
    reserve_output=4096,
    strategy="truncate_middle",
)