cascade.cascade()

Execute a cascade consensus: start with one model, fan out if confidence is low.

Usage

Source

cascade.cascade(
    prompt,
    responder,
    *,
    confidence_threshold=0.6,
    max_fan_out=3,
    fan_out_strategy=ConsensusStrategy.MAJORITY,
    candidates=None,
    routing_strategy=RoutingStrategy.BALANCED
)

The cascade works in two phases:

1. Initial query: Routes the prompt to the best available model and queries it. If the response confidence is above confidence_threshold, returns immediately.

2. Fan-out: If confidence is low, queries up to max_fan_out additional models from the routing alternatives, then runs consensus across all responses.

Parameters

prompt: str

The task or prompt text.

responder: Responder

A callable that takes (model_key, prompt) and returns the response text. This keeps the cascade framework-agnostic — you provide the actual LLM call.

confidence_threshold: float = 0.6

Minimum confidence to accept the initial response without fan-out (default 0.6).

max_fan_out: int = 3

Maximum number of additional models to query during fan-out (default 3).

fan_out_strategy: ConsensusStrategy = ConsensusStrategy.MAJORITY


Consensus strategy to use when fan-out occurs (default MAJORITY).

candidates: list[str] | None = None

Specific model keys to consider for routing. None uses all registered models.

routing_strategy: RoutingStrategy = RoutingStrategy.BALANCED


Strategy for the initial model selection (default BALANCED).

Returns

CascadeResult

The cascade outcome including winner, confidence, rounds, and whether fan-out occurred.

Raises

ValueError

If no candidate models are available.

Examples

import talk_box as tb

def ask_model(model_key: str, prompt: str) -> str:
    # Your LLM call here
    return "The answer is 42."

result = tb.cascade("What is the meaning of life?", ask_model)
result.winner          # "The answer is 42."
result.fanned_out      # False (if initial response was confident)
result.confidence      # ~0.75
result.rounds          # [CascadeRound(round_number=1, ...)]