find_disagreements()

Detect disagreements between model responses.

Usage

Source

find_disagreements(responses)

Compares each pair of responses and flags significant differences. Uses word-level similarity to classify severity.

Parameters

responses: list[ModelResponse]

List of model responses to compare.

Returns

list[Disagreement]

Detected disagreements, sorted by severity (major first).

Examples

import talk_box as tb

responses = [
    tb.ModelResponse(model="model_a", text="Python was created in 1991."),
    tb.ModelResponse(model="model_b", text="Python was created in 1989."),
]

disagreements = tb.find_disagreements(responses)
disagreements[0].severity  # "major"