game_revenue = pb.load_dataset(dataset="game_revenue", tbl_type="polars")13 Thresholds and Actions
The earlier chapters showed how to write checks, but knowing that 74 rows failed a check does not tell you what to do about it. Seventy-four failures out of 2,000 might be normal background noise, or it might be the first sign of a broken feed. A plain pass or fail result cannot tell those apart. This chapter covers the two features that can: thresholds, which classify how serious a failure is, and actions, which respond to it automatically.
Together, thresholds and actions let validation run without someone reading every report. You set thresholds that decide when a failure counts as a warning, an error, or a critical problem, and you attach actions that respond at each level, such as logging a warning or stopping a pipeline on a critical failure. We use game_revenue throughout, with checks whose failure rates land at different severity levels.
We start with thresholds, since they are what give a failure its severity.
13.1 Classifying failures with thresholds
Without a threshold, a check has only two outcomes: every test unit passed, or at least one failed. In practice some imperfection is usually expected, and the real question is how much. Pointblank has three severity levels. A warning is a minor problem worth looking into when you have time, an error is a real problem that should be fixed soon, and a critical result is serious enough that you might stop downstream work or alert someone right away. What each level means is up to you and your team, but using them consistently means a warning in one plan means the same thing as a warning in another.
NoteMapping levels to responses
It helps to think of each level in terms of the response it should trigger. A warning might write a log entry for later review, an error might alert the team that owns the data, and a critical result might stop the pipeline or page someone on call. The actions covered later in this chapter attach responses like these to each level, and the notifiers in Chapter 16 can send them to Slack, email, or a monitoring system. Choosing the threshold value and the response together is what turns a severity label into a working rule.
A threshold can be a proportion or a count. A proportion like 0.05 means 5 percent of test units, so it scales with the data: 5 failures in 100 rows, or 50,000 in a million. A count like 100 stays the same no matter how big the table is. Use a proportion when the rate of failure is what matters, and a count when even a few bad records are a problem. Thresholds are usually created with pb.Thresholds(warning=..., error=..., critical=...). You do not have to set all three, and any level you leave out is never checked. You can also pass a tuple, like (0.02, 0.05, 0.10), for the three levels in order, or a dictionary like {"warning": 0.02, "error": 0.05} to set only the levels you name.
13.1.1 Global thresholds
Thresholds set on the Validate object apply to every step. The plan below uses a warning at 0.02, an error at 0.05, and a critical level at 0.10, with three checks chosen to land at different levels.
(
pb.Validate(
data=game_revenue,
thresholds=pb.Thresholds(warning=0.02, error=0.05, critical=0.10),
)
.col_vals_lt(columns="session_duration", value=40)
.col_vals_in_set(columns="acquisition", set=["organic", "google"])
.col_vals_lt(columns="item_revenue", value=100)
.interrogate()
)The report now has status indicators showing which level each step reached. The first step, which flags long sessions, fails on 74 rows. That is a rate of 0.037, which is above the 0.02 warning but below the 0.05 error, so it shows a warning. The second step, which allows only two acquisition channels, fails on 574 rows (a rate of about 0.287), which is past all three levels, so it shows as critical. The third step, which flags very large revenue, fails on only 6 rows. That is a rate of 0.003, below even the warning level, so it has failures but does not reach any threshold. That third case is worth noticing: a step can have failing test units and still be below every threshold.
13.1.2 Step-level thresholds
Not every check needs the same tolerance, and a threshold set on an individual step overrides the global one for that step. To see this, we set a global warning of 0.05 and then give the long-session check its own, stricter warning of 0.02.
strict_step = (
pb.Validate(data=game_revenue, thresholds=pb.Thresholds(warning=0.05))
.col_vals_lt(
columns="session_duration", value=40,
thresholds=pb.Thresholds(warning=0.02),
)
.interrogate()
)
strict_step.warning(){1: True}The step reaches the warning level. Its failure rate of 0.037 is below the global 0.05, so with the global threshold alone it would not have. The step’s own 0.02 threshold replaced the global one. Step-level thresholds are how you hold your most important checks to a higher standard, such as allowing almost no failures on a primary key while allowing more on an optional field.
13.1.3 Counts versus proportions
A proportion and a count can classify the same failures differently. The large-revenue check fails on 6 rows out of 2,000. As a proportion that is 0.003, well below a warning at 0.05, so no warning is raised.
prop = (
pb.Validate(data=game_revenue, thresholds=pb.Thresholds(warning=0.05))
.col_vals_lt(columns="item_revenue", value=100)
.interrogate()
)
prop.warning(){1: False}No warning. A count threshold asks a different question, which is whether the number of bad records has reached a certain size, no matter how small a share of the table they are. Setting the warning at a count of 5 catches the same six failures.
count = (
pb.Validate(data=game_revenue, thresholds=pb.Thresholds(warning=5))
.col_vals_lt(columns="item_revenue", value=100)
.interrogate()
)
count.warning(){1: True}Now the warning is raised, because 6 failures is more than 5. (A threshold is reached when failures meet it, so warning=6 would also be raised here, but warning=7 would not.) The same six rows are ignored under the proportion and flagged under the count. Use a proportion when you care about the rate and expect it to scale with the data. Use a count when even a few bad records need attention, such as duplicate primary keys or negative account balances.
13.2 Reading threshold results
After interrogation, you can see threshold results in the report and in code. The report shows each level as an indicator, empty if the threshold was set but not reached and filled if it was reached. When the levels are set in the usual increasing order, a step that reaches critical also shows error and warning. In code, the warning(), error(), and critical() methods each return a dictionary keyed by step number, telling you which steps reached that level.
graded = (
pb.Validate(
data=game_revenue,
thresholds=pb.Thresholds(warning=0.02, error=0.05, critical=0.10),
)
.col_vals_lt(columns="session_duration", value=40)
.col_vals_in_set(columns="acquisition", set=["organic", "google"])
.interrogate()
)
graded.critical(){1: False, 2: True}The first step did not reach critical and the second did. These methods are what you check when a plan controls whether a pipeline continues. For example, if any(graded.critical().values()) lets you stop processing if any step reached the critical level.
13.3 Responding with actions
Thresholds classify a problem, and actions respond to it. Without actions, a plan just produces a report that someone has to remember to read. With actions, something happens as soon as a threshold is reached during interrogation. An action can be a string, which is printed, a function, which is called, or a list of these, which run in order. An action set on the Validate object applies to every step. An action set on an individual step replaces the global action for that step.
The most useful actions look at the failure that triggered them. Inside an action, the get_action_metadata() function returns a dictionary describing the current step, which your function can use to build a specific message.
def log_issue():
m = pb.get_action_metadata()
print(f"[warn] step {m['step']} ({m['type']}) on '{m['column']}' reached {m['level']}")
_ = (
pb.Validate(
data=game_revenue,
thresholds=pb.Thresholds(warning=0.02),
actions=pb.Actions(warning=log_issue),
)
.col_vals_lt(columns="session_duration", value=40)
.interrogate()
)[warn] step 1 (col_vals_lt) on 'session_duration' reached warningThe action runs because the long-session check reaches the 0.02 warning, and it prints a line built from the metadata. The metadata includes the step number (step), the assertion type (type), the column (column), the comparison value (value), and the level reached (level). It also has a numeric level_num, a time stamp, a generated description in autobrief, and a ready-made message in failure_text that you can print as is. The failure counts are not part of the metadata. If you need them, read them from the validation object after interrogation with methods like n_failed().
13.3.1 Choosing which actions fire
When a step reaches several levels at once, you can choose whether only the most severe action runs or all of them do. The highest_only= argument of Actions controls this, and it defaults to True. The check below fails on most rows, so it reaches warning, error, and critical together, and with the default only the critical action runs.
actions_highest = pb.Actions(
warning=lambda: print("warning"),
error=lambda: print("error"),
critical=lambda: print("critical"),
highest_only=True,
)
_ = (
pb.Validate(
data=game_revenue,
thresholds=pb.Thresholds(warning=0.05, error=0.10, critical=0.20),
actions=actions_highest,
)
.col_vals_eq(columns="item_type", value="iap")
.interrogate()
)criticalOnly “critical” is printed, so you get one alert for one problem instead of three. With highest_only=False, every level that was reached runs its own action, starting with the most severe, so the same check would print “critical”, then “error”, then “warning”. That is useful when the responses build on each other, such as always logging at warning, also emailing at error, and also paging someone at critical.
13.4 Final actions
Step actions respond to individual steps as they run. A final action runs once after every step has finished, whether or not anything failed, which makes it the place for a summary report or an overall go or no-go decision. Inside a final action, get_validation_summary() returns a dictionary describing the whole run.
def summary_report():
s = pb.get_validation_summary()
print(f"{s['n_steps']} steps, warning={s['n_warning_steps']} critical={s['n_critical_steps']}, highest={s['highest_severity']}")
_ = (
pb.Validate(
data=game_revenue,
thresholds=pb.Thresholds(warning=0.02, error=0.05, critical=0.10),
final_actions=pb.FinalActions(summary_report),
)
.col_vals_lt(columns="session_duration", value=40)
.col_vals_in_set(columns="acquisition", set=["organic", "google"])
.interrogate()
)2 steps, warning=2 critical=1, highest=criticalThe final action reports that two steps ran, both reached the warning level, one reached critical, and the highest severity in the run was critical. The summary dictionary has much more than this, including the table name and dimensions, per-step counts and results, how long the validation took, and an overall health score. Because a final action always runs, it is a reliable place for anything that must happen on every run, such as writing an audit log entry.
13.5 Asserting quality in tests
Thresholds are also useful in a test suite, where you want a validation failure to fail a test. The assert_passing() method raises an AssertionError if any test unit in any step failed. On a plan where everything passes, it does nothing.
clean = (
pb.Validate(data=game_revenue)
.col_vals_gt(columns="session_duration", value=0)
.interrogate()
)
clean.assert_passing()
print("all checks passed")all checks passedEvery session duration is positive, so the assertion passes without a sound and the message prints. On a plan with failures, it raises an error instead, and the error message lists the failing steps.
try:
graded.assert_passing()
except AssertionError as e:
print(str(e).splitlines()[0])The following assertions failed:We print only the first line of the message here. The full message goes on to list both failing steps, each described in words. The assert_below_threshold() method is more forgiving. It raises only when a given level was reached, not whenever anything failed, so use it when some failure is acceptable but a serious one is not. The strict_step plan from earlier reaches only the warning level, so asserting that it stayed below the error level passes, even though the check had failures.
strict_step.assert_below_threshold(level="error")
print("stayed below the error level")stayed below the error levelThe assertion passes because no step reached error. Checking the graded plan against the critical level gives the opposite result, because its acquisition check reached critical.
try:
graded.assert_below_threshold(level="critical")
except AssertionError as e:
print(str(e).splitlines()[0])The following steps exceeded the critical threshold level:In a testing framework such as pytest, these methods let you check test data and transformation outputs the same way you check production data. Use assert_passing() when no failures are acceptable and assert_below_threshold() when some are.
13.6 Automatic notifications
Actions can do more than print, and Pointblank includes ready-made actions for sending notifications. The send_slack_notification() action posts to a Slack channel and works as either a step action or a final action, and the emit_otel() action exports results as OpenTelemetry signals for a monitoring system. Setting these up involves details beyond thresholds and actions, so they are covered in Chapter 16, along with how to build your own notifier from a final action.
13.7 Summary
Thresholds and actions let validation classify and respond to problems without someone reading every report. A threshold, set with pb.Thresholds() as either a proportion or a count, grades a failure as a warning, an error, or critical. Thresholds on the Validate object apply to every step, and a threshold on an individual step replaces the global one for that step. After interrogation, the warning(), error(), and critical() methods tell you which steps reached each level.
Actions run when a threshold is reached. An action can be a string, a function, or a list, and a function can call get_action_metadata() to learn about the step that triggered it. The highest_only= argument decides whether only the most severe action runs or every level’s action does. A final action runs once after the whole plan and can use get_validation_summary() to see the entire run. The assert_passing() and assert_below_threshold() methods bring the same ideas into test suites. The next chapter, Chapter 14, moves from responding to results to presenting them, through reports and extracts of the failing rows.