Skip to main content

16  Notifications and Observability

A validation only helps if someone finds out about the results. When a check fails on a scheduled run at three in the morning, a report saved to a folder does not help anyone. When validation runs unattended, a person or a system needs to be told when something goes wrong. This chapter covers sending results out of the validation process, whether as a message in a chat channel, as metrics in a monitoring system, or as a custom alert for whatever tools you already use.

These notifications are sent through the actions from Chapter 13, so you already know when they run. A step action runs when an individual check reaches a threshold, which suits an urgent alert about one check. A final action runs once after the whole plan and sees the complete results, which suits a summary of the run. Pointblank includes two ready-made notifiers, one for Slack and one for OpenTelemetry, and you can write your own for anything else. We use game_revenue for the examples that run.

game_revenue = pb.load_dataset(dataset="game_revenue", tbl_type="polars")

The most common place to send notifications is a chat channel, so we start with Slack.

16.1 Slack notifications

The send_slack_notification() function creates an action that posts to a Slack channel through an incoming webhook. Before you connect it to a plan, it helps to see what the messages will look like. If you call it with debug=True and no webhook, it prints sample messages instead of sending anything, so you can check them without a real channel.

pb.send_slack_notification(debug=True)

=== Step Notification Preview ===
*Data Validation Alert*
• Type: col_vals_gt
• Level: critical
• Step: 1
• Column: column_name
• Time: 2026-11-07 04:34:33.806127

=== Summary Notification Preview ===
*Data Validation Summary*
• Highest Severity: critical
• Total Steps: 5
• Failed Steps: 2
• Time: 2026-11-07 04:34:33.806130

The preview shows the two kinds of messages the notifier sends. The step message describes a single failing check, with its type, severity level, step number, column, and time. The summary message describes the whole run, with the highest severity, the number of steps, and how many steps failed. The values here are made-up samples, such as column_name for the column. Both messages use default templates, and you can replace either one. The summary_msg= and step_msg= arguments take template strings with placeholders in braces, which are filled in when the message is sent. Here is a custom summary template.

pb.send_slack_notification(
    debug=True,
    summary_msg="Validation of {tbl_name}: {highest_severity} ({n_failing_steps} of {n_steps} steps failed)",
)

=== Step Notification Preview ===
*Data Validation Alert*
• Type: col_vals_gt
• Level: critical
• Step: 1
• Column: column_name
• Time: 2026-11-07 04:34:33.812690

=== Summary Notification Preview ===
Validation of example_table: critical (2 of 5 steps failed)

The summary preview now reads “Validation of example_table: critical (2 of 5 steps failed)”, using the same sample values. The step message is unchanged, since we only replaced the summary template. Step templates can use placeholders such as {step}, {column}, {type}, {level}, {value}, {autobrief}, and {failure_text}. Summary templates can use {tbl_name}, {highest_severity}, {n_steps}, {n_failing_steps}, the per-level counts like {n_critical_steps}, and a few more, such as {tbl_row_count} and {validation_duration}. A custom step template works the same way.

pb.send_slack_notification(
    debug=True,
    step_msg="[{level}] step {step}: {type} failed on {column}",
)

=== Step Notification Preview ===
[critical] step 1: col_vals_gt failed on column_name

=== Summary Notification Preview ===
*Data Validation Summary*
• Highest Severity: critical
• Total Steps: 5
• Failed Steps: 2
• Time: 2026-11-07 04:34:33.819190

The step preview now reads “[critical] step 1: col_vals_gt failed on column_name”. Using debug=True like this is a quick way to get both templates right, which is much easier than adjusting them after they are connected to a real channel.

To send real messages, you give the function a webhook URL, and it returns a function that you attach as an action. The same notifier works in both roles. It sends a step message when used as a step action and a summary message when used as a final action.

notify = pb.send_slack_notification(
    webhook_url="https://hooks.slack.com/services/XXX/YYY/ZZZ"
)

validation = (
    pb.Validate(
        data=game_revenue,
        thresholds=pb.Thresholds(critical=0.10),
        actions=pb.Actions(critical=notify),
        final_actions=pb.FinalActions(notify),
    )
    .col_vals_not_null(columns="player_id")
    .interrogate()
)

As a critical step action, the notifier posts an alert as soon as any check reaches the critical threshold. As a final action, it posts a summary at the end of every run. This example is not run here, because it would post to a real channel.

Where you attach a notifier matters as much as what its messages say. A step action runs once for each step that reaches its threshold, not once per failing row, so a check on a million rows still sends only one alert. But attaching a notifier at the warning level across a large plan can still produce a flood of messages on a routine run, and people soon learn to ignore a channel like that. A better setup is to use step notifications only for the critical level, where an immediate alert is justified, and to let a single final action send the summary. That way the channel gets one summary per run plus the occasional urgent alert.

16.2 Exporting to OpenTelemetry

Chat messages are for people. For monitoring systems, the emit_otel() function exports validation results as OpenTelemetry signals, so they end up in whatever observability platform you already use. Used as a final action, it can send metrics, traces, and log records about the run.

NoteRequires the OpenTelemetry SDK and a collector

The examples in this section are not run here, because emit_otel() needs the opentelemetry libraries installed and a collector or backend set up to receive the signals. To run them, install the SDK alongside Pointblank and configure it to send to your collector.

otel = pb.emit_otel(enable_metrics=True, enable_tracing=True)

validation = (
    pb.Validate(
        data=game_revenue,
        tbl_name="game_revenue",
        final_actions=pb.FinalActions(otel),
    )
    .col_vals_not_null(columns="player_id")
    .col_vals_lt(columns="session_duration", value=40)
    .interrogate()
)

Metrics are on by default (enable_metrics=True is written out here for clarity). They include counts of steps and test units passed and failed, the pass rate, step durations, and how many steps reached each threshold level, all named with a pb.validation prefix that you can change with metric_prefix=. With enable_tracing=True, the run is recorded as a trace with a span for the whole validation and a child span for each step. The enable_logging=True option also sends log records for steps that reach a threshold, starting at the level set by log_level= (the default is "warning"). The extra_attributes= argument adds your own labels to every signal, such as an environment or a pipeline name, which makes the data easier to filter once it reaches your monitoring system. The data engineering scenario in Chapter 30 shows this kind of setup in a larger pipeline.

16.3 Building your own notifier

Slack and OpenTelemetry are the two built-in notifiers, but you can send results anywhere with a final action of your own. Inside the action, get_validation_summary() returns a dictionary describing the run, and you use its values to decide what to send and where. To start, this action just prints a few of the summary’s keys so we can see what it contains.

def show_summary():
    s = pb.get_validation_summary()
    print({k: s[k] for k in [
        "tbl_name", "highest_severity", "n_steps",
        "n_failing_steps", "n_critical_steps", "overall_health_score",
    ]})

_ = (
    pb.Validate(
        data=game_revenue,
        tbl_name="game_revenue",
        thresholds=pb.Thresholds(warning=0.02, error=0.05, critical=0.10),
        final_actions=pb.FinalActions(show_summary),
    )
    .col_vals_lt(columns="session_duration", value=40)
    .col_vals_in_set(columns="acquisition", set=["organic", "google"])
    .interrogate()
)
{'tbl_name': 'game_revenue', 'highest_severity': 'critical', 'n_steps': 2, 'n_failing_steps': 2, 'n_critical_steps': 1, 'overall_health_score': 83.8}

Both steps failed, and one of them (the acquisition check) reached critical, so highest_severity is "critical". The overall health score from Chapter 15 is 83.8, pulled down mostly by the 574 rows that fail the acquisition check. The full summary has more than this, including counts of steps at each level, lists of passing and failing step numbers, the dimension scores, the table’s size, and how long the run took. Once you know what is in the summary, you can make the action decide what to do based on highest_severity.

def notify_on_severity():
    s = pb.get_validation_summary()
    if s["highest_severity"] == "critical":
        print(f"PAGE: critical data quality failure in {s['tbl_name']} "
              f"({s['n_failing_steps']} of {s['n_steps']} steps failed)")
    elif s["highest_severity"] in ("error", "warning"):
        print(f"LOG: {s['highest_severity']} in {s['tbl_name']}")
    else:
        print(f"OK: {s['tbl_name']} passed all checks")

_ = (
    pb.Validate(
        data=game_revenue,
        tbl_name="game_revenue",
        thresholds=pb.Thresholds(warning=0.02, error=0.05, critical=0.10),
        final_actions=pb.FinalActions(notify_on_severity),
    )
    .col_vals_lt(columns="session_duration", value=40)
    .col_vals_in_set(columns="acquisition", set=["organic", "google"])
    .interrogate()
)
PAGE: critical data quality failure in game_revenue (2 of 2 steps failed)

The action runs at the end of the interrogation. Since the acquisition check reached critical, it takes the first branch and prints the “PAGE” message. In a real notifier, the print() calls would be replaced with a call to an email service, a webhook, or a ticketing system, but the structure is the same: read the summary, check the severity, and send the message. This works for any destination that does not have a built-in notifier.

NoteOther destinations

At the time of writing, Slack and OpenTelemetry are the only built-in notifiers. For email, generic webhooks, PagerDuty, Discord, or anything else, use a custom final action like the one above.

16.4 Summary

Notifications are what make unattended validation useful, since they get the results to the people and systems that need to act on them. They are sent through the actions from Chapter 13, with step actions for urgent alerts about individual checks and final actions for run summaries. A step action runs once per step that reaches its threshold, not once per failing row, but attaching notifiers too widely still leads to too many alerts. A good default is to use step notifications only at the critical level and send everything else in a single summary. The send_slack_notification() function posts to Slack, and its debug=True mode lets you preview the step_msg= and summary_msg= templates before sending anything.

The emit_otel() function exports metrics, traces, and log records to an OpenTelemetry backend, so data quality shows up alongside the rest of your monitoring. For any other destination, write a final action that reads get_validation_summary(), checks highest_severity, and sends the message. This chapter ends the part of the book on responding to results. The next part covers validation that uses language models, starting with Chapter 17.