5 Validation in the Analysis Loop

The previous chapter built a formal validation plan, a named object that accumulates steps and produces a report you save, schedule, and share. That is one way to use Pointblank, and much of this book develops it, but it is not the only way and often not the first. In the middle of an analysis, working through a notebook cell by cell, you reach for validation in a lighter register: a quick check dropped between two steps to confirm that what you think just happened actually happened. These checks are cheap, they are frequently thrown away, and they catch mistakes the moment they occur rather than letting a bad assumption propagate through the rest of the analysis.

This chapter is about that sort of habit. The tools are the same ones the rest of the book covers, but the code is more minimal and straight-to-the-point, with the goal of making validation a commonplace activity during interactive work (rather than a separate and more involved phase that happens later). We’ll use the following orders table for the upcoming examples.

orders = pl.DataFrame({
    "order_id":    [1, 2, 3, 4, 5],
    "customer_id": ["C1", "C2", "C1", "C3", "C2"],
    "amount":      [50.0, 30.0, 20.0, 75.0, 40.0],
    "status":      ["paid", "paid", "refunded", "paid", "paid"],
})

5.1 A check as a single line

A really light use of Pointblank is through a one-line assertion that confirms an assumption. It should do nothing nothing visible when the assumption holds. Using a single check and then the sequence of interrogate() and then assert_passing() turns a interrogation plan into a quick check that’ll remain silent on success and raise an error on failure (ideal as it’s own notebook cell, for example).

pb.Validate(orders).col_vals_not_null(columns="customer_id").interrogate().assert_passing()

Every order has a customer identifier, so the check passes quietly. The reason to write it this way, instead of looking at a report, is that it stops your work the moment something is wrong. You find out now, not three steps later when a result looks odd.

The same pattern catches other wrong assumptions. Say you thought every amount was at least twenty-five, but the refunded order is actually twenty.

try:
    pb.Validate(orders).col_vals_ge(columns="amount", value=25).interrogate().assert_passing()
    print("all amounts are at least 25")
except AssertionError:
    print("stopped: an amount below 25 slipped through")

stopped: an amount below 25 slipped through

The error happens right here and it stops you before you build on something that isn’t true. These checks are fairly easy to compose, and inserting them wherever you make an assumption is a simple and great way to catch mistakes early.

5.2 Guarding a join

The single most valuable inline check guards a join, because a join is where silent, hard-to-spot corruption most often enters an analysis. The classic failure is a join key that is not unique on the table you join to. When a dimension table has a duplicate key, a join against it does not error, it quietly multiplies rows, and the inflated result flows into every count and sum downstream. The defense is to check the key for uniqueness before joining, not after. Consider a customer dimension that, through some upstream mistake, lists one customer twice.

customers = pl.DataFrame({
    "customer_id": ["C1", "C2", "C2", "C3"],
    "segment":     ["retail", "smb", "smb", "enterprise"],
})

pb.Validate(customers).rows_distinct(columns_subset=["customer_id"]).interrogate()

		STEP	COLUMNS	VALUES	TBL	EVAL	UNITS	PASS	FAIL	W	E	C	EXT
#4CA64C66	1	rows_distinct()	customer_id	—		✓	4	2 0.50	2 0.50	—	—	—

The check flags two rows, the duplicated C2 pair. This is exactly the warning you want before the join, because if you joined without checking, the duplicate would silently create extra rows in your result.

joined = orders.join(customers, on="customer_id", how="left")

orders.height, joined.height

(5, 7)

Five orders have become seven, because each of the two C2 orders matched both copies of the duplicated customer, inventing revenue that does not exist. Deduplicating the dimension first restores the expected behavior.

customers_clean = customers.unique(subset=["customer_id"])
joined_clean = orders.join(customers_clean, on="customer_id", how="left")

joined_clean.height

The join now preserves the five orders, because the key is unique on the dimension. Checking rows_distinct() on a join key is a habit worth forming, since it turns a whole class of silent row-multiplication bugs into an explicit, up-front check.

5.3 Confirming a join preserved its rows

The other side of checking the key beforehand is checking the row count afterward. A left join should not change the number of rows on the left side, so if the joined result has more rows than the original, something duplicated. But if it has fewer, then something was dropped. The row_count_match() check tests this directly. Here we run it on joined_clean (the join against the deduplicated customers) and on joined (the join that hit the duplicate key):

preserved = pb.Validate(joined_clean).row_count_match(count=orders.height).interrogate()
preserved.all_passed()

True

duplicated = pb.Validate(joined).row_count_match(count=orders.height).interrogate()
duplicated.all_passed()

False

The clean join passes and the duplicated one fails. We used orders.height (the original row count) as the expected value for count=, so the check is asking: did the join produce exactly as many rows as we started with? If you run this after every join, it catches both duplication that inflates a result and unexpected inner-join behavior that silently drops unmatched rows. Between checking the key for uniqueness before the join and checking the row count after it, you know exactly what happened.

5.4 Sanity-checking a transform

The same reflex applies to any step that reshapes the data. After a filter, you usually have an expectation about how many rows should survive, and stating it catches a predicate that did not mean what you thought. Filtering the orders to the paid ones should leave four.

paid = orders.filter(pl.col("status") == "paid")
pb.Validate(paid).row_count_match(count=4).interrogate().assert_passing()

paid.height

The assertion confirms the filter kept exactly the four paid orders, so a typo in the predicate or an unexpected status value would have stopped you here. The same idea covers a derived column that should fall in a known range, an aggregation that should produce one row per group, or a type conversion that should not introduce nulls. Each is a quick check that pins down what a transform was supposed to do, and each is most useful written immediately after the transform while the expectation is fresh.

Checking row count within a range

Sometimes you don’t know the exact number of rows to expect but you know roughly how many there should be. The tol= parameter in row_count_match() lets you allow some wiggle room. A single integer gives you a plus-or-minus range, a float between 0 and 1 gives a percentage, and a tuple lets you set different lower and upper bounds:

# Pass if the row count is within 2 of the expected (i.e., 3 to 7)
pb.Validate(data).row_count_match(count=5, tol=2).interrogate()

# Pass if the row count is within 10% of the expected
pb.Validate(data).row_count_match(count=100, tol=0.1).interrogate()

# Pass if the row count is between 95 and 110 (count=100, lower=-5, upper=+10)
pb.Validate(data).row_count_match(count=100, tol=(5, 10)).interrogate()

5.5 When to use a formal plan

The inline checks in this chapter and the formal plans in the rest of the book use the same tools, just with different levels of effort. An inline check is for catching your own mistakes as you work, and it is written, run, and often deleted within a single session. A formal plan is for data you receive repeatedly, for a pipeline that needs to pass before moving forward, or for expectations you want to document and share (written once and run many times). The two work well together, and often a quick check that keeps proving useful is a good candidate for a saved plan. Starting with the simple version first and promoting the checks that keep being useful, that’s how validation can become a commonplace part of an analysis rather than an extra step added on later.

Promoting an inline check

When you find yourself repeating the same inline check, it’s worth turning it into a formal plan. Gather those checks into a single Validate plan, give each step a label and a brief so the report is easy to read, add thresholds to define how much failure is acceptable, and save the plan to YAML with to_yaml() so it can be version-controlled. The full details of this are covered in Chapter 20.

5.6 Summary

Validation doesn’t have to be a separate phase that produces a saved report. It can also be something you do lightly and often while working. A single check ending with assert_passing() stays quiet when things are fine and raises an error when they’re not, which makes it a low-effort way to confirm the assumptions your analysis depends on. The most useful place for these checks is around joins: use rows_distinct() on the key before the join and row_count_match() on the result after it to catch duplicate or dropped rows. The same idea works for filters, derived columns, and aggregations—state what you expect right after the step that produces it.

These quick checks use the same methods as the formal plans covered in the rest of the book. When one keeps proving useful, it’s a good candidate for promotion into a saved plan. With this habit in place, the next chapter (Chapter 6) begins the tour of the validation methods that both inline checks and formal plans are built from.