Many validation methods have a columns= argument that can be used to specify the columns for validation (e.g., col_vals_gt(), col_vals_regex(), etc.). The last_n() selector function can be used to select n columns positioned at the end of the column list. So if the set of table columns consists of
[age, rev_01, rev_02, profit_01, profit_02]
and you want to validate the last two columns, you can use columns=last_n(2). This will select the profit_01 and profit_02 columns and a validation step will be created for each.
The offset= parameter can be used to skip a certain number of columns from the end of the column list. So if you want to select the third and fourth columns from the end, you can use columns=last_n(2, offset=2).
Parameters
n:int
The number of columns to select from the end of the column list. Should be a positive integer value. If n is greater than the number of columns in the table, all columns will be selected.
offset:int=0
The offset from the end of the column list. The default is 0. If offset is greater than the number of columns in the table, no columns will be selected.
Returns
:LastN
A LastN object, which can be used to select the last n columns.
Relevant Validation Methods where last_n() can be Used
This selector function can be used in the columns= argument of the following validation methods:
col_vals_gt()
col_vals_lt()
col_vals_ge()
col_vals_le()
col_vals_eq()
col_vals_ne()
col_vals_between()
col_vals_outside()
col_vals_in_set()
col_vals_not_in_set()
col_vals_null()
col_vals_not_null()
col_vals_regex()
col_exists()
The last_n() selector function doesn’t need to be used in isolation. Read the next section for information on how to compose it with other column selectors for more refined ways to select columns.
Additional Flexibilty through Composition with Other Column Selectors
The last_n() function can be composed with other column selectors to create fine-grained column selections. For example, to select all column names starting with “rev” along with the last two columns, you can use the last_n() and starts_with() functions together. The only condition is that the expressions are wrapped in the col() function, like this:
col(last_n(2) | starts_with("rev"))
There are four operators that can be used to compose column selectors:
& (and)
| (or)
- (difference)
~ (not)
The & operator is used to select columns that satisfy both conditions. The | operator is used to select columns that satisfy either condition. The - operator is used to select columns that satisfy the first condition but not the second. The ~ operator is used to select columns that don’t satisfy the condition. As many selector functions can be used as needed and the operators can be combined to create complex column selection criteria (parentheses can be used to group conditions and control the order of evaluation).
Examples
Suppose we have a table with columns name, paid_2021, paid_2022, paid_2023, and paid_2024 and we’d like to validate that the values in the last four columns are greater than 10. We can use the last_n() column selector function to specify that the last four columns in the table are the columns to validate.
From the results of the validation table we get four validation steps. The values in all those columns were all greater than 10.
We can also use the last_n() function in combination with other column selectors (within col()) to create more complex column selection criteria (i.e., to select columns that satisfy multiple conditions). For example, to select the last four columns but also omit those columns that end with "2023", we can use the - operator to combine column selectors.