API Reference

Validate

If going down the road of preparing a data quality analysis, you’ll need the Validate class to get the process started. The object is given the target table along with a stepwise validation plan composed via validation methods.

Validate	Workflow for defining a set of validations on a table and interrogating for results.
Thresholds	Definition of threshold values.

Validation Steps

The validation steps are the individual validation methods that can be used to build a validation plan. These methods are used to interrogate the data and report on the results.

Validate.col_vals_gt	Are column data greater than a fixed value or data in another column?
Validate.col_vals_lt	Are column data less than a fixed value or data in another column?
Validate.col_vals_ge	Are column data greater than or equal to a fixed value or data in another column?
Validate.col_vals_le	Are column data less than or equal to a fixed value or data in another column?
Validate.col_vals_eq	Are column data equal to a fixed value or data in another column?
Validate.col_vals_ne	Are column data not equal to a fixed value or data in another column?
Validate.col_vals_between	Do column data lie between two specified values or data in other columns?
Validate.col_vals_outside	Do column data lie outside of two specified values or data in other columns?
Validate.col_vals_in_set	Validate whether column values are in a set of values.
Validate.col_vals_not_in_set	Validate whether column values are not in a set of values.
Validate.col_vals_null	Validate whether values in a column are NULL.
Validate.col_vals_not_null	Validate whether values in a column are not NULL.
Validate.col_vals_regex	Validate whether column values match a regular expression pattern.
Validate.col_exists	Validate whether one or more columns exist in the table.

Interrogation and Reporting

The interrogation and reporting methods are used to extract data and report on the results.

Validate.interrogate	Execute each validation step against the table and store the results.
Validate.get_data_extracts	Get the rows that failed for each validation step.
Validate.get_json_report	Get a report of the validation results as a JSON-formatted string.
Validate.get_tabular_report	Validation report as a GT table.
Validate.get_sundered_data	Get the data that passed or failed the validation steps.
Validate.all_passed	Determine if every validation step passed perfectly, with no failing test units.
Validate.n	Provides a dictionary of the number of test units for each validation step.
Validate.n_passed	Provides a dictionary of the number of test units that passed for each validation step.
Validate.n_failed	Provides a dictionary of the number of test units that failed for each validation step.
Validate.f_passed	Provides a dictionary of the fraction of test units that passed for each validation step.
Validate.f_failed	Provides a dictionary of the fraction of test units that failed for each validation step.
Validate.warn	Provides a dictionary of the warning status for each validation step.
Validate.stop	Provides a dictionary of the stopping status for each validation step.
Validate.notify	Provides a dictionary of the notification status for each validation step.

Utilities

The utilities group will contain functions that are helpful for the validation process.

load_dataset

Load a dataset hosted in the library as specified DataFrame type.