From 683028b9eb8829663f1972f47eb106c649533af9 Mon Sep 17 00:00:00 2001 From: shinichitazawa Date: Sun, 11 Oct 2026 23:21:56 +0900 Subject: [PATCH] docs: fix kl_divergence and chi_square audit arguments and semantics The documented examples used argument names that the built-in audits do not accept, so copying them fails with "Macro variable 'source_column' is undefined": - kl_divergence takes source_column / target_column (not column / target_column) - chi_square takes column_a / column_b (not column / target_column) The chi_square description also said the audit ensures the statistic does not exceed the critical value. The implementation does the opposite by default: with dependent := true (the default) it asserts that the columns are dependent and fails when the statistic does not exceed the critical value. Document the dependent argument and both behaviours, and fix the "statistic0" typo. Also align the kl_divergence usage comment in builtin.py with its arguments. Signed-off-by: shinichitazawa --- docs/concepts/audits.md | 10 ++++++---- sqlmesh/core/audit/builtin.py | 2 +- 2 files changed, 7 insertions(+), 5 deletions(-) diff --git a/docs/concepts/audits.md b/docs/concepts/audits.md index c7c7cbd190..3945a5c5b1 100644 --- a/docs/concepts/audits.md +++ b/docs/concepts/audits.md @@ -664,13 +664,15 @@ This example asserts that the symmetrised KL Divergence between columns `age` an ```sql linenums="1" MODEL ( audits ( - kl_divergence(column := age, target_column := reference_age, threshold := 0.1) + kl_divergence(source_column := age, target_column := reference_age, threshold := 0.1) ) ); ``` #### chi_square, chi_square_non_blocking -Ensures that the [chi-square](https://en.wikipedia.org/wiki/Chi-squared_test) statistic for two categorical columns does not exceed a critical value. +Performs a [chi-square](https://en.wikipedia.org/wiki/Chi-squared_test) test of independence between two categorical columns, `column_a` and `column_b`, and compares the statistic with `critical_value`. + +By default (`dependent := true`), the audit asserts that the columns are dependent: it fails when the chi-square statistic does not exceed the critical value. Set `dependent := false` to assert that the columns are independent: the audit then fails when the statistic exceeds the critical value. You can look up the critical value corresponding to a p-value with a table (such as [this one](https://www.medcalc.org/manual/chi-square-table.php)) or by using the Python [scipy library](https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.chi2.html): @@ -680,12 +682,12 @@ from scipy.stats import chi2 # critical value for p-value := 0.95 and degrees of freedom := 1 chi2.ppf(0.95, 1) ``` -This example asserts that the chi-square statistic0 for columns `user_state` and `user_type` does not exceed 6.635: +This example asserts that columns `user_state` and `user_type` are dependent, i.e. that their chi-square statistic exceeds 6.635: ```sql linenums="1" MODEL ( audits ( - chi_square(column := user_state, target_column := user_type, critical_value := 6.635) + chi_square(column_a := user_state, column_b := user_type, critical_value := 6.635) ) ); ``` diff --git a/sqlmesh/core/audit/builtin.py b/sqlmesh/core/audit/builtin.py index b4eaab8d50..8aae9ba89a 100644 --- a/sqlmesh/core/audit/builtin.py +++ b/sqlmesh/core/audit/builtin.py @@ -524,7 +524,7 @@ def create_non_blocking_copy(audit: Audit) -> Audit: """, ) -# kl_divergence(column=age, target_column=normalized_age, threshold=0.1) +# kl_divergence(source_column=age, target_column=normalized_age, threshold=0.1) kl_divergence_audit = ModelAudit( name="kl_divergence", defaults={"condition": exp.true()},