Skip to content

Support string categorical features in PartialDependence - #684

Open
raashish1601 wants to merge 2 commits into
interpretml:mainfrom
raashish1601:fix/217-pdp-categorical
Open

raashish1601 wants to merge 2 commits into
interpretml:mainfrom
raashish1601:fix/217-pdp-categorical

Conversation

@raashish1601

Copy link
Copy Markdown
Contributor

Fixes #217

PartialDependence converted the whole data matrix to float64 before building the curves, so any string categorical column (for example a model that is a pipeline with a OneHotEncoder) failed with ValueError: could not convert string to float, with or without feature_types.

X = pd.DataFrame({"age": rng.choice(["20-29", "30-39", "40-49"], n), "size": rng.normal(size=n)})
model = make_pipeline(ColumnTransformer([("onehot", OneHotEncoder(), ["age"])], remainder="passthrough"), LogisticRegression()).fit(X, y)
PartialDependence(model, X)  # ValueError: could not convert string to float: '40-49'

_gen_pdp already uses the unique values as grid points for nominal and ordinal features, so the change only keeps the data as an object array when it can't be converted, and converts each continuous column to float on its own. Numeric data still goes through the float64 path as before.

Tests: added tests/blackbox/test_partialdependence.py, which checks the curves for a string feature and a continuous feature against the mean prediction with that feature set to each grid value. It fails on main and passes with this change. ruff format is clean; ruff check reports only the three warnings that are already on main in _partialdependence.py.

Signed-off-by: Raashish Aggarwal <94279692+raashish1601@users.noreply.github.com>
@codecov

codecov Bot commented Oct 7, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 67.38%. Comparing base (d82ea62) to head (a3afa04).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #684      +/-   ##
==========================================
+ Coverage   67.32%   67.38%   +0.05%     
==========================================
  Files          77       77              
  Lines       11739    11745       +6     
==========================================
+ Hits         7903     7914      +11     
+ Misses       3836     3831       -5     
Flag Coverage Δ
bdist_linux_311_python 67.13% <100.00%> (+0.07%) ⬆️
bdist_linux_312_python ?
bdist_linux_313_python ?
bdist_linux_314_python 67.04% <100.00%> (+0.07%) ⬆️
bdist_linuxarm_311_python 67.15% <100.00%> (+0.07%) ⬆️
bdist_linuxarm_312_python ?
bdist_linuxarm_313_python ?
bdist_linuxarm_314_python 67.05% <100.00%> (+0.07%) ⬆️
bdist_mac_311_python 67.29% <100.00%> (+0.07%) ⬆️
bdist_mac_312_python ?
bdist_mac_313_python ?
bdist_mac_314_python 67.18% <100.00%> (+0.05%) ⬆️
bdist_win_311_python 67.27% <100.00%> (+0.02%) ⬆️
bdist_win_312_python ?
bdist_win_313_python ?
bdist_win_314_python 67.21% <100.00%> (+0.06%) ⬆️
sdist_linux_311_python ?
sdist_linux_312_python ?
sdist_linux_313_python ?
sdist_linux_314_python ?
sdist_linuxarm_311_python ?
sdist_linuxarm_312_python ?
sdist_linuxarm_313_python ?
sdist_linuxarm_314_python ?
sdist_mac_311_python ?
sdist_mac_312_python ?
sdist_mac_313_python ?
sdist_mac_314_python ?
sdist_win_311_python ?
sdist_win_312_python ?
sdist_win_313_python ?
sdist_win_314_python ?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@raashish1601

Copy link
Copy Markdown
Contributor Author

The failing jobs don't come from this change: the scheduled interpret-CI runs on main fail the same way (e.g. run 37461974875 on 560f8dd). The test jobs fail while importing interpret.glassbox._aplr (Cannot create a consistent method resolution order (MRO) for bases RegressorMixin, LocalExplainer, GlobalExplainer, BaseEstimator, APLRRegressor), the docs job fails in the same notebooks, and test_powerlift fails on sqlalchemy.orm.attributes.ScalarAttributeImpl.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The focused changes preserve numeric behavior, include regression coverage, and have no identified blocking issues.

No open issues 🎉

What changed in this PR

Enables PartialDependence to handle string categorical features, addressing #217 while preserving the existing numeric conversion path.

Changes:

  • Retains mixed-type data when whole-matrix float conversion fails.
  • Converts continuous columns individually for grid and density calculations.
  • Adds regression coverage comparing categorical and continuous curves against model predictions.
File Description
python/​interpret-core/​tests/​blackbox/​test_partialdependence.py Tests partial dependence curves with mixed string and numeric features.
python/​interpret-core/​interpret/​blackbox/​_partialdependence.py Adds mixed-type fallback and per-column numeric conversion.

Review effort: Balanced

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

Categorical features in PartialDependence()?

3 participants