Skip to content

feat!: pass input schema to ExprPlanner::plan_scalar - #26177

Merged
alamb merged 1 commit into
apache:mainfrom
fengys1996:feat/plan-scalar-schema
Oct 16, 2026
Merged

alamb merged 1 commit into
apache:mainfrom
fengys1996:feat/plan-scalar-schema

Conversation

@fengys1996

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

  • Closes #.

Rationale for this change

Scalar function planners need access to the input schema to inspect argument types and field metadata, enabling expression rewrites.

What changes are included in this PR?

Add a &DFSchema parameter to ExprPlanner::plan_scalar and pass the input schema during SQL planning.

What is the testing strategy for this PR?

Updated select_scalar_func_with_expr_planner to verify that the planner can resolve a column argument’s type using the input schema and correctly rewrite the function call.

Are there any user-facing changes?

Yes. Implementations overriding ExprPlanner::plan_scalar must accept an additional &DFSchema parameter. This is a breaking API change.

@github-actions github-actions Bot added sql SQL Planner logical-expr Logical plan and expressions labels Oct 10, 2026
@github-actions

Copy link
Copy Markdown

Thank you for opening this pull request!

Reviewer note: cargo-semver-checks reported the current version number is not SemVer-compatible with the changes in this pull request (compared against the base branch).

Details
     Cloning apache/main
    Building datafusion-expr v55.1.0 (current)
       Built [  23.208s] (current)
     Parsing datafusion-expr v55.1.0 (current)
      Parsed [   0.067s] (current)
    Building datafusion-expr v55.1.0 (baseline)
       Built [  23.782s] (baseline)
     Parsing datafusion-expr v55.1.0 (baseline)
      Parsed [   0.067s] (baseline)
    Checking datafusion-expr v55.1.0 -> v55.1.0 (no change; assume patch)
     Checked [   1.141s] 223 checks: 222 pass, 1 fail, 0 warn, 31 skip

--- failure trait_method_parameter_count_changed: pub trait method parameter count changed ---

Description:
A trait method now takes a different number of parameters.
        ref: https://doc.rust-lang.org/cargo/reference/semver.html#trait-item-signature
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.50.0/src/lints/trait_method_parameter_count_changed.ron

Failed in:
  ExprPlanner::plan_scalar now takes 2 instead of 1 parameters, in file /home/runner/work/datafusion/datafusion/datafusion/expr/src/planner.rs:269

     Summary semver requires new major version: 1 major and 0 minor checks failed
    Finished [  50.122s] datafusion-expr
    Building datafusion-sql v55.1.0 (current)
       Built [  31.991s] (current)
     Parsing datafusion-sql v55.1.0 (current)
      Parsed [   0.025s] (current)
    Building datafusion-sql v55.1.0 (baseline)
       Built [  32.749s] (baseline)
     Parsing datafusion-sql v55.1.0 (baseline)
      Parsed [   0.023s] (baseline)
    Checking datafusion-sql v55.1.0 -> v55.1.0 (no change; assume patch)
     Checked [   0.185s] 223 checks: 223 pass, 31 skip
     Summary no semver update required
    Finished [  66.126s] datafusion-sql

@github-actions github-actions Bot added the auto detected api change Auto detected API change label Oct 10, 2026
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 83.33333% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 82.77%. Comparing base (d137137) to head (c8cab03).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
datafusion/sql/src/expr/function.rs 0.00% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #26177      +/-   ##
==========================================
- Coverage   82.78%   82.77%   -0.01%     
==========================================
  Files        1148     1148              
  Lines      451075   451079       +4     
  Branches   451075   451079       +4     
==========================================
- Hits       373400   373392       -8     
- Misses      54927    54939      +12     
  Partials    22748    22748              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@fengys1996
fengys1996 marked this pull request as ready for review October 12, 2026 01:53
@fengys1996

Copy link
Copy Markdown
Contributor Author

@alamb PTAL when you have time.

@alamb alamb left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @fengys1996 -- this PR makes sense to me; I think a small note in the upgrade guide, and perhaps a small example use / test that shows using the input schema would be valuable.

///
/// Returns the original scalar function if not possible
fn plan_scalar(&self, expr: RawScalarExpr) -> Result<PlannerResult<RawScalarExpr>> {
fn plan_scalar(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you please add a note in the upgrade guide for this change as it is an API change?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I took another look and checked the 55.0.0 and 55.2.0 tags, and ExprPlanner::plan_scalar isn't present in either. It was introduced in #24846 for the upcoming 56.0.0 release, so this signature change doesn't affect an API from a previous release. Given that, could we consider this PR non-breaking? 😄

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

perfect!

@alamb alamb removed the auto detected api change Auto detected API change label Oct 16, 2026
@alamb

alamb commented Oct 16, 2026

Copy link
Copy Markdown
Contributor

FYI @MichaelScofield

@alamb
alamb enabled auto-merge October 16, 2026 14:13

@alamb alamb left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @fengys1996

@alamb
alamb added this pull request to the merge queue Oct 16, 2026
Merged via the queue into apache:main with commit 396ea7c Oct 16, 2026
42 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

logical-expr Logical plan and expressions sql SQL Planner v56.0.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants