Skip to content

Preserve timestamp casts when unit or timezone changes - #128

Merged
osipovartem merged 1 commit into
embucket-sync-df55.0.0from
fix-df55-timestamp-cast-unwrap
Oct 9, 2026
Merged

osipovartem merged 1 commit into
embucket-sync-df55.0.0from
fix-df55-timestamp-cast-unwrap

Conversation

@osipovartem

Copy link
Copy Markdown
Collaborator

Summary

  • Keep timestamp CAST/TRY_CAST in comparison and IN expressions when the unit or timezone changes. Widening an i64 timestamp can overflow, narrowing loses precision, and attaching a non-UTC timezone can shift values. Removing the cast changes strict errors, NULLs, and comparison results.
  • Preserve the existing unwrap fast path for unchanged timestamp types and other supported casts.

Validation

  • cargo test -p datafusion-optimizer --lib --locked: 763 passed.
  • cargo clippy -p datafusion-optimizer --lib --tests --locked: passed.
  • Formatting and diff checks passed. Independent read-only review approved correctness, performance tradeoff, API compatibility, and tests.

Performance

  • Retaining the cast costs CPU for in-range rows. A local Arrow 60 release microbenchmark (1M rows, 20 iterations, median of 7) measured direct equality at 3.46-3.53B rows/s versus cast plus equality at 1.19-1.23B rows/s (~2.9x slower kernel throughput). This is not an end-to-end SQL result; pruning and I/O can dominate. A safe fast path requires proven source-value bounds.

The same change has been prepared for Apache DataFusion upstream. Rustice currently pins a divergent older fork SHA, so its dependency update will be handled separately rather than including unrelated fork changes here.

@osipovartem
osipovartem merged commit 81bf4c5 into embucket-sync-df55.0.0 Oct 9, 2026
17 of 19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant