Skip to content

Align Arrow pin for CSV Boolean alias support - #131

Merged
osipovartem merged 2 commits into
rustice-df55-case-physical-pinfrom
df55-arrow-csv-snowflake-boolean-aliases
Oct 10, 2026
Merged

osipovartem merged 2 commits into
rustice-df55-case-physical-pinfrom
df55-arrow-csv-snowflake-boolean-aliases

Conversation

@osipovartem

Copy link
Copy Markdown
Collaborator

Which issue does this PR close?

No issue; this synchronizes the DataFusion fork used by Rustice with Embucket/arrow-rs#18.

Rationale for this change

Rustice COPY INTO must accept Snowflake CSV BOOLEAN aliases. Pinning only Rustice to the updated Arrow revision creates two Arrow 59.2.0 sources because DataFusion directly pins the older commit. The entire dependency graph must use one revision.

What changes are included in this PR?

  • Pin the DataFusion workspace Arrow/Parquet dependencies to the tested Arrow CSV revision.
  • Update the opt-in CSV Boolean rustdocs to list all accepted aliases.
  • Correct three stale ASOF diagnostic expectations and one timezone comparison fixture; these were pre-existing local suite failures and do not change runtime code.

Are these changes tested?

  • cargo +1.97.0 fmt --all and cargo +1.97.0 clippy --all-targets --all-features --locked --offline -- -D warnings passed.
  • Full DataFusion extended workspace suite passed with RUST_BACKTRACE=1 and --test-threads 4, including 508/508 SQLLogicTest files. The initial run exposed missing test-data submodules; those were initialized. A 32-thread run had a parallelism-sensitive spill failure, so the final full run used four threads.
  • cargo metadata --locked --offline confirms exactly one Arrow 59.2.0 git source. Arrow CSV unit and doc tests passed in the dependency PR; release microbenchmark showed no material common-token regression.
  • Independent read-only review approved the final diff, including the timestamp instant and ASOF assertions.

Are there any user-facing changes?

No DataFusion runtime API change. Rustice opt-in CSV scans gain the Snowflake Boolean aliases from the Arrow dependency; the default strict CSV reader is unchanged.

@osipovartem
osipovartem merged commit fd6e2c6 into rustice-df55-case-physical-pin Oct 10, 2026
9 of 17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant