Skip to content

fix(NoTicket): keep fractional seconds in datetime query parameters - #527

Merged
lorenzhs merged 2 commits into
firebolt-db:mainfrom
aminghadersohi:fix-datetime-param-fractional-seconds
Oct 7, 2026
Merged

lorenzhs merged 2 commits into
firebolt-db:mainfrom
aminghadersohi:fix-datetime-param-fractional-seconds

Conversation

@aminghadersohi

@aminghadersohi aminghadersohi commented Sep 26, 2026 •

Copy link
Copy Markdown

Summary

StatementFormatter.format_value renders datetime parameters with '%Y-%m-%d %H:%M:%S', which silently drops fractional seconds:

cursor.execute("INSERT INTO t (ts) VALUES (?)", [datetime(2024, 2, 29, 23, 59, 58, 123456)])
# sent as: INSERT INTO t (ts) VALUES ('2024-02-29 23:59:58')

Firebolt TIMESTAMP/TIMESTAMPTZ store microseconds, so inserted values are truncated. Equality filters on sub-second timestamps also match the wrong rows. The same thing happens through SQLAlchemy, where the dialect passes Python datetimes straight to the cursor.

Change

Datetimes with a non-zero microsecond now render with .%f ('2024-02-29 23:59:58.123456'). Whole-second values render exactly as before, so existing queries and tests are unchanged. The aware-datetime handling (convert to UTC) is unchanged too.

Tests

  • New test_format_value / test_format_value_v1 cases for naive and aware datetimes with microseconds, plus a format_statement test. The 5 new cases fail on main and pass with this change.
  • pytest tests/unit: 782 passed, 1 skipped. mypy is clean on the changed module.
  • Checked against a live Firebolt 2.0 engine (4.32.6): 'YYYY-MM-DD HH:MM:SS.ffffff' literals insert into TIMESTAMP/TIMESTAMPTZ columns and compare with microsecond precision.

Related observation (not changed here)

Decimal parameters are rendered as bare literals (str(value)). The server types these as DOUBLE PRECISION, so values with more than ~15-17 significant digits lose precision, e.g. 99999999999999999999999999999.999999999 -> 99999999999999991434602339384.939855190.

I did not find a safe fix at the formatter level. The formatter does not know the target type, and on 4.32.6:

  • a typed literal such as CAST('…' AS NUMERIC(38, 9)) is rejected when it meets a different precision/scale (numeric(38, 9) can't be assigned to column … numeric(10, 2); Operations between decimals with different precision and scale is not supported);
  • a quoted string literal fails against integer operands (Unable to cast text '1.5' to integer).

Layers that know the column type (e.g. a SQLAlchemy dialect casting to the column's declared NUMERIC(p, s)) can make this exact. Happy to follow up if you have a preferred approach, e.g. via the server-side parameter path.


Note

Low Risk
Narrow change to datetime literal formatting with backward-compatible output for whole-second values; improves correctness for microsecond timestamps.

Overview
Fixes silent truncation of sub-second precision when Python datetime values are inlined into SQL via StatementFormatter.

format_value now appends .%f when microsecond is non-zero (after the existing UTC normalization for aware datetimes). Whole-second datetimes keep the previous 'YYYY-MM-DD HH:MM:SS' format, so behavior for existing callers should be unchanged except where fractional seconds matter for TIMESTAMP/TIMESTAMPTZ inserts and filters.

Unit tests cover naive and timezone-aware microsecond cases for both formatter versions, plus format_statement placeholder substitution.

Reviewed by Cursor Bugbot for commit afb38ea. Bugbot is set up for automated code reviews on this repo. Configure here.

StatementFormatter.format_value rendered datetime parameters with
'%Y-%m-%d %H:%M:%S', silently dropping microseconds. Inserted values were
truncated, and equality filters on sub-second timestamps matched the wrong
rows. Datetimes with a non-zero microsecond now render with '.%f'.
Whole-second values render exactly as before.
@aminghadersohi
aminghadersohi requested a review from a team as a code owner September 26, 2026 06:01

@lorenzhs lorenzhs left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, good catch!

@lorenzhs
lorenzhs merged commit c0c929b into firebolt-db:main Oct 7, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants