Repository navigation
feat: SQLite and Cloudflare D1 query dialects (#49, fuzzed) - #119
Merged
Merged
Conversation
…ractions
Found by running the fuzzer and the core builder suite on SQLite and D1:
- LIKE has no default escape on SQLite, so `like("org\\_1")` matched a
literal backslash and never `org_1`. The dialect now writes
`ESCAPE '\'`, so `\` escapes `%`, `_` and `\` as on every dialect.
- SQLite drivers end text at U+0000: node:sqlite reads `'a' || char(0)`
back as `'a'`, bound or literal. A string with U+0000 is refused at
compile time instead of being stored and read back cut short. (The
hex-cast literal kept NUL in the SQL, but not in the value.)
- SQLite parses some decimals to a neighbouring double, as a literal or
through json_extract (`1.23456789e-95`). A fraction in an expression
is bound instead of written as text; integers are still written.
- `Amount > 9.75` was refused because SQLite.integer only encodes whole
numbers. A fraction compared with an integer column now compiles, as
on Postgres and ClickHouse; inserting one is still refused.
docs/sqlite.md now states SQLite's ASCII-case-insensitive LIKE.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…QLite and D1 - tests/core.sqlite.test.ts runs every shared core case on node:sqlite and on a local D1 binding (Miniflare), with SQL snapshots for both. - tests/fuzz/dialects/sqlite.ts registers `sqlite` and `d1` with the fuzzer: query plans against the reference model, literal round-trips for every SQLite column type, compile invariants and hostile values. Their semantics: NULL first ascending, truncating integer division, `%` on integers, ASCII-case-insensitive LIKE. - The real reference row is built from the double's mantissa and powers of two, since SQLite cannot parse some decimals exactly. - New core cases pin LIKE escapes and case on every dialect; ILIKE moves to its own case so `patterns-and-lists` also runs on SQLite. - Counterexamples pinned in tests/fuzz/regressions.ts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Caution Review failedThe pull request is closed. Finishing Touches
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This was referenced Oct 25, 2026
…st 2^53
`SQLite.timestamp` stored and compared ISO text (`…T16:28:54.750Z`), while
SQLite's functions write `YYYY-MM-DD HH:MM:SS`. SQLite compares text by
bytes, and ' ' sorts before 'T', so a row a DEFAULT stamped just now was
not later than an hour ago: wrong rows, no error. Timestamps now use
SQLite's own form, `YYYY-MM-DD HH:MM:SS.SSS` in UTC (what strftime('%f')
and datetime(..., 'subsec') write), for inserts, comparisons and params;
both forms decode, and zoneless text is UTC. The literal fuzz's reference
row is now written by SQLite's datetime(), so builder values must equal
the database's own text. `sqliteTimestampText` is exported for raw SQL.
`SQLite.integer` decoded any integer-valued number. D1 rounds an INTEGER
past 2^53 to a different number, which then passed; it is a decode error
now.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Unions: `unionAll` of two or three row plans that select the same aliases (each alias one sort in every branch), each branch with its own join, filter, DISTINCT, ORDER BY and LIMIT, under the union's ORDER BY, LIMIT and OFFSET. The model concatenates the branches' rows and pages them; the union pages only when it orders by every alias, so ties are equal rows. New properties `unions match the model (<target>, …)`. Writes: a new suite. Each input gets a fresh scratch table on the dialect's first target and runs a sequence of builder writes: multi-row inserts leaving defaulted or nullable columns out, INSERT … SELECT, ON CONFLICT DO NOTHING / DO UPDATE (with the existing and the excluded row), UPDATE and DELETE with a filter or allRows(), and RETURNING. After every write the table is read back and compared with a JS model of its rows, and RETURNING with the rows the write touched. A write the dialect does not support is still generated and must be refused at compile time. `FuzzDialect.writes` declares the capabilities and the scratch table (query `Table`, create/drop DDL, `exec`); `DialectModule` gains unionAll, compileUnion, insertInto, update and deleteFrom. ClickHouse targets pass mutations_sync=2 and lightweight_deletes_sync=2 so an ALTER TABLE UPDATE or DELETE is visible to the next read. Found: a ClickHouse 26 UNION ALL column whose branches have numeric types with no common supertype (UInt64 and Int64, Int64 and Float64) becomes a Variant, and ORDER BY on it fails. Recorded in known-issues (clickhouse-union-variant-order-by) and pinned. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A UNION ALL column of numbers whose branches have no common ClickHouse
type (`Id + 1` is UInt64 and `Id - 1` Int64; Int64 and Float64) becomes a
Variant, and an ORDER BY on it fails ("Variant/Dynamic are not allowed in
ORDER BY keys"); with use_variant_as_common_type = 0 the union itself is
refused. The union fuzzer found it.
On a dialect with `unionNumberCast` (ClickHouse), a column is selected as
toFloat64(…) in every branch when every branch selects a number and their
types may differ: different declared types, or different computations.
Every numeric ClickHouse type decodes to a JS number, so Float64 loses
nothing the decode keeps. The known-issues entry is gone; the pinned
counterexample now passes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Registers the write capabilities from #122 for sqlite and d1: every write the dialect has (INSERT … SELECT, UPDATE, DELETE, ON CONFLICT, RETURNING), and per-cell DEFAULT refused, as SQLite has none in VALUES. D1 compiles its unions with the D1 dialect. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ant for that org
The tenant oracle took "single-tenant" to mean the orgId param's tenant.
`OrgId IN ('org_2')` pins a query to org_2, which the builder rightly
reports as single-tenant; the oracle called reading org_2 a leak. Where
the plan's WHERE filters OrgId itself, any one org is right; otherwise
it must still be the param's. Found by a 20x soak on D1; pinned.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts: # tests/fuzz/regressions.ts
…t inside it Inside a branch the cast ran before that branch's DISTINCT, so two UInt64 values past 2^53 (one Float64) became one row. The cast now wraps each branch, SELECT toFloat64(x) AS x, … FROM (<branch>), so DISTINCT, GROUP BY and LIMIT see exact values (CodeRabbit). The writes suite now accepts only a QueryBuilderError or a QueryBuilderDefect as the refusal of a clause a dialect lacks, not any defect. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…never cast its values The previous fix cast such a column to Float64, which an independent review showed corrupts integers past 2^53 wherever the values stay on the server: INSERT ... SELECT from the union stored 9007199254740992 for 9007199254740993, and an IN over it matched nothing. Only the union's ORDER BY was ever broken (a Variant is not a valid sort key), so only the sort key changes now: toFloat64(x), then accurateCastOrNull(x, 'Int128'), which keeps integers past 2^53 in exact order. Checked on ClickHouse 26.8 with UInt64/Int64 values around 2^53 and 2^64 - 1. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…dialect flag From an independent review: - A timestamp past year 9999 (new Date(8.64e15), a common max-date sentinel) was written as '+275760-09 13T00:00:00.', which then failed to decode, and `at > year 10000` bound text that sorts before every real timestamp, so it matched every row. Inserts, comparisons and params now refuse it at compile time; sqliteTimestampText throws. - `integer > 9.75` keyed off the dialect name, so a profile spread from sqliteDialect under another name lost it. It is the dialect flag integerComparesFraction now. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…er a throw The range check ran after a union member's encoder, which threw a RangeError for a Date before year 0, so compile died instead of failing with a QueryBuilderError (found by the hostile-values fuzz property). Each member of the timestamp codec now checks the range on its decoded side, and the encoder never throws; dateTimeLiteral fails with a typed error too. Pinned. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… accepts on a Variant toFloat64 refuses a Variant on ClickHouse 26.2 (CI); CAST(x AS Nullable(Float64)) works on 26.2 and 26.8, Variant or not. Checked both versions with the full suite and every fuzz property at 20x. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rite, rows still checked The known-issue matcher for this ClickHouse server bug covered two shapes; a 20x soak on 26.2 found a third (`(a OR 0 OR a) AND (a OR 0)`). Rather than chase shapes, a ClickHouse target that gets exactly this error runs the query again with optimize_extract_common_expressions = 0, and its rows are compared with the model as usual: these inputs are now checked, not skipped. The matcher and its helpers are gone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Under a 20x soak Miniflare's local socket occasionally drops ('fetch
failed'). Like the ClickHouse helper, the D1 helper retries transport
errors only; a database error still fails.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fails with TOO_FEW_ARGUMENTS_FOR_FUNCTION if the rerun without the rewrite is removed; passes, rows checked, with it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts: # tests/fuzz/regressions.ts
Contributor
Author
|
@coderabbitai review |
|
Makisuo
added this pull request to stack #124
October 25, 2026 19:16
This was referenced Oct 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is #49 by @tao-io, with review fixes on top. Their 9 commits are unchanged. GitHub doesn't let maintainers push to a fork owned by an organization, so the fixes couldn't go onto #49 directly. Refs #48.
What #49 adds
See #49: a
/sqliteentry with codecs, functions, positional?binding,sqliteDialect(client transactions and savepoints) andd1Dialect(100 bindings, no interactive transactions), plus compile-time refusals for what SQLite lacks.What this adds
Fixes, found by running the fuzzer (#118) and the core builder suite on SQLite and D1:
like("org\\_1")matched a literal backslash and neverorg_1. The dialect now writesESCAPE '\', so\escapes the same way on every dialect.node:sqlitereads'a' || char(0) || 'b'back as'a', whether bound or literal. feat: add SQLite and Cloudflare D1 query dialects #49's own NUL test failed on Node 24 (ourmise.tomlversion) for this reason. The builder now refuses such a string at compile time instead of storing a value that reads back cut short.json_extract(1.23456789e-95). Fractions in expressions are now bound instead of written as text.Amount > 9.75compiles. It was refused becauseSQLite.integeronly encodes whole numbers. It now behaves as on Postgres and ClickHouse; inserting a fraction is still refused.docs/sqlite.mdnow states that SQLite's LIKE ignores ASCII case.Tests:
tests/core.sqlite.test.tsruns every shared core case onnode:sqliteand on a local D1 binding (Miniflare), with SQL snapshots for both. This is the "shared builder suite" criterion from MySQL / MariaDB dialect #53/Support SQLite and Cloudflare D1 through the existing dialect and Effect SQL APIs #48.tests/fuzz/dialects/sqlite.tsregisterssqliteandd1with the fuzzer: query plans against the reference model, literal round-trips for every SQLite column type, compile invariants and hostile values.patterns-and-listsnow runs on SQLite too.Verification
bun run typecheck: pass.bun run testwith ClickHouse 26.8.2.7: 64 files, 1,219 tests passed, none skipped. Doc citations, export catalogs (151/sqlite) and doc examples ok.FUZZ_SCALE=20on all six targets: 101/101 pass.bun scripts/check-package.ts: ok.Decisions
likestays SQLite-native (ignores ASCII case), and the difference is documented and pinned. ILIKE stays refused, as feat: add SQLite and Cloudflare D1 query dialects #49 chose.QueryBuilderDefect, as row locks and FORMAT already do on main.Second round: gaps closed (after review)
SQLite.timestampstored ISO text (…T16:28:54.750Z), while SQLite's own functions writeYYYY-MM-DD HH:MM:SS. SQLite compares text by bytes, and' 'sorts before'T', so a row a DEFAULT had just stamped was not "later than an hour ago": wrong rows, no error.YYYY-MM-DD HH:MM:SS.SSSin UTC, the formstrftime('%f')anddatetime(…,'subsec')write.new Date(8.64e15)used to be written as garbage.datetime().integerrefuses values past 2^53. D1 rounds such an INTEGER to a different JS number, which then decoded silently. It is now a decode error.integer > 9.75rule is a dialect flag (integerComparesFraction), so a custom profile spread fromsqliteDialectkeeps it.bun run test(1,239 tests on this branch) and every fuzz property atFUZZ_SCALE=20.🤖 Generated with Claude Code
Need help on this PR? Tag
@codesmith-botwith what you need.