feat(sqlite): generate migrations

.changeset/quiet-sqlite-dialects.md

@@ -0,0 +1,8 @@
+---
+"@maple-dev/effect-orm": minor
+---
+
+Add the `/sqlite` query entry with SQLite codecs, functions and positional bindings. Use
+`sqliteDialect` with an Effect SQLite SqlClient, or `d1Dialect` for Cloudflare D1's binding limit
+and explicit refusal of interactive transactions. `table` is a query schema only: SQLite DDL and
+migration generation are not implemented yet.

.changeset/sqlite-generate.md

@@ -0,0 +1,10 @@
+---
+"@maple-dev/effect-orm": minor
+---
+
+`effect-orm generate` supports `dialect: "sqlite"`. It diffs `SQLite.table` definitions against
+the migrations folder and writes `<timestamp>_<name>/migration.sql` plus `snapshot.json`, the
+drizzle-kit 1.0 layout that Alchemy's D1 resource applies. `ALTER TABLE` adds, drops and renames
+where SQLite allows it; other column, primary key and UNIQUE changes rebuild the table. Renames
+need a `confirm_rename` hint. `generate --baseline --from-drizzle` adopts a drizzle-kit 1.0
+SQLite folder. Snapshots gain a `sqlite` dialect. `/migrate` refuses SQLite folders for now.

.changeset/sqlite-table-ddl.md

@@ -0,0 +1,10 @@
+---
+"@maple-dev/effect-orm": minor
+---
+
+`SQLite.table(name, { columns, primaryKey, indexes })` declares a SQLite table with its DDL:
+`NOT NULL` from the column type, single and composite primary keys, literal and expression
+defaults, unique columns, and named indexes and unique indexes. `sqliteEntitiesOf` and
+`renderSqliteSchema` from `/schema` check a schema and write its `CREATE TABLE` and
+`CREATE INDEX` statements. `table(name, columns, options?)` and
+`table(name, { external: true, columns })` stay query-only. SQLite migrations are not generated yet.

README.md

@@ -4,10 +4,13 @@
 > **Alpha.** The API is still changing and minor releases may break it. Pin an exact version,
 > read the [changelog](./CHANGELOG.md) before upgrading, and expect rough edges.
 
-Type-safe ClickHouse and Postgres queries, result decoding, and reproducible benchmarks for Effect and TypeScript.
+Type-safe ClickHouse, Postgres and SQLite queries, result decoding, and reproducible benchmarks for Effect and TypeScript.
 
 Formerly `@maple-dev/effect-clickhouse`.
 
+SQLite and Cloudflare D1 use the same builder and row decoder through the new
+[`/sqlite` entry](./docs/sqlite.md), which also writes SQLite table DDL. `effect-orm generate` writes SQLite migrations; effect-orm does not apply them yet.
+
 [Read the documentation](https://effect-clickhouse.maple.dev) ·
 [Getting started](./docs/getting-started.md) · [Recipes](./docs/recipes.md)
 

docs/README.md

@@ -99,3 +99,7 @@ The optional `@maple-dev/effect-orm/benchmark` entry point and bundled
 `ch-bench` CLI measure real queries, compare fixed workloads, and save evidence.
 See [Benchmarking](./benchmarking.md) and the
 [agent playbook](./benchmark-agent.md). The SQL builders remain driver-free.
+
+## SQLite and D1
+
+[SQLite and D1](./sqlite.md) covers `@maple-dev/effect-orm/sqlite`, its codecs and functions, table DDL, Effect SQL drivers, tenant checks and transaction limits. `effect-orm generate` writes SQLite migrations; effect-orm does not apply them yet.

docs/migrations.md

@@ -1,6 +1,6 @@
 # Schema and migrations
 
-Every table is declared with its DDL: `table` from `/clickhouse` or `/postgres` carries the
+Every table is declared with its DDL: `table` from `/clickhouse`, `/postgres` or `/sqlite` carries the
 engine, keys, indexes, and defaults beside the columns the query builder reads. You can ignore
 that and manage the database elsewhere, or let three opt-in entry points turn the definitions
 into migrations:
@@ -17,6 +17,7 @@ check and folder loading are shared; table definitions, the diff, the DDL, the r
 drift detection are per database, because the two differ where it matters: ClickHouse DDL is
 not transactional and cannot change most things in place, and Postgres DDL is and can. The
 sections below describe ClickHouse first; [Postgres](#postgres) covers what differs.
+[SQLite](#sqlite) has `generate` only: another tool applies its migrations for now.
 
 ## Defining tables
 
@@ -420,3 +421,74 @@ drizzle-kit recorded, so every place they disagree (a constraint name, a default
 op to accept or fix. `verify` against the baseline also finds objects the database has and
 drizzle-kit's snapshot does not, such as a table a hand-written migration created and nothing
 dropped.
+
+## SQLite
+
+Set `dialect: "sqlite"` and export `SQLite.table` values with their DDL from the schema module:
+
+```ts title="sqlite-kit-config.ts"
+import { defineConfig } from "@maple-dev/effect-orm/kit"
+
+export default defineConfig({ dialect: "sqlite", schema: "./src/db/schema.ts", out: "./migrations" })
+```
+
+`effect-orm generate` writes `<timestamp>_<name>/migration.sql` and `snapshot.json`, the
+drizzle-kit 1.0 layout. Statements are split by `--> statement-breakpoint`, each under a comment
+naming its label. A SQLite migration is always SQL; there is no `migration.json` for it. Tools
+that apply drizzle-kit folders apply it as it is, for example Alchemy's
+`Cloudflare.D1.Database({ migrations: "./migrations" })`, which runs each folder's statements as one
+D1 batch. `snapshot.json` is for `generate` and `check`; those tools ignore it.
+
+What each change becomes:
+
+| Change | SQL |
+| --- | --- |
+| New table, dropped table | `CREATE TABLE`, `DROP TABLE` (confirm the data loss) |
+| New column | `ALTER TABLE ... ADD COLUMN`, when SQLite allows it: the column is nullable or has a literal default, and is not in the primary key or a UNIQUE constraint. Otherwise a rebuild |
+| Dropped column | `ALTER TABLE ... DROP COLUMN` (confirm the data loss), unless it is in the primary key or a UNIQUE constraint. Otherwise a rebuild |
+| A column's type, nullability or default; the primary key; a UNIQUE column | A table rebuild |
+| New or changed index | `DROP INDEX`, `CREATE [UNIQUE] INDEX` |
+| Renamed table or column | `ALTER TABLE ... RENAME`, only with a `confirm_rename` hint |
+
+A rebuild follows [SQLite's procedure](https://www.sqlite.org/lang_altertable.html#otheralter)
+the way drizzle-kit writes it: `PRAGMA foreign_keys=OFF`, `CREATE TABLE "__new_<table>"` with the
+new definition, `INSERT INTO "__new_<table>" (...) SELECT ...` for every column both tables
+have, `DROP TABLE`, `ALTER TABLE "__new_<table>" RENAME TO "<table>"`, `PRAGMA foreign_keys=ON`,
+then every index of the table again. A copy fails when the rows do not fit the new table, such
+as a column made NOT NULL over existing NULLs or a new NOT NULL column with no default. A
+column a rebuild leaves out needs the same `confirm_data_loss` hint as a dropped column.
+
+A rename is never guessed. Without a hint it reads as a drop plus an add, and the drop asks for
+confirmation. Pass the rename instead:
+
+```json
+[
+	{ "type": "confirm_rename", "kind": "table", "entity": "links", "to": "urls" },
+	{ "type": "confirm_rename", "kind": "column", "entity": "urls.url", "to": "target" }
+]
+```
+
+`entity` is the old table name, or `<table>.<old column>` with the table's new name. Table
+renames run first. Indexes keep their names and follow the rename.
+
+### Adopting a drizzle-kit SQLite folder
+
+`effect-orm generate --baseline --from-drizzle` reads drizzle-kit 1.0's last `snapshot.json`
+(version 7, `dialect: "sqlite"`): tables, columns, primary keys, indexes and UNIQUE
+constraints. Foreign keys, checks, views, generated and AUTOINCREMENT columns and partial
+indexes are listed, and nothing is written. The baseline runs nothing. A later `generate`
+compares your `SQLite.table` definitions with it; when they declare the same tables, it writes
+nothing. The tests prove this with a folder drizzle-kit 1.0.0-rc.5 generated.
+
+drizzle-kit 1.0 leaves `NOT NULL` out of a primary key column (`"id" text PRIMARY KEY`), so
+SQLite then accepts a NULL key in a `text` key column. Its snapshot marks the column
+`notNull: false`, but drizzle-kit treats a key column as not null. The baseline reads it as not
+null too, so a `SQLite.table` key column (always `NOT NULL`) matches it and no rebuild is
+written. The database keeps accepting a NULL key until that table is rebuilt for another change.
+
+The drizzle-kit 0.x layout (`meta/_journal.json` and flat `.sql` files) is not read. Run
+`drizzle-kit up` to convert it first.
+
+`/migrate` does not run SQLite folders yet: `run`, `status`, `verify` and `baseline` fail with a
+`MigrateSourceError` for one. Keep applying them with the tool you use now.
+

docs/reference.md

@@ -5,9 +5,10 @@ The package has one entry per database, each self-contained:
 ```ts
 import * as CH from "@maple-dev/effect-orm/clickhouse"
 import * as PG from "@maple-dev/effect-orm/postgres"
+import * as SQLite from "@maple-dev/effect-orm/sqlite"
 ```
 
-Both carry the whole [query builder](#query-builder-both-entries). Each adds its own column
+All carry the whole [query builder](#query-builder-every-entry). Each adds its own column
 types, functions, table definitions and a `compile` that defaults to its dialect:
 [`/clickhouse`](#clickhouse) and [`/postgres`](#postgres). Everything else is on a
 [subpath](#other-subpaths) for a narrower job.
@@ -32,9 +33,9 @@ uniformly, which some codebases prefer for exactly this reason.
 
 ---
 
-## Query builder (both entries)
+## Query builder (every entry)
 
-Everything in this section is exported, identically, from both `/clickhouse` and `/postgres`.
+Everything in this section is exported, identically, from `/clickhouse`, `/postgres` and `/sqlite`.
 
 ### Query construction
 
@@ -46,7 +47,7 @@ Everything in this section is exported, identically, from both `/clickhouse` and
 | `unionAll`  | `(...queries) => CHUnionQuery`            |
 | `update`     | `(table) => CHUpdateStart`, then `CHUpdate`: `.set(record \| fn)`, `.where(fn)` or `.allRows()`, `.returning(...)`, `.settings(record)`. See [Updating and deleting](./updates-and-deletes.md) |
 | `deleteFrom` | `(table) => CHDelete`: `.where(fn)` or `.allRows()`, `.returning(...)`, `.settings(record)` |
-| `insertInto` | `(table) => CHInsertStart`, then `CHInsert`; `.values(row \| rows)` or `.select(query)` sets its rows, `.settings(record)` ClickHouse `SETTINGS`, `.returning(...)` the RETURNING list, `.onConflictDoNothing(options?)` / `.onConflictDoUpdate(options)` the ON CONFLICT clause (Postgres). See [Inserting rows](./inserts.md) |
+| `insertInto` | `(table) => CHInsertStart`, then `CHInsert`; `.values(row \| rows)` or `.select(query)` sets its rows, `.settings(record)` ClickHouse `SETTINGS`, `.returning(...)` the RETURNING list, `.onConflictDoNothing(options?)` / `.onConflictDoUpdate(options)` the ON CONFLICT clause (Postgres and SQLite). See [Inserting rows](./inserts.md) |
 
 Tables come from the dialect's `table`: [`CH.table`](#tables-and-ddl) or
 [`PG.table`](#tables-and-ddl-1).
@@ -240,7 +241,7 @@ Tag `"@maple-dev/effect-orm/CompiledQueryDecodeError"`. Fails the `decodeRows` /
 
 #### `SchemaDefinitionError`
 
-Tag `"@maple-dev/effect-orm/SchemaDefinitionError"`. Fails `S.entitiesOf`, `S.pgEntitiesOf` and
+Tag `"@maple-dev/effect-orm/SchemaDefinitionError"`. Fails `S.entitiesOf`, `S.pgEntitiesOf`, `S.sqliteEntitiesOf` and
 `TB.buildProject` when a definition cannot render: a name that is not a plain identifier, a
 MergeTree-family engine with no `orderBy`, a view reading from a union, a view writing to a table
 outside the schema. Fields: `problems` (every `DefinitionProblem`, `{ object, message }`) and
@@ -509,7 +510,7 @@ placeholders, double-quoted identifiers.
 | `SqlQuery`, `compileQuery`                                                                       | `/sql`            |
 | `ClickHouseStatement`, `parseStatement`, `renderStatement`, `withSettings`, `withFormat`         | `/sql`            |
 | `ClickHouseStatementFromString`, `splitTerminalClauses`, `maskLiteralsAndComments`               | `/sql`            |
-| Schema tooling: `isSchemaObject`, `makeSnapshot`, `renderSchema`, `diffSchemas`, `diffPgSchemas`, `fromDrizzleSnapshot`, snapshot and entity schemas (`PgEnumEntity`, `PgUniqueEntity`, …), `Hint` | `/schema`         |
+| Schema tooling: `isSchemaObject`, `makeSnapshot`, `renderSchema`, `diffSchemas`, `diffPgSchemas`, `fromDrizzleSnapshot`, snapshot and entity schemas (`PgEnumEntity`, `PgUniqueEntity`, …), `Hint`; for SQLite, `sqliteEntitiesOf`, `renderSqliteSchema`, `renderSqliteCreateTable`, `renderSqliteCreateIndex`, `diffSqliteSchemas`, `fromDrizzleSqliteSnapshot`, `SqliteMigrationOp`, `renderSqliteOp`, `renderSqliteMigrationSql`, `labelOfSqlite`, `rebuildName`, the `SqliteSnapshot` schema and the `SqliteTableEntity`, `SqliteColumnEntity`, `SqliteIndexEntity`, `SqliteUniqueEntity` schemas | `/schema`         |
 | `run`, `status`, `verify`, `baseline`, `MigrationDriver` and the migrate errors                  | `/migrate`        |
 | `Database`, `run`, `sql`, `query`, `execute`, `transaction`, `requireTransaction`, `retryContention`, `Transaction` | `/database` |
 | `DatabaseError`, `TransactionCommitFailed`, `TransactionRollbackFailed` and the other transaction errors | `/database` |
@@ -521,12 +522,76 @@ placeholders, double-quoted identifiers.
 | `makeHttpClient`, `makeHttpTransport`, `httpConfigFromEnv`                                       | `/benchmark/http` |
 | `runCli`                                                                                         | `/benchmark/cli`  |
 
-`/schema` reads tables; it does not declare them. Declare tables with `table` from `/clickhouse`
-or `/postgres`. See [Schema and migrations](./migrations.md), [Database](./database.md),
+`/schema` reads tables; it does not declare them. Declare tables with `table` from `/clickhouse`,
+`/postgres` or `/sqlite`. See [Schema and migrations](./migrations.md), [Database](./database.md),
 [Running a query](./running-queries.md) for what the `/sql` statement helpers are for, and
 [Benchmarking](./benchmarking.md) for the complete benchmark API, command workflow, result
 verification, and JSON protocol.
 
 Note `/sql` exports a `compile` (fragment → string) distinct from the dialect entries' `compile`
 (query → `CompiledQuery`), and a `when` distinct from the query builder's `when` (optional
 conditions).
+
+## SQLite and D1
+
+`@maple-dev/effect-orm/sqlite` exports the shared query builder and the four compile functions.
+`sqliteDialect` is the default; `d1Dialect` uses the same SQL with a 100-binding limit and no
+interactive transactions. See [SQLite and D1](./sqlite.md) for driver wiring and limits.
+
+| Export | Purpose |
+| --- | --- |
+| `SqliteType` | A SQLite SQL type with its Effect codec |
+| `text`, `integer`, `real` | Text, safe integer and finite number codecs |
+| `boolean` | A boolean encoded as 0/1; other integers fail decoding |
+| `timestamp` | An instant stored as ISO text |
+| `blob` | Bytes from a Uint8Array or D1's array wire representation |
+| `json(schema)` | JSON text decoded through the supplied schema |
+| `nullable(type)`, `custom`, `brand` | Type composition |
+| `count`, `countDistinct`, `countIf`, `sum`, `avg`, `min`, `max` | SQLite aggregates; the last four can return SQL NULL |
+| `lower`, `upper`, `length`, `coalesce` | SQLite scalar functions |
+
+### Tables and DDL
+
+```ts
+const Links = SQLite.table("short_link", {
+	columns: {
+		id: SQLite.text,
+		code: SQLite.column(SQLite.text, { unique: true }),
+		orgId: SQLite.column(SQLite.nullable(SQLite.text), { name: "org_id" }),
+		clickCount: SQLite.column(SQLite.integer, { name: "click_count", default: 0 }),
+		createdAt: SQLite.column(SQLite.text, { name: "created_at", defaultExpr: "(CURRENT_TIMESTAMP)" }),
+	},
+	primaryKey: ["id"],
+	indexes: [SQLite.index("short_link_org", ["orgId"])],
+	tenantColumn: "orgId",
+})
+```
+
+| Export | Signature |
+| --- | --- |
+| `table` | `(name, TableDefinition) => SqliteSchemaTable`, `(name, ExternalTableDefinition) => Table`, or `(name, columns, TableOptions?) => Table`, a query schema with no DDL |
+| `column` | `(type, ColumnOptions) => ColumnSpec` |
+| `index` | `(name, on) => IndexSpec`: columns, or a callback building expressions |
+| `uniqueIndex` | The same, `UNIQUE` |
+
+**`TableDefinition`** — `columns` (required), `primaryKey` (column names: one is written inline,
+several as a table constraint), `indexes`, `tenantColumn`. A column is `NOT NULL` unless its
+type is `nullable(...)`; a primary key column cannot be nullable.
+
+**`ExternalTableDefinition`** — `{ external: true, columns, tenantColumn? }`: a view, or a table
+another tool migrates. No DDL. **`TableOptions`** — `tenantColumn`, `defaults`, `computed`, for
+the bare-columns form.
+
+**`ColumnOptions`** — one of `default` (a literal, encoded through the column type; a boolean
+is written `true` / `false`) or `defaultExpr` (SQL as written, so add SQLite's parentheses, or a
+DSL callback, parenthesized for you); either lets an insert leave the column out
+(`DefaultedColumnsOf`). Also `name` (the column's name in SQL) and `unique` (`true` or a
+constraint name), an inline `UNIQUE` constraint named `<table>_<column>_unique` by default, as
+drizzle-kit 1.0 writes and names it.
+
+**Types** — `ColumnInput`, `ColumnSpec`, `ColumnsOf`, `IndexSpec`, `TableDdl`, `SqliteSchemaTable`,
+`DdlExpr`, `DdlKey`, `DefinitionProblem`. A definition that cannot render records a problem;
+`S.sqliteEntitiesOf` fails with a `SchemaDefinitionError`, and `S.renderSqliteSchema` writes
+the `CREATE TABLE` and `CREATE INDEX` statements. `effect-orm generate` with `dialect: "sqlite"`
+writes migrations for them; see [SQLite migrations](./migrations.md#sqlite). The `/migrate`
+runner does not apply SQLite migrations yet.