feat(sqlite): table DDL — columns, keys, indexes (#50, reviewed)

.changeset/quiet-sqlite-dialects.md

@@ -7,5 +7,4 @@ Add the `/sqlite` query entry with SQLite codecs, functions and positional bindi
 and explicit refusal of interactive transactions. `timestamp` stores SQLite's own
 `YYYY-MM-DD HH:MM:SS.SSS` UTC text, so it compares correctly with what `strftime` writes; `LIKE`
 escapes with `\` as on every dialect; text with U+0000 and integers past 2^53 are refused rather
-than silently changed. `table` is a query schema only: SQLite DDL and
-migration generation are not implemented yet.
+than silently changed. SQLite migration generation is not implemented yet.

.changeset/sqlite-table-ddl.md

@@ -0,0 +1,11 @@
+---
+"@maple-dev/effect-orm": minor
+---
+
+`SQLite.table(name, { columns, primaryKey, indexes })` declares a SQLite table with its DDL:
+`NOT NULL` from the column type, single and composite primary keys, literal and expression
+defaults (`SQLite.timestampNow` for a `timestamp` column), unique columns, and named indexes and
+unique indexes. `sqliteEntitiesOf` and
+`renderSqliteSchema` from `/schema` check a schema and write its `CREATE TABLE` and
+`CREATE INDEX` statements. `table(name, columns, options?)` and
+`table(name, { external: true, columns })` stay query-only. SQLite migrations are not generated yet.

README.md

@@ -9,7 +9,7 @@ Type-safe ClickHouse, Postgres and SQLite queries, result decoding, and reproduc
 Formerly `@maple-dev/effect-clickhouse`.
 
 SQLite and Cloudflare D1 use the same builder and row decoder through the new
-[`/sqlite` entry](./docs/sqlite.md). SQLite schema/migration generation is not implemented.
+[`/sqlite` entry](./docs/sqlite.md), which also writes SQLite table DDL. SQLite migration generation is not implemented.
 
 [Read the documentation](https://effect-clickhouse.maple.dev) ·
 [Getting started](./docs/getting-started.md) · [Recipes](./docs/recipes.md)

docs/README.md

@@ -102,4 +102,4 @@ See [Benchmarking](./benchmarking.md) and the
 
 ## SQLite and D1
 
-[SQLite and D1](./sqlite.md) covers `@maple-dev/effect-orm/sqlite`, its codecs and functions, Effect SQL drivers, tenant checks and transaction limits. SQLite migration generation is not implemented.
+[SQLite and D1](./sqlite.md) covers `@maple-dev/effect-orm/sqlite`, its codecs and functions, table DDL, Effect SQL drivers, tenant checks and transaction limits. SQLite migration generation is not implemented.

docs/reference.md

@@ -241,7 +241,7 @@ Tag `"@maple-dev/effect-orm/CompiledQueryDecodeError"`. Fails the `decodeRows` /
 
 #### `SchemaDefinitionError`
 
-Tag `"@maple-dev/effect-orm/SchemaDefinitionError"`. Fails `S.entitiesOf`, `S.pgEntitiesOf` and
+Tag `"@maple-dev/effect-orm/SchemaDefinitionError"`. Fails `S.entitiesOf`, `S.pgEntitiesOf`, `S.sqliteEntitiesOf` and
 `TB.buildProject` when a definition cannot render: a name that is not a plain identifier, a
 MergeTree-family engine with no `orderBy`, a view reading from a union, a view writing to a table
 outside the schema. Fields: `problems` (every `DefinitionProblem`, `{ object, message }`) and
@@ -510,7 +510,7 @@ placeholders, double-quoted identifiers.
 | `SqlQuery`, `compileQuery`                                                                       | `/sql`            |
 | `ClickHouseStatement`, `parseStatement`, `renderStatement`, `withSettings`, `withFormat`         | `/sql`            |
 | `ClickHouseStatementFromString`, `splitTerminalClauses`, `maskLiteralsAndComments`               | `/sql`            |
-| Schema tooling: `isSchemaObject`, `makeSnapshot`, `renderSchema`, `diffSchemas`, `diffPgSchemas`, `fromDrizzleSnapshot`, snapshot and entity schemas (`PgEnumEntity`, `PgUniqueEntity`, …), `Hint` | `/schema`         |
+| Schema tooling: `isSchemaObject`, `makeSnapshot`, `renderSchema`, `diffSchemas`, `diffPgSchemas`, `fromDrizzleSnapshot`, snapshot and entity schemas (`PgEnumEntity`, `PgUniqueEntity`, …), `Hint`; for SQLite, `sqliteEntitiesOf`, `renderSqliteSchema`, `renderSqliteCreateTable`, `renderSqliteCreateIndex` and the `SqliteTableEntity`, `SqliteColumnEntity`, `SqliteIndexEntity`, `SqliteUniqueEntity` schemas | `/schema`         |
 | `run`, `status`, `verify`, `baseline`, `MigrationDriver` and the migrate errors                  | `/migrate`        |
 | `Database`, `run`, `sql`, `query`, `execute`, `transaction`, `requireTransaction`, `retryContention`, `Transaction` | `/database` |
 | `DatabaseError`, `TransactionCommitFailed`, `TransactionRollbackFailed` and the other transaction errors | `/database` |
@@ -522,8 +522,8 @@ placeholders, double-quoted identifiers.
 | `makeHttpClient`, `makeHttpTransport`, `httpConfigFromEnv`                                       | `/benchmark/http` |
 | `runCli`                                                                                         | `/benchmark/cli`  |
 
-`/schema` reads tables; it does not declare them. Declare tables with `table` from `/clickhouse`
-or `/postgres`. See [Schema and migrations](./migrations.md), [Database](./database.md),
+`/schema` reads tables; it does not declare them. Declare tables with `table` from `/clickhouse`,
+`/postgres` or `/sqlite`. See [Schema and migrations](./migrations.md), [Database](./database.md),
 [Running a query](./running-queries.md) for what the `/sql` statement helpers are for, and
 [Benchmarking](./benchmarking.md) for the complete benchmark API, command workflow, result
 verification, and JSON protocol.
@@ -545,13 +545,54 @@ interactive transactions. See [SQLite and D1](./sqlite.md) for driver wiring and
 | `boolean` | A boolean encoded as 0/1; other integers fail decoding |
 | `timestamp` | An instant stored as SQLite's `YYYY-MM-DD HH:MM:SS.SSS` UTC text; reads ISO-8601 too |
 | `sqliteTimestampText(epochMillis)` | That text for an instant, for raw SQL and seeds |
+| `timestampNow` | A `defaultExpr` for a `timestamp` column: the current time in that text form |
 | `blob` | Bytes from a Uint8Array or D1's array wire representation |
 | `json(schema)` | JSON text decoded through the supplied schema |
 | `nullable(type)`, `custom`, `brand` | Type composition |
-| `table(name, columns, options?)`, `TableOptions` | A query schema, including `tenantColumn`, defaults and computed-column metadata; no generated DDL |
-| `table(name, ExternalTableDefinition)`, `ExternalTableDefinition` | `{ external: true, columns, tenantColumn? }`, the same query schema in the `/clickhouse` and `/postgres` form |
 | `count`, `countDistinct`, `countIf`, `sum`, `avg`, `min`, `max` | SQLite aggregates; the last four can return SQL NULL |
 | `lower`, `upper`, `length`, `coalesce` | SQLite scalar functions |
 
-SQLite's table definition is for query typing and decoding. The schema/kit/migrate entries
-currently support ClickHouse and Postgres DDL only.
+### Tables and DDL
+
+```ts
+const Links = SQLite.table("short_link", {
+	columns: {
+		id: SQLite.text,
+		code: SQLite.column(SQLite.text, { unique: true }),
+		orgId: SQLite.column(SQLite.nullable(SQLite.text), { name: "org_id" }),
+		clickCount: SQLite.column(SQLite.integer, { name: "click_count", default: 0 }),
+		createdAt: SQLite.column(SQLite.text, { name: "created_at", defaultExpr: "(CURRENT_TIMESTAMP)" }),
+	},
+	primaryKey: ["id"],
+	indexes: [SQLite.index("short_link_org", ["orgId"])],
+	tenantColumn: "orgId",
+})
+```
+
+| Export | Signature |
+| --- | --- |
+| `table` | `(name, TableDefinition) => SqliteSchemaTable`, `(name, ExternalTableDefinition) => Table`, or `(name, columns, TableOptions?) => Table`, a query schema with no DDL |
+| `column` | `(type, ColumnOptions) => ColumnSpec` |
+| `index` | `(name, on) => IndexSpec`: columns, or a callback building expressions |
+| `uniqueIndex` | The same, `UNIQUE` |
+
+**`TableDefinition`** — `columns` (required), `primaryKey` (column names: one is written inline,
+several as a table constraint), `indexes`, `tenantColumn`. A column is `NOT NULL` unless its
+type is `nullable(...)`; a primary key column cannot be nullable.
+
+**`ExternalTableDefinition`** — `{ external: true, columns, tenantColumn? }`: a view, or a table
+another tool migrates. No DDL. **`TableOptions`** — `tenantColumn`, `defaults`, `computed`, for
+the bare-columns form.
+
+**`ColumnOptions`** — one of `default` (a literal, encoded through the column type; a boolean
+is written `true` / `false`) or `defaultExpr` (SQL as written, so add SQLite's parentheses, or a
+DSL callback, parenthesized for you); either lets an insert leave the column out
+(`DefaultedColumnsOf`). Also `name` (the column's name in SQL) and `unique` (`true` or a
+constraint name), an inline `UNIQUE` constraint named `<table>_<column>_unique` by default, as
+drizzle-kit 1.0 writes and names it.
+
+**Types** — `ColumnInput`, `ColumnSpec`, `ColumnsOf`, `IndexSpec`, `TableDdl`, `SqliteSchemaTable`,
+`DdlExpr`, `DdlKey`, `DefinitionProblem`. A definition that cannot render records a problem;
+`S.sqliteEntitiesOf` fails with a `SchemaDefinitionError`, and `S.renderSqliteSchema` writes
+the `CREATE TABLE` and `CREATE INDEX` statements. There is no SQLite diff, snapshot or migration
+runner yet.

docs/sqlite.md

@@ -24,9 +24,68 @@ export const compiled = SQLite.compileUnsafe(byOrg, { orgId: "org_1" })
 // compiled.parameters is ["org_1"]; compiled.tenantScope is "single-tenant".
 ```
 
-`table` describes the query schema. Manage its actual DDL and migrations outside this package.
-Unlike the ClickHouse and Postgres entries, SQLite has no schema diff, introspection, migration
-ledger or kit dialect yet. A `table` value here cannot be passed to the DDL generators.
+`table(name, columns, options?)` is a query schema only: it carries no DDL.
+
+## Declare a table with its DDL
+
+Pass `{ columns, primaryKey, indexes }` to `table` to declare the table's DDL too. The value is
+still a query `Table`. `sqliteEntitiesOf` from `/schema` checks a set of tables as one schema,
+and `renderSqliteSchema` writes their `CREATE TABLE` and `CREATE INDEX` statements:
+
+```ts title="sqlite-ddl.ts"
+import { Effect } from "effect"
+import * as S from "@maple-dev/effect-orm/schema"
+import * as SQLite from "@maple-dev/effect-orm/sqlite"
+
+export const ShortLinks = SQLite.table("short_link", {
+	columns: {
+		id: SQLite.text,
+		code: SQLite.text,
+		orgId: SQLite.column(SQLite.nullable(SQLite.text), { name: "org_id" }),
+		clickCount: SQLite.column(SQLite.integer, { name: "click_count", default: 0 }),
+		createdAt: SQLite.column(SQLite.text, { name: "created_at", defaultExpr: "(CURRENT_TIMESTAMP)" }),
+	},
+	primaryKey: ["id"],
+	indexes: [SQLite.uniqueIndex("short_link_code", ["code"]), SQLite.index("short_link_org", ["orgId"])],
+	tenantColumn: "orgId",
+})
+
+export const statements = S.sqliteEntitiesOf([ShortLinks]).pipe(Effect.map(S.renderSqliteSchema))
+// CREATE TABLE IF NOT EXISTS "short_link" (
+// 	"id" text PRIMARY KEY NOT NULL,
+// 	"code" text NOT NULL,
+// 	"org_id" text,
+// 	"click_count" integer NOT NULL DEFAULT 0,
+// 	"created_at" text NOT NULL DEFAULT (CURRENT_TIMESTAMP)
+// )
+// CREATE UNIQUE INDEX IF NOT EXISTS "short_link_code" ON "short_link" ("code")
+// CREATE INDEX IF NOT EXISTS "short_link_org" ON "short_link" ("org_id")
+```
+
+- A column is `NOT NULL` unless its type is `nullable(...)`. A primary key column cannot be
+  nullable: SQLite would accept NULL keys in it.
+- One primary key column is written inline (`PRIMARY KEY`); several become a table constraint
+  `PRIMARY KEY ("a", "b")`. SQLite stores no constraint name, so there is none to give.
+- `default` takes a literal of the column type. `defaultExpr` takes SQL as written, so add the
+  parentheses SQLite requires around an expression, or a DSL callback, which is parenthesized
+  for you. Either lets an insert leave the column out.
+- `unique: true` writes an inline `UNIQUE` constraint, as drizzle-kit 1.0 does for `.unique()`.
+  Its name, `<table>_<column>_unique` or the string you give, is kept in the schema entities;
+  SQLite itself names the backing index `sqlite_autoindex_...`.
+- Column types declare their SQLite type: `text`, `timestamp` and `json` are `text`; `integer`
+  and `boolean` are `integer`; `real` is `real`; `blob` is `blob`. A `custom(sql, schema)` type
+  declares `sql`, for example `numeric`.
+- `table(name, { external: true, columns })` declares a table another tool owns: no DDL.
+
+The tests apply this DDL and drizzle-kit's SQL for the same tables to real SQLite, and compare
+`PRAGMA table_info`, `index_list` and `index_xinfo`. One difference is deliberate: drizzle-kit
+1.0 drops `NOT NULL` from a primary key column, so SQLite then accepts a NULL key in a `text`
+key column. This entry keeps `NOT NULL`.
+
+There is no SQLite schema diff, introspection, migration ledger or kit dialect yet. Run the
+rendered statements yourself, or keep migrating with your current tool. `WITHOUT ROWID`,
+`STRICT`, `AUTOINCREMENT`, partial indexes, foreign keys and `CHECK` constraints are not
+declared yet.
 
 The entry exports `text`, `integer` (a safe JavaScript integer), `real` (finite numbers),
 `boolean` (0/1), `blob`, `timestamp`, `json(schema)`, `nullable`, `custom` and `brand`.
@@ -40,7 +99,8 @@ form `strftime('%Y-%m-%d %H:%M:%f')` and `datetime(..., 'subsec')` write. It rea
 and ISO-8601; text without a zone is UTC. SQLite compares text by bytes, so a timestamp column
 orders by time only when every value in it has one form: write them through the builder or with
 `strftime('%Y-%m-%d %H:%M:%f', ...)`. `CURRENT_TIMESTAMP` writes no milliseconds
-(`2026-01-01 00:00:00`), which compares as earlier than the same instant written with them.
+(`2026-01-01 00:00:00`), which compares as earlier than the same instant written with them, so
+a `timestamp` column refuses it as a `defaultExpr`; use `defaultExpr: SQLite.timestampNow`.
 `blob` accepts the Uint8Array wire form from Node and the array-of-bytes form from D1.
 `json` always uses the supplied schema to validate decoded JSON.
 Use `text` when a stored date/time must remain a string.

src/schema.ts

@@ -87,5 +87,21 @@ export {
 	type PgOpLabel,
 } from "./schema/pg-ops"
 export { diffPgSchemas } from "./schema/pg-diff"
+export { sqliteEntitiesOf, type SqliteSchemaTable } from "./schema/sqlite-define"
+export {
+	SqliteColumnEntity,
+	SqliteIndexEntity,
+	SqliteSchemaEntity,
+	SqliteTableEntity,
+	SqliteUniqueEntity,
+	canonicalSqliteType,
+} from "./schema/sqlite-entities"
+export {
+	renderSqliteColumnDefinition,
+	renderSqliteCreateIndex,
+	renderSqliteCreateTable,
+	renderSqliteSchema,
+	sqliteIdent,
+} from "./schema/sqlite-render"
 export { fromDrizzleSnapshot, type DrizzleImport } from "./schema/drizzle"
 export { Hint, Hints, diffSchemas, type DiffResult, type UnsupportedChange } from "./schema/diff"

src/schema/ddl-expr.ts

@@ -0,0 +1,34 @@
+// Expressions in DDL: defaults, index keys and predicates, written with the
+// query DSL or as SQL strings, and rendered once with the dialect the table is
+// for. Shared by the Postgres and SQLite `table`.
+
+import { compile as compileFragment } from "../sql/sql-fragment"
+import { withDialect, type Dialect } from "../ch/dialect"
+import type { Condition, Expr } from "../ch/expr"
+import { createColumnAccessor, type ColumnAccessor } from "../ch/query"
+import { columnSqlName, type ColumnDefs } from "../ch/types"
+import type { DdlExpr, DdlKey } from "./define"
+
+/** A partial index's predicate: SQL, or a condition built with the DSL. */
+export type DdlPredicate<Cols extends ColumnDefs> = string | (($: ColumnAccessor<Cols>) => Condition | Expr<any> | string)
+
+export interface DdlRenderer {
+	/** SQL as written, or a DSL value rendered with the dialect. */
+	readonly value: (value: Expr<any> | Condition | string) => string
+	readonly expr: <Cols extends ColumnDefs>(expr: DdlExpr<Cols> | DdlPredicate<Cols>, columns: Cols) => string
+	/** Key parts: a column name is written quoted, an expression as it renders. */
+	readonly keyParts: <Cols extends ColumnDefs>(key: DdlKey<Cols>, columns: Cols) => ReadonlyArray<string>
+}
+
+export const ddlRenderer = (dialect: Dialect): DdlRenderer => {
+	const value = (v: Expr<any> | Condition | string): string =>
+		typeof v === "string" ? v : withDialect(dialect, () => compileFragment(v.toFragment()))
+	return {
+		value,
+		expr: (expr, columns) => (typeof expr === "string" ? expr : value(expr(createColumnAccessor(columns)))),
+		keyParts: (key, columns) =>
+			typeof key === "function"
+				? key(createColumnAccessor(columns)).map(value)
+				: key.map((column) => dialect.quoteIdent(columnSqlName(columns, column))),
+	}
+}

src/schema/entities.ts

@@ -9,6 +9,7 @@
 
 import { Schema } from "effect"
 import { PgSchemaEntity } from "./pg-entities"
+import type { SqliteSchemaEntity } from "./sqlite-entities"
 
 /** `DEFAULT`, `MATERIALIZED`, or `ALIAS`, with its SQL expression. */
 export const ColumnDefault = Schema.Struct({
@@ -112,8 +113,11 @@ export type SchemaDialect = Snapshot["dialect"]
 /** An entity of either dialect. Everything dialect-neutral (keys, ordering, hashing, the branch graph) takes this. */
 export type AnySchemaEntity = SchemaEntity | PgSchemaEntity
 
+/** What `entityKey` and `sortEntities` take: a snapshot dialect's entity, or a SQLite one (no snapshot yet). */
+type KeyedEntity = AnySchemaEntity | SqliteSchemaEntity
+
 /** A stable key per entity, unique within a snapshot: `<kind>:<name>`, or `<kind>:<table>.<name>` for a table's parts. */
-export const entityKey = (entity: AnySchemaEntity): string =>
+export const entityKey = (entity: KeyedEntity): string =>
 	"table" in entity
 		? `${entity.kind}:${entity.table}.${entity.name}`
 		: entity.kind === "enum" && entity.schema !== null
@@ -131,7 +135,7 @@ const kindOrder: Record<AnySchemaEntity["kind"], number> = {
 }
 
 /** Enums, tables, then columns in declaration order, then indexes, unique constraints, foreign keys, views. Deterministic. */
-export const sortEntities = <E extends AnySchemaEntity>(entities: ReadonlyArray<E>): ReadonlyArray<E> =>
+export const sortEntities = <E extends KeyedEntity>(entities: ReadonlyArray<E>): ReadonlyArray<E> =>
 	[...entities].sort((a, b) => {
 		const byKind = kindOrder[a.kind] - kindOrder[b.kind]
 		if (byKind !== 0) return byKind