feat(database): typed constraint violations across dialects

.changeset/constraint-errors.md

@@ -0,0 +1,5 @@
+---
+"@maple-dev/effect-orm": minor
+---
+
+Constraint violations are typed errors. `refineConstraints(effect)` turns a `DatabaseError` or `TransactionCommitFailed` for a broken constraint into `UniqueViolation`, `ForeignKeyViolation`, `NotNullViolation` or `CheckViolation`, so a caller can `catchTag` a duplicate key. Each carries the `constraint`, `table` and `column` the server named. Postgres maps SQLSTATE 23505, 23503, 23502 and 23514; ClickHouse maps code 469 (`VIOLATED_CONSTRAINT`) to `CheckViolation`. `constraintViolation(error)` reads one without failing. `DatabaseError` and `TransactionCommitFailed` now keep the server's numeric error code as `errorCode`, which ClickHouse reports and `sqlState` dropped. `ClickhouseHttpClient` classifies code 469 as `ConstraintError` instead of `StatementTimeoutError`.

design/gap-review.md

@@ -32,7 +32,7 @@ builder; **P1** commonly used; **P2** niche.
 | ~~`FOR UPDATE` / `FOR SHARE` / `SKIP LOCKED` / `NOWAIT`~~ (built) | 7 | S |
 | jsonb and array operators (`@>`, `->`, `?`, `&&`, `ANY`) | ~12 | M |
 | ~~`isNull` / `isNotNull` / `between`; variadic `and` / `or` that skip `undefined`~~ (built) | everywhere | S |
-| Constraint error helpers (unique, foreign key, not null); keep ClickHouse's numeric error codes, which `sqlStateOf` drops today | all upserts | S |
+| ~~Constraint error helpers (unique, foreign key, not null); keep ClickHouse's numeric error codes, which `sqlStateOf` drops today~~ (built: `refineConstraints`, `DatabaseError.errorCode`; Postgres and ClickHouse) | all upserts | S |
 | Tenant-scope enforcement in `Database`, opt in, with an explicit cross-tenant entry point | safety | S |
 | ~~Postgres `defineTable` (indexes, unique, FKs), Postgres migrations, a drizzle-kit importer~~ Done: `S.pg.table`, `dialect: "postgres"`, `--baseline --from-drizzle` (design/migrations.md section 8) | 68 tables, 90 indexes, 47 unique, 75 folders | L |
 

docs/database.md

@@ -92,6 +92,8 @@ Calling `withdraw(1, 30)` outside `transfer` does not compile: `requireTransacti
 | `retryContention(options?)` | Re-run on serialization failure or deadlock |
 | `Transaction` | The open transaction (`depth`, `isolationLevel`, `accessMode`) |
 | `isContention(error)` | Whether an error is a serialization failure or deadlock |
+| `refineConstraints(effect)` | Fail with a typed `ConstraintViolation` where a statement broke a constraint ([Constraint violations](#constraint-violations)) |
+| `constraintViolation(error)` | The `ConstraintViolation` an error reports, as an `Option` |
 
 `run`, `query`, `execute`, `transaction` and `retryContention` are also methods on
 `DatabaseApi`.
@@ -214,12 +216,48 @@ again. `read uncommitted` is not offered because Postgres runs it as `read commi
 
 | Error | When |
 | --- | --- |
-| `DatabaseError` | A statement failed. `reason` is the driver's classification (`UniqueViolation`, `SerializationError`, ...) and `sqlState` the server's code |
-| `TransactionCommitFailed` | COMMIT failed: a deferred constraint, a serialization failure at commit, a dropped connection. Carries `reason` and `sqlState` |
+| `DatabaseError` | A statement failed. `reason` is the driver's classification (`UniqueViolation`, `SerializationError`, ...), `sqlState` the server's five-character code (Postgres) and `errorCode` its numeric code (ClickHouse) |
+| `TransactionCommitFailed` | COMMIT failed: a deferred constraint, a serialization failure at commit, a dropped connection. Carries `reason`, `sqlState` and `errorCode` |
 | `TransactionRollbackFailed` | ROLLBACK failed after the body failed. `bodyCause` is the body's failure, `cause` the rollback's |
 | `TransactionOptionsRejected` | A setting the dialect does not support, or settings on a nested transaction |
 | `TransactionUnsupported` | The dialect has no transactions (ClickHouse) |
 
+### Constraint violations
+
+`refineConstraints(effect)` turns a `DatabaseError` or `TransactionCommitFailed` for a broken
+constraint into one tagged error per kind, so you can `catchTag` a duplicate key instead of
+reading SQLSTATEs. Every other failure passes through unchanged.
+
+| Error | Postgres SQLSTATE | ClickHouse code |
+| --- | --- | --- |
+| `UniqueViolation` | 23505 | — |
+| `ForeignKeyViolation` | 23503 | — |
+| `NotNullViolation` | 23502 | — |
+| `CheckViolation` | 23514 | 469 (`VIOLATED_CONSTRAINT`) |
+
+Each carries `message`, `sql`, the server's `sqlState` or `errorCode`, and `cause` (the error it
+came from). It also carries `constraint`, `table` and `column` when the server named them:
+Postgres names the constraint and the table, and the column for NOT NULL. ClickHouse names the
+CHECK constraint and the table (`db.table`) in its message. ClickHouse has no unique, foreign key
+or NOT NULL constraints to violate.
+
+```ts title="database-constraints.ts"
+import { Effect } from "effect"
+import * as Db from "@maple-dev/effect-orm/database"
+
+// An idempotent write: a duplicate email is "already exists", not a failure.
+export const register = (email: string) =>
+	Db.execute(Db.sql`INSERT INTO users (email) VALUES (${email})`).pipe(
+		Db.refineConstraints,
+		Effect.as("created" as const),
+		Effect.catchTag("@maple-dev/effect-orm/UniqueViolation", () => Effect.succeed("exists" as const)),
+	)
+// Effect<"created" | "exists", DatabaseError | ForeignKeyViolation | NotNullViolation | CheckViolation, Database>
+```
+
+A deferred constraint fails at COMMIT, so put `refineConstraints` outside the `transaction`
+to see it.
+
 Effect 4.0.0 turns a failed COMMIT or ROLLBACK into a defect and, when ROLLBACK fails, drops
 the body's own failure. `transaction` turns both back into the typed errors above. Defects
 from `body` stay defects.

docs/reference.md

@@ -513,6 +513,7 @@ placeholders, double-quoted identifiers.
 | `run`, `status`, `verify`, `baseline`, `MigrationDriver` and the migrate errors                  | `/migrate`        |
 | `Database`, `run`, `sql`, `query`, `execute`, `transaction`, `requireTransaction`, `retryContention`, `Transaction` | `/database` |
 | `DatabaseError`, `TransactionCommitFailed`, `TransactionRollbackFailed` and the other transaction errors | `/database` |
+| `refineConstraints`, `constraintViolation`, and the `ConstraintViolation` errors `UniqueViolation`, `ForeignKeyViolation`, `NotNullViolation`, `CheckViolation` | `/database` |
 | `withQuerySummary`, `QuerySummary` — the `db.query.summary` of the statements an effect runs, for spans | `/database` |
 | `ClickhouseHttpClient`, `layer`, `make`, `ClickhouseHttpClientConfig` — a ClickHouse `SqlClient` over `fetch`, for Workers | `/clickhouse-http` |
 | `defineConfig`, `generate`, `check`, `pull`, `loadSchema`, `readMigrations`, `analyze`, `KitError` | `/kit`            |

src/clickhouse-http/client.test.ts

@@ -106,6 +106,7 @@ describe("ClickhouseHttpClient", () => {
 		[516, "AuthenticationError"],
 		[497, "AuthorizationError"],
 		[159, "StatementTimeoutError"],
+		[469, "ConstraintError"],
 		[241, "UnknownError"],
 	] as const)("classifies ClickHouse code %s as %s", ([code, reason]) => {
 		const { live } = fake(() => ({
@@ -161,6 +162,25 @@ describe("ClickhouseHttpClient", () => {
 		)
 	})
 
+	it.effect("keeps the numeric code on DatabaseError, and a violated CHECK is CheckViolation", () => {
+		const body =
+			"Code: 469. DB::Exception: Constraint `positive` for table default.rents (7fee639d-39fa-44e2-936d-107bc35ca49b) is violated at row 1. Expression: (cents > 0). Column values: cents = -5. (VIOLATED_CONSTRAINT) (version 26.10.2.103 (official build))"
+		const { live } = fake(() => ({ status: 500, body, headers: { "x-clickhouse-exception-code": "469" } }))
+		return Effect.gen(function* () {
+			const insert = Db.execute(Db.sql`INSERT INTO rents VALUES (${1}, ${-5})`)
+			const raw = yield* Effect.flip(insert)
+			expect(raw).toMatchObject({ errorCode: 469, reason: "ConstraintError" })
+			expect(raw.sqlState).toBeUndefined()
+			const error = yield* Effect.flip(insert.pipe(Db.refineConstraints))
+			expect(error).toBeInstanceOf(Db.CheckViolation)
+			expect(error).toMatchObject({ constraint: "positive", table: "default.rents", errorCode: 469 })
+		}).pipe(
+			Effect.provide(Layer.effect(Db.Database, Effect.gen(function* () {
+				return Db.fromSqlClient(yield* SqlClient.SqlClient, { dialect: CH.clickhouseDialect })
+			})).pipe(Layer.provideMerge(live))),
+		)
+	})
+
 	it.effect("kills the query by id when its fiber is interrupted", () => {
 		const sent: Array<Sent> = []
 		const http = HttpClient.make((request, url) => {

src/clickhouse-http/client.ts

@@ -20,6 +20,7 @@ import {
 	AuthenticationError,
 	AuthorizationError,
 	ConnectionError,
+	ConstraintError,
 	SqlError,
 	SqlSyntaxError,
 	StatementTimeoutError,
@@ -82,15 +83,18 @@ const Settings = Context.Reference<Readonly<Record<string, ClickhouseSetting>>>(
 )
 
 // The codes Effect's Node driver classifies, so a failure means the same thing
-// whichever client ran it.
+// whichever client ran it. One exception: that driver reads 469
+// (VIOLATED_CONSTRAINT, a failed CHECK) as a statement timeout, which says
+// retry; here it is a ConstraintError.
 const SYNTAX_ERROR_CODES = new Set([36, 60, 62, 242])
 
 const classify = (code: number | undefined, message: string, cause: unknown, operation: string) => {
 	const props = { cause, message, operation }
 	if (code === 516) return new AuthenticationError(props)
 	if (code === 497) return new AuthorizationError(props)
 	if (code !== undefined && SYNTAX_ERROR_CODES.has(code)) return new SqlSyntaxError(props)
-	if (code === 159 || code === 469) return new StatementTimeoutError(props)
+	if (code === 159) return new StatementTimeoutError(props)
+	if (code === 469) return new ConstraintError(props)
 	return new UnknownError(props)
 }
 

src/database.ts

@@ -28,14 +28,20 @@ export {
 	type TransactionInfo,
 	type TransactionOptions,
 } from "./database/database"
+export { constraintViolation, refineConstraints } from "./database/constraint"
 export {
+	CheckViolation,
 	DatabaseError,
+	ForeignKeyViolation,
+	NotNullViolation,
+	UniqueViolation,
 	TenantScopeViolation,
 	TransactionClosed,
 	TransactionCommitFailed,
 	TransactionOptionsRejected,
 	TransactionRollbackFailed,
 	TransactionUnsupported,
+	type ConstraintViolation,
 	type TransactionError,
 } from "./database/errors"
 export { QuerySummary, withQuerySummary } from "./database/tracing"

src/database/constraint.test.ts

@@ -0,0 +1,89 @@
+// Constraint violations as typed errors, against a real Postgres (PGlite) for
+// the SQLSTATE side. ClickHouse's live CHECK test is in
+// tests/database.clickhouse.test.ts; here its errors are built by hand.
+
+import { PgliteClient } from "@effect/sql-pglite"
+import { describe, expect, it, layer } from "@effect/vitest"
+import { Effect, Layer } from "effect"
+import * as SqlClient from "effect/sql/SqlClient"
+import { postgresDialect } from "../pg/dialect"
+import * as Db from "../database"
+
+const Live = Layer.effect(
+	Db.Database,
+	Effect.gen(function* () {
+		return Db.fromSqlClient(yield* SqlClient.SqlClient, { dialect: postgresDialect })
+	}),
+).pipe(Layer.provideMerge(PgliteClient.layer()))
+
+layer(Live, { excludeTestServices: true })("constraint violations on PGlite", (it) => {
+	it.effect("a duplicate key is UniqueViolation with the constraint name", () =>
+		Effect.gen(function* () {
+			yield* Db.execute(Db.sql`CREATE TABLE users_u (id int4 PRIMARY KEY, email text CONSTRAINT users_u_email_key UNIQUE)`)
+			yield* Db.execute(Db.sql`INSERT INTO users_u VALUES (${1}, ${"a@x"})`)
+			const error = yield* Effect.flip(
+				Db.execute(Db.sql`INSERT INTO users_u VALUES (${2}, ${"a@x"})`).pipe(Db.refineConstraints),
+			)
+			expect(error._tag).toBe("@maple-dev/effect-orm/UniqueViolation")
+			if (error._tag !== "@maple-dev/effect-orm/UniqueViolation") return
+			expect(error.constraint).toBe("users_u_email_key")
+			expect(error.table).toBe("users_u")
+			expect(error.sqlState).toBe("23505")
+			expect(error.cause).toBeInstanceOf(Db.DatabaseError)
+		}),
+	)
+
+	it.effect("a missing referenced row is ForeignKeyViolation", () =>
+		Effect.gen(function* () {
+			yield* Db.execute(Db.sql`CREATE TABLE orgs_f (id int4 PRIMARY KEY)`)
+			yield* Db.execute(
+				Db.sql`CREATE TABLE units_f (id int4 PRIMARY KEY, org_id int4 CONSTRAINT units_f_org_fk REFERENCES orgs_f (id))`,
+			)
+			const error = yield* Effect.flip(Db.execute(Db.sql`INSERT INTO units_f VALUES (${1}, ${9})`).pipe(Db.refineConstraints))
+			expect(error).toBeInstanceOf(Db.ForeignKeyViolation)
+			expect(error).toMatchObject({ constraint: "units_f_org_fk", table: "units_f", sqlState: "23503" })
+		}),
+	)
+
+	it.effect("NULL in a NOT NULL column is NotNullViolation with the column", () =>
+		Effect.gen(function* () {
+			yield* Db.execute(Db.sql`CREATE TABLE tours_n (id int4 PRIMARY KEY, starts_at timestamptz NOT NULL)`)
+			const error = yield* Effect.flip(Db.execute(Db.sql`INSERT INTO tours_n VALUES (${1}, ${null})`).pipe(Db.refineConstraints))
+			expect(error).toBeInstanceOf(Db.NotNullViolation)
+			expect(error).toMatchObject({ table: "tours_n", column: "starts_at", sqlState: "23502" })
+		}),
+	)
+
+	it.effect("a row that fails a CHECK is CheckViolation with the constraint name", () =>
+		Effect.gen(function* () {
+			yield* Db.execute(Db.sql`CREATE TABLE rents_c (id int4 PRIMARY KEY, cents int4 CONSTRAINT rents_c_positive CHECK (cents > 0))`)
+			const error = yield* Effect.flip(Db.execute(Db.sql`INSERT INTO rents_c VALUES (${1}, ${-5})`).pipe(Db.refineConstraints))
+			expect(error).toBeInstanceOf(Db.CheckViolation)
+			expect(error).toMatchObject({ constraint: "rents_c_positive", table: "rents_c", sqlState: "23514" })
+		}),
+	)
+
+	it.effect("a deferred foreign key found at COMMIT is ForeignKeyViolation too", () =>
+		Effect.gen(function* () {
+			yield* Db.execute(Db.sql`CREATE TABLE orgs_d (id int4 PRIMARY KEY)`)
+			yield* Db.execute(
+				Db.sql`CREATE TABLE units_d (id int4, org_id int4 CONSTRAINT units_d_org_fk REFERENCES orgs_d (id) DEFERRABLE INITIALLY DEFERRED)`,
+			)
+			const error = yield* Effect.flip(
+				Db.transaction(Db.execute(Db.sql`INSERT INTO units_d VALUES (${1}, ${9})`)).pipe(Db.refineConstraints),
+			)
+			expect(error).toBeInstanceOf(Db.ForeignKeyViolation)
+			expect(error).toMatchObject({ constraint: "units_d_org_fk", sqlState: "23503" })
+			expect((error as Db.ForeignKeyViolation).cause).toBeInstanceOf(Db.TransactionCommitFailed)
+		}),
+	)
+
+	it.effect("every other failure passes through unchanged", () =>
+		Effect.gen(function* () {
+			const syntax = yield* Effect.flip(Db.execute({ sql: "SELEC 1" }).pipe(Db.refineConstraints))
+			expect(syntax).toBeInstanceOf(Db.DatabaseError)
+			const domain = new Error("domain")
+			expect(yield* Effect.flip(Effect.fail(domain).pipe(Db.refineConstraints))).toBe(domain)
+		}),
+	)
+})

src/database/constraint.ts

@@ -0,0 +1,63 @@
+// Constraint violations as typed errors.
+//
+// A `DatabaseError` keeps the driver's classification and the server's code; a
+// caller who wants "this email is taken" should not read either. These helpers
+// read them once and give one tagged error per constraint kind, the same on
+// every dialect that reports it.
+
+import { Effect, Option } from "effect"
+import { CheckViolation, DatabaseError, ForeignKeyViolation, NotNullViolation, TransactionCommitFailed, UniqueViolation, type ConstraintViolation } from "./errors"
+import { serverFieldOf } from "./sql-error"
+
+type Kind = typeof UniqueViolation | typeof ForeignKeyViolation | typeof NotNullViolation | typeof CheckViolation
+
+const kindOf = (error: DatabaseError | TransactionCommitFailed): Kind | undefined => {
+	if (error.sqlState === "23505" || error.reason === "UniqueViolation") return UniqueViolation
+	if (error.sqlState === "23503") return ForeignKeyViolation
+	if (error.sqlState === "23502") return NotNullViolation
+	if (error.sqlState === "23514") return CheckViolation
+	// ClickHouse VIOLATED_CONSTRAINT: its only constraints are CHECKs.
+	if (error.sqlState === undefined && error.errorCode === 469) return CheckViolation
+	return undefined
+}
+
+// ClickHouse names the constraint and table only in its message:
+// "Constraint `positive` for table default.rents (uuid) is violated at row 1".
+const fromClickHouseMessage = (message: string) => {
+	const match = /Constraint `([^`]+)` for table ([^\s(]+)/.exec(message)
+	return match === null ? {} : { constraint: match[1], table: match[2] }
+}
+
+/**
+ * The constraint violation `error` reports, if it is a `DatabaseError` or
+ * `TransactionCommitFailed` for one. Anything else is `None`.
+ */
+export const constraintViolation = (error: unknown): Option.Option<ConstraintViolation> => {
+	if (!(error instanceof DatabaseError || error instanceof TransactionCommitFailed)) return Option.none()
+	const Kind = kindOf(error)
+	if (Kind === undefined) return Option.none()
+	const named = error.errorCode === undefined ? {} : fromClickHouseMessage(error.message)
+	const constraint = serverFieldOf(error.cause, "constraint") ?? named.constraint
+	const table = serverFieldOf(error.cause, "table") ?? named.table
+	const column = serverFieldOf(error.cause, "column")
+	return Option.some(
+		new Kind({
+			message: error.message,
+			...(error instanceof DatabaseError ? { sql: error.sql } : undefined),
+			...(constraint === undefined ? undefined : { constraint }),
+			...(table === undefined ? undefined : { table }),
+			...(column === undefined ? undefined : { column }),
+			...(error.sqlState === undefined ? undefined : { sqlState: error.sqlState }),
+			...(error.errorCode === undefined ? undefined : { errorCode: error.errorCode }),
+			cause: error,
+		}),
+	)
+}
+
+/**
+ * Fail with a `ConstraintViolation` where `effect` fails with a
+ * `DatabaseError` or `TransactionCommitFailed` for one, so a caller can
+ * `catchTag` a duplicate key. Every other failure passes through unchanged.
+ */
+export const refineConstraints = <A, E, R>(effect: Effect.Effect<A, E, R>): Effect.Effect<A, E | ConstraintViolation, R> =>
+	Effect.mapError(effect, (error): E | ConstraintViolation => Option.getOrElse(constraintViolation(error), () => error))

src/database/database.test-d.ts

@@ -77,3 +77,10 @@ expectTypeOf(Db.run(upsert, { count: 1 })).toEqualTypeOf<Effect.Effect<ReadonlyA
 // @ts-expect-error the param is still required
 Db.run(upsert)
 expectTypeOf(PG.compileUnsafe(upsert, { count: 1 })).toEqualTypeOf<CH.CompiledQuery<{ readonly count: number }, undefined>>()
+
+// `refineConstraints` adds the violations; catching one removes it.
+const write = Db.execute(Db.sql`INSERT INTO t VALUES (1)`).pipe(Db.refineConstraints)
+expectTypeOf(write).toEqualTypeOf<Effect.Effect<void, Db.DatabaseError | Db.ConstraintViolation, Db.Database>>()
+expectTypeOf(write.pipe(Effect.catchTag("@maple-dev/effect-orm/UniqueViolation", (error) => Effect.succeed(error.constraint)))).toEqualTypeOf<
+	Effect.Effect<string | undefined | void, Db.DatabaseError | Db.ForeignKeyViolation | Db.NotNullViolation | Db.CheckViolation, Db.Database>
+>()

src/database/database.ts

@@ -30,7 +30,7 @@ import {
 	type TransactionError,
 } from "./errors"
 import { isSqlTemplate, renderTemplate, type SqlTemplate } from "./sql"
-import { firstLine, reasonOf, sqlStateOf, toDatabaseError } from "./sql-error"
+import { errorCodeOf, firstLine, reasonOf, sqlStateOf, toDatabaseError } from "./sql-error"
 import { traceStatement, type TracedStatement } from "./tracing"
 
 /** SQL and the values bound to its placeholders. A `CompiledQuery` is one. */
@@ -445,11 +445,13 @@ const restoreCause = <E>(
 		const error = defect.success
 		if (bodyExit !== undefined && Exit.isSuccess(bodyExit)) {
 			const sqlState = sqlStateOf(error)
+			const errorCode = errorCodeOf(error)
 			return Effect.fail(
 				new TransactionCommitFailed({
 					message: `${depth === 0 ? "COMMIT" : "releasing the savepoint"} failed: ${firstLine(error)}`,
 					reason: reasonOf(error),
 					...(sqlState === undefined ? undefined : { sqlState }),
+					...(errorCode === undefined ? undefined : { errorCode }),
 					cause: error,
 				}),
 			)