From a4428de54ae71bb72378c7e1e39737b3dc14778f Mon Sep 17 00:00:00 2001 From: Owen Mansel-Chan Date: Thu, 8 Oct 2026 14:46:47 +0100 Subject: [PATCH 1/2] Update QlBuiltins::NewEntity for db upgrades --- docs/prepare-db-upgrade-downgrade.md | 53 ++++++++++++++++++++++++++++ 1 file changed, 53 insertions(+) diff --git a/docs/prepare-db-upgrade-downgrade.md b/docs/prepare-db-upgrade-downgrade.md index 1ea1dcffd45d..a76227be3391 100644 --- a/docs/prepare-db-upgrade-downgrade.md +++ b/docs/prepare-db-upgrade-downgrade.md @@ -101,6 +101,59 @@ relation1.rel: run upgrade.qlo predicate1 relation2.rel: run upgrade.qlo predicate2 ``` +### Creating new database entities + +An upgrade or downgrade query can create database entities that do not exist in +the source database by using the compiler-provided `QlBuiltins::NewEntity` +module. Define a `newtype` whose values uniquely identify the entities to +create, and instantiate `NewEntity` with that type: + +```ql +class Element extends @element { + string toString() { none() } +} + +// Create one wrapper for each existing element. +newtype TAddedElement = TWrapper(Element element) + +module Fresh = QlBuiltins::NewEntity; +``` + +`Fresh::map` maps each value of `TAddedElement` to a distinct new entity ID. +Using an algebraic data type with multiple constructors allows several distinct +entities to be created for the same source entity. + +The query is compiled against the source schema, so a fresh ID cannot belong to +an entity type that exists only in the target schema. Define a union of the +appropriate source-schema type and `Fresh::EntityId`, then use a class extending +that union in the output predicates: + +```ql +class TNewElement = @element or Fresh::EntityId; + +class NewElement extends TNewElement { + string toString() { none() } +} + +query predicate new_wrappers(NewElement wrapper, Element element) { + wrapper = Fresh::map(TWrapper(element)) +} +``` + +Run the predicate to populate a relation in the target schema: + +```properties +wrappers.rel: run upgrade.qlo new_wrappers +``` + +The target relation's column type determines the database type of each fresh +ID. Emit every target relation needed to describe the new entity, including +relationships such as its parent, location, or type. When rewriting an existing +relation, its output predicate will usually need to preserve all existing rows +as well as add rows containing the fresh IDs. Reuse the same +`Fresh::map(...)` expression in each predicate that refers to a particular new +entity. + ### Testing your scripts Although we have some automated testing of the scripts (e.g. to test that you can upgrade databases all the way from an initial dbscheme to the newest, and back), it's essential that you apply some more rigorous testing for any non-trivial upgrade or downgrade. You might do so as follows: From 7952f3cc06fb25178cd18bd9585cda855ba1a394 Mon Sep 17 00:00:00 2001 From: Owen Mansel-Chan Date: Thu, 8 Oct 2026 15:10:43 +0100 Subject: [PATCH 2/2] Clarify fresh-only columns and column types --- docs/prepare-db-upgrade-downgrade.md | 42 ++++++++++++++++------------ 1 file changed, 24 insertions(+), 18 deletions(-) diff --git a/docs/prepare-db-upgrade-downgrade.md b/docs/prepare-db-upgrade-downgrade.md index a76227be3391..113f49ce3083 100644 --- a/docs/prepare-db-upgrade-downgrade.md +++ b/docs/prepare-db-upgrade-downgrade.md @@ -123,19 +123,12 @@ module Fresh = QlBuiltins::NewEntity; Using an algebraic data type with multiple constructors allows several distinct entities to be created for the same source entity. -The query is compiled against the source schema, so a fresh ID cannot belong to -an entity type that exists only in the target schema. Define a union of the -appropriate source-schema type and `Fresh::EntityId`, then use a class extending -that union in the output predicates: +The query is compiled against the source schema, so its static QL types cannot +refer to an entity type that exists only in the target schema. An output column +that contains only fresh IDs can use `Fresh::EntityId` directly: ```ql -class TNewElement = @element or Fresh::EntityId; - -class NewElement extends TNewElement { - string toString() { none() } -} - -query predicate new_wrappers(NewElement wrapper, Element element) { +query predicate new_wrappers(Fresh::EntityId wrapper, Element element) { wrapper = Fresh::map(TWrapper(element)) } ``` @@ -146,13 +139,26 @@ Run the predicate to populate a relation in the target schema: wrappers.rel: run upgrade.qlo new_wrappers ``` -The target relation's column type determines the database type of each fresh -ID. Emit every target relation needed to describe the new entity, including -relationships such as its parent, location, or type. When rewriting an existing -relation, its output predicate will usually need to preserve all existing rows -as well as add rows containing the fresh IDs. Reuse the same -`Fresh::map(...)` expression in each predicate that refers to a particular new -entity. +If an output column can contain both existing source IDs and fresh IDs, define a +union of the source-schema type and `Fresh::EntityId`, then use a class extending +that union as the column's static QL type: + +```ql +class TExistingOrFreshElement = @element or Fresh::EntityId; + +class ExistingOrFreshElement extends TExistingOrFreshElement { + string toString() { none() } +} +``` + +These QL types only describe values while evaluating the transformation against +the source schema. The corresponding column type in the target database schema +determines the database type assigned to each fresh ID. Emit every target +relation needed to describe the new entity, including relationships such as its +parent, location, or type. When rewriting an existing relation, its output +predicate will usually need to preserve all existing rows as well as add rows +containing the fresh IDs. Reuse the same `Fresh::map(...)` expression in each +predicate that refers to a particular new entity. ### Testing your scripts