diff --git a/.github/workflows/release-golang.yml b/.github/workflows/release-golang.yml index 0db79e34a..b8b919d59 100644 --- a/.github/workflows/release-golang.yml +++ b/.github/workflows/release-golang.yml @@ -20,7 +20,8 @@ name: Release (Go) # The ref must carry the EQL v4 bundle (packages/eql's v4 build), or the # build job stops at "Build the EQL SQL bundles". A version with a pre-release # suffix may be cut from any branch and is published as a GitHub pre-release; -# a final version is cut from main only (scripts/go-release-version.mjs). +# a final version is cut from main only, and not while go.mod pins vitaminc's +# Go bindings at a pseudo-version (scripts/go-release-version.mjs). # # Three jobs, split by what they may touch: # diff --git a/AGENTS.md b/AGENTS.md index a482387cc..7a795d2e8 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -92,7 +92,7 @@ Every npm package except EQL lives under `languages/typescript/`: packages in `l links are provenance only. - `packages/stack-auth`, `packages/stack-profile`, `packages/stack-kms`, `packages/stack-encrypt`, `packages/stack-encrypt-derive`, `packages/stack-guest-abi`: The Rust crates imported from `cipherstash/cipherstash-suite` with their history — `stack-auth` and `stack-profile` (published to crates.io, one version group), `stack-kms` (published to crates.io from 0.1.0, its own version group, re-exported by `stack-encrypt` as `stack_encrypt::kms`), `stack-encrypt` and `stack-encrypt-derive` (published to crates.io from 0.1.0, one version group; `eql-bindings`' `stack-encrypt` feature depends on them from the registry), and `stack-guest-abi` (`publish = false`). They are the members of the **root Cargo workspace**, with the three node binding crates below. See "Working on the Rust crates". - `languages/typescript/packages/auth`, `languages/typescript/packages/profile`, `languages/typescript/packages/stack-auth-wasm`: The node bindings of those crates. `@cipherstash/auth` (napi-rs v2) and its six `platforms/*` packages are published to npm from this repository by `release.yml` (`auth-artifacts`, `publish-auth`); a change to what it ships, the `stack-auth` crate included, needs an `@cipherstash/auth` changeset (`require-auth-npm-changeset.yml`). `@cipherstash/profile` and its platforms are private and never published; `@cipherstash/stack-auth-wasm` is private and builds the wasm that `@cipherstash/auth` ships. Their `build` and `test` scripts never invoke cargo; `build:native`, `build:debug` and `test:cargo` do. -- `languages/golang`: The Go module — the SDK `encrypt` with its generated-code support `encrypt/gensupport` and the policy packages `encrypt/policy` and `encrypt/policy/protosource`; the credential package `auth`; the generator `stashgen` and its command `cmd/stashgen`; and `internal` (the shared guest plumbing and the `record` wire model). A wazero host with no cgo. Its two WASI guests (`encrypt/guest`, `auth/guest`) are detached Cargo workspaces built by `mise run wasm:guest:build` and `mise run wasm:auth-guest:build`; `mise run wasm:guest:build:deterministic` builds the seeded TEST build the hermetic Go tests use; the `.wasm` files they embed are gitignored. The stack-encrypt guest has a second build with the EQL types (`mise run wasm:guest:build:eql`, cargo feature `eql`, linking `eql-bindings` by path; `wasm:guest:build:eql:deterministic` is its test build), embedded by `encrypt/eql` and registered on import, so a program whose generated code names an EQL type runs it (ADR-0007, amended). `encrypt/eql`'s types and `Types` table are generated from the EQL catalog by `eql-codegen` (`eql_gen.go`, written by `mise run types:generate` in `packages/eql` and drift-gated by `types:check` and `cargo test -p eql-codegen`). Generated `*_stash.go` files are committed and CI fails when `go generate ./...` changes one. Releases are cut by hand with `.github/workflows/release-golang.yml` (`gh workflow run release-golang.yml -f version=X.Y.Z[-pre] --ref `): because the guests are gitignored on main and Go fetches a version as the files committed at its tag, the workflow builds them, commits them on a release-only commit no branch points at, tags it `languages/golang/vX.Y.Z` and pushes the tag alone, then attaches the EQL v4 SQL bundle to the GitHub release. The ref must carry the EQL v4 bundle, or the build stops at "Build the EQL SQL bundles". `scripts/go-release-version.mjs` refuses, before anything is built, every version Go would refuse (non-canonical semver, `+build` metadata, a v2+ major on a module path with no `/vN` suffix, a version shaped like a pseudo-version), and a final version dispatched from anywhere but `main`; a pre-release may come from a branch. The build job also refuses a version the Go proxy already lists in `@v/list`, which catches a tag deleted after the proxy fetched it; it never asks the proxy for the version itself before the tag exists, because the proxy would cache "not found" for up to 30 minutes and `verify` would fail after the push. The build checks the `Cargo.lock` of the EQL workspace and both guest workspaces with `--locked` first, since none of those builds passes it. It runs in a job with no write token and hands the guests to a `publish` job that holds `contents: write` and runs only git and gh. `publish` fetches the commit with plain unauthenticated git rather than `actions/checkout`, whose `token` input defaults to the job's writable token, and refuses a release commit whose tree is not the one the build tested. `publish` takes the paths it commits from the workflow's fixed `GUESTS` list, never from the artifact, and refuses a `guests.sha256` that is not exactly one SHA-256 line for each of those three guests: the build job runs third-party code after writing that manifest, so a path read from it could put any file on the tag (`scripts/__tests__/go-release-artifact.test.mjs`). A version is immutable once the Go proxy has fetched it, so a tag is never moved or deleted: if `publish` or `verify` fails after the tag is pushed, use "Re-run failed jobs" on the same run, which keeps a tag that run pushed and creates or completes the release; a new dispatch of the same version is refused. Never tag a main commit by hand: `go get` would fetch a module with no guests. +- `languages/golang`: The Go module — the SDK `encrypt` with its generated-code support `encrypt/gensupport` and the policy packages `encrypt/policy` and `encrypt/policy/protosource`; the credential package `auth`; the generator `stashgen` and its command `cmd/stashgen`; and `internal` (the shared guest plumbing and the `record` wire model). A wazero host with no cgo. Its two WASI guests (`encrypt/guest`, `auth/guest`) are detached Cargo workspaces built by `mise run wasm:guest:build` and `mise run wasm:auth-guest:build`; `mise run wasm:guest:build:deterministic` builds the seeded TEST build the hermetic Go tests use; the `.wasm` files they embed are gitignored. The stack-encrypt guest has a second build with the EQL types (`mise run wasm:guest:build:eql`, cargo feature `eql`, linking `eql-bindings` by path; `wasm:guest:build:eql:deterministic` is its test build), embedded by `encrypt/eql` and registered on import, so a program whose generated code names an EQL type runs it (ADR-0007, amended). `encrypt/eql`'s types and `Types` table are generated from the EQL catalog by `eql-codegen` (`eql_gen.go`, written by `mise run types:generate` in `packages/eql` and drift-gated by `types:check` and `cargo test -p eql-codegen`). Generated `*_stash.go` files are committed and CI fails when `go generate ./...` changes one. Releases are cut by hand with `.github/workflows/release-golang.yml` (`gh workflow run release-golang.yml -f version=X.Y.Z[-pre] --ref `): because the guests are gitignored on main and Go fetches a version as the files committed at its tag, the workflow builds them, commits them on a release-only commit no branch points at, tags it `languages/golang/vX.Y.Z` and pushes the tag alone, then attaches the EQL v4 SQL bundle to the GitHub release. The ref must carry the EQL v4 bundle, or the build stops at "Build the EQL SQL bundles". `scripts/go-release-version.mjs` refuses, before anything is built, every version Go would refuse (non-canonical semver, `+build` metadata, a v2+ major on a module path with no `/vN` suffix, a version shaped like a pseudo-version), a final version dispatched from anywhere but `main`, and a final version while `languages/golang/go.mod` pins a `github.com/cipherstash/vitaminc/…` module at a pseudo-version (vitaminc's Go bindings have no tags, and a TEMPORARY pin on an unreleased commit must be gone before the module is tagged); a pre-release may come from a branch and may carry such a pin. The build job also refuses a version the Go proxy already lists in `@v/list`, which catches a tag deleted after the proxy fetched it; it never asks the proxy for the version itself before the tag exists, because the proxy would cache "not found" for up to 30 minutes and `verify` would fail after the push. The build checks the `Cargo.lock` of the EQL workspace and both guest workspaces with `--locked` first, since none of those builds passes it. It runs in a job with no write token and hands the guests to a `publish` job that holds `contents: write` and runs only git and gh. `publish` fetches the commit with plain unauthenticated git rather than `actions/checkout`, whose `token` input defaults to the job's writable token, and refuses a release commit whose tree is not the one the build tested. `publish` takes the paths it commits from the workflow's fixed `GUESTS` list, never from the artifact, and refuses a `guests.sha256` that is not exactly one SHA-256 line for each of those three guests: the build job runs third-party code after writing that manifest, so a path read from it could put any file on the tag (`scripts/__tests__/go-release-artifact.test.mjs`). A version is immutable once the Go proxy has fetched it, so a tag is never moved or deleted: if `publish` or `verify` fails after the tag is pushed, use "Re-run failed jobs" on the same run, which keeps a tag that run pushed and creates or completes the release; a new dispatch of the same version is refused. Never tag a main commit by hand: `go get` would fetch a module with no guests. - `e2e/*`: Cross-package end-to-end tests (package managers, supply chain, Prisma example README) - `languages/typescript/examples/*`: Working apps (basic, prisma, supabase-worker) - `docs/plans/*`: Internal design plans. User-facing documentation lives at https://cipherstash.com/docs (not in this repo). diff --git a/languages/golang/CHANGELOG.md b/languages/golang/CHANGELOG.md index 6a2a4f4d2..ca56396a0 100644 --- a/languages/golang/CHANGELOG.md +++ b/languages/golang/CHANGELOG.md @@ -187,6 +187,20 @@ listed under **Breaking**. Releases are tagged `languages/golang/vX.Y.Z`. ### Changed +- **vitaminc's Go bindings at an untagged vitaminc commit (temporary).** + `vcffi` and `vcvalue` move to vitaminc `06568ba93` on its `main`, which + carries the storage profile (`vcffi.MarshalStorage`, from + cipherstash/vitaminc#526); vitaminc's Go bindings have no tags, so the + commit is the pin. One encoding changes + with them, and it comes from vitaminc `e9c3837b` (on `main`, in no Go + tag yet), not from the storage profile: a nil pointer to a fixed-width + value (`(*int64)(nil)`) is written as a typed null where it was written + as a null. The engine reads a typed null as a null, and generated code + passes an untyped nil for a nil pointer, so this only shows in the bytes + a hand-built `record.Source` or query input sends. The pin returns to a + vitaminc release before this module is tagged: + `scripts/go-release-version.mjs` refuses a final version while `go.mod` + names a vitaminc pseudo-version. - **An EQL v4 block ORE term is one hex string.** The `ob` of a value encrypted into an `_ord_ore` or `text_search_ore` EQL type is `"ob": "0202…"`, where it was a one-element array, and the EQL v4 bundle diff --git a/languages/golang/encrypt/gensupport/gensupport_test.go b/languages/golang/encrypt/gensupport/gensupport_test.go index 68bd64ea0..7d178828e 100644 --- a/languages/golang/encrypt/gensupport/gensupport_test.go +++ b/languages/golang/encrypt/gensupport/gensupport_test.go @@ -5,6 +5,7 @@ import ( "log/slog" "strings" "testing" + "time" ) func TestRedactedHidesSealedFieldsAndSortsShown(t *testing.T) { @@ -73,3 +74,31 @@ func TestVersionConstantIsTyped(t *testing.T) { t.Fatalf("GeneratedVersion1 = %d", GeneratedVersion1) } } + +// Deref is how generated code passes a field declared over a pointer, and +// it is why a nil pointer never reaches the codec typed: the untyped nil it +// gives is == nil, so Seal leaves a null=sql field out and the engine sees a +// plain null elsewhere, while the typed nil itself is not == nil and would be +// sent as a typed null of the pointer's type (see encrypt's typed_nil_test). +func TestDerefGivesAnUntypedNilForANilPointer(t *testing.T) { + // The contrast Deref exists for: the same pointer, as an interface + // value, is not == nil. (Through a call, so the checker does not fold + // the comparison away.) + asAny := func(v any) any { return v } + if asAny((*int64)(nil)) == nil { + t.Fatal("a typed nil pointer is == nil as an interface; this test pins the opposite") + } + if v := Deref((*int64)(nil)); v != nil { + t.Fatalf("Deref((*int64)(nil)) = %#v, want an untyped nil", v) + } + if v := Deref((*time.Time)(nil)); v != nil { + t.Fatalf("Deref((*time.Time)(nil)) = %#v, want an untyped nil", v) + } + n, when := int64(7), time.Unix(1_700_000_000, 0).UTC() + if v := Deref(&n); v != int64(7) { + t.Fatalf("Deref(&7) = %#v, want the int64 7", v) + } + if v := Deref(&when); v != when { + t.Fatalf("Deref(&when) = %#v, want the time", v) + } +} diff --git a/languages/golang/encrypt/typed_nil_test.go b/languages/golang/encrypt/typed_nil_test.go new file mode 100644 index 000000000..a1020c126 --- /dev/null +++ b/languages/golang/encrypt/typed_nil_test.go @@ -0,0 +1,206 @@ +package encrypt_test + +import ( + "bytes" + "context" + "errors" + "reflect" + "testing" + "time" + + "github.com/cipherstash/stack/languages/golang/encrypt" + "github.com/cipherstash/stack/languages/golang/internal/record" + "github.com/cipherstash/vitaminc/bindings/go/vcffi" +) + +// A typed nil pointer on the two raw paths. Generated code passes a nil +// pointer through gensupport.Deref, which gives an untyped nil; the two +// paths that take a caller's value as it is are Cipher.Seal with a +// hand-built record.Source and Cipher.QueryTerm's input. There a +// (*int64)(nil) is not == nil, so it is sent, and vcffi, from vitaminc +// e9c3837b on (on main, in no Go tag yet), writes it as a typed null, +// padded to the width of an int64, where it wrote a one-byte null. These +// tests pin what the engine does with it: a typed null is a null to the +// engine, so each path answers a typed nil as it answers an untyped one, on +// a field of every null mode. + +// typedNilPlan has one int64 field in each null mode and a timestamp field +// that seals its nulls, so both encodings meet every answer the engine has. +var typedNilPlan = &record.Plan{Context: []string{"typed_nil"}, Fields: []record.Field{ + {Name: "every", Kind: record.Int64, Outputs: []record.Output{record.Ciphertext, record.Equality}}, + {Name: "sealed", Kind: record.Int64, Outputs: []record.Output{record.Ciphertext, record.Equality}, Null: record.SealedNull}, + {Name: "sql", Kind: record.Int64, Outputs: []record.Output{record.Ciphertext, record.Ore}, Null: record.SQLNull}, + {Name: "when", Kind: record.Timestamp, Outputs: []record.Output{record.Ciphertext, record.Equality}, Null: record.SealedNull}, +}} + +// The pinned vcffi behaviour itself, so the engine tests below are read +// against the bytes that reach it: a nil pointer to a fixed-width type is a +// typed null (tag 0x15, then the type, padded to the type's width), not the +// one-byte null an untyped nil is, and it decodes as nil like every null. +func TestANilFixedWidthPointerMarshalsAsATypedNull(t *testing.T) { + untyped, err := vcffi.Marshal(nil) + if err != nil || !bytes.Equal(untyped, []byte{0x00}) { + t.Fatalf("an untyped nil marshals as %x, %v; want the one-byte null", untyped, err) + } + for name, tc := range map[string]struct { + nilPointer any + present any + }{ + "*int64": {(*int64)(nil), int64(7)}, + "*float64": {(*float64)(nil), 1.5}, + "*time.Time": {(*time.Time)(nil), time.Unix(1_700_000_000, 0).UTC()}, + } { + got, err := vcffi.Marshal(tc.nilPointer) + if err != nil { + t.Fatalf("%s: %v", name, err) + } + if bytes.Equal(got, untyped) || got[0] != 0x15 { + t.Fatalf("%s: a nil pointer marshals as %x; want a typed null (tag 0x15)", name, got) + } + value, err := vcffi.Marshal(tc.present) + if err != nil { + t.Fatal(err) + } + if len(got) != len(value) { + t.Fatalf("%s: the typed null is %d bytes, a value %d; want them as long", name, len(got), len(value)) + } + back, err := vcffi.Unmarshal(got) + if err != nil || back != nil { + t.Fatalf("%s: the typed null decodes as %#v, %v; want nil", name, back, err) + } + } +} + +// Seal with a hand-built Source: a typed nil seals as the untyped nil does +// on a field of every null mode. Sealed, it is a sealed null, as long as a +// value, with the term every nil shares; sql, it is left out of the record; +// not null, it is refused as a value of the wrong type, the same refusal. +// What it sealed opens as nil, as the untyped nil's does. +func TestSealReadsATypedNilAsANull(t *testing.T) { + c := deterministicClient(t) + ctx := context.Background() + cipher := c.DefaultKeyset() + when := time.Unix(1_700_000_000, 0).UTC() + rows := []record.Source{ + {"every": int64(1), "sealed": int64(2), "sql": int64(3), "when": when}, + {"every": int64(1), "sealed": nil, "sql": nil, "when": nil}, + {"every": int64(1), "sealed": (*int64)(nil), "sql": (*int64)(nil), "when": (*time.Time)(nil)}, + } + sealed, err := cipher.Seal(ctx, typedNilPlan, rows) + if err != nil { + t.Fatal(err) + } + present, untyped, typed := sealed[0], sealed[1], sealed[2] + + // A sealed null: both nils are sealed as long as a value, and they + // share the equality term, which no value has. + for _, field := range []string{"sealed", "when"} { + if len(typed[field].Ciphertext) == 0 || len(typed[field].Ciphertext) != len(untyped[field].Ciphertext) || len(typed[field].Ciphertext) != len(present[field].Ciphertext) { + t.Fatalf("%s: a typed nil seals to %d bytes, an untyped nil to %d, a value to %d; want all three as long", + field, len(typed[field].Ciphertext), len(untyped[field].Ciphertext), len(present[field].Ciphertext)) + } + typedTerm, untypedTerm, presentTerm := typed[field].Terms[record.Equality], untyped[field].Terms[record.Equality], present[field].Terms[record.Equality] + if len(typedTerm) == 0 || !bytes.Equal(typedTerm, untypedTerm) || bytes.Equal(typedTerm, presentTerm) { + t.Fatalf("%s: a typed nil's equality term is %x, an untyped nil's %x, a value's %x; want the nils to share one and the value another", field, typedTerm, untypedTerm, presentTerm) + } + } + // A SQL null: a typed nil stores nothing, as an untyped nil does; the + // host sends the typed one (it is not == nil) and the engine leaves it + // out. + for name, rec := range map[string]record.Sealed{"a typed nil": typed, "an untyped nil": untyped} { + if outputs, ok := rec["sql"]; ok { + t.Fatalf("%s on a null=sql field stored %+v; want no entry", name, outputs) + } + } + if _, ok := present["sql"]; !ok { + t.Fatal("a value on a null=sql field is stored") + } + + // Both open as nil, and the two opened rows are the same row. + opened, err := cipher.Open(ctx, typedNilPlan, sealed) + if err != nil { + t.Fatal(err) + } + for _, field := range []string{"sealed", "sql", "when"} { + v, ok := opened[2][field] + if !ok || v != nil { + t.Fatalf("%s sealed from a typed nil opened as %#v (present %v); want an untyped nil", field, v, ok) + } + } + if !reflect.DeepEqual(opened[1], opened[2]) { + t.Fatalf("a typed nil opened as %#v, an untyped nil as %#v; want the same row", opened[2], opened[1]) + } + if got := opened[0]; got["sealed"] != int64(2) || got["sql"] != int64(3) || got["when"] != when { + t.Fatalf("the values opened as %#v", got) + } + + // A field every row has refuses a typed nil as it refuses an untyped + // one: a value that is not of the field's type, named. + for name, v := range map[string]any{"a typed nil": (*int64)(nil), "an untyped nil": nil} { + _, err := cipher.Seal(ctx, typedNilPlan, []record.Source{{"every": v, "sealed": int64(2), "sql": int64(3), "when": when}}) + var d *encrypt.Diagnostic + if !errors.Is(err, encrypt.ErrEncoding) || !errors.As(err, &d) || d.Field() != "every" || d.Reason() != "field_type" { + t.Fatalf("%s on a field every row has: %v; want ErrEncoding naming \"every\" with reason field_type", name, err) + } + } +} + +// QueryTerm with a typed nil input: the engine reads it as the null it is. +// On a field that seals its nulls, eq derives the term the stored nils +// share, the same term for a typed and an untyped nil; on any other field, +// or operator, a null is no query, and both nils are refused alike. +func TestQueryTermReadsATypedNilAsANull(t *testing.T) { + c := deterministicClient(t) + ctx := context.Background() + cipher := c.DefaultKeyset() + when := time.Unix(1_700_000_000, 0).UTC() + sealed, err := cipher.Seal(ctx, typedNilPlan, []record.Source{{"every": int64(1), "sealed": nil, "sql": nil, "when": nil}}) + if err != nil { + t.Fatal(err) + } + for _, tc := range []struct { + field string + typedNil any + aValue any + valueTerm []byte + }{ + {"sealed", (*int64)(nil), int64(2), nil}, + {"when", (*time.Time)(nil), when, nil}, + } { + typed, err := cipher.QueryTerm(ctx, typedNilPlan, tc.field, "eq", tc.typedNil) + if err != nil { + t.Fatalf("%s eq a typed nil: %v", tc.field, err) + } + untyped, err := cipher.QueryTerm(ctx, typedNilPlan, tc.field, "eq", nil) + if err != nil { + t.Fatalf("%s eq an untyped nil: %v", tc.field, err) + } + if typed.Operator != "eq" || typed.Index != record.Equality || len(typed.Terms) != 1 || !reflect.DeepEqual(typed, untyped) { + t.Fatalf("%s eq a typed nil = %+v, an untyped nil = %+v; want the same eq term", tc.field, typed, untyped) + } + if stored := sealed[0][tc.field].Terms[record.Equality]; !bytes.Equal(typed.Terms[0], stored) { + t.Fatalf("%s eq a typed nil = %x, the stored nil's term is %x; want the query to find the stored nils", tc.field, typed.Terms[0], stored) + } + value, err := cipher.QueryTerm(ctx, typedNilPlan, tc.field, "eq", tc.aValue) + if err != nil || bytes.Equal(value.Terms[0], typed.Terms[0]) { + t.Fatalf("%s eq a value: %v, %x; want a term other than the nils'", tc.field, err, value.Terms) + } + } + // Everywhere else a null is no query: on a field every row has and on a + // field whose nulls are SQL NULL. (A field that seals its nulls answers + // eq alone, so there is no other operator to put a null under.) + for name, tc := range map[string]struct { + field, operator string + input any + }{ + "eq on a field every row has": {"every", "eq", (*int64)(nil)}, + "lt on a null=sql field": {"sql", "lt", (*int64)(nil)}, + "the untyped nil, for comparison": {"every", "eq", nil}, + } { + _, err := cipher.QueryTerm(ctx, typedNilPlan, tc.field, tc.operator, tc.input) + var d *encrypt.Diagnostic + if !errors.Is(err, encrypt.ErrEncoding) || !errors.As(err, &d) || d.Field() != tc.field || d.Reason() != "null_query" { + t.Fatalf("%s: %v; want ErrEncoding naming %q with reason null_query", name, err, tc.field) + } + } +} diff --git a/languages/golang/go.mod b/languages/golang/go.mod index 2df28f1c2..ac66bd9a5 100644 --- a/languages/golang/go.mod +++ b/languages/golang/go.mod @@ -2,9 +2,17 @@ module github.com/cipherstash/stack/languages/golang go 1.26.10 +// TEMPORARY: vcffi and vcvalue at vitaminc 06568ba93, the commit on vitaminc +// main that carries the storage profile (vcffi.MarshalStorage, from +// cipherstash/vitaminc#526 by way of cipherstash/vitaminc#544). vitaminc's Go +// bindings have no tags, so the commit is the pin. Only Go writes and reads +// that profile; the guests stay on vitaminc 0.8.0 from crates.io, whose +// transport is the one this commit writes. The pin moves to a tag when +// vitaminc provides one; until then scripts/go-release-version.mjs refuses a +// final version of this module. require ( - github.com/cipherstash/vitaminc/bindings/go/vcffi v0.0.0-20261017042657-ca2298d3e7e8 - github.com/cipherstash/vitaminc/bindings/go/vcvalue v0.0.0-20261017042657-ca2298d3e7e8 + github.com/cipherstash/vitaminc/bindings/go/vcffi v0.0.0-20261026122913-06568ba93bd9 + github.com/cipherstash/vitaminc/bindings/go/vcvalue v0.0.0-20261026122913-06568ba93bd9 github.com/tetratelabs/wazero v1.12.0 golang.org/x/oauth2 v0.37.0 ) diff --git a/languages/golang/go.sum b/languages/golang/go.sum index dfafe678f..6c30aed4a 100644 --- a/languages/golang/go.sum +++ b/languages/golang/go.sum @@ -1,7 +1,7 @@ -github.com/cipherstash/vitaminc/bindings/go/vcffi v0.0.0-20261017042657-ca2298d3e7e8 h1:kXpXvBsE5P35aw8yN8l2k3ARuoPdjQkXvxocg4D2WUc= -github.com/cipherstash/vitaminc/bindings/go/vcffi v0.0.0-20261017042657-ca2298d3e7e8/go.mod h1:6jpaqAo6f7rjWuxoti0Ymoi9DQZyxpnieWVudhXv38Y= -github.com/cipherstash/vitaminc/bindings/go/vcvalue v0.0.0-20261017042657-ca2298d3e7e8 h1:72kmUdUUZENerHctDhFMD31/mWtE80cRbpjLPZOeLCs= -github.com/cipherstash/vitaminc/bindings/go/vcvalue v0.0.0-20261017042657-ca2298d3e7e8/go.mod h1:RJODA1DCSm4H+1pbmCzDdT2pAIkxhinwIu6ewjh5sPs= +github.com/cipherstash/vitaminc/bindings/go/vcffi v0.0.0-20261026122913-06568ba93bd9 h1:ZF7YOnal03J9uj5ggsqF3vUlPgM3affKpz3DHIyO4UE= +github.com/cipherstash/vitaminc/bindings/go/vcffi v0.0.0-20261026122913-06568ba93bd9/go.mod h1:7hPdj2tN3YpcPZYdZ+Fe7+Fsc7dnSyH5bsL9y+LbRuM= +github.com/cipherstash/vitaminc/bindings/go/vcvalue v0.0.0-20261026122913-06568ba93bd9 h1:y1+c1HV4FeZ+EiX9k3imPIzQYYK3QZD7HoZU8YORa6Q= +github.com/cipherstash/vitaminc/bindings/go/vcvalue v0.0.0-20261026122913-06568ba93bd9/go.mod h1:RJODA1DCSm4H+1pbmCzDdT2pAIkxhinwIu6ewjh5sPs= github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= diff --git a/scripts/__tests__/go-release-version.test.mjs b/scripts/__tests__/go-release-version.test.mjs index 1eca3c37c..ed2639573 100644 --- a/scripts/__tests__/go-release-version.test.mjs +++ b/scripts/__tests__/go-release-version.test.mjs @@ -6,6 +6,7 @@ import { allowedMajors, goReleaseVersionErrors, modulePathOf, + vitamincPseudoVersions, } from '../go-release-version.mjs' import { REPO_ROOT } from './lib/repo-root.mjs' import { readWorkflow } from './lib/workflows.mjs' @@ -19,8 +20,26 @@ const MODULE = 'github.com/cipherstash/stack/languages/golang' const MAIN = 'refs/heads/main' const BRANCH = 'refs/heads/ci/golang-release' -const errorsFor = (version, ref = MAIN, modulePath = MODULE) => - goReleaseVersionErrors(version, { modulePath, ref }) +const errorsFor = (version, ref = MAIN, modulePath = MODULE, goMod = '') => + goReleaseVersionErrors(version, { modulePath, ref, goMod }) + +const VCFFI = 'github.com/cipherstash/vitaminc/bindings/go/vcffi' +const VCVALUE = 'github.com/cipherstash/vitaminc/bindings/go/vcvalue' +const PSEUDO = 'v0.0.0-20261026030854-d7de535c4288' + +// A go.mod whose vitaminc modules are at `version`, with the TEMPORARY +// comment the real one carries when that is a pseudo-version. +const goModPinning = (version) => `module ${MODULE} + +go 1.26.10 + +// TEMPORARY: vcffi and vcvalue at vitaminc d7de535c4 (v0.0.0-20261026030854-d7de535c4288). +require ( + ${VCFFI} ${version} + ${VCVALUE} ${version} + github.com/tetratelabs/wazero v1.12.0 +) +` describe('goReleaseVersionErrors', () => { it.each([ @@ -80,6 +99,62 @@ describe('goReleaseVersionErrors', () => { }) }) +describe('a TEMPORARY vitaminc pin', () => { + const temporary = goModPinning(PSEUDO) + const released = goModPinning('v0.9.0') + + it('is every vitaminc module go.mod names at a pseudo-version', () => { + expect(vitamincPseudoVersions(temporary)).toEqual([ + `${VCFFI} ${PSEUDO}`, + `${VCVALUE} ${PSEUDO}`, + ]) + // The three pseudo-version forms, and a replace directive, count too. + expect( + vitamincPseudoVersions( + `require ${VCFFI} v0.8.1-0.20261026030854-d7de535c4288\n` + + `replace ${VCVALUE} v0.9.0 => ${VCVALUE} v0.9.1-alpha.1.0.20261026030854-d7de535c4288\n`, + ), + ).toEqual([ + `${VCFFI} v0.8.1-0.20261026030854-d7de535c4288`, + `${VCVALUE} v0.9.1-alpha.1.0.20261026030854-d7de535c4288`, + ]) + }) + + it('is not a tag, a pseudo-version quoted in a comment, or another module', () => { + expect(vitamincPseudoVersions(released)).toEqual([]) + expect( + vitamincPseudoVersions( + `// ${VCFFI} ${PSEUDO}\nrequire github.com/tetratelabs/wazero ${PSEUDO}\n`, + ), + ).toEqual([]) + expect(vitamincPseudoVersions('')).toEqual([]) + }) + + it('refuses a final version, naming the pins', () => { + const errors = errorsFor('0.1.0', MAIN, MODULE, temporary) + expect(errors).toHaveLength(1) + expect(errors[0]).toContain('pseudo-version') + expect(errors[0]).toContain(`${VCFFI} ${PSEUDO}`) + expect(errors[0]).toContain(`${VCVALUE} ${PSEUDO}`) + expect(errors[0]).toContain('0.1.0-alpha.1') + }) + + it('lets a pre-release through, from main or a branch', () => { + expect(errorsFor('0.1.0-alpha.1', MAIN, MODULE, temporary)).toEqual([]) + expect(errorsFor('0.1.0-rc.1', BRANCH, MODULE, temporary)).toEqual([]) + }) + + it('does not hold a final version once the pin is a tag', () => { + expect(errorsFor('0.1.0', MAIN, MODULE, released)).toEqual([]) + }) + + it('is reported alongside the other reasons, not instead of them', () => { + const errors = errorsFor('0.1.0', BRANCH, MODULE, temporary).join('\n') + expect(errors).toContain('released from main only') + expect(errors).toContain('pseudo-version') + }) +}) + describe('allowedMajors', () => { it('reads the /vN suffix and nothing else', () => { expect(allowedMajors(MODULE)).toEqual([0, 1]) @@ -125,6 +200,21 @@ describe('the script as the workflow runs it', () => { expect(out).toContain('::error::') }) + it("reads the tree's go.mod, so a final version is refused while it carries a TEMPORARY vitaminc pin", () => { + const goMod = readFileSync( + join(REPO_ROOT, 'languages/golang/go.mod'), + 'utf8', + ) + const pins = vitamincPseudoVersions(goMod) + const { status, out } = run('0.1.0', MAIN) + if (pins.length > 0) { + expect(status).toBe(1) + for (const pin of pins) expect(out).toContain(pin) + } else { + expect(status).toBe(0) + } + }) + it('is what release-golang.yml checks the version with, before any build', () => { // The check is only worth something if it runs before the guests are // built and long before the tag is pushed. diff --git a/scripts/go-release-version.mjs b/scripts/go-release-version.mjs index bb5e739c0..780339ff7 100644 --- a/scripts/go-release-version.mjs +++ b/scripts/go-release-version.mjs @@ -9,7 +9,7 @@ // pushed, rather than by the proxy check at the end of the run, after the tag // exists and the version number is spent. // -// Four rules: +// Five rules: // // 1. CANONICAL SEMVER. Go accepts only `vMAJOR.MINOR.PATCH[-PRERELEASE]` with // no leading zeros in a numeric part or a numeric pre-release identifier, @@ -29,6 +29,14 @@ // the dispatcher picks. A pre-release (a version with a `-suffix`) may be // cut from a branch, which is how a partner alpha is built from an // unmerged stack; a final version may not. +// 5. A FINAL VERSION SHIPS NO TEMPORARY VITAMINC PIN. vitaminc's Go bindings +// (`github.com/cipherstash/vitaminc/bindings/go/…`) have no tags, so +// `go.mod` names them by pseudo-version, and from time to time that is an +// unreleased commit taken ahead of a vitaminc release, marked TEMPORARY in +// `go.mod` and promised in the CHANGELOG to be gone before the module is +// tagged. `release-golang.yml` tags whatever `go.mod` names, so the promise +// is held here: a final version is refused while any vitaminc module is +// pinned to a pseudo-version; a pre-release may carry one. import { readFileSync } from 'node:fs' import { join, resolve } from 'node:path' @@ -43,6 +51,9 @@ const NUMERIC = /^(0|[1-9][0-9]*)$/ const PSEUDO_VERSION = /^v[0-9]+\.(0\.0-|\d+\.\d+-([^+]*\.)?0\.)\d{14}-[A-Za-z0-9]+$/ const IDENTIFIER = /^[0-9A-Za-z-]+$/ +// A module of vitaminc's Go bindings and the version it is required or +// replaced at, in a go.mod stripped of its comments. +const VITAMINC_MODULE = /github\.com\/cipherstash\/vitaminc(?:\/\S+)?\s+(v\S+)/g /** The `module` line of a go.mod. */ export function modulePathOf(goMod) { @@ -51,6 +62,20 @@ export function modulePathOf(goMod) { return match[1] } +/** + * Every `github.com/cipherstash/vitaminc/…` module a go.mod names at a + * pseudo-version, as `path version` strings. Comments are dropped first, so + * the TEMPORARY note explaining a pin does not count as one. + */ +export function vitamincPseudoVersions(goMod) { + const code = goMod.replace(/\/\/[^\n]*/g, '') + const pins = [] + for (const match of code.matchAll(VITAMINC_MODULE)) { + if (PSEUDO_VERSION.test(match[1])) pins.push(match[0].trim()) + } + return pins +} + /** The major version a module path may publish: `/vN` gives N, otherwise 0 or 1. */ export function allowedMajors(modulePath) { const suffix = modulePath.match(/\/v([2-9]|[1-9][0-9]+)$/) @@ -59,9 +84,12 @@ export function allowedMajors(modulePath) { /** * Every reason `version` may not be released from `ref` for the module at - * `modulePath`. Empty when it may. + * `modulePath`, whose go.mod text is `goMod`. Empty when it may. */ -export function goReleaseVersionErrors(version, { modulePath, ref }) { +export function goReleaseVersionErrors( + version, + { modulePath, ref, goMod = '' }, +) { const errors = [] if (version.startsWith('v')) { @@ -125,6 +153,13 @@ export function goReleaseVersionErrors(version, { modulePath, ref }) { ) } + const pins = vitamincPseudoVersions(goMod) + if (pre === null && pins.length > 0) { + errors.push( + `a final version cannot ship vitaminc's Go bindings at a pseudo-version (go.mod names ${pins.join(', ')}); pin a vitaminc tag or a commit on vitaminc main that a release carries first, or give a pre-release version such as ${core}-alpha.1`, + ) + } + return errors } @@ -134,8 +169,9 @@ function main() { console.error('usage: go-release-version.mjs ') process.exit(2) } - const modulePath = modulePathOf(readFileSync(GO_MOD, 'utf8')) - const errors = goReleaseVersionErrors(version, { modulePath, ref }) + const goMod = readFileSync(GO_MOD, 'utf8') + const modulePath = modulePathOf(goMod) + const errors = goReleaseVersionErrors(version, { modulePath, ref, goMod }) for (const error of errors) console.log(`::error::${error}`) if (errors.length > 0) process.exit(1) console.log(`${modulePath} v${version} may be released from ${ref}`)