Repository navigation
feat(cli): load external projector modules in local mode - #1431
Conversation
Maple review🟢 Confidence 9/10 · safe to merge Adds a repeatable
What was checked
Observability coverage: 1 of 1 changes observable
|
Maple review🟢 Confidence 9/10 · safe to merge Moves module validation and store preparation into the detached child and forwards
Production impactProduction traffic of the changed files (last 7 days)
What was checked
Observability coverage: 1 of 1 changes observable
|
|
I'm intrigued what you use this for @robbiemu |
oh, I’ve just got a self-hosted GitLab setup feeding events into Maple, with custom projectors that turn those into updates for Matrix rooms and threads. I’d been carrying that stuff in my fork, and since my earlier PR got merged I haven't had time to revisit using main instead of my fork until today. I realized I couldn't stop using a custom branch without a loader, and a generic loader obviously is best. this lets me keep them in a separate module and just plug them in at startup. so yeah I'm basically trying to stop maintaining a whole fork for what is really just configuration. 😅 |
Resolves conflicts with --in-memory (MapleTechLabs#1764), the self-hosted ingest key and access token (MapleTechLabs#1725) and the native probe fan-out in CI. Review follow-ups: - The moved background branch passes the defaulted --on-dirty-store and --checkpoint-interval values and the auth summary line. - startServer takes the loaded ProjectorRegistry in its options instead of loading module paths itself, so modules are loaded only under the CLI's PID claim. - A ProjectorModuleError is a deterministic start refusal (exit 78), so supervisors and container restart policies stop retrying a broken module. - The module loader uses Effect.forEach and Effect.ensuring instead of a for loop, a mutable flag and try/finally; duplicate paths are rejected before any module is imported. - The native probe decodes its JSON responses, fixing the CLI typecheck.
CI runs it directly with bun, like the other native probes.
Maple Local already supports registering projectors in its eventing runtime, but its standalone server has no way to load external implementations. Deployments currently need a source patch to populate that registry. This adds repeatable
maple start --projector-module /path/to/projectors.mjsflags so domain-specific projectors can ship separately.The module's default export implements a small versioned contract:
{ apiVersion: 1, register(registerProjector) }. Registration can be synchronous or asynchronous. The serving process builds a fresh registry under its PID claim before any reset, checkpoint reconciliation, migration abandonment, dirty-store recovery, or chDB open. It reuses that registry in the managed eventing runtime and compiles saved projections before accepting traffic. Background parents hand their PID claim, canonical module paths and reset option to the child; only that child validates modules and prepares the store. Invalid modules, duplicate registrations, and saved projections with missing implementations fail startup.Modules are explicitly selected trusted local code, with the server's privileges. This does not add package discovery, URL loading, hot reload, or domain-specific projectors. Runtime dependencies should be bundled into ESM
.mjsfiles. The new documentation covers the contract, checkpoint/restart requirements, and trust boundary.Validation with Bun 1.4.0:
src/core/release-signature.test.ts:202, which is unchanged from the base commitf73ef32c1.The native probe is also added to the existing Linux native CI job. No existing installation or data migration is part of this change.
Summary by CodeRabbit
--projector-moduleoptions, including in background starts.