Repository navigation
feat(skills): Agent Skills - #87
Merged
Merged
Conversation
First of five slices splitting the Agent Skills feature for review. This one
adds the layer with no I/O in it at all: the types, the validation, and the
projection from a resolved AI Config to the skills it references.
- `skill_refs(config)` projects a config's `skills` array into
`list[SkillReference]`. Pure — no client, no store, no network, no telemetry.
- `Skill` and `SkillReference` are frozen dataclasses, exported from the package
root. `Skill.content` is `bytes` — the verified verbatim bytes LaunchDarkly
delivered, exactly what was hashed. Skills are opaque byte buffers by
construction: the SDK never parses, decodes, or interprets skill content
anywhere. `content_hash` is the sha256 (lowercase hex) over those bytes, and
the optional display metadata comes from LaunchDarkly, never from the content.
- `parse_ai_config` now validates the optional `skills` array and fails closed
on a malformed one. Key grammar, length bound, and the version predicate live
in `types_validation.py` as one canonical rejection reason, so every layer
added on top rejects a key for the same stated reason.
Note one behaviour change for existing users: `parse_ai_config` fails closed on
a `skills` value that is not a list of `{key, version}` objects, where before
any value parsed and was ignored. A variation carrying its own differently
shaped `skills` field must rename it before upgrading.
The two layers that follow — retrieval through a store seam, and
materialization onto disk — are separate slices. `agents.md` names all three
and the one-way dependencies between them.
Testing: `uv run pytest` → 1048 passed. `ruff check`,
`ruff format --check`, and `mypy packages/*/src` all clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Second of five slices. Adds the layer that turns a reference into content: an
injectable store seam, integrity verification of everything it serves, and a
body-free telemetry seam for the failures.
- `get_skill(key, *, version=None)` returns one verified skill, or None.
- `get_skills(refs)` is the batch form, accepting references and bare keys.
- `all_skills()` returns every verified skill the store holds, one per key.
- `SkillStore` is the structural interface content arrives through —
`get_object(kind, key, version=None)`, `all_objects(kind)`, and an optional
`add_listener(kind, fn)` — configured with
`init_client(options={"skillStore": store})`. `InMemorySkillStore` ships for
local development and testing. A delivery transport drops in behind the same
seam with no public API change.
Store data is untrusted. Key and version are revalidated, size is bounded, and
the sha256 of the verbatim bytes must match the delivered `contentHash`;
anything that does not verify is withheld and treated as missing, so no
unverified content is ever returned. The wire object delivers content as a JSON
string; the UTF-8 encode happens exactly once, inside verification, and the
`Skill` handed to user code carries the verified verbatim bytes
(`Skill.content: bytes`) — the exact byte sequence that was hashed, never a
re-derived value. Content carrying an unpaired surrogate has no UTF-8 encoding
at all and is withheld too — `str.encode` is called strictly, never with an
error handler that would fabricate bytes a hash comparison could then accept.
`verified_bytes` also accepts already-bytes content, hashing it directly, for
the pre-write re-verification pass a later slice adds.
Integrity failures are reported through a private telemetry seam carrying
hashes and byte counts only, never the skill body. The two properties copied
off the wire, `skill_key` and `expected_hash`, are shape-checked and replaced
when malformed, so a hostile store cannot use either one to publish the body
through a signal that is otherwise body-free. The default emitter is a no-op:
nothing leaves the process in this release, and the three signal names are an
allowlist maintained in one section of one module.
Version is part of the lookup identity rather than a filter applied to the
answer. A delivery payload carries the newest version of every skill plus every
version any variation currently pins, so two versions of one key coexist
routinely; a seam keyed by key alone would answer a pinned reference with the
newest object and then reject it, turning the primary use case into a missing
skill. `InMemorySkillStore` holds several versions of a key, `get_object` takes
the wanted version, and `version=None` means "the newest you hold". The
equality check afterwards is kept as a defense — the store is untrusted, so an
answer that is not the version asked for is withheld.
`all_objects` returns one entry per key-and-version under keys that are opaque
to this SDK; identity is read off each object's own fields. `newest_by_key` is
the single place that collapses the result to one object per key.
A run that withheld anything now logs a count at WARN. Every individual
withholding already records a signal and an error line, but a caller reading
logs at WARN saw neither, and a payload where nothing verifies otherwise
returns an empty result indistinguishable from "this project has no skills".
`SKILL_OBJECT_KIND` is deliberately **not** exported from the package root. It
is the string this SDK hands a store, and an adapter maps whatever the transport
underneath calls a skill onto it; publishing it would advertise an SDK-side seam
value as the wire contract. An adapter that needs to agree with it reaches it
through `skills_core`. `MAX_SKILL_CONTENT_BYTES` stays internal for the
adjacent reason.
Note one behaviour change: `shutdown()` clears the configured skill store along
with the client. `init_client` applies `skillStore` on every successful call,
even the idempotent ones, which is what lets a lazily auto-initialized client be
given a store afterwards.
Testing: `uv run pytest` → 1145 passed. `ruff check`,
`ruff format --check`, and `mypy packages/*/src` all clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
An integrity failure now writes a structured, machine-parseable ERROR record on the SDK's own logger, designed to be ingested by a SIEM and alerted on. This is the detection path that works when telemetry is off, and the only one that exists at all in an instance with no telemetry destination — so it is a documented contract rather than a debugging aid. The LD-side counter is left exactly as designed: opt-out respecting, no-op by default, property set unchanged. `reason_code` lives in the log record only. - `ld.skills.integrity_failure` is the stable event name, and it appears in the message text rather than only in `extra`. Severity cannot discriminate — a raising store also logs ERROR from this module — and the stdlib's default formatter drops `extra`, so an `extra`-only record is invisible under a plain `logging.basicConfig()`. - The message is the event name plus compact key-sorted JSON, so the line is greppable, `jq`-able, and byte-identical across LaunchDarkly's AI SDKs for the same input. The same mapping is attached as `extra["ld_skills"]`. - `reason_code` is a closed vocabulary of eight tokens, one per `record_integrity_failure` call site, typed as a `Literal` so a typo at a call site is a type error. - The record spreads the signal's properties rather than rebuilding them, so the two cannot drift on which fields are redacted or omitted. Optional fields are omitted, never nulled. No new untrusted value, and no path. Documented for customers in the README and for contributors in agents.md, including the full vocabulary, so a ninth reason cannot land in one language only.
Third of five slices. Adds `safe_fs.py`, the "write a file under a directory something else may be racing you for" problem solved once. Nothing here knows what a skill is; the materialization layer is its only caller, and it lands next. A path check is only as good as the last path resolution after it. Every `lstat` and containment check validates an inode, but a following `os.replace(tmp, dir / name)` re-resolves `dir` from its name — so anything holding write permission there can move the validated directory aside, leave a symlink in its place, and redirect the write or the unlink somewhere else. Narrowing that window is not a fix; the race is winnable at any width. So the checks hand off to a descriptor and nothing re-resolves a path afterwards. - `open_directory_nofollow` opens with `O_RDONLY | O_DIRECTORY | O_NOFOLLOW` and confirms `S_ISDIR` on the `fstat`, since not every platform defines `O_DIRECTORY`. `open_or_create_directory` adds `os.mkdir` plus an `lstat` on the `FileExistsError` path, because `Path.mkdir(exist_ok=True)` accepts a symlink-to-directory as "already there" and would reopen the hole the caller's check just closed. `pinned_directory` holds either for a block, so a caller states the platform split once and cannot forget the close. - `atomic_write` creates its temp file with `O_CREAT | O_EXCL | O_NOFOLLOW` at that descriptor, `fchmod`s the descriptor rather than a path, writes, fsyncs, renames, and fsyncs the directory so the rename survives a crash. Mode is set explicitly at 0644, never inherited from the umask and never executable. `os.replace` is the single rename call site and `os.rename` must not be substituted for it. - `unlink_file` probes and unlinks descriptor-relative. `unlink` never follows a trailing symlink but it does resolve the directory above it, so the same swap turns a removal into a delete of an arbitrary file. A symlink found where this SDK expects its own file raises `SymlinkRefused` rather than being tidied away — the state on disk is not what the caller believes, and that is the caller's to report. `SUPPORTS_DIR_FD` gates all of it, and the probe deliberately names `os.rename`/`os.stat` rather than the `os.replace`/`os.lstat` this module calls: `os.supports_dir_fd` is populated per underlying syscall, and CPython registers `renameat` under `rename` only and `fstatat` under `stat` only. Probing the names actually called reports "unsupported" on every POSIX platform and silently turns the defense off. Where the family is absent, every operation falls back to the identical full-path sequence. The tests here exercise the module directly, on its own terms. The TOCTOU races these primitives exist to close are proved through the materialization layer, which is what holds a descriptor across a sequence of operations. Testing: `uv run pytest` → 1260 passed, 11 skipped. `ruff check`, `ruff format --check`, and `mypy packages/*/src` all clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Fifth and last of five slices. The adversary. Every filesystem defense the previous two slices introduced now has a test that fails if the defense is removed, plus the materialization telemetry allowlist. - **Path traversal.** A key that escapes the root, a key that is the manifest filename, an over-long key, and a key that resolves outside after `realpath` — each refused before any filesystem call, and the resolved containment check asserted on inode identity rather than on path strings. - **Symlink attacks.** A symlinked skill directory, a symlinked target file, and the `<root>/<key>` directory swapped for a symlink at the exact instant of the rename and of the unlink — the narrowest version of the window the descriptor pinning exists to close, fired from the interception point rather than from implementation internals. - **The no-`*at()` shape.** The path fallback Windows takes for every operation, exercised with the capability probe forced off, so the platform that cannot pin a descriptor is not the untested one. The TOCTOU tests skip off that same flag deliberately: a probe that wrongly reported "unsupported" cannot also silently skip the tests that would have caught it. - **Non-regular files and clobber protection.** A fifo or a directory where `SKILL.md` belongs, and a file at a managed path with no matching manifest entry — reported and left alone, never overwritten and never removed. - **Corrupt manifests.** Unreadable, unparseable, not an object, malformed entries, and a `manifestVersion` this release cannot read: no overwrites, no prunes, an error action naming the manifest, and the manifest itself left as it was found. - **Atomicity.** A crash injected between the write and the rename leaves neither a partial file nor a temp file, and the one recorded rename is proved to have moved `SKILL.md` within the target's own directory — by descriptor identity where the platform has `renameat`, which also rules out the descriptor having been redirected between the check and the rename. - **Telemetry.** The three signal names are asserted as an allowlist rather than a floor: any other name reaching the emitter fails, the two deliberately excluded names are called out by name, no signal carries a filesystem path or the skill body, an emitter that raises never fails the reconcile, and `client.track()` is never reached. Two of these are worth naming, because the obvious test does not reach the guard. The unencodable-content cases pin `contentHash` to the sha256 of the bytes a non-strict encoder would have fabricated, since an arbitrary wrong hash is rejected by the mismatch check first and never exercises the encoder guard. The redaction cases smuggle the body through `contentHash` and through `key`, since a sweep using a well-formed digest under a valid key reaches neither replacement branch. Testing: `uv run pytest` → 1280 passed. `ruff check`, `ruff format --check`, and `mypy packages/*/src` all clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Fourth of five slices. Adds `write_skills`, which writes
`<root>/<key>/SKILL.md` and reconciles against a manifest recording what the
SDK owns, so it overwrites or removes only files it wrote — a file you placed
yourself is reported and left untouched.
report = await write_skills(refs, ".claude/skills")
`skills` accepts `Skill` values, references, bare keys, or the literal `"*"` for
everything the store holds. Every outcome is visible in the returned
`ReconcileReport`: one `ReconcileAction` per skill, carrying `written`,
`updated`, `skipped_current`, `removed` or `error`, plus `.ok` and `.errors`.
A failure belonging to the run rather than to one skill — an unreadable
manifest, a retrieval that failed before any key was known — carries the empty
string as its key.
Writes are atomic, at mode 0644, and every destructive step runs against a
descriptor pinned to a directory that was already checked, so a path swapped
after the check cannot redirect a write or an unlink out of the managed root.
Where the platform has no `*at()` family the identical sequence runs against
full paths.
The defenses, all of them deliberate and all of them tested:
- The key is re-validated here regardless of upstream validation, before any
filesystem call, because a key becomes a directory name. The data model
allows 256 characters and `NAME_MAX` is 255 bytes, so an over-long key is
refused too.
- Never write or unlink through a symlink, on the write path or the prune path.
- Destruction only on manifest-listed paths whose key matches.
- A corrupt manifest fails closed: no overwrites and no prunes, brand-new paths
may still be written, an error action names the manifest, and the manifest is
not rewritten.
- An incomplete retrieval suppresses pruning, so a transport outage cannot read
as "everything was revoked".
- Content is re-verified immediately before the write, because a `Skill` can
also be constructed directly by a caller.
Pruning removes formerly-managed skills that are no longer referenced, which is
how revocation takes effect. `timeout` bounds retrieval, the writes and the
pruning; only the final manifest rewrite runs past it, so files already written
are never orphaned.
`write_skills` performs synchronous filesystem I/O and does not yield — it is
`async` for signature parity with the other accessors. Reconcile one root at a
time: a run is atomic against the rest of the loop today, so wrapping it to run
concurrently makes two runs against one root race on the manifest.
The `"*"` form collapses to one object per key at its newest version, since
`<root>/<key>/SKILL.md` is a single path and writing it twice in one run is a
bug rather than a policy, and it reports a withholding count at WARN for the
same reason the accessors do.
The security abuse matrix — path traversal, symlink attacks, clobber
protection, corrupt manifests, atomicity under an injected crash, and the
materialization telemetry allowlist — is the next slice. The guards it exercises
are all here; what lands next is the adversary that proves each one fails
without them.
Testing: `uv run pytest` → 1216 passed. `ruff check`,
`ruff format --check`, and `mypy packages/*/src` all clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
``test_key_at_the_data_model_bound_is_reported_not_raised`` read ``dst_dir_id`` directly, but that field is only populated when ``os.replace`` is called with ``dir_fd`` kwargs. On the path fallback (``SUPPORTS_DIR_FD`` false — the shape Windows takes) it stays ``None``, so the assertion failed even though the valid skill had been renamed correctly into its own directory. ``_assert_atomic_rename_of`` already branches on both call shapes and asserts the same containment property, plus the single-rename count the list comparison implied. Use it. Verified by forcing the probe off for a whole session: this was the only test in the module that broke under the no-``*at()`` shape, and the helper-based check passes under both. Reported by Cursor Bugbot on #54. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…esystem can hold Two gaps from the Agent Skills security design review, both in `skills_fs.py`: AV-3 (partial reconciles are unrecoverable) and DV-2 (the key grammar admits Windows device names). **AV-3 — adopt a file whose bytes already are the resolved content.** `_write_all` reconciles every skill and only then rewrites the manifest, once, last. A process killed in that window leaves a skill file at a managed path with no manifest entry — exactly the condition `_write_one` treats as an unmanaged-file collision, so the skill was wedged permanently: every later reconcile took the same refusal branch. Boot-time execution under a ten-second budget makes the crash window realistic. `_write_one` now reads and hashes first and decides from the bytes. Content byte-identical to what LaunchDarkly resolved is adopted — manifest entry recorded, reported `skipped_current` — and anything else falls through to the same refusal as before. This cannot weaken the clobber guarantee: differing unmanaged bytes are never overwritten, and the existing clobber tests pass unchanged. Three details carry the safety: - A read that fails is a refusal, never an overwrite, with a message distinguishable from the byte-mismatch refusal — it is the comparison that would otherwise authorize the write, and a file that could not be read has not been shown to be ours. - The read stays on `_read_regular_file`. Adoption widens it to genuinely foreign files, so its refusal of FIFOs and other non-regular files is now load-bearing rather than defensive. It gains a `max_bytes` bound of `len(content) + 1` — enough to prove inequality for anything longer, and what keeps a foreign file of arbitrary size out of memory. A bound of exactly `len(content)` would adopt every file that merely begins with the resolved content. - `skipped_current` is reused rather than adding an `adopted` action kind, so `ReconcileActionKind` — public, and owned by an approved PR — does not change. Its documented meaning already fits. Adoption also makes the file prunable later. That is correct rather than a weakening: only byte-identical LaunchDarkly content is ever adopted, so a later prune removes content LaunchDarkly delivered anyway — what would have happened had the crash not occurred. The review also floats a write-intent journal. Assessed as over-engineered; not built. **AV-3, secondary — sweep orphaned temp files.** `atomic_write` unlinks its temp file on any exception but not after a `SIGKILL`, and `_prune` walks manifest entries, which an orphan never has, so nothing would ever notice one. The second-order effect is worse than the disk: `_prune_one`'s `rmdir` only succeeds on an empty directory, so a single orphan pins a skill's directory forever. The sweep runs on both the write and the prune path, and is the one place this SDK removes a file the manifest does not list, so it is bounded on every axis: inside `<root>/<key>/` only, for a key that passes `_key_rejection_reason`; only names `safe_fs` itself recognizes, via a new `is_temp_name` beside the naming code rather than a copy of the format string that could drift from it; only regular files, with the type read off the descriptor; unlinked through the pinned descriptor. It never raises and never aborts a run. **DV-2 — reject the 22 Windows reserved device names.** `con`, `prn`, `aux`, `nul`, `com1`–`com9`, `lpt1`–`lpt9` are all valid skill keys and none can be a directory name on Windows. Rejected in `_key_rejection_reason`, which the write and prune paths already share, and *not* in the key grammar: `parse_ai_config` fails closed, so a grammar-level rejection would invalidate an entire AI Config for a Linux customer over a Windows-only constraint, and would silently shrink `skill_refs` — which is what authorizes a prune, turning "fails to write on Windows" into "gets deleted on Linux". The 255-byte component bound is in this layer for the same reason. Unconditional, not platform-gated: a root written from a Linux container is routinely read from a Windows host, and neither repository has a Windows CI runner, so a gated branch would be untestable — the condition that produced the gap. No suffix stripping and no case folding: the grammar admits no `.` and no `$`, so `con.txt` and `CONIN$` are unreachable, and keys are lowercase-only. `com0` and `lpt0` are not reserved and are not included. The trade is real and belongs in the release notes: a customer who legitimately names a skill `aux` now gets a reported `error` action on Linux where it previously worked. Neither gap emits the integrity-failure log record. A key rejection and a clobber refusal are `ReconcileAction` errors, not integrity failures. Tests cover crash-mid-reconcile recovery end to end (adopted, reported, recorded, and the next reconcile an ordinary no-op), byte-differing content still refused untouched, an unmanaged FIFO refused without hanging, a read failure refusing rather than overwriting, the `len + 1` off-by-one, the sweep and the `rmdir` it unblocks, lookalike temp names and a symlink wearing one left alone, all 22 reserved names through both destructive paths, and — the point of the layer choice — each reserved name still valid to `is_valid_skill_key`, `parse_ai_config`, and `skill_refs`. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…failure ``get_skill`` returns ``None`` for four unrelated outcomes: no such skill, the store raised, the requested version is not the one held, and content that failed hash verification. A caller cannot fail closed on suspected tampering while tolerating a merely-absent skill, so no automated customer-side response is possible — finding LA-2 of the Agent Skills security design review. The information already existed internally, as prose in ``Resolution.error``. This gives it a token: ``Resolution`` grows a typed ``reason``, set explicitly at every construction site and declared without a default so a sixth outcome added later has to choose which public token it maps to. ``get_skill_result`` maps that straight through to a frozen ``SkillOutcome`` (``skill``, ``reason``, ``detail``). Deriving the public reason by matching the error string is the fragility LA-2 is about, so the mapping is readable in one table. ``get_skill`` is untouched — its ``None``-for-every-failure contract is documented in its docstring and in the README, and a test now pins that all four failures still collapse to ``None`` and still never raise. Nothing new is emitted: Gap 1's integrity record already fired inside verification before ``resolve_from_store`` returned, and a test asserts one failed retrieval still produces exactly one record and one signal. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… remainder The four items left open in the response to the Agent Skills security design review. Three are documentation, one is tests; no behavior changes, and the code halves of rows 9 and 2 are deliberately untouched. **Privilege separation** (row 9 docs half, and the agreed counter-proposal for row 26). The recommended deployment runs the reconcile as a different identity than the agent, which is the whole reason the ``0644``/``0755`` modes deny anything: the agent reads its instructions and cannot rewrite them, or the manifest. That is the mitigation for AZ-1, a prompt-injected agent editing its own skills. Write access to the manifest is the worse half — it is what tells the *next* reconcile which paths the SDK may delete — which is why ``_prune`` re-validates every entry from scratch rather than trusting it. The README documents the pattern and hands the operator the check to run, because the SDK cannot run it: it knows only its own identity, which trivially has write access, having just written there. So ``ReconcileReport`` grows no writability field — the review asked for one and we declined, since any check the SDK could make would answer a different question than the one asked and manufacture false confidence exactly where caution is wanted. ``agents.md`` records that reasoning so the field is not added later by someone reading its absence as an oversight. **Three hostile-manifest prune tests** (row 12 remainder): a well-formed manifest listing ``/etc/passwd``, ``../../../etc/passwd``, and a path under a parent that has since become a symlink. ``_prune`` already refuses all three, so these turn asserted into verified. Two things make them worth more than their line count. They are deliberately *well-formed* — the corrupt-manifest suite above them proves nothing here, because a corrupt manifest suppresses every destructive action wholesale, whereas these manifests give the implementation everything it needs to prune. And "deleted nothing" is asserted through an unlink spy rather than by checking that ``/etc/passwd`` still exists: the test process cannot delete that file anyway, so the obvious assertion would pass against an implementation with no path check at all. **One sentence on** ``"*"`` (row 16 remainder). It materializes the whole project library, so every skill's ``description`` enters the agent's context — including skills no AI Config references and skills belonging to other teams. **The Windows platform bound is now explicit** (row 2 residual), in ``safe_fs.py``, ``agents.md`` and the README. Reparse-point checks (``GetFileAttributesW`` / ``FILE_FLAG_OPEN_REPARSE_POINT``) are not implemented, by decision: Windows is not a supported or tested platform for this release, neither repository has a Windows CI runner so the checks would ship unverified, and the TypeScript SDK could not match them in any case because Node exposes no ``*at()`` family on *any* platform. Implementing them in Python alone would break cross-language parity and trade a documented bound for an unverified one. Two consequences are recorded rather than left to be rediscovered: on Windows, write permission on the managed root is the only boundary, which is what makes privilege separation the mitigation and not merely advice; and this retroactively lowers the priority of the row 25 reserved-device-name work, noted where that code lives. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…does not close Three expected-failure tests (TestRootSwapRaces) for SEC-8985 row 2. The SDR response says every destructive operation runs relative to a descriptor held for the duration of the reconcile. The code pins <root>/<key> per operation and never holds the root: _resolve_root validates it once and returns a path, and each write and prune re-opens <root>/<key> by path with O_NOFOLLOW, which guards only the final component. A root swapped for a symlink after validation redirects the open, and every descriptor-relative step behind it, into the attacker's directory. Precondition is write permission on the root's parent, which the README checklist does not mention. The tests state the contract (nothing lands outside the root, no outside file is overwritten, no outside file is removed) and are marked xfail(strict=True, raises=AssertionError) so the suite stays green, the gap is recorded next to the other race tests, and the fix cannot land without removing the marker. Run with --runxfail to see the three escapes. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> (cherry picked from commit c22688b)
The expected-failure marker made the suite green while the contract was violated. A red run is the demonstration: the tests assert the contract, the code does not meet it, and they go green when the root is pinned for the reconcile, with nothing to remove. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> (cherry picked from commit 37ddb6a)
SEC-8985 row 2. The SDR response says every destructive operation runs relative to a descriptor held for the duration of the reconcile. It did not. `_resolve_root` validated the root and returned a plain `Path`; nothing held it open. Each write and each prune then opened `<root>/<key>` by path with `O_NOFOLLOW|O_DIRECTORY` and pinned that — and `O_NOFOLLOW` guards only the final component, so the root and every ancestor were re-resolved on every such open. A root renamed aside and replaced with a symlink after validation redirected the open, and with it every descriptor-relative step behind it, into the attacker's directory; on the create path `os.mkdir(<root>/<key>)` followed the link as well, and `mkdir` follows a symlink at its parent. The manifest write was the one operation that pinned the root, and it ran last, by which time the skill files were already outside it. The attacker precondition is write access to the root's *parent* — `.claude` for a root of `.claude/skills` — which the README checklist did not mention. `write_skills` now opens the root once, immediately after `_resolve_root`, with `O_RDONLY|O_DIRECTORY|O_NOFOLLOW`, confirms `S_ISDIR` on the descriptor, and holds it until the call returns. The descriptor is threaded through `_write_all`, `_prune`, `_rewrite_manifest`, the orphan sweep and the per-skill helpers, and every destructive step names a bare component against it: `os.mkdir(key, dir_fd=root_fd)`, `os.open(key, ..., dir_fd=root_fd)` for the skill directory, `os.rmdir(key, dir_fd=root_fd)`, and `atomic_write(..., dir_fd=root_fd)` for the manifest. A root swapped in the one interval left — after validation, before the open — fails `O_NOFOLLOW` and is reported as a run-level error with nothing touched, rather than as the `ValueError` an unusable root raises. `safe_fs`'s three openers take a `dir_fd` for the *parent* rather than growing a parallel API, and `SUPPORTS_DIR_FD` now probes `os.mkdir` and `os.rmdir` alongside the four it already named. Where the `*at()` family is absent the per-component `lstat` floor runs exactly as before: the root open returns `None` there, and every call site keeps its full-path branch. `_unsafe_path_reason` stays and still runs, but it is documented as defense in depth rather than the boundary — every check in it inspects a path, so each is a check-then-use against anything that can rename a component of that path. Security's three tests fired only when the intercepted `os.mkdir`/`os.open` was handed the absolute `<root>/<key>`, which the fix stops passing — they would have gone green while asserting nothing. The trigger now matches the bare key as well, so the swap fires in both worlds and the tests fail before the fix and pass after it. Two tests added: the root swapped before the pin is refused at the run level, and an audit that across a full reconcile no destructive call names an absolute path. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The privilege-separation checklist denied the agent identity the managed root, the per-skill directories, the files and the manifest, and said nothing about the root's parent. That was the precondition for the SEC-8985 row 2 root swap: renaming any ancestor is what lets the root be replaced with a symlink, and in the documented `<app>/.claude/skills` layout the parent is `.claude`, which an agent identity is otherwise likely to own outright. The checklist and its shell snippet now walk every ancestor up to `/`. Write access to one of them is a strictly larger capability than racing the reconcile — no timing is involved, it persists until someone notices, and descriptor pinning inside `write_skills` cannot address it, because the substituted tree is what the agent reads rather than what the SDK wrote. The platform-bound paragraph claimed a descriptor held for the whole reconcile before that was true of the root; it now describes what is actually held and for how long, and names the root's ancestors alongside the root as the security boundary on every platform. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
write_skills is a one-shot reconcile, so a revocation takes effect at the next process restart. watch_skills runs that reconcile now and again whenever the configured store reports a change, so a revoked skill's files leave the disk within a debounce interval of the store learning about it. It is wired to the SkillStore interface, not to any one transport: it needs a store that implements add_listener and nothing more, and refuses loudly when the store does not, since a watcher that silently never fires looks exactly like one whose skills never changed. Above the interface, remove_listener joins add_listener as the optional second half of change notification, on the SkillStore contract and on InMemorySkillStore. SkillWatcher.close needs it to detach; without it a store held every watcher ever created for the rest of its life. The watcher probes for it, so a store without it keeps working. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…network The half of the delivery transport that has no I/O: identifying a skill object on the wire by kind inline-resource plus category skill, translating it into the raw object shape the SkillStore interface defines, holding it by (key, objectVersion), and applying a payload's events as one commit at payload-transferred. The store that puts a connection underneath this follows separately, so the three decisions that matter most can be reviewed on their own: - objectVersion is the skill's version; version is the payload's. The translation happens in one place and TestVersionTranslation asserts it in both directions, because confusing them fails silently. - Changes commit at payload-transferred, not per object. A half-applied full transfer would briefly empty the store, which with pruning on is the difference between a reconcile and deleting a customer's files. - A hashless object is held, not dropped, so verification withholds it with a reason code rather than the transport reporting it absent. Flag and segment objects share the connection and are skipped and counted, not rejected. Nothing here is exported yet; the store exports it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
``InMemorySkillStore.get_object`` consulted the version-less entry on any pinned miss, while the unpinned path consulted it only when nothing well-formed was filed under the key. A key holding both well-formed versions and one malformed object therefore answered a pin for an undelivered version with the malformed object, and verification recorded an integrity failure — an alert pointed at a skill whose integrity was never in question — where the honest answer is that the version is not held. Both paths now follow the one rule: the version-less entry answers only when nothing well-formed is filed under the key, which is the case it exists for. A malformed object that is all the store holds still reaches verification and is still withheld with a signal, so tampering cannot read as a skill that was never delivered. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two untrusted-store answers were read as data rather than as failures.
``list_raw_objects`` collapsed a non-mapping listing to ``{}`` with no
error, so a store that served nothing usable was indistinguishable from
one holding no skills. ``resolve_from_store`` read identity off the
object without checking it against the key that was asked for, so an
answer served under a different key came back under the caller's key
while carrying its own.
Both are now withheld and reported, alongside the version check that
already guarded the same way.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…oncile watch_skills awaited write_skills and only then constructed SkillWatcher, which is where the store listener attaches. The reconcile snapshots the store as its first step and then spends the rest of its time on the filesystem, so every write, fsync, prune and manifest rewrite in that first pass ran with nothing listening. A change delivered in that window was never seen, and since nothing re-reconciles on a timer, a revocation that landed there waited for the next unrelated change — on a quiet root, the next restart. Exactly the gap watch_skills exists to close. The watcher now attaches its listener before the initial reconcile and starts its worker after. notify only sets an event, so a change arriving mid-reconcile is recorded and picked up by the worker's first pass, while holding the thread back keeps write_skills's one-root-one-reconcile contract: the worker cannot race the caller's own reconcile over the same manifest. A reconcile that raises detaches the listener on the way out, since the caller is handed an exception rather than a watcher to close. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Brings in the withheld-answer fix for broken store answers, which the materialization path above this branch has regression tests for. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Brings in the withheld-answer fix the reconcile regression tests below depend on. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Covers the reconcile side of the withheld-answer fix: a listing that is not a mapping leaves every managed file alone rather than reading as a full revocation, and an answer served under a different key writes nothing, is reported against the key that was asked for, and does not reach that other key's file. Each one previously deleted a file and reported a clean run. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The protocol reader took payloads[0]'s intentCode and applied it to the skill object set, which is what the delivery protocol requires — one payload per credential, read the first intent, tolerate the rest — but it left the assumption behind that rule undocumented and unguarded. If the one-payload guarantee ever widens, an xfer-full for another payload would start an empty pending set and the next payload-transferred would publish it: every skill reported revoked, and with pruning on, a customer's files deleted. The first payload is still the payload that is read. What is new is that the reader now knows which payload skills actually arrive on — learnt from the intent's id, or from the (p:<id>:<version>) selector, since no object or transfer event carries a payload id of its own — and declines to apply a transfer of any other, holding last known good, warning once, and counting it in diagnostics.payloads_ignored. An intent describing more than one payload warns once on its own, because that is the one case the comparison cannot catch: another payload's transfer arriving before any skill has been seen has nothing to be compared against. Behaviour under one-payload delivery is unchanged, and a full transfer of the skill payload still empties it — every skill deleted is a real state the guard must not mask. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The SDK-facing FDv2 channel now delivers skills the way streamer #4681 and gonfalon #70638 spell them: object kinds are open strings, the agent-skill payload is classified `generic`, and every generic object carries only `key`, `kind`, `version` and `object`, exactly like a flag. A skill arrives under kind `skill` with its own version folded into the key as `<key>:<version>`. There is no `category` field and no `objectVersion` field; both came from an earlier streamer draft that never shipped. Identification is now the kind alone. The wire key is split in one place, `_split_wire_key`, and both the put and the delete translation go through it. A key that will not split cleanly is held rather than dropped — version-less, or with the offending text as its version — so verification withholds it with `invalid_version` under a key the caller recognises; only a key with nothing before the delimiter is dropped, since there is no identity to hold it under. `SDK_DATA_MODEL_VERSION` goes with it: the connection's `mv` parameter only accepts flag model versions, and generic payloads ignore it. The transport stops sending it in the following change. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
FDv2SkillStore puts LaunchDarkly's SDK-facing FDv2 channel underneath the protocol layer: GET /sdk/poll and GET /sdk/stream, authenticated with the environment's server-side SDK key, streaming by default. It carries basis across requests, sends If-None-Match and treats 304 as a current answer, retries with capped jittered backoff, honours Retry-After only up to max_backoff, gives up after a bounded run of consecutive failures where a committed payload resets the count, and keeps serving last known good through every failure. A mobile key or client-side environment ID is refused in the constructor. Standard library only. close interrupts the socket rather than only setting a flag, because the delivery thread lives in a read no flag can reach; without that every shutdown of a healthy stream waited out the full join timeout. The no-store message now names FDv2SkillStore first, and watch_skills points at it as the store with a delivery transport. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
_Requester.stream wrapped only the connect as recoverable, so a read timeout, reset or truncated chunk in the body reached the delivery loop as whatever the socket raised. The loop read that as a bug and gave up: delivery stopped for the process lifetime, taking updates and revocations with it, the first time a socket died. read_timeout exists to bound a stream that has gone quiet so the loop can reconnect, and tripping it did the opposite. The body now carries the same promise the connect already did. Wrapping the line source rather than the whole read keeps protocol reader errors out of it: those are raised from the consumer's loop body, where they still surface as the bugs they are. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The client-side cap was 64 KiB, close enough to the platform's own limit that any backend increase would force an SDK release. Raise it to 10 MiB so the guard stays a backstop against absurd input rather than a second enforcement of a bound this side does not own, and the real limit can grow without the SDKs moving. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…n promptly Three faults in the delivery loop, all of which left the store reporting itself healthy while doing less than it claimed. **An up-to-date stream tripped the failure cap.** The consecutive-failure count reset only at a commit, and an environment whose skills are not changing answers every reconnect with `intentCode: "none"` and transfers nothing. A stream only ever ends by being dropped, so each recycle of a perfectly healthy idle connection counted as a failure — announced with a `goodbye` or not — and `max_consecutive_failures + 1` of them stopped delivery for the process lifetime, revocations included. The reset on commit covered only the case where content had changed, which is the case that was easy to test and not the case that runs in production. `_TransferOutcome` now reports `up_to_date`, and a complete answer that transfers nothing breaks the row of failures exactly as a commit does. An intent this module does not recognise is still not an answer. **`close` could not interrupt a poll.** The interrupt reached the streaming connection only, so polling parked in its request with nothing to reach and `close` returned when its join timed out — on a 300s-class request, long after the process meant to exit. `_Requester` now tracks the response of a poll in flight and offers `interrupt`, which `close` calls alongside the stream's own. A request still inside its connect has no response to reach; that one is bounded by `read_timeout`, and `start` no longer leaves the store inert when a join times out around it. An interrupt we asked for is no longer recorded as a delivery failure. **`close` left a waiter parked.** `wait_for_skills` waited on the first payload alone, so a shutdown racing a waiter added the waiter's whole timeout to it. Delivery ending is now its own event: a waiter is released by a payload, a give-up or a close, and reports whether a payload actually arrived rather than merely that it was let go. That also settles what `_give_up` had been quietly asserting — it set the first-payload flag to unblock waiters, which made `wait_for_skills` answer `True` for a store holding nothing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A stream only ever ends by being dropped, and LaunchDarkly — and any proxy in between — recycles a long-lived one. Every reconnect therefore logged "Skill delivery failed" at WARNING, for as long as the process ran. Until the previous commit that noise was bounded, because an idle stream gave up after eleven recycles and went quiet; now that delivery correctly survives them, it would run forever and describe a healthy store as failing. A connection that got a complete answer before it ended — a committed payload, or an up-to-date intent — delivered everything it was asked for, so its reconnect is now DEBUG and says so. A connection that ended without answering is the case the warning exists for and still gets it: a connect that never landed, or a transfer that died part-way through. Filling a customer's logs with a fault they do not have is not merely untidy; it teaches them that the level which means something can be ignored. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Stacked on the protocol-layer PR (`xie/skills-fdv2-protocol`). Third of three PRs split out of #69, and the one that makes the feature real: the network underneath the protocol reader, exported as `FDv2SkillStore`. ## Why this shape Skill content arrives over `GET /sdk/poll` and `GET /sdk/stream`, authenticated with the environment's server-side SDK key. These are the SDK-facing endpoints the base SDK's FDv2 data source uses, and the channel that payload signing will eventually cover. No private route is involved, and no credential other than the environment's own SDK key ships to a customer host. Standard library only, so the content path adds no dependency to a package whose sole runtime dependency is `opentelemetry-api`. ## What's here **`FDv2SkillStore`.** Authenticates, streams (default) or polls, carries `basis` across requests, sends `If-None-Match` and treats 304 as a first-class current answer, and serves `get_object` / `all_objects` / `add_listener` / `remove_listener` from what the protocol reader has committed. Capped jittered backoff; `Retry-After` honoured but clamped to `max_backoff` and rejected when non-finite; bounded consecutive-failure retries, where a committed payload resets the count. One network timeout, `read_timeout`, whose default follows the mode: 10s for a whole poll, 300s between reads on a stream. A mobile key or client-side environment ID raises from the constructor. Last known good survives every failure; `diagnostics` and `failed` report the degradation. **`close` interrupts the socket.** The delivery thread parks in a read no flag can reach, and closing a urllib response from another thread does not unblock CPython's buffered reader, so `_interrupt_read` shuts the socket down underneath it. Without that every shutdown of a *healthy* stream blocked for the full join timeout. **Above the interface**, two strings: `NO_STORE_MESSAGE` now names `FDv2SkillStore` first, since it is the first thing a user sees on a missing store and offering only the development store was wrong once a production transport existed; and `watch_skills`' refusal message names it as the store with a delivery transport. ## Bugs found and fixed while testing the loop Five, all sharing one shape: the store stopped delivering while continuing to report itself healthy. - **The consecutive-failure counter never reset in stream mode.** `_stream_once` always ends by raising, so a reset on return was unreachable and `failures` grew for the whole process lifetime. Eleven *fully successful* payload transfers were enough to trip `max_consecutive_failures` and stop delivery for good, revocations included. A commit now resets the count, in `_apply`. - **A non-finite `Retry-After` killed the delivery thread.** `float("inf")` parses, and `Event.wait(inf)` raises `OverflowError` from inside the recoverable-error handler. Non-finite values are rejected and every honoured delay is clamped to `max_backoff`. - **`close()` during the initial connect waited out its full join timeout.** `self._connection` was assigned after the connect returned, so a `close()` in that window found nothing to interrupt. The stop flag is re-checked immediately after the assignment. - **`close()` blocked for the full join timeout on every healthy stream.** See `_interrupt_read` above. - **A stream interrupted by our own `close` was reported as a delivery failure.** Also: `connect_timeout` was accepted and never used, so a poll against a black-holed host hung for 300s rather than 10. It is gone, with the request timeout now chosen by mode, and `TestTimeouts` measures the bound against a socket that accepts and never answers. ## Tests > **Rebase note.** The previous push of this branch had silently reverted the protocol PR's last commit (payload identity: `payloads_ignored`, `_is_foreign_payload`, `TestPayloadIdentity`). Rebasing onto the updated protocol branch restored it; the full suite passes with it present. `_FakeFDv2Endpoint` is an in-process `ThreadingHTTPServer` implementing the wire contract, so request construction and header handling are exercised over real sockets rather than mocked. Covers skill put/delete over the wire, mixed payloads, 304, `basis` round-tripping, reconnect/backoff in both modes, `Retry-After` including non-finite and oversized values, bounded retries and the reset on commit, prompt shutdown during connect and during a healthy stream, hashless envelopes end to end through the accessors, server-side-only credentials, timeouts, and `watch_skills` over the transport: a wire-level revocation pruning a file without a restart. Full suite 1629 passing, 11 skipped; `ruff`, `ruff format`, and `mypy` clean. ## Open items, none in this PR's scope - 🔴 **`contentHash` is not on the wire yet.** Against a real environment today every skill resolves to nothing. This PR makes that loud (an error per hashless object, a summary per wholly-hashless payload, `diagnostics.hashless_objects`) rather than surviving it. - 🔴 **Server-side skill delivery is not deployed.** The wire shape this store reads — kind `skill`, key `<key>:<version>`, generic payload — is what [streamer #4681](launchdarkly/streamer#4681) and [gonfalon #70638](launchdarkly/gonfalon#70638) emit; both are still open. No account can receive skill objects until they ship and the producer is enabled. - 🟡 **FDv2 is opt-in per account.** A real environment returns 403 today; the store reports it as fatal and explains what to do. - ~~🟡 **`mv` is a guess.**~~ Resolved: the request sends no `mv`. That parameter selects the *flag* data model and the connection rejects any value but the flag default, while the generic agent-skill payload is served regardless of it. The `data_model_version` constructor argument is gone with it. - 🟡 **No payload signing** on this channel yet, so Beta is TLS-only. - 🟡 **The connection also carries the environment's flags.** Skipped and counted; a transport property, not fixable here. - 🟡 **`ld-relay` does not speak the FDv2 endpoints**, so relay-only deployments cannot receive skills in Beta. **Nothing here has touched a real LaunchDarkly environment**, because it cannot yet. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Overview** > Adds **`FDv2SkillStore`**, a production `SkillStore` that pulls agent skills over LaunchDarkly’s SDK FDv2 **`/sdk/poll`** and **`/sdk/stream`** endpoints (stdlib HTTP, background delivery thread, stream-by-default). It implements **`SkillStore`** (`get_object`, listeners, etc.) on top of the existing protocol reader, plus **`StoreDiagnostics`**, **`wait_for_skills`**, capped backoff with **`Retry-After`**, and **`close`** that interrupts blocked socket reads so shutdown is prompt. > > **Public surface:** `FDv2SkillStore` and `StoreDiagnostics` are exported from the package; README documents production setup with `init_client` and `watch_skills`. Server-side SDK keys only; mobile/client credentials are rejected. Outages keep last-known-good content; accessors above the store are unchanged. > > **Delivery-loop fixes** bundled here: reset consecutive-failure counts on successful commits / up-to-date answers (so healthy stream recycling does not stop delivery), safe handling of non-finite **`Retry-After`**, and not treating intentional **`close`** interrupts as transport failures. Removed unused **`connect_timeout`**; **`read_timeout`** is the single knob with mode-specific defaults. > > **Tests:** in-process fake FDv2 server exercises poll/stream, basis/ETag/304, revocations, retries, timeouts, hashless payloads, and **`watch_skills`** over the transport. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 88c225e. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
The docstring listed a give-up as ending delivery with no hint that start() undoes it. Matches TESTING.md §3.25 (ai-sdks-monorepo#40). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tarted again (#139) ## TL;DR After a store gives up and you call `start()` again, `diagnostics.connection_failures` kept the old run's count. So a healthy restarted store looked like it was already in an outage. This is a one-line fix, and it matches the TypeScript SDK. Found in review of launchdarkly/ai-sdks-monorepo#40 (review). ## Changes - `_rearm_waiters` resets `diagnostics.connection_failures`, alongside the internal `_failures` count it already reset. Before this, two recoverable failures and then a 401, followed by `start()`, gave `failed is None` and `connection_failures == 2`, with nothing failed in the new run. TypeScript resets the counter in `start()` (js-ai-sdk#107). - New test `test_a_restart_reports_no_failures_before_the_new_run_has_any`. It holds the new run's first request open and reads the counter right after `start()`. The existing `test_a_restart_resets_the_failure_count` couldn't catch this, because it reads the counter only after the new run fails, and that failure overwrites the stale value. The new test fails with `2 == 0` when the reset is reverted. - README (recovery after a 422) and `agents.md` now say the counter resets on restart. - The `wait_for_skills` docstring now says a store that gave up waits again once `start()` runs, and only `close` is final. Same change for TypeScript: launchdarkly/js-ai-sdk#113. Spec: TESTING.md §3.25 in launchdarkly/ai-sdks-monorepo#40. ## Test plan - [x] `make lint`, `make format-check`, `make typecheck` - [x] `make test`: 2201 passed, 11 skipped - [x] New test fails with the fix reverted - [ ] Note: `TestWatchSkillsOverTheTransport::test_a_revocation_prunes_without_a_restart` failed once in my runs. It also fails on `xie/agent-skills` without this change (1 in 25 runs), so it's an existing flaky test. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Agent Skills is exported only from launchdarkly_ai_server.experimental.skills, and none of its names is exported from the package root. Its names may change in a minor release while it is experimental. - Add set_skill_store(store) and remove the skillStore option from init_client: core client options do not name an experimental feature. - shutdown() still clears the skill store, through an internal hook whose failure is logged rather than raised. - Pin the experimental surface to an explicit allowlist in tests, and assert that no Skills name is reachable from the package root. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…_skill_store - init_client logs a warning for any option key it does not read, so a retired option such as skillStore is reported rather than dropped silently. - Inline _resolve_client into init_client; the split existed only to install the skill store after a successful resolve. - set_skill_store takes a SkillStore and raises TypeError on None. - _reset_for_testing clears skills state directly so a failed reset fails the test instead of being logged. - Soften the changelog wording for experimental names, and note in the launchdarkly-ai-python README where experimental features are imported from. - Clarify in agents.md that AiConfigRep still documents skills references: the experimental part is the SDK API, not the data model. - Drop init_client calls that tests no longer need before set_skill_store. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2 tasks done
…tal move - Pin the root `__all__` to a checked-in snapshot, so a root export such as a `frontmatter` accessor fails a test rather than relying on a denylist. The experimental module's `dir()` is pinned too, not only its `__all__`. - Test that a failure clearing experimental state in `shutdown()` is logged, not raised, and the client is still closed. - `set_skill_store` raises `TypeError` for a value without callable `get_object` and `all_objects`, instead of failing later as `store_unavailable`. Its docstring says replacing a store does not close the old one and a running watcher keeps the store it started with. - Document all seven `init_client` options in the docstring and README. - The README quickstart stops before `write_skills` when `inspect_config` returns no config, since an empty reference list would prune every skill. - `AiConfigRep` points at `experimental.skills.skill_refs`, and the `watch_skills` no-store message names the import path. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`parse_ai_config` rejected a whole AI Config when its `skills` field was malformed, so a bad experimental field failed `config().invoke()`, `stream()`, graph nodes and `extract_variation` for apps that never use Agent Skills. That breaks TESTING.md §0.3: experimental behaviour must not break a core call. The parser now passes `skills` through unchecked. `skill_refs` validates it instead and raises `ValueError` when the field is present but malformed, including `skills: null` and a single bad entry. It used to drop bad entries with a warning, which was safe only because the parser had already rejected them. Raising keeps the original reason for the check: `write_skills` with `prune=True` never receives an empty or partial list for a field that is present. An absent field, or a config that is not a dict, still returns `[]`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…146) ## Summary A malformed `skills` field no longer fails core config calls. Today `parse_ai_config` rejects the whole AI Config when `skills` is malformed. `config().invoke()`, `stream()`, graph nodes and `extract_variation` then raise, and `inspect_config` returns `config: None`, even for apps that never use Agent Skills. Agent Skills is experimental, and TESTING.md §0.3 says experimental behaviour must not break a core call. - **`parse_ai_config` passes `skills` through unchecked.** - **`skill_refs` validates the field and raises `ValueError`** when it is present but malformed, including `skills: null` and a single bad entry. Before, it dropped bad entries with a warning, which was only safe because the parser had already rejected them. - **An absent field, or a config that is not a dict, still returns `[]`.** Raising rather than dropping keeps the reason the parser checked in the first place. `write_skills` with the default `prune=True` deletes every managed skill missing from the list it is given, so `skill_refs` never returns an empty or partial list for a field that is present. Stacked on #144. JS counterpart: launchdarkly/js-ai-sdk#117. ## Tests - **Parser:** a malformed `skills` field of every kind (null, non-list, bad entry, one bad entry among good ones) parses successfully and passes through unchanged. - **`skill_refs`:** the cases the parser used to reject, including `null`, now raise `ValueError` naming the field. One bad entry rejects the whole field. The error does not echo the rejected key. - **Core calls:** `inspect_config` and `extract_variation` succeed on a variation with malformed `skills`. ## Open question `skill_refs(None)` still returns `[]`, as documented. That is the case where a failed `inspect_config` feeds an empty list to a pruning `write_skills`. The README quickstart guards against it, but raising would remove the trap for every caller. That is a contract change, so it is not in this PR. ## Test plan - [x] `make lint`, `make format-check`, `make typecheck` - [x] `uv run pytest`: 2209 passed, 11 skipped 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Overview** > **Agent Skills validation moves off the core config parser** so experimental `skills` data cannot break `config().invoke()`, `inspect_config`, or `extract_variation` when apps never use skills (TESTING.md §0.3). > > `parse_ai_config` now **passes `skills` through unchanged**; `_parse_skills` was removed from the parse path and exposed as `skills_field_rejection_reason` for reference discovery only. **`skill_refs` validates the field** and raises `ValueError` when `skills` is present but malformed (including `null` or one bad entry in a list), instead of dropping bad entries with warnings—avoiding a shortened list that would authorize `write_skills(..., prune=True)` to delete on-disk skills the config still references. > > Docs (`README.md`, `agents.md`, `AiConfigRep` docstring) and tests were updated: parser tests expect malformed `skills` to parse successfully; `skill_refs` and lifecycle tests cover the new raise behavior and that core calls succeed with invalid `skills`. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 0f303f4. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
…144) ## Summary Reorganizes Agent Skills to ship as an **experimental** feature. Every Skills name is exported from `launchdarkly_ai_server.experimental.skills`, and none from the package root. Applications opt in by importing from that module. While the feature is experimental, its names may change in a minor release, so review the changelog when you upgrade. Additionally, the store is configured via a `set_skill_store(store)` setter rather than injected into `init_client`, which logs a warning for any option key it does not read, so a retired `skillStore` option is reported rather than dropped silently. The check is generic and does not name the feature. Other changes: - **Shutdown:** `shutdown()` clears the skill store in a way that logs a failure rather than raising it into the core call. The test reset clears it directly, so a failed reset fails the test. - **Messages and docs:** error messages, docstrings, the README and `agents.md` now point at `set_skill_store` and the experimental import path. The `init_client` docstring and README list all seven options it reads. The README quickstart stops before `write_skills` when `inspect_config` returns no config, since an empty reference list would prune every managed skill. The `launchdarkly-ai-python` README notes that experimental features are not re-exported and are imported from `launchdarkly_ai_server.experimental`. `agents.md` clarifies that the experimental part is the SDK's API, not the LaunchDarkly data model, so `AiConfigRep` still documents a config's `skills` references. - **Tests:** - The experimental exports are pinned to an exact list, so a new export fails. The module's public `dir()` is pinned too. - The root `__all__` is pinned to a checked-in snapshot (`tests/test_public_surface.py`), so a new root export fails. - A failure clearing experimental state in `shutdown()` is logged, and the client is still closed. - `set_skill_store` raises `TypeError` for a value without callable `get_object` and `all_objects`, as in the JS SDK. - No Skills name is reachable from the package root. - `init_client` ignores a `skillStore` option and warns about it; unknown options warn and documented ones do not. - `set_skill_store(None)` is ignored and leaves the configured store in place, as in the JS SDK. - `SKILL_OBJECT_KIND` and `MAX_SKILL_CONTENT_BYTES` stay unexported from both the root and the experimental module. ## Open questions - **The config parser still validates `skills`.** `parse_ai_config` fails a whole config when its `skills` array is malformed, so a bad Skills field can fail an ordinary `config().invoke()`. That conflicts with "experimental behaviour must not break a core call". The behaviour is unchanged here, pending a decision. - **Evaluations is still exported from the package root.** It was already released from there, so moving it to `experimental.evaluations` needs deprecated aliases. That is left for a separate change. ## Test plan - [x] `make lint`, `make format-check`, `make typecheck` - [x] `uv run pytest`: 2217 passed, 11 skipped 🤖 Generated with [Claude Code](https://claude.com/claude-code), updated by @XieX <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Overview** > **Agent Skills** are no longer exported from `launchdarkly_ai_server`; callers must import from `launchdarkly_ai_server.experimental.skills`, which may change in minor releases. Store wiring moves from `init_client(options={"skillStore": ...})` to **`set_skill_store(store)`** (callable before or after init; `None` is ignored; invalid stores raise `TypeError`). > > **`init_client`** now allowlists option keys (`sdkKey`, URIs, OTel fields) and **logs a warning** for anything else—including retired `skillStore`—instead of applying it silently. **`shutdown()`** clears experimental state (e.g. the skill store) via a path that logs failures without breaking core teardown. > > Docs (`README`, `agents.md`, `launchdarkly-ai-python` README) and error messages point at the experimental import path and `set_skill_store`. Tests add a frozen snapshot of the **root `__all__`**, an allowlist for **experimental.skills** exports, and lifecycle coverage for unknown options and experimental clear failures. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit f5c9aa2. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
Add a store with only `get_object` and one with only `all_objects` to the `set_skill_store` rejection cases, so a check that skips either method fails. Matches the JS SDK's `setSkillStore` tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Resolve packages/client/README.md: keep the init_client options table after the client export table it describes, followed by main's "Give the evaluation tools" section. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`main` exports `EvalTool` from the package root. Add it to `ROOT_SURFACE` so the snapshot test passes after the merge. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Author
|
The repo squashes with the PR title and body as the commit message, and release-please reads the (Moved here from the PR description so it doesn't end up in the squash commit message.) |
XieX
changed the title
Fixes the must-fix-before-merge findings from the pre-release review of Agent Skills (#87, reviewed at `0306c69`). One commit per fix. Each code fix has a test that fails on `0306c69` and passes here. ## 1. A stream event whose data isn't JSON no longer commits the rest of its transfer `_iter_sse` used to log the event and skip it. The `payload-transferred` that followed then committed the transfer without the event and advanced `basis`. A corrupted `delete-object` left a revoked skill served indefinitely, because the reconnect asked only for changes after a basis that already covered the revocation. A corrupted `put-object` in an `xfer-full` revoked that skill by omission. It now raises `_RecoverableTransportError`, matching `ldclient`'s FDv2 stream, which interrupts on `JSONDecodeError`. The in-flight payload is dropped and the reconnect sends the last committed basis. The drop is logged at WARNING in `_iter_sse`, because `_run` logs a disconnect after a completed exchange at debug, and a revocation usually arrives after a commit on the same connection. Only the six events `handle` reads are parsed (`server-intent`, `put-object`, `delete-object`, `payload-transferred`, `goodbye`, `error`). A `heart-beat` or an unknown event is yielded with `None` data, so its data can't drop the connection. `ldclient` likewise calls `json.loads` only inside its known-event branches. Tests: `test_a_malformed_event_abandons_its_transfer_and_keeps_the_basis`, parametrized over two cases: - `delete-object-in-xfer-changes` commits `a:1` and `revoked:1`, then sends an `xfer-changes` containing a clean `put fresh:1` and a truncated `delete revoked:1`, then `payload-transferred`. At the reconnect, nothing from that transfer is committed: `revoked` is still held and `fresh` is absent. - `put-object-in-xfer-full` commits `a:1`, then sends an `xfer-full` with a clean `put b:1` and a truncated `put a:1`. At the reconnect, `a` is still held and `b` is absent. In both cases the reconnect carries `basis-1`, a clean retransmission then applies in full, and a WARNING names the event. `test_an_event_handle_does_not_read_is_not_parsed` covers `heart-beat` and `x-future-event` with `data: not json`. `test_an_event_handle_reads_is_parsed` pins the parse set (all six), and the store-level catastrophe test goes through `_iter_sse`. Empty data on a known event is not JSON either (`bf559fb`), as `json.loads("")` raises in `ldclient`. Read as `None`, a `delete-object` was ignored and a `payload-transferred` committed with no selector. The malformed-event test has a third case, `empty-delete-object-in-xfer-changes`, and `test_an_event_handle_reads_with_no_data_drops_the_connection` covers all six names. Two reader tests pin the experimental-stage rule for a put or delete with no usable `key`: it is warned and ignored, and its transfer commits (TESTING.md §3.25, ai-sdks-monorepo#48). ## 2. A `goodbye` with `catastrophe: true` no longer stops delivery It was raised as fatal, so delivery stopped until something called `start()`. Python `ldclient` 9.16 never reads `catastrophe`, and the Go SDK only logs it. It is now a recoverable disconnect, logged at ERROR. It is not marked `recycled`, so it counts toward `connection_failures` even after a completed exchange. `_TransferOutcome.fatal` had no other producer, so it is removed. Tests: `test_a_catastrophic_goodbye_is_fatal` is flipped to `test_a_catastrophic_goodbye_is_a_counted_disconnect`. New: `test_a_catastrophic_goodbye_reconnects_and_is_counted`. No docstring or README said the goodbye was fatal; only the code and that test did. ## 3. `skill_refs` raises on a config that is not a dict `skill_refs(None)` returned `[]`, and `None` is the config a failed `inspect_config` returns. So `write_skills(skill_refs(info["config"]), root)` pruned every managed skill during an outage. A non-dict config now raises `ValueError`, as a malformed `skills` field already does. A dict without a `skills` key still returns `[]`. The docstring, the README (quick-start comment, validation paragraph, API table), and `agents.md` are updated. Test: `test_a_config_that_is_not_a_dict_raises`, parametrized over `None`, a list, and a string. ## 4. Docs: the `ld.skills.integrity_failure` commitment The README said the event "will not be renamed". It now says that during the experimental stage the name, fields, and `reason_code` values may change in a minor release, with a changelog entry under **Experimental** (TESTING.md §0.1's wording, and js-ai-sdk's). `agents.md` is updated to match. Docs only, so there is no test. ## Checks `uv run pytest` (repo root): 2321 passed, 11 skipped. `make lint`, `make format-check`, and `make typecheck` are clean. Spec: launchdarkly/ai-sdks-monorepo#48. JS counterparts: launchdarkly/js-ai-sdk#120. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Overview** > Addresses pre-release review must-fixes for **Agent Skills** in the Python client: safer reference discovery, FDv2 stream consistency, and doc alignment. > > **`skill_refs`** now raises `ValueError` when the config is not a dict (including `None` from a failed `inspect_config`), instead of returning `[]`. That stops `write_skills(skill_refs(...), root)` from pruning every managed skill during a config outage. A dict with no `skills` field still returns `[]`. > > **FDv2 streaming** treats non-JSON data on the six events the protocol reader consumes as a recoverable disconnect (`_RecoverableTransportError`), matching base `ldclient` behavior and abandoning the in-flight payload so a later `payload-transferred` cannot commit a partial transfer and advance `basis`. `heart-beat` and unknown events are not parsed. **`goodbye` with `catastrophe: true`** reconnects with backoff (ERROR log, counted failure) instead of stopping delivery fatally; `_TransferOutcome.fatal` is removed. > > **Docs** (`README.md`, `agents.md`) describe the `skill_refs` contract, stream/keyless-object behavior, and soften the `ld.skills.integrity_failure` stability wording for the experimental stage (minor-release changes with changelog). Tests cover malformed SSE, catastrophic goodbye, non-dict configs, and keyless put/delete experimental rules. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit bf559fb. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
andrewklatzke
pushed a commit
that referenced
this pull request
Oct 8, 2026
…gelog heading (#150) ## Summary Lists changes to the two experimental features under an **Experimental** heading in each package's changelog, as TESTING.md §0.1 promises. The features are Agent Skills (scope `skills`) and Evaluations from Code (scope `evaluations`). Spec: launchdarkly/ai-sdks-monorepo#50. JS twin: launchdarkly/js-ai-sdk#123. ## How `release-please-config.json` gains `changelog-sections`. Rows matching `feat`, `fix` and `perf` with scope `skills` or `evaluations` map to "Experimental". The rest of the list reproduces release-please 17.6.0's Python-strategy defaults (`strategies/python.ts:31-44`), which keep `deps` and a visible "Documentation" section, so every other commit renders exactly as before. Matching on scope works because release-please hands `changelog-sections` to `conventional-changelog-conventionalcommits` 6.1.0 unchanged (`manifest.ts:1397`), and the preset's `findTypeEntry` checks an entry's `scope` (`writer-opts.js:60-71`). The first match wins, so the scoped rows come first, and "Experimental" is therefore the first heading. `scope` isn't in release-please's documented config schema. Rerun the checks below when upgrading release-please. `AGENTS.md` gains a "PR Titles and the Changelog" section with the convention: - an experimental change uses its feature's scope, and only that one scope; - a PR that mixes core and experimental changes is split, or uses `BEGIN_COMMIT_OVERRIDE`; - a breaking experimental change uses `!` while the package is 0.x, and drops it after 1.0. ## Effect on the pending release `release-please release-pr --dry-run` against this branch proposes the same five packages and versions as release PR #142. The one difference is the point of this PR: the `evaluations` commits move out of "Features" and "Bug Fixes" into "Experimental" for `launchdarkly-ai-server` 0.3.0 and `launchdarkly-ai-python` 0.2.0. They also still appear in "⚠ BREAKING CHANGES". `docs(evaluations)` stays under "Documentation". If this merges before #142, the 0.3.0 changelog will show that split. ## Verification - **Rendering:** I ran release-please 17.6.0's own commit parser and `DefaultChangelogNotes` against this config, with synthetic commits. Non-experimental commits render byte-for-byte the same as with the Python defaults, and the scoped commits land under Experimental. - **Dry run:** compared with #142, as above. - **Agent Skills:** #87 is retitled `feat(skills): Agent Skills`, so its squash commit lands under Experimental. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Overview** > Adds **`changelog-sections`** to `release-please-config.json` so squash-merged `feat`/`fix`/`perf` commits with scope **`skills`** or **`evaluations`** appear under an **Experimental** heading in package changelogs; other commit types keep the usual Python release-please sections (Features, Bug Fixes, etc.), with scoped rules listed first so they win matching. > > Documents the same conventions in **`AGENTS.md`** under **PR Titles and the Changelog**: single-scope experimental PR titles, splitting or `BEGIN_COMMIT_OVERRIDE` when mixing core and experimental work, and how `!` applies to breaking experimental changes before/after 1.0. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 0df5fb1. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Brings in #145 (helper usage), #150 (Experimental changelog heading) and #133 (strip tools from LangChain model constructor kwargs). lifecycle.py merged without a conflict and keeps both sides: the usage flush sits beside each flush_ai_sdk_info call, so on the BYOC path it runs only after telemetry setup succeeds and the client is adopted, and on the options path only after a successful setup. shutdown() runs _clear_experimental_state() before client teardown, then resets usage with sdk-info. Root surface: add main's report_usage to ROOT_SURFACE. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 0345fd5. Configure here.
This was referenced Oct 9, 2026
XieX
added a commit
to launchdarkly/js-ai-sdk
that referenced
this pull request
Oct 9, 2026
…handle (#129) Same finding as the Bugbot comment on launchdarkly/python-ai-sdk#87 (comment 4232816271). The gap is identical in this SDK. **What was wrong:** `openOrCreateDirectory` in `safe-fs.ts` created `<root>/<key>/` with `mkdir(directory, 0o755)`, but the process umask still masks that mode, and nothing set it afterwards. Under `umask 0077` the directory came out `0700`, so an agent running as a different identity couldn't read `SKILL.md`, even though `writeSkills` reported success. The README's privilege-separation section promises per-skill directories at `0755` no matter what the umask is. **Fix:** after a successful create, the code sets the mode to `0755` with `FileHandle.chmod` (that is, `fchmod`) on the handle `openDirectoryNoFollow` already returns, the same way `FILE_MODE` is applied to files. It never chmods the path, because a swap between the mkdir and the open could redirect a path-based chmod. - The mode is only changed when this call created the directory. An existing `<root>/<key>/` keeps its mode. - The managed root's mode is still up to the user, as the README says. It's created by a separate `mkdir(root)` in `skills-fs.ts` and isn't touched here. - **A setgid bit the new directory inherited is kept.** On Linux, a directory created under a setgid parent gets the parent's `S_ISGID` bit, and `fchmod` uses exactly the bits passed, so a bare `0755` cleared it. Files written inside would then get the reconcile identity's primary group instead of the shared group. The mode is now `(mode & S_ISGID) | 0755`, with the current mode read through `handle.stat()` on the same descriptor. macOS doesn't propagate the bit, so nothing changes there. - The symlink and not-a-directory refusals are unchanged. If the chmod fails, the handle is closed before the error is rethrown. **Tests (skipped on win32):** - `creates per-skill directories as 0755 even under a restrictive umask` runs `writeSkills` under `process.umask(0o077)` and asserts the directory is `0755` and `SKILL.md` is `0644`. It fails without the fix. - `keeps a setgid bit the new skill directory inherited` makes the root `2755` and asserts the new skill directory is `2755`. It's skipped unless a probe directory under that root actually inherits the bit, so it runs on Linux and skips on macOS. On Linux (Docker, `node:22`) it fails without the setgid change. `fs.constants` has no `S_ISGID`, so `safe-fs.ts` defines it locally. - `leaves the mode of an existing directory alone` checks that an existing `0700` directory stays `0700`. Full suite: 1394 passed on macOS (the setgid test skips) and 1405 passed on Linux. biome and tsc are clean. The Python fix is in a separate PR, linked below. The spec lines are in launchdarkly/ai-sdks-monorepo#53 (merged) and its setgid follow-up. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…descriptor (#154) Addresses the Bugbot finding on #87 (comment 4232816271). **What was wrong:** `safe_fs.open_or_create_directory` created `<root>/<key>/` with `os.mkdir(..., 0o755)`, but the process umask still masks that mode, and nothing set it afterwards. Under `umask 0077` the directory came out `0700`, so an agent running as a different identity couldn't read `SKILL.md`, even though `write_skills` reported success. The README's privilege-separation section promises per-skill directories at `0755` no matter what the umask is. **Fix:** after a successful create, the code sets the mode to `0755` with `os.fchmod` on the directory descriptor it already pins. This is gated on `_SUPPORTS_FCHMOD`, the same way `_FILE_MODE` is applied to files. It never chmods the path, because a swap between the mkdir and the open could redirect a path-based chmod. - The mode is only changed when this call created the directory. An existing `<root>/<key>/` keeps its mode. - The managed root's mode is still up to the user. It's created by `Path.mkdir()` in `skills_fs` and isn't touched here. - **A setgid bit the new directory inherited is kept.** On Linux, a directory created under a setgid parent gets the parent's `S_ISGID` bit, and `fchmod` uses exactly the bits passed, so a bare `0755` cleared it. Files written inside would then get the reconcile identity's primary group instead of the shared group. The mode is now `(mode & S_ISGID) | 0755`, with the current mode read through `os.fstat` on the same descriptor. macOS doesn't propagate the bit, so nothing changes there. - The symlink and not-a-directory refusals are unchanged. - When there's no `*at()` family (Windows), there's no descriptor, so the mode isn't corrected there. Windows has no POSIX directory modes anyway, and that fallback still only does the existing `lstat` check. **Tests (POSIX only):** - `test_created_skill_directory_is_0755_under_a_restrictive_umask` runs `write_skills` under `umask 0o077` and asserts `<root>/<key>/` is `0755` and `SKILL.md` is `0644`. It fails without the fix (`0700`). - `test_created_skill_directory_keeps_an_inherited_setgid_bit` makes the root `2755` and asserts the new skill directory is `2755`. It's skipped unless a probe directory under that root actually inherits the bit, so it runs on Linux and skips on macOS. On Linux (Docker, `python:3.12`) it fails without the setgid change. - `test_create_leaves_an_existing_directory_mode_alone` checks that an existing `0700` directory stays `0700`. Full suite: 1521 passed on macOS (the setgid test skips) and 1522 passed on Linux. ruff and mypy are clean. The JS fix is in a separate PR, linked below. The spec lines are in launchdarkly/ai-sdks-monorepo#53 (merged) and its setgid follow-up. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…#153) Corrects one sentence in the "Experimental features" section of `packages/ai/README.md` (`launchdarkly-ai-python`). The README said `launchdarkly_ai_server.experimental` is something "this package installs". That isn't guaranteed. `packages/ai/pyproject.toml` depends on an unpinned `launchdarkly-ai-server`. A user who upgrades only `launchdarkly-ai-python` keeps whatever `launchdarkly-ai-server` they already have installed or locked, because that version still satisfies the requirement. The latest published release, 0.2.4, has no `experimental` subpackage, so on it `import launchdarkly_ai_server.experimental.skills` raises `ModuleNotFoundError`. Nothing else breaks: `launchdarkly-ai-python` is `from launchdarkly_ai_server import *` and never imports `experimental`. The new text: - says the features live in `launchdarkly-ai-server` and need a release of it that includes them - tells users to upgrade `launchdarkly-ai-server` itself if the import fails 🤖 Generated with [Claude Code](https://claude.com/claude-code), updated by @XieX <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Overview** > **Documentation-only** update to the **Experimental features** section in `packages/ai/README.md`. > > The README no longer implies that installing `launchdarkly-ai-python` always installs a `launchdarkly-ai-server` build with `experimental`. It now states that experimental APIs live in **`launchdarkly-ai-server`**, depend on a server release that actually ships them, and that **`import launchdarkly_ai_server.experimental` failures should be fixed by upgrading `launchdarkly-ai-server`**, not only the barrel package—because the dependency is unpinned and an older server can remain after upgrading `launchdarkly-ai-python`. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 167b370. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Agent Skills
Skills are versioned
SKILL.mddocuments managed in LaunchDarkly and attached to AI Config variations by reference. This adds the full server-side surface for them: the SDK reports which skills a resolved config references, retrieves their content over LaunchDarkly's FDv2 delivery channel, verifies it, and materializes it onto disk as<root>/<key>/SKILL.md— where the Claude Agent SDK and anything else following that convention discovers it.Experimental. Every name below is imported from
launchdarkly_ai_server.experimental.skills; none is exported from the package root (test_public_surface.pypins that), and any of them may change in a minor release.launchdarkly-ai-pythondoes not re-export them either; its README points at the experimental module.Release notes
release-please reads the block below in place of this PR's title, one changelog entry per line. The skills feature comes first; the rest are the core, non-skills behaviour changes this branch also makes.
BEGIN_COMMIT_OVERRIDE
feat(skills): add experimental Agent Skills under
launchdarkly_ai_server.experimental.skillsfeat(client): warn about unrecognized
init_clientoptions instead of ignoring them silentlyfix(client):
shutdown()releases the global OpenTelemetry tracer provider the SDK registered, so a laterinit_clientexports spans againfix(client):
init_clientadopts a client only after telemetry setup succeeds, and closes a client it built when setup failsEND_COMMIT_OVERRIDE
Layers
types.py/types_validation.pySkill,SkillReference,SkillOutcome,ReconcileReport; the canonical key grammar (^[a-z0-9][a-z0-9-]*$) and version predicateexperimental/skills.py__all__, nothing elseskills.pyset_skill_store, the accessors, andInMemorySkillStoreskills_core.pySkillStoreinterface, verification, and the structured integrity log recordskills_fdv2.pyFDv2SkillStore— delivery over the SDK-facingGET /sdk/pollandGET /sdk/streamendpointsskills_fs.py/safe_fs.pyskills_watch.pywatch_skills— re-reconcile on every delivery changeDependencies flow downward only; nothing above the store can tell which store produced an object.
API
All from
launchdarkly_ai_server.experimental.skills. The accessors,write_skillsandwatch_skillsareasync.set_skill_store(store)write_skillsandwatch_skillsread from. Applies on every call, including afterinit_client.Noneis ignored and never clears a configured store;shutdown()does. Replacing a store does not close the old oneskill_refs(config)skillsarray intolist[SkillReference]. Pure — no client, store, or network. RaisesValueErrorwhenskillsis present but malformed (includingnull)get_skill(key, *, version=None)None.version=Nonemeans newest availableget_skill_result(key, *, version=None).skill,.reason(ok/absent/integrity_failure/store_unavailable/wrong_version),.detailget_skills(refs)/all_skills()refstakesSkillReferences or bare keys. Unresolvable entries are omitted; a run that omitted anything logs a count at WARNwrite_skills(skills, root, *, prune=True, timeout=10.0, on_unavailable="keep")root, returning aReconcileReport. Pass"*"for the whole librarywatch_skills(skills, root, *, prune=True, timeout=10.0, on_unavailable="keep", debounce=0.5, on_reconcile=None)write_skillsplus re-reconcile on delivery change. Returns(initial report, SkillWatcher)SkillWatcherclose(timeout=15.0),reconciles; also a context managerSkillStoreget_object,all_objects; optionalis_initialized()andadd_listener/remove_listenerInMemorySkillStore(objects=None)put(raw), for tests and bring-your-own-contentFDv2SkillStore(sdk_key, *, base_uri=None, stream_uri=None, mode="stream", poll_interval=30.0, read_timeout=None, initial_backoff=1.0, max_backoff=30.0)start()(returns the store),wait_for_skills(timeout=10.0) -> bool,close(timeout=5.0),diagnostics,failed; also a context managerStoreDiagnosticsPlus the value types (
Skill,SkillReference,SkillOutcome,ReconcileAction,ReconcileReport), the closed-set types (ReconcileActionKind,OnUnavailable,SkillOutcomeReason) and the fixed on-disk constants (SKILL_FILENAME,MANIFEST_FILENAME,MANIFEST_VERSION). The store is configured withset_skill_store(store), not throughinit_client: askillStoreoption is logged as unrecognized and ignored.launchdarkly_ai_server.shutdown()detaches the store from the accessors (it does not close it).The primary use case
Notes for reviewers
parse_ai_configpassesskillsthrough unvalidated, so a malformed field never failsconfig(),invoke()or any other core call.skill_refsis where it is validated: it raisesValueErrorwhenskillsis present but is not a list of{key, version}objects, and one bad entry rejects the whole field, sowrite_skillsnever receives a partial list that would prune skills the config still references.init_clientwarns about option keys it does not read; it adopts a client (BYOC or its own) only after telemetry setup succeeds, closing one it built when setup fails; andshutdown()releases the global tracer provider when this SDK was the one that registered it, so init → shutdown → init exports spans again. When another library registered a tracer provider first, the SDK logs a warning and leaves it alone. Each is its own line in the release notes above.contentHash, its key and version revalidate, and its size is within 10 MiB. There is no fallback that skips it. Every withheld skill emits one structured ERROR record —ld.skills.integrity_failure, a stability commitment, byte-identical across LaunchDarkly's AI SDKs — on the SDK's own logger, independent of telemetry configuration.write_skillstouches only what it owns. It writes<root>/<key>/SKILL.md, records what it owns in<root>/.launchdarkly-skills.json, and will overwrite or delete only manifest-recorded paths. It treats that manifest as untrusted input and re-validates every entry. The one adoption exception — a byte-identical file at a managed path — is what makes a crashed reconcile recoverable.lstat, which is a check-then-use race rather than a closed window. Write permission on the managed root or any ancestor is the security boundary on every platform, and on Windows the only one. The README's privilege-separation section is the deployment contract — the recommended shape runs the reconcile as a different identity than the agent."*".watch_skills("*", root)removes a revoked skill'sSKILL.mdwithout a restart. With an explicit list such asskill_refs(config), the list is fixed: a skill the store answersabsentfor is reported as anerrorand its files are kept, and the watcher listens only to the skill store, so unpinning a skill or moving it to a new version is not seen until the refs are read again (at the next boot, typically). Making the watcher follow config changes is a planned follow-up and is additive.write_skillsblocks. It isasyncfor parity with the other accessors and with the TypeScript SDK, but awaits nothing. Wrap it inasyncio.to_threadif a large reconcile holding the event loop matters.ld-relaydoes not speak the FDv2 endpoints.Testing
551 skills test functions across five files —
test_skills(151),test_skills_fdv2(230),test_skills_fs(130),test_safe_fs(23),test_skills_watch(17) — including a filesystem abuse matrix and root-swap race tests that fail outright rather than skip.test_lifecyclecovers theinit_client/shutdownchanges, andtest_public_surfacepins that the package root exports no skills names.🤖 Generated with Claude Code, reviewed by @XieX.
Note
Overview
Adds experimental Agent Skills under
launchdarkly_ai_server.experimental.skills(not re-exported from the package root). VersionedSKILL.mdcontent can be discovered from AI Config viaskill_refs, fetched through a pluggableSkillStore(includingFDv2SkillStoreover LaunchDarkly FDv2 poll/stream), verified with mandatorycontentHashchecks and structuredld.skills.integrity_failurelogging, and materialized to<root>/<key>/SKILL.mdwith manifest-scoped reconcile/prune (write_skills, optionalwatch_skills). Supporting modules cover retrieval (skills/skills_core), delivery (skills_fdv2), descriptor-pinned disk I/O (safe_fs/skills_fs), and extensive README/agents.mdguidance (privilege separation, POSIX vs Windows bounds).Core lifecycle fixes:
init_clientdocuments supported options and warns on unknown keys; client adoption happens only after telemetry setup succeeds (built SDK clients are closed on setup failure);shutdown()clears skill store state and, when this SDK owned it, releases the global OTel tracer provider so a laterinit_clientcan export spans again.Reviewed by Cursor Bugbot for commit dbd15da. Bugbot is set up for automated code reviews on this repo. Configure here.