Repository navigation
Test harness CLI v1.0.32 incompatible with permission kind values and session.auth from commit dd2dcbc #1146
Description
Summary
Commit dd2dcbc (Per-session GitHub authentication for all SDK languages, plus update runtime and permissions to match (#1124)) introduced two protocol changes:
- New permission kind values:
approve-once,reject,user-not-available,no-result(replacingapproved,denied-by-rules,denied-no-approval-rule-and-could-not-request-from-user,denied-interactively-by-user) - New RPC method:
session.auth.getStatusfor per-session authentication
However, the test harness (test/harness/package.json) still pins "@github/copilot": "^1.0.32", and CLI v1.0.32 does not support either of these changes:
- CLI v1.0.32 only accepts the old permission kind strings. Sending the new values causes
Unhandled permission result kind: [object Object], which makes every tool execution fail with permission errors. - CLI v1.0.32 does not implement
session.auth.getStatus, returningJsonRpcException: Unhandled method session.auth.getStatus.
Impact
Any SDK that faithfully ports commit dd2dcbc will have broken E2E tests when run against the test harness. We observed this in the Java SDK (github/copilot-sdk-java) where 14 E2E tests fail (11 tool/permission failures + 3 auth errors).
Failing test categories
Permission/Tool failures (11): The CLI rejects the new permission kind values, so all tool executions fail. The model responds with apology messages instead of executing tools. Examples:
ToolsTest.testInvokesBuiltInTools— expects tool output, gets permission errorPermissionsTest.testPermissionHandlerForWriteOperations— permission handler never invokedCopilotSessionTest.testShouldCreateSessionWithCustomTool— custom tool never executes
Per-session auth errors (3): All PerSessionAuthTest methods throw Unhandled method session.auth.getStatus. Additionally, no test snapshots exist for auth tests under test/snapshots/.
Expected fix
Either:
- Bump the CLI dependency in
test/harness/package.jsonto a version that supports the new permission protocol andsession.auth.getStatus - Or add auth test snapshots and backport the new permission kind values to the CLI
Reproduction
# In any SDK that has ported commit dd2dcbc:
# Run E2E tests against the test harness with @github/copilot@1.0.32
# Observe 14 test failuresReferences
- Reference impl commit: dd2dcbc
- CLI version in harness:
@github/copilot@1.0.32 - Affected downstream: github/copilot-sdk-java (PR Add GitHub Actions workflow for .NET Core Desktop #122)
Activity
brunoborges commentedon Apr 27, 2026
This is also affecting real users of the Java SDK in production — see github/copilot-sdk-java#126. Users are having to manually send the old
"approved"string as a workaround.github-actions commentedon Apr 27, 2026
Investigation findings
Label applied:
bug— this is a genuine regression in the test infrastructure.What I investigated
test/harness/package.json— confirms@github/copilotis pinned to^1.0.32; the lock file also resolves to exactly1.0.32nodejs/src/generated/rpc.ts— shows the new permission result kinds:approve-once,reject,user-not-available(noapproved/denied-*variants)nodejs/src/types.ts—PermissionRequestResultuses the new kinds;approveAllreturns{ kind: "approve-once" }nodejs/src/session.ts— emitsuser-not-availablewhen no handler is registeredWhat I found
Commit
dd2dcbc(Per-session GitHub authentication for all SDK languages, plus update runtime and permissions to match #1124) made two breaking protocol changes that require a newer CLI build:approve-once/reject/user-not-availablereplace the oldapproved/denied-*strings. CLI v1.0.32 does not recognise the new values, responding withUnhandled permission result kind: [object Object], which causes every tool invocation to fail with a permission error.session.auth.getStatus: not implemented in CLI v1.0.32, returnsJsonRpcException: Unhandled method session.auth.getStatus.The test harness was not bumped to a compatible CLI version when these protocol changes landed, so any SDK that faithfully ports the commit (e.g. the Java SDK) will hit 14+ E2E failures immediately.
Fix needed
The
test/harness/package.jsondependency"@github/copilot": "^1.0.32"needs to be bumped to the minimum CLI version that supports both the new permission protocol andsession.auth.getStatus. Auth test snapshots undertest/snapshots/may also need to be added so that the replay harness can coverPerSessionAuthscenarios.brunoborges commentedon Apr 27, 2026
Update: Tested with CLI v1.0.36 (latest available on npm) — same failures. The new permission kind values (
approve-once,reject,user-not-available,no-result) andsession.auth.getStatusare not supported in any released CLI version up to and including v1.0.36.The CLI's permission handler still only accepts the old values:
approved,denied-by-rules,denied-no-approval-rule-and-could-not-request-from-user,denied-interactively-by-user,denied-by-content-exclusion-policy,denied-by-permission-request-hook.This means the protocol changes in commit
dd2dcbccannot be used by any downstream SDK until a new CLI version ships with support for them.edburns commentedon Apr 27, 2026
See PR #1150
SteveSandersonMS commentedon Apr 28, 2026
I'm not sure what's causing the behavior you're observing, but 1.0.36 is the version where
approve-onceetc. is added. The E2E tests in CI are using this runtime version and passing (e.g., https://github.com/github/copilot-sdk/actions/runs/24973224129/job/73120080224).I don't think that's the case. The test harness only uses that package to extract a TypeScript type. It's not actually invoking the binary bundled there at runtime in the tests.
edburns commentedon Apr 30, 2026
I attribute this problem to an ordering problem regarding the agentic workflow sync. This problem has been resolved in github/copilot-sdk-java#137 and related PRs.
This issue may be closed.