[claude-code-user-docs-review] 🔍 Claude Code User Documentation Review - 2026-10-08

[github-actions[bot]]

Executive Summary

Claude Code users can fully set up gh-aw (API key or WIF), but the docs consistently treat Copilot as first-class and Claude/Codex as secondary: Copilot gets two fully inline auth paths and 10 smoke-test auth variants, while Claude gets one inline path, a link-out for WIF, and 6 variants. No critical blocker exists today, but a specific trap remains: a claude login subscription token (CLAUDE_CODE_OAUTH_TOKEN) is silently ignored, failing with an error that never names the token.

Severity Findings

Major Obstacles

1. CLAUDE_CODE_OAUTH_TOKEN silently ignored. A token from standard claude login is not supported; the run fails with a generic Claude CLI auth error that never mentions the token, so a subscription user has no signal to switch credentials. (docs/src/content/docs/setup/quick-start.mdx:151, docs/src/content/docs/setup/cli.md:232)
2. gh aw init scaffolding parity gap. Copilot gets a custom agent file plus MCP wiring (.github/mcp.json, copilot-setup-steps.yml) automatically; Claude/Codex users are told only to "author an agent file in your own agent's format" with no template or concrete steps. (docs/src/content/docs/setup/cli.md:127-132)
3. Example/smoke-test parity gap. Copilot has 116 workflow declarations vs Claude's 64 (~1.8x) and Codex's 78; smoke-test auth-variant coverage is 10 files for Copilot vs 6 for Claude and only 3 for Codex — fewer worked examples for non-Copilot users to copy from.
4. Anthropic WIF is link-only. Copilot's auth section gives two fully inline, step-by-step paths (org billing, PAT); Claude's WIF path is one summary paragraph plus a link to a separate page, with no inline trust-policy/federation-rule walkthrough. (docs/src/content/docs/setup/quick-start.mdx:149)

Minor Confusion

1. tools.timeout default (60s) is documented only for Claude and Codex; Copilot/Gemini/Pi defaults are unstated, leaving no comparison baseline. (docs/src/content/docs/reference/tools.md:239)
2. The CLI reference's add-wizard section spells out Copilot's auth-method decision tree (org billing vs PAT) in prose, but Claude/Codex/Gemini wizard behavior isn't given equivalent inline treatment — readers must cross-reference quick-start.mdx. (docs/src/content/docs/setup/cli.md:153)
3. Web-search/web-fetch default-disabled behavior differs silently per engine: declaring web-search for Gemini/Pi triggers only a compile warning and has no effect, which could confuse a user porting a Claude workflow. (docs/src/content/docs/reference/tools.md:126-132)

Engine & Tool Matrix

Engine	Setup	Examples	Auth	Score
Copilot	Two fully inline paths (org billing + PAT)	116 workflows, 10 smoke-auth variants	COPILOT_GITHUB_TOKEN or copilot-requests:write, fully inline	9/10
Claude	API key inline; WIF link-only; OAuth token explicitly unsupported	64 workflows, 6 smoke-auth variants	ANTHROPIC_API_KEY or WIF (link-out); CLAUDE_CODE_OAUTH_TOKEN rejected	6/10
Codex	API key inline, CODEX_API_KEY optional override	78 workflows, 3 smoke-auth variants	OPENAI_API_KEY or CODEX_API_KEY, straightforward	7/10
Custom	No scaffolding or dedicated auth docs found	0 declarations in current .github/workflows/*.md	N/A	n/a

Tool-level parity gaps feeding the above: gh aw init only scaffolds agent/MCP files for Copilot (cli.md:127-132); tools.timeout default is documented for Claude/Codex only (tools.md:239); web-search/web-fetch silently no-ops for Gemini/Pi (tools.md:126-132). Minor engines (aider, goose, opencode, cursor, crush, kiro, deepseek-harness, pydantic-ai, gemini) each follow a consistent 3-4 file pattern (engine-conformance-*.md, smoke-*.md, shared/*.md) despite low absolute counts (2-6), so they're evenly documented relative to each other — the real asymmetry is Copilot vs. everyone else.

Auth Gaps

• No supported path for Claude subscription users (claude login / CLAUDE_CODE_OAUTH_TOKEN); only API key or WIF, with subscription auth rejected via a confusing downstream error. (quick-start.mdx:151)
• No inline, step-by-step Anthropic WIF/OIDC walkthrough (trust policy, federation fields) comparable to Copilot's two detailed inline paths — only a summary paragraph plus a link. (quick-start.mdx:149)
• No guidance on replicating Copilot's auto-scaffolded custom agent/MCP wiring for Claude Code users who want the same "author workflows via agent" ergonomics. (cli.md:131-132)
• cli.md's add-wizard section documents only Copilot's auth-method decision tree in prose; Claude/Codex/Gemini wizard behavior isn't spelled out at the same level in the CLI reference. (cli.md:153)

Recommended Actions

Priority 1

• Add a prominent callout (not just prose) in both quick-start.mdx:151 and cli.md:232 that CLAUDE_CODE_OAUTH_TOKEN is unsupported, placed before the auth steps, not after — so Claude Pro/Max users don't attempt it first.

Priority 2

• Expand the Anthropic WIF section (quick-start.mdx:149) to inline the same level of step-by-step detail Copilot gets, or add a concrete example federation-rule JSON snippet.
• Provide a minimal Claude Code subagent template for gh aw init non-Copilot users (cli.md:131), even if not auto-scaffolded, so "author an agent file" isn't a blank-page problem.

Priority 3

• Document tools.timeout defaults for Copilot/Gemini/Pi alongside Claude/Codex (tools.md:239).
• Add 2-3 more Claude-specific smoke-test auth variants to close some of the 10-vs-6 example gap.

Generated by 📝 Claude Code User Documentation Review · claude · agent · 195.3 AIC · ⊞ 5.2K · ◷

• expires on Oct 9, 2026, 4:51 AM UTC-08:00

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-10-09T12:51:57.902Z.

Closed by Workflow