Repository navigation
Image request cap ignores modelCapabilities.limits.vision.max_prompt_images, and its newest-first eviction invalidates the prompt cache on every request #2823
Agent suggestions
Public previewDescription
The runtime caps images per model request at 50 and evicts the overflow newest-first across the whole history. Two separable problems; the second is the expensive one and stands on its own.
1. max_prompt_images is accepted and ignored
modelCapabilities.limits.vision.max_prompt_images is declared in types.d.ts:2929, reachable via ModelCapabilitiesOverride, accepted by session.open and setModel, and declared in the runtime's own wire schema (api.schema.json → ModelCapabilitiesOverrideLimitsVision). Setting it to 1000000 on createSession changes nothing — the cap still applies at 50 and the runtime still emits:
Removed {n} images from the model request to stay within the 50-image request limit. User-provided images were prioritized over tool-generated images, with newer messages kept first.
Expected: honour it, or document it as ignored and reject it at the API boundary instead of silently accepting it.
Same defect class as #2756 (maxContextWindowTokens accepted and ignored, only an undocumented provider.* field works).
2. Newest-first eviction invalidates the prompt cache on every request
Eviction keeps the newest 50 images across the whole history. When tool results carry images the count keeps growing, so the retained set changes every request. Each change rewrites a message early in the array, so the cacheable prefix differs every time and the provider re-writes it instead of reading it.
No fixed-size window that keeps the newest entries can be prefix-stable. A sticky oldest-first policy — once dropped, stays dropped — would be, at the same cap.
Expected: eviction is monotonic and oldest-first, so mutations land at the tail rather than the head.
Measured (Anthropic via BYOK)
| cache-creation tokens | cost | |
|---|---|---|
| Unmodified SDK, one run | 16.0M | $120.63 |
| Comparable runs, median | — | ~$6 |
| Same paper, worked around in a loopback proxy | 375K | ~$24 |
43x fewer cache writes from the workaround; with it, cache creation is 1.0–1.2% of cached traffic.
Reproduction
- BYOK Anthropic session with
modelCapabilities: { limits: { vision: { max_prompt_images: 1000000 } } }. - Return images from a tool until the history exceeds 50.
- The removal warning appears — the override had no effect (1).
- Log outbound bodies: the retained image set, and therefore an early message, differs between consecutive requests;
cache_creation_input_tokensstays high andcache_read_input_tokensnear zero (2).
Notes
- 50 is a Copilot-side policy, not a provider limit — the Anthropic API accepted 120 images in one request in our probe. Preserve image replay compatibility across CLI image limits #2670 calls the cap an "intentional temporary policy change".
- Ask 2 is the unresolved half of One pasted image and claude-opus-5 won't look at any more images copilot-cli#4831, which hit this at the old cap of 1 and asked for old images to drop out rather than hold the slot. Raising 1 → 50 moved the threshold, not the retention policy.
- Not Bug: Copilot CLI 1.0.82 BYOK silently disables prompt caching (~5x cost) copilot-cli#4720 (there BYOK sends no cache declaration at all). Here the declaration is present and defeated by prefix mutation — same shape as BYOK: autopilot nudge turn re-serializes previously-sent transcript items, breaking prompt caching copilot-cli#4500 and SDK 0.3.0 regression: per-turn mutating state in <system_reminder> block defeats Anthropic prompt cache #1296, both fixed.
Environment: @github/copilot-sdk 1.0.15, copilot-sdk-linux-x64 1.0.15, Linux x64, BYOK Anthropic.
Activity
github-actions commentedon Oct 8, 2026
Thanks for the detailed report. I labeled this
bug.What I checked: I searched this repo for
max_prompt_images/maxPromptImages. It appears only in the generated RPC types (Node, Python, Go, .NET, Rust, Java), in the hand-written per-SDKModelCapabilitiesOverride/ vision-limits types, and indocs/features/image-input.md. The SDKs only pass the override through in thesession.open/setModelpayloads. No SDK code enforces or drops the 50-image cap, and the "Removed {n} images..." message is not generated here.Findings:
modelCapabilities.limits.maxContextWindowTokensis ignored by the runtime; only the undocumentedprovider.maxContextWindowTokensworks #2756. I couldn't reproduce the runtime behavior from this repo, since the runtime source isn't here.Next steps: The fix belongs in the runtime (
github/copilot-cli), so this needs routing to the runtime team. Please link it with github/copilot-cli#4831. On the SDK side, once the runtime's behavior is settled,docs/features/image-input.mdshould document the image cap and whethermax_prompt_imagesis honoured. Tell us if you have reproduction logs of the outbound request bodies. Those would help the runtime team.IslamTayeb commentedon Oct 10, 2026
Routed to the runtime as github/copilot-cli#5111, which carries this full report, since #4831 there is closed. Please follow up on that issue; this comment stays as the SDK-side record.
Linked with github/copilot-cli#4831 as requested, with a comment there pointing back here. That issue was auto-closed as resolved in CLI v1.0.88, which raised the cap from 1 to 50; the retention half it asked for ("some way for an old image to drop out instead of holding the slot") is what this issue is about.
Reproduction logs of the outbound request bodies
https://gist.github.com/IslamTayeb/46cfa44887d318ecec4efe1d3fabb44a
probe.mjs— the reproduction.@github/copilot-sdk1.0.15 +copilot-sdk-linux-x641.0.15, Node v25.2.1,claude-opus-5, BYOK Anthropic through a loopback proxy that records every outbound/v1/messagesbody.createSessionpassesmodelCapabilities: { supports: { vision: true }, limits: { vision: { max_prompt_images: 1000000 } } }. The workspace holds 61 tiny 8×8 PNGs; the model is asked toviewtile_01 … tile_60 (10 per response), then replyok(no new image), thenviewtile_61.request_bodies.json— all 10 request bodies as the runtime sent them. Image base64 is replaced bysha256+ length, the Copilot system prompt and tool schemas by digests; messages are otherwise verbatim,cache_controlmarkers included. Each entry also carries theassistant.usageevent for that request.session_events.jsonl—model.log,assistant.usage,assistant.message,tool.execution_startevents.What the bodies show:
1.
max_prompt_imagesis ignored. With the override at 1,000,000, request 7 (60 images in history) carries exactly 50 and the runtime loggedRemoved 10 images from the model request to stay within the 50-image request limit…. The sametool_resultin message index 2, request 6 → request 7:{"type":"tool_result","tool_use_id":"toolu_011jisLSffC4K4nSk18SbGs5","content":[{"type":"text","text":"Viewed image file successfully."},{"type":"image","source":{"type":"base64","media_type":"image/png","data":"<sha256:06e76d363b3c678f base64_chars:100>"}}]}{"type":"tool_result","tool_use_id":"toolu_011jisLSffC4K4nSk18SbGs5","content":[{"type":"text","text":"Viewed image file successfully."},{"type":"text","text":"You've reached the maximum number of images you can view (50) so I can't provide the image for you to see."}]}2. Each admission past the cap rewrites an already-sent message. The replacement lands in the oldest tool batch (message 2 at request 7, message 4 at request 10), so the cacheable prefix ends there:
cacheReadTokensdrops from 20,697 to 13,912 (system prompt + tools + first user message) and the rest is re-written. Requests 8 and 9 add no image and nothing is rewritten (48 and 58 write tokens). Request 10 admits one more image, replaces one more oldtool_result, and rewrites again.3. A correction to my ask 2. The bodies show the runtime's eviction is already oldest-first and sticky: the dropped set goes 10 → 10 → 10 → 11 and never shrinks or reorders. So the problem is not that the retained set is unstable between requests; it is that every image admitted past the cap replaces one
tool_resultthat the provider has already cached. A "sticky oldest-first" policy is what runs today and does not help. The only prefix-stable behaviours are (a) no eviction, i.e. honourmax_prompt_images(our workaround: a proxy that puts the evicted images back from earlier requests; the Anthropic API accepted 120 images in one request), or (b) refuse the new image at the tail and keep the old ones, which starves the model of what it just rendered. Which one is the runtime team's call; (a) is the one the SDK already advertises.4. The warning under-reports. It fired on every eviction here (4 of 4), but in the production run below it fired twice for 51 evictions, and
runtime.nodecontainsmark_image_limit_warning_emitted, so counting warnings does not measure the problem.Same mechanism at production scale (same SDK, no override)
Two runs of the same paper on the same harness; the only difference is whether our proxy restored the evicted images before the request reached the API.
In the first run every one of the 51 requests that followed an image arrival past the cap shows
cacheReadTokens≤ 16,144 (system + tools) and a 189K–381K write; no other request does. Those 51 rewrites are 15.68M of the 16.04M write tokens. (This is a different replicate from the table in the issue body, hence the slightly different numbers.)