Repository navigation
perf(bedrock): accumulate native block text in a strings.Builder - #4667
Merged
Merged
Conversation
Appending each streamed delta to an immutable string copied the whole block per delta, quadratic in response length. Text and reasoning deltas now go to a per-block builder, materialized once in state(). Signed-off-by: Arnaud Héritier <aheritier@apache.org>
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
The focused optimization preserves recording behavior, includes regression coverage, and has no unresolved findings.
No open issues 🎉
What changed in this PR
Optimizes Bedrock native-block recording by replacing repeated string concatenation with per-block builders, preserving captured content for replay.
Changes:
- Accumulates text and reasoning deltas in
strings.Builder. - Adds regression coverage for allocation counts and serialized block contents.
| File | Description |
|---|---|
pkg/model/provider/bedrock/native_blocks.go |
Buffers deltas and materializes text in state(). |
pkg/model/provider/bedrock/native_blocks_test.go |
Checks allocation limits and reconstructed blocks. |
Review effort: Balanced
1 of 6 tasks
dgageot
approved these changes
Oct 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #4666 (part of #4653): addresses the Copilot review comment #4666 (comment), which arrived as #4666 entered the merge queue.
What
The Bedrock native-block recorder appended each streamed text or reasoning delta to an immutable string, copying the whole block on every delta: quadratic in response length, on every Bedrock stream. Deltas now accumulate in a per-block
strings.Builder, materialized once instate().Test
TestBlockRecorder_AccumulatesDeltasWithoutPerDeltaCopiesrecords 1000 reasoning plus 1000 text deltas and asserts allocations stay below n/10. Before the change it measured 2002 allocations, one per delta. It also checks the rebuilt blocks.References
strings.Builderminimizes memory copying when building a string incrementallyValidation
task buildpasses.task testpasses.golangci-lintv2.14.0 reports 0 issues, andgo run ./lint .reports no offenses.go test -race -shuffle=on ./pkg/model/provider/... ./pkg/session/ ./pkg/chat/passes.All of this ran on the identical change before rebasing onto
main; after rebasing, the Bedrock tests and lint were rerun.