Skip to content

perf(bedrock): accumulate native block text in a strings.Builder - #4667

Merged
dgageot merged 1 commit into
mainfrom
perf/bedrock-native-blocks-builder
Oct 20, 2026
Merged

dgageot merged 1 commit into
mainfrom
perf/bedrock-native-blocks-builder

Conversation

@aheritier

Copy link
Copy Markdown
Collaborator

Follow-up to #4666 (part of #4653): addresses the Copilot review comment #4666 (comment), which arrived as #4666 entered the merge queue.

What

The Bedrock native-block recorder appended each streamed text or reasoning delta to an immutable string, copying the whole block on every delta: quadratic in response length, on every Bedrock stream. Deltas now accumulate in a per-block strings.Builder, materialized once in state().

Test

TestBlockRecorder_AccumulatesDeltasWithoutPerDeltaCopies records 1000 reasoning plus 1000 text deltas and asserts allocations stay below n/10. Before the change it measured 2002 allocations, one per delta. It also checks the rebuilt blocks.

References

Change Documentation
strings.Builder minimizes memory copying when building a string incrementally Go: strings.Builder

Validation

  • task build passes.
  • task test passes.
  • golangci-lint v2.14.0 reports 0 issues, and go run ./lint . reports no offenses.
  • go test -race -shuffle=on ./pkg/model/provider/... ./pkg/session/ ./pkg/chat/ passes.

All of this ran on the identical change before rebasing onto main; after rebasing, the Bedrock tests and lint were rerun.

Appending each streamed delta to an immutable string copied the whole block per delta, quadratic in response length. Text and reasoning deltas now go to a per-block builder, materialized once in state().

Signed-off-by: Arnaud Héritier <aheritier@apache.org>
Copilot AI balanced review requested due to automatic review settings October 20, 2026 09:29
@aheritier
aheritier requested a review from a team as a code owner October 20, 2026 09:29
@aheritier aheritier added area/providers/bedrock AWS Bedrock provider support kind/refactor PR refactors code without behavior change labels Oct 20, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The focused optimization preserves recording behavior, includes regression coverage, and has no unresolved findings.

No open issues 🎉

What changed in this PR

Optimizes Bedrock native-block recording by replacing repeated string concatenation with per-block builders, preserving captured content for replay.

Changes:

  • Accumulates text and reasoning deltas in strings.Builder.
  • Adds regression coverage for allocation counts and serialized block contents.
File Description
pkg/​model/​provider/​bedrock/​native_blocks.go Buffers deltas and materializes text in state().
pkg/​model/​provider/​bedrock/​native_blocks_test.go Checks allocation limits and reconstructed blocks.

Review effort: Balanced

@dgageot
dgageot added this pull request to the merge queue Oct 20, 2026
Merged via the queue into main with commit 003555a Oct 20, 2026
17 checks passed
@dgageot
dgageot deleted the perf/bedrock-native-blocks-builder branch October 20, 2026 09:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/providers/bedrock AWS Bedrock provider support kind/refactor PR refactors code without behavior change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants