Repository navigation
buffer: document evacuate_limit_size - #715
Merged
Merged
Conversation
fluent/fluentd#5473 adds evacuate_limit_size to limit the total size of the evacuated chunk files. Describe how chunks are chosen, why the total can exceed the limit, and why @id is needed. Signed-off-by: Shizuo Fujita <fujita@clear-code.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
✅ Deploy Preview for fluentd-docs ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
Watson1978
added a commit
to fluent/fluentd
that referenced
this pull request
Oct 20, 2026
… the disk (#5473) **Which issue(s) this PR fixes**: Fixes #5352 **What this PR does / why we need it**: When the retry limit is reached, all queued chunks are evacuated to `${root_dir}/buffer/${plugin_id}/`, and a long outage repeats this until the disk is full (#5352). This PR adds `evacuate_limit_size`, which limits the total size of the files in that directory, including the ones evacuated before. Chunks that do not fit are purged without being evacuated, `0` disables evacuation, and the default `nil` keeps the current behavior. **Docs Changes**: Describe `evacuate_limit_size` in "Handling Successive Failures" of buffer/README.md. It is an approximate limit. The total can exceed it by the `.meta` files of the evacuated chunks, or when multiple workers of the same plugin evacuate chunks at the same time. Chunks are evacuated from the oldest, so once the directory reaches the limit, newer chunks are not evacuated until the evacuated files are moved or removed. `@id` is recommended, and the parameter is ignored by the memory buffer. I will open a PR on fluentd-docs-gitbook. fluent/fluentd-docs-gitbook#715 **Release Note**: buffer: add `evacuate_limit_size` to limit the total size of the evacuated chunk files. Set `0` to disable evacuation. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: Chris (ChrisJr404) <11917633+ChrisJr404@users.noreply.github.com> Signed-off-by: Shizuo Fujita <fujita@clear-code.com> Co-authored-by: Chris (ChrisJr404) <11917633+ChrisJr404@users.noreply.github.com> Co-authored-by: Shizuo Fujita <fujita@clear-code.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
kenhys
requested changes
Oct 20, 2026
"disable the evacuation" did not say what happens to the queued chunks. Signed-off-by: Shizuo Fujita <fujita@clear-code.com> Co-authored-by: Kentaro Hayashi <kenhys@gmail.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
kenhys
approved these changes
Oct 20, 2026
Contributor
Author
|
Thanks |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fluent/fluentd#5473 adds
evacuate_limit_sizeto limit the total size of the evacuated chunk files, and the docs assume that it is released in v1.20.0. This describes the parameter in "Handling Successive Failures" and the buffer section, including that it is an approximate limit, that newer chunks are not evacuated once the directory is full, and that@idis needed. It also notes for buffer plugin authors where the base class counts the evacuated files.This should be merged after fluent/fluentd#5473.
🤖 Generated with Claude Code