Repository navigation
Conversation
Contributor
|
Hi @craffit, thanks for the PR! I added you as contributor in ASF Jira and assigned the ticket to you. |
…ains more than one ServerCrashProcedure for the same server Merge duplicate SCPs per server with Math::min instead of throwing on the duplicate key. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Author
|
@petersomogyi thanks for pointer. I rebased onto master and retargeted this PR. Let me know what more i can do to facilitate review/improvements. Also happy to offer backported PRs if that is useful. |
Contributor
|
@craffit For some reason Yetus does not handle correctly this PR due to the changed base branch. Could you open a new PR? |
Author
|
@petersomogyi i opened a new PR under #8765 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
https://issues.apache.org/jira/browse/HBASE-30463
Since HBASE-28158,
HMaster#finishActiveMasterInitializationcollects the unfinishedServerCrashProcedures with
Collectors.toMap(ServerCrashProcedure::getServerName, Procedure::getSubmittedTime)and no merge function. If the procedure store holds twounfinished SCPs for the same
ServerName, the collector throwsIllegalStateException: Duplicate key ...and the master aborts. It aborts again onevery restart, because the same store is reloaded. Before HBASE-28158 these procedures
were collected into a
Set, which tolerated duplicates.Duplicate SCPs can be persisted because the submission guards rely on in-memory
DeadServerstate.DeadServer#cleanPreviousInstanceremoves the old server name whileits SCP is still running, as soon as a new instance on the same host and port reports in.
The fix adds
Math::minas the merge function. That keeps the earliest submit time asthe crash time, matching
DeadServer#putIfAbsent. Both SCPs are still loaded and runone after the other, because they take the same exclusive server lock.
TestMasterStartupWithDuplicateSCPsholds the server lock so that two SCPs for the sameserver stay unfinished in the store. It then restarts the master and verifies that the
master initializes and both procedures complete.
Affects 2.5.14+, 2.6.5+, 2.7.0 and 3.0.0. This patch is on master
🤖 Generated with Claude Code