[IoTV2] Add ConfigNode-side consensus pipe guardian and remove deprecated DataNode-side checking#17277
Open
Pengzna wants to merge 4 commits intoapache:masterfrom
Open
[IoTV2] Add ConfigNode-side consensus pipe guardian and remove deprecated DataNode-side checking#17277Pengzna wants to merge 4 commits intoapache:masterfrom
Pengzna wants to merge 4 commits intoapache:masterfrom
Conversation
jt2594838
approved these changes
Mar 11, 2026
...rc/test/java/org/apache/iotdb/confignode/persistence/pipe/PipeTaskInfoConsensusPipeTest.java
Outdated
Show resolved
Hide resolved
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
NOTE: This PR is genereated by Claude, I proposed the entire plans and prompts and have carefully reviewed this PR.
Summary
This PR introduces a periodic consensus pipe reconciliation mechanism on ConfigNode and removes the now-redundant detection-only checking logic on DataNode.
As a follow-up to #17233 (which delegated consensus pipe lifecycle management from DataNode to ConfigNode), DataNode-side checking only logged warnings like "should be created/dropped by ConfigNode" but never performed actual repairs. With ConfigNode now owning the full lifecycle, we add a proactive guardian on ConfigNode that can create missing pipes, drop unexpected pipes, and restart stopped pipes, then clean up the obsolete DataNode code.
Changes
1. ConfigNode: Add consensus pipe guardian (
2bd1c26b)PipeMetaSyncer: Added aconsensusPipeCheckRoundCounterthat triggerscheckAndRepairConsensusPipes()every 5 sync rounds.PipeTaskCoordinator/PipeTaskInfo: AddedgetConsensusPipeStatusMap()to query all consensus pipe names and their current status fromPipeMetaKeeper.RegionMaintainHandler.checkAndRepairConsensusPipes(): The core reconciliation logic that:PartitionManagerreplica sets (all node-pair permutations per DataRegion)PipeTaskInfo2. Unit tests (
ff3d8e39)PipeTaskInfoConsensusPipeTest: TestsgetConsensusPipeStatusMap()— verifying filtering by pipe type (CONSENSUS vs USER vs SUBSCRIPTION), correct status mapping, and multi-region scenarios.RegionMaintainHandlerConsensusPipeTest: TestscheckAndRepairConsensusPipes()with Mockito — covering no-op for non-IoTConsensusV2, all-pipes-match, create-missing, drop-unexpected, restart-stopped, mixed scenarios, and 3-node replica sets.3. Remove deprecated DataNode-side code (
5c93839b)Deleted files:
ConsensusPipeDataNodeRuntimeAgentGuardian.javaConsensusPipeGuardian.java(interface)ConsensusPipeSelector.java(interface)Cleaned up:
PipeConsensus.java— removed guardian field,checkAllConsensusPipe()method, and guardian start/stop callsPipeConsensusServerImpl.java— removedcheckConsensusPipe()methodPipeConsensusConfig.java— removed guardian, selector, and interval config fromPipeclass and its builderDataRegionConsensusImpl.java— removed guardian/selector/interval configurationPipeDataNodeTaskAgent.java— removedgetAllConsensusPipe()methodTest Plan
PipeTaskInfoConsensusPipeTestpasses (5 test cases)RegionMaintainHandlerConsensusPipeTestpasses (7 test cases)