Skip to content

Document exact AI reviewer deviation label formats#197

Merged
igerber merged 2 commits intomainfrom
docs/deviation-label-formats
Mar 14, 2026
Merged

Document exact AI reviewer deviation label formats#197
igerber merged 2 commits intomainfrom
docs/deviation-label-formats

Conversation

@igerber
Copy link
Owner

@igerber igerber commented Mar 14, 2026

Summary

  • Add "Documenting Deviations (AI Review Compatibility)" section to CLAUDE.md with the exact label patterns the AI reviewer recognizes
  • Update dev-checklists.md methodology item to specify reviewer-recognized labels instead of generic "add a Note"
  • Add Section 2.5 (Methodology Documentation Check) to pre-merge-check.md as a co-change warning
  • Add Methodology Deviation Documentation checklist under "If Methodology Files Changed"
  • Remove redundant REGISTRY.md line from Documentation Sync checklist (now covered by methodology checklist)

Methodology references (required if estimator / math changes)

  • N/A — no methodology changes

Validation

  • Tests added/updated: No test changes (documentation-only PR)
  • Backtest / simulation / notebook evidence: N/A

Security / privacy

  • Confirm no secrets/PII in this PR: Yes

Generated with Claude Code

Add recognized REGISTRY.md label patterns (`**Note**:`, `**Deviation from R**:`,
`**Note (deviation from R):**`) to CLAUDE.md, dev-checklists.md, and
pre-merge-check.md so that deviations are documented in formats the AI reviewer
recognizes, avoiding unnecessary P1 findings.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@github-actions
Copy link

Overall Assessment

⚠️ Needs changes

Executive Summary

  • No estimator, weighting, variance/SE, identification, or default-behavior code changed; the risk here is whether the new documentation accurately states the reviewer contract.
  • The new “exact reviewer-recognized labels” guidance is wrong in three places: it documents **Note**: and **Deviation from R**: instead of **Note:** and **Deviation from R:**.
  • Because the guidance is framed as exact, following it can still leave a methodology deviation unmitigated and trigger a P1 “undocumented methodology deviation” review finding.
  • The new TODO guidance is aligned with the current table structure and categories in TODO.md:38-60.
  • No runtime, performance, or security issues were introduced by this docs-only PR.

Methodology

  • Severity: P1. Impact: CLAUDE.md:94-106, .claude/commands/dev-checklists.md:48-55, and .claude/commands/pre-merge-check.md:115-120 plus .claude/commands/pre-merge-check.md:152-155 all claim the reviewer-recognized labels are **Note**: and **Deviation from R**:. This PR is explicitly documenting the exact accepted formats, but those strings do not match the review contract used here (**Note:**, **Deviation from R:**, **Note (deviation from R):**). A contributor who follows the new docs can still get a P1 “undocumented methodology deviation” finding. Concrete fix: replace the first two label spellings everywhere in this PR with the exact accepted strings **Note:** and **Deviation from R:**, keep **Note (deviation from R):** as-is, and use literal copy/paste examples in CLAUDE.md rather than variant formatting.

Code Quality

No findings.

Performance

No findings.

Maintainability

No findings.

Tech Debt

No findings.

Security

No findings.

Documentation/Tests

No additional findings beyond the methodology-documentation mismatch above. This is a docs-only PR; I did not see a separate test gap created by the changed behavior once the label strings are corrected.

Path to Approval

  1. Update CLAUDE.md, .claude/commands/dev-checklists.md, and .claude/commands/pre-merge-check.md to use the exact reviewer-recognized label spellings: **Note:**, **Deviation from R:**, and **Note (deviation from R):**.
  2. Re-scan the changed text for any remaining **Note**: / **Deviation from R**: variants so the PR no longer claims exact support for unrecognized forms.

The AI reviewer contract expects `**Note:**` and `**Deviation from R:**`
(colon inside bold), not `**Note**:` and `**Deviation from R**:` (colon
outside). Fixes P1 finding from AI review.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@igerber
Copy link
Owner Author

igerber commented Mar 14, 2026

/ai-review

@github-actions
Copy link

🔁 AI review rerun (requested by @igerber)

Head SHA: 9eac65058bac1cdb923ea7557e46c27138b15f6d


Overall Assessment

✅ Looks good

Executive Summary

Methodology

Code Quality

  • No findings. The edits are internally consistent, and removing the redundant REGISTRY reminder from the generic documentation checklist is a reasonable deduplication.

Performance

  • No findings. Documentation-only change.

Maintainability

  • No findings. Centralizing the reviewer-compatible label guidance in CLAUDE.md:L94-L115 should reduce future drift across contributor instructions and review checklists.

Tech Debt

  • No findings. The added deferral instructions align with the current TODO table layout and category split in TODO.md:L38-L60.

Security

  • No findings. No executable code, dependency surface, or secret-handling paths changed.

Documentation/Tests

  • No findings. For a docs-only PR, the lack of test changes is acceptable. Residual risk is limited to future doc/reviewer drift, and this PR reduces that risk rather than introducing it.

@igerber igerber merged commit f1fdda3 into main Mar 14, 2026
@igerber igerber deleted the docs/deviation-label-formats branch March 14, 2026 20:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant