docs: add API contract verification to extension model skill#861
Merged
docs: add API contract verification to extension model skill#861
Conversation
Add an API contract verification phase and failure pattern to the
swamp-extension-model skill's smoke testing protocol. This addresses a
real bug where a GCP IAM extension model shipped with an incorrect POST
body that mocked unit tests couldn't catch — the endpoint required an
empty body but the model sent JSON.stringify({}).
Changes:
- Add Phase 0.5 (API Contract Verification) to smoke_testing.md with a
5-item checklist: endpoint URLs, HTTP methods, request bodies, response
schemas, and field naming conventions
- Add "API contract mismatches" to Common Failure Patterns in
smoke_testing.md with symptoms, causes, and fix guidance
- Add contextual reminder in SKILL.md's Smoke Testing section prompting
model authors to verify against provider REST API docs before pushing
Co-authored-by: Blake Irvin <bixu@users.noreply.github.com>
There was a problem hiding this comment.
Code Review
No blocking issues. Documentation-only change — no code, no runtime behavior, no domain model impact.
Suggestions
- Phase 0.5 naming: The "0.5" numbering works but could become awkward if another phase needs to be inserted later. Consider renaming phases to use whole numbers (0 → Pre-flight, 1 → API Contract Verification, 2 → List methods, etc.) in a future pass. Not worth doing in this PR since it would be a larger diff for no functional gain.
Otherwise the changes are well-placed and follow the existing Symptom/Cause/Fix format for failure patterns. The new phase sits logically between pre-flight setup and live API calls, and the SKILL.md reminder is a clean contextual nudge without being redundant with the reference doc.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #855
Adds API contract verification guidance to the
swamp-extension-modelskill's smoke testing protocol. This addresses a real bug where a GCP IAM extension model shipped withJSON.stringify({})as the POST body toserviceAccounts.getIamPolicy— an endpoint that requires an empty body per GCP docs. Mocked unit tests passed because they validated internal logic, not the actual API contract.Changes
smoke_testing.md) — New phase between Pre-flight and Phase 1 with a 5-item checklist: endpoint URL patterns, HTTP methods, request body schemas, response schemas, and field naming conventionssmoke_testing.md) — New entry in Common Failure Patterns covering wrong HTTP methods, incorrect request bodies, and field naming mismatches, with symptoms/causes/fix guidanceSKILL.md) — Prompt in the Smoke Testing section reminding model authors to verify against provider REST API docs before pushingWhy this is correct
The
swamp-extension-modelskill is what Claude follows when building extension models. By adding an API contract verification step to the skill, Claude will execute that step during model development — cross-referencing each endpoint URL, HTTP method, and request/response body against the provider's official docs. This catches the exact class of bug described in the issue (contract mismatches that mocked tests cannot detect) at the right point in the workflow (before push, after implementation).Impact
Test plan
🤖 Generated with Claude Code