docs: add skill review guidance to CLAUDE.md#460
Open
vanceingalls wants to merge 1 commit intomainfrom
Open
Conversation
Comment on lines
+72
to
+88
| ### Reviewing skill changes | ||
|
|
||
| When reviewing a changeset that touches `skills/`, review as an agent-instruction | ||
| designer who has shipped many skills and watched agents fail on each one. Focus on: | ||
|
|
||
| - **Where will agents misinterpret?** Ambiguous vocabulary, instructions that assume | ||
| context the agent won't have 200 lines later, field names that don't match across | ||
| files (e.g., a picker emits "Canvas" but an eval checks for "bg"). | ||
| - **Where will agents silently skip?** Soft suggestions agents will deprioritize, | ||
| gates that lack enforcement, steps with no verification that they ran. | ||
| - **Where will agents produce wrong output on first attempt?** Missing examples | ||
| (agents need to see the shape, not just read the rules), under-specified formats, | ||
| instructions that require multi-step reasoning to combine. | ||
|
|
||
| Assume the agent reads the full SKILL.md before authoring and has no memory | ||
| between compositions. Consider: will this instruction survive in a 300+ line | ||
| file? Will the agent connect Step 0's context to a quality check 250 lines later? |
Collaborator
There was a problem hiding this comment.
Also add a ci check that each skill change should have a SbS HTML attached comparing prev skill/s vs updated skill/s
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
skills/diffsWhy
Skill review requires a different mindset than code review — instructions that look correct to a human reader can still fail when an agent processes them. This codifies the review heuristics we've been applying manually.
Test plan
🤖 Generated with Claude Code