Claude Setup Audit

Comprehensive quality assessment of Claude Code configuration and components in a repository. Evaluates skills, agents, CLAUDE.md rules, commands, hooks, and MCP configuration against established best practices. Produces a graded health report with concrete good/bad example references and actionable fixes.

Assessment Scope

| Component | Primary Locations | Max Score | |-----------|------------------|-----------| | Skills | .claude/skills/*/SKILL.md, plugins/*/skills/*/SKILL.md | 32 pts each | | Agents | .claude/agents/*.md, plugins/*/agents/*.md | 32 pts each | | CLAUDE.md | CLAUDE.md, .claude/CLAUDE.md, **/CLAUDE.md (subdirs), CLAUDE.local.md | 36 pts each | | Commands | .claude/commands/**/*.md, plugins/*/commands/**/*.md | 20 pts each | | Hooks | .claude/settings.json (hooks array) | 20 pts total | | MCP Config | .mcp.json, .claude/mcp.json | 20 pts |

5-Phase Workflow

Phase 1: Discovery

Run scripts/scan-repo.sh or manually scan:

find . \( \
  -path '*/.claude/agents/*.md' -o \
  -path '*/skills/*/SKILL.md' -o \
  -path '*/.claude/commands/**/*.md' -o \
  -name 'CLAUDE.md' -o \
  -name 'CLAUDE.local.md' -o \
  -name '.mcp.json' -o \
  -path '*/.claude/settings.json' \
\) -not -path '*/node_modules/*' 2>/dev/null

CLAUDE.md knowledge organisation scan (run separately):

# Find all CLAUDE.md files
find . -name 'CLAUDE.md' -o -name 'CLAUDE.local.md' \
  | grep -v node_modules | sort

# Find dedicated guide directories
find . -type d \( -name 'guides' -o -name 'references' -o -name '.codemie' \) \
  | grep -v node_modules | sort

Detect which pattern is in use — both are valid, mutually exclusive:

| Pattern | Signals | Scoring note | |---------|---------|-------------| | A — Single CLAUDE.md + Guide Files | 1 CLAUDE.md at root + guide files in guides/, .codemie/guides/, references/, docs/ | R5.3 full credit if CLAUDE.md references guide paths | | B — Hierarchical CLAUDE.md | Multiple CLAUDE.md files at different directory depths | R5.3 N/A (hierarchy IS the delegation); run hierarchy checks |

Print discovery summary:

Found: X skills, Y agents, Z commands
CLAUDE.md pattern: A (single + guides) | B (hierarchical) | unknown
CLAUDE.md locations: [list each path found]
Guide dirs: [list if present]
Hooks: ✓/✗ | MCP: ✓/✗

Phase 2: Per-Component Scoring

Apply checklists from references/component-checklists.md. Mark each criterion:

✅ Pass — full points
⚠️ Partial — half points (present but incomplete or weak)
❌ Fail — 0 points

Skills — classify type first, then score:

Before scoring any skill, detect its type:
grep -i "codemie.*assistant\|codemie.*chat\|assistants chat" SKILL.md
Match found → Codemie-delegating skill (API wrapper): applies reduced rubric (21 pts max)

No match → Standard skill: full 32-pt rubric applies

Standard Skills (32 pts max):

Structure (3×): Valid SKILL.md filename, [a-z0-9-]+ name, description >20 chars, allowed-tools field
Content (2×): Methodology/workflow section, output format, examples, checklists (- [ ])
Technical (1×): No absolute paths, no hardcoded UUIDs/secrets, dependencies documented
Design (2×): Single responsibility, "When to use" triggers, no >50% overlap, <8K tokens

Codemie-delegating Skills (21 pts max — exemptions apply):

Structure (3×): Valid SKILL.md, valid name, description >20 chars — allowed-tools is N/A (no local tool execution)
Content: All content criteria N/A — output format, workflow, examples, checklists are determined by the backend assistant
Technical (1×): No absolute paths, no secrets, deps documented
Design (2×): Single responsibility, "When to use" triggers, no overlap, token budget
Score = (pts obtained / 21) × 100 — do not penalise for exempted criteria

Agents (32 pts max):

Identity (3×): Descriptive name, description has "when"/"use"/"trigger" keywords, model specified, tools justified
Prompt Quality (2×): "You are…" role statement, output format section, scope/limits, anti-hallucination keywords
Validation (1×): 3+ examples, edge cases, error handling described
Design (2×): Single responsibility, composable (references skills), <8K tokens

CLAUDE.md Rules (36 pts max):

Coverage (3×): Coding standards, workflow policies, explicit critical rules with triggers
Clarity (3×): Actionable instructions, trigger conditions stated, prohibitions explicit
Structure (2×): Section organization, quick-reference tables, correct/incorrect pattern examples
Maintenance (1×): No contradictions, technology versions noted, project context stated
Efficiency (2×): Concise body (≤400 non-code words), no anti-pattern noise, delegates detail via @import

CLAUDE.md hierarchy — score each file separately, then run hierarchy analysis (Phase 3).

Commands (20 pts max):

Structure (3×): Frontmatter (name + description), argument-hint if uses $ARGUMENTS, numbered phases, usage examples
Quality (2×): Error/failure handling, output format defined, validation gates, argument parsing shown

Hooks (20 pts max):

Validity (3×): Valid JSON, recognized event types, valid matchers
Security (2×): No hardcoded credentials, bash scripts use set -e/trap
Quality (2×): Hook purpose documented, tool restriction scope appropriate

MCP Config (20 pts max):

Structure (3×): Valid JSON, mcpServers key, each server has command/url + type
Security (2×): No hardcoded tokens/passwords, env vars used for credentials
Documentation (2×): Descriptive server names, required env vars listed

Phase 3: Cross-Component Analysis

After individual scoring, check repository-wide health:

Coverage gaps — CLAUDE.md present if agents exist? Skills referenced by agents available?
Naming consistency — All files use lowercase-kebab-case?
Duplication — Any agents/skills with >50% description keyword overlap?
Security sweep — Grep all .claude/ content for /Users/, /home/, password=, api_key, token=
Integration — Agents reference the skills they use? Commands invoke relevant agents?
CLAUDE.md knowledge organisation analysis — see checklist in references/component-checklists.md (Pattern A and B checks are separate)

Phase 4: Grading

Component Score = (Points Obtained / Max Points) × 100
Overall Score   = Average of all component scores

| Grade | Range | Label | |-------|-------|-------| | A | 90–100% | Production-ready | | B | 80–89% | Good — meets production threshold | | C | 70–79% | Needs improvement | | D | 60–69% | Significant gaps | | F | <60% | Critical issues — rewrite needed |

Production threshold: Grade B (≥80%) required per component.

Phase 5: Report Generation

Generate a report following examples/sample-report.md structure.

For each failed criterion, cite the relevant example:

❌ No role statement ("You are...") found
   → See examples/bad-agent.md (line 10) vs examples/good-agent.md (line 8)

Prioritize findings:

🔴 Critical (fix immediately): Security issues OR grade F
🟡 Important (fix soon): Grade D–C OR missing key structural elements
🔵 Minor (polish): Grade B OR optional improvements

Quick Wins (High-Impact, Low-Effort Fixes)

| Issue | Score Impact | Fix Effort | |-------|-------------|------------| | Missing model: in agent | +3 pts | 1 line | | No allowed-tools in standard skill | +3 pts | 1 line |

Reference Files

Scoring Details

references/component-checklists.md — Full rubric: all criteria, point values, detection patterns
references/best-practices.md — Rationale per criterion, anti-patterns to avoid, quick-fix examples

Examples

| Component | Good Example | Bad Example | |-----------|-------------|-------------| | Skill | examples/good-skill.md | examples/bad-skill.md | | Agent | examples/good-agent.md | examples/bad-agent.md | | CLAUDE.md | examples/good-claude-md-snippet.md | examples/bad-claude-md-snippet.md | | Command | examples/good-command.md | examples/bad-command.md | | Hooks config | examples/good-hooks.json | examples/bad-hooks.json | | Full report | examples/sample-report.md | — |

Utilities

scripts/scan-repo.sh — Discover and inventory all Claude Code components