Baseline Check Skill

Purpose

This skill maintains code quality throughout development by:

Establishing baseline quality metrics before development
Detecting regressions after code changes with automated delta analysis
Tracking quality trends over time
Providing clear, actionable feedback on code health with visual indicators

When to Use

Invoke this skill:

Before starting development work - Establish current baseline (capture mode)
After completing changes - Check for regressions (compare or auto mode)
After GREEN phase - Verify tests exist and pass
After REFACTOR phase - Verify tests still pass after refactoring
Before committing code - Verify quality standards (compare mode)
When explicitly requested - User asks for quality check, baseline, or regression analysis
During code review - Validate changes meet quality bar

Execution Modes

The skill supports three execution modes for different scenarios:

Mode 1: Capture (Default)

Purpose: Establish a quality baseline

When to use:

Starting new development work
After syncing with main branch
Before making changes
Creating a reference point for later comparison

Command:

./.claude/skills/baseline-check/baseline-check.sh --capture [--issue SPI-XXX]

What it does:

Runs ./gradlew clean check --rerun-tasks (comprehensive, cache-proof execution)
Captures test metrics (total, passed, failed, skipped)
Captures detekt metrics (offenses by severity)
Stores JSON + log file in .claude/baseline/
Creates latest symlink for easy access

Output:

JSON file: .claude/baseline/{branch}-{timestamp}.json
Log file: .claude/baseline/{branch}-{timestamp}.log
Symlink: .claude/baseline/latest-{branch}.json

Mode 2: Compare

Purpose: Compare current state against a specific baseline

When to use:

After completing implementation
Before creating a pull request
Verifying no regressions introduced
Explicit regression check requested

Command:

# Auto-detect latest baseline for current branch
./.claude/skills/baseline-check/baseline-check.sh --compare

# Compare against specific baseline file
./.claude/skills/baseline-check/baseline-check.sh --compare .claude/baseline/main-20241120-100000.json

What it does:

Runs comprehensive tests (fresh execution)
Auto-detects latest baseline if not specified
Calculates deltas (total, passed, failed, skipped)
Classifies changes (new tests, fixed tests, regressions)
Generates visual comparison report with interpretation
Returns exit code 1 if regressions detected

Output:

═══════════════════════════════════════════════════════════
  BASELINE vs CURRENT COMPARISON
═══════════════════════════════════════════════════════════

BASELINE:
  • Total: 861 | Passed: 861 | Failed: 0 | Skipped: 0

CURRENT:
  • Total: 873 | Passed: 868 | Failed: 5 | Skipped: 0

DELTA (Current - Baseline):
  • Total: +12 | Passed: +7 | Failed: +5 | Skipped: 0

───────────────────────────────────────────────────────────
  INTERPRETATION
───────────────────────────────────────────────────────────
  ✅ 12 new tests added
  ❌ 5 NEW test failures introduced
     📋 Analyze if failures are:
        - Expected (testing edge cases, deprecated endpoints)
        - Bugs requiring fixes
        - Pre-existing issues incorrectly attributed

═══════════════════════════════════════════════════════════

Mode 3: Auto (Autonomous)

Purpose: Capture baseline and automatically compare if previous exists

When to use:

Continuous quality monitoring
Agent-driven autonomous checks
When unsure if baseline exists
Workflow automation

Command:

./.claude/skills/baseline-check/baseline-check.sh --auto [--issue SPI-XXX]

What it does:

Captures current baseline (same as capture mode)
Checks if previous baseline exists for this branch
If found, automatically performs comparison
Reports both baseline capture and comparison results
Returns exit code 1 if regressions detected

Decision logic:

First run on branch: Only captures baseline, no comparison
Subsequent runs: Captures baseline AND compares against previous

Command-Line Reference

# Capture baseline (default mode)
./.claude/skills/baseline-check/baseline-check.sh
./.claude/skills/baseline-check/baseline-check.sh --capture
./.claude/skills/baseline-check/baseline-check.sh --capture --issue SPI-866

# Compare against baseline
./.claude/skills/baseline-check/baseline-check.sh --compare
./.claude/skills/baseline-check/baseline-check.sh --compare .claude/baseline/main-20241120-100000.json

# Autonomous mode (capture + optional compare)
./.claude/skills/baseline-check/baseline-check.sh --auto
./.claude/skills/baseline-check/baseline-check.sh --auto --issue SPI-866

# Adjust timeout (default: 300 seconds)
./.claude/skills/baseline-check/baseline-check.sh --capture --timeout 600

# Show help
./.claude/skills/baseline-check/baseline-check.sh --help

JSON Output Format

The skill generates structured JSON for programmatic access:

{
  "timestamp": "2025-11-20T14:30:22Z",
  "branch": "feat/spi-866-baseline-enhancement",
  "commit": "abc123d",
  "issue_id": "SPI-866",
  "checks": {
    "detekt": {
      "available": true,
      "status": "passed",
      "total_offenses": 5,
      "offenses_by_severity": {
        "convention": 2,
        "warning": 3,
        "error": 0
      },
      "execution_time": 8
    },
    "tests": {
      "available": true,
      "status": "passed",
      "total": 873,
      "passed": 873,
      "failures": 0,
      "skipped": 0,
      "execution_time": 145
    }
  },
  "overall_status": "passed",
  "summary": "All quality checks passed"
}

File Storage Structure

.claude/baseline/
├── main-20241120-100000.json          # Metrics JSON
├── main-20241120-100000.log           # Full console log
├── main-20241120-110000.json          # Later baseline
├── main-20241120-110000.log           # Later log
├── latest-main.json -> main-20241120-110000.json  # Symlink to latest
├── feat-spi-866-20241120-120000.json  # Feature branch baseline
├── feat-spi-866-20241120-120000.log   # Feature branch log
└── latest-feat-spi-866.json           # Feature branch latest symlink

Key features:

Dual storage: JSON (metrics) + log (full output)
Timestamped: Sortable, preserves history
Issue-based naming: Optional --issue flag for Linear integration
Latest symlinks: Easy access to most recent baseline per branch

Interpretation Guidelines

Visual Indicators

The skill uses these indicators in comparison reports:

✅ Improvements - Metrics got better (tests fixed, offenses reduced)
⚠️ Changes - Neutral changes (tests removed, skipped tests changed)
❌ Regressions - Serious issues (new failures, error offenses)
Stable - No change in metrics

Delta Classifications

New Tests Added (Positive):

Delta Total > 0 indicates new test coverage

Tests Fixed (Positive):

Delta Failures < 0 indicates previously failing tests now pass

Regressions (Critical):

Delta Failures > 0 indicates NEW test failures introduced
Script exits with code 1 (blocks commit)

Tests Removed (Warning):

Delta Total < 0 may indicate deleted tests (review needed)

Regression Analysis

When regressions are detected, the report provides:

Clear identification: "❌ 5 NEW test failures introduced"
Classification guidance: Expected vs bugs vs misattribution
Actionable next steps: Review logs, fix failures, re-run check
Exit code 1: Signals failure for CI/CD integration

Workflow Examples

Example 1: Starting New Feature (Capture Mode)

# Agent invokes before starting work
./.claude/skills/baseline-check/baseline-check.sh --capture --issue SPI-866

# Output shows baseline established
# Agent proceeds with implementation
# After implementation, agent runs compare mode

Agent prompt:

Before starting SPI-866, I'll establish a quality baseline to track regressions.

Running baseline check...

✓ Baseline captured: 861/861 tests passing, 5 detekt offenses
✓ Saved to: .claude/baseline/main-20241120-140000.json

Proceeding with implementation...

Example 2: Post-Implementation Check (Compare Mode)

# Agent invokes after completing work
./.claude/skills/baseline-check/baseline-check.sh --compare

# Output shows delta analysis
# Agent reports findings to user

Agent prompt:

Implementation complete. Running regression check against baseline...

═══════════════════════════════════════════════════════════
  BASELINE vs CURRENT COMPARISON
═══════════════════════════════════════════════════════════

BASELINE:
  • Total: 861 | Passed: 861 | Failed: 0 | Skipped: 0

CURRENT:
  • Total: 873 | Passed: 873 | Failed: 0 | Skipped: 0

DELTA (Current - Baseline):
  • Total: +12 | Passed: +12 | Failed: 0 | Skipped: 0

───────────────────────────────────────────────────────────
  INTERPRETATION
───────────────────────────────────────────────────────────
  ✅ 12 new tests added

═══════════════════════════════════════════════════════════

✓ No regressions detected
✓ Quality improved: 12 new tests for new functionality
✓ Ready for code review

Example 3: Autonomous Monitoring (Auto Mode)

# Agent uses auto mode for continuous monitoring
./.claude/skills/baseline-check/baseline-check.sh --auto

# First run: only captures
# Subsequent runs: captures + compares

Agent prompt (first run):

Running autonomous quality check...

✓ Baseline captured: 861/861 tests passing
ℹ️  No previous baseline for comparison (first run on this branch)

Baseline established for future comparisons.

Agent prompt (subsequent run):

Running autonomous quality check...

✓ Current: 873/873 tests passing
✓ Previous baseline detected, performing comparison...

  ✅ 12 new tests added
  ✅ No regressions detected

Quality maintained!

Example 4: Regression Detected (Compare Mode)

# After changes, comparison detects regressions
./.claude/skills/baseline-check/baseline-check.sh --compare

Agent prompt:

Running regression check...

═══════════════════════════════════════════════════════════
  BASELINE vs CURRENT COMPARISON
═══════════════════════════════════════════════════════════

BASELINE:
  • Total: 873 | Passed: 873 | Failed: 0 | Skipped: 0

CURRENT:
  • Total: 885 | Passed: 880 | Failed: 5 | Skipped: 0

DELTA (Current - Baseline):
  • Total: +12 | Passed: +7 | Failed: +5 | Skipped: 0

───────────────────────────────────────────────────────────
  INTERPRETATION
───────────────────────────────────────────────────────────
  ✅ 12 new tests added
  ❌ 5 NEW test failures introduced
     📋 Analyze if failures are:
        - Expected (testing edge cases, deprecated endpoints)
        - Bugs requiring fixes
        - Pre-existing issues incorrectly attributed

═══════════════════════════════════════════════════════════

⚠️  REGRESSIONS DETECTED

I've detected 5 new test failures. Let me analyze the failure logs...

[Agent reviews .claude/baseline/*.log file]

The failures are in SessionManagerTest. These appear to be legitimate bugs
introduced by the recent changes. I recommend:

1. Review the failing tests in the log file
2. Fix the SessionManager implementation
3. Re-run baseline check to verify fixes

Should I help investigate the failures?

TDD Integration

baseline-check plays a critical role in TDD workflow:

During GREEN Phase:

Verify tests exist for new implementation
Confirm tests are passing
Measure test coverage increase
Flag if implementation has no corresponding tests

During REFACTOR Phase:

Confirm all tests still pass after refactoring
Verify no test coverage regression
Compare before/after refactor metrics

Quality Gate:

No implementation accepted without tests
Test count should increase with new features
Test pass rate should remain 100%
No detekt errors allowed

Best Practices

Always capture before starting work - Clean baseline prevents confusion
Use auto mode for continuous monitoring - Simplifies workflow
Compare before creating PRs - Catch regressions early
Review logs when failures occur - Full console output preserved
Use issue-based naming for Linear integration - --issue SPI-XXX
Don't ignore regressions - Exit code 1 should block commits
Leverage symlinks for quick access - latest-{branch}.json

Technical Notes

Comprehensive execution: Uses ./gradlew clean check --rerun-tasks
Cache-proof: --rerun-tasks forces fresh execution, no FROM-CACHE false positives
Cross-platform: Works on macOS and Linux
Dual parsing: Console output (primary) + XML (fallback)
Timeout support: Default 300s, adjustable with --timeout
Exit codes: 0 = success, 1 = failures/regressions, 2 = script error
Storage: .claude/baseline/ (should be in .gitignore)

Success Criteria

The skill succeeds when it:

Executes in the appropriate mode (capture/compare/auto)
Runs comprehensive tests with forced fresh execution
Captures metrics in structured JSON format
Compares against baseline with delta calculation (compare/auto modes)
Provides clear, actionable reports with visual indicators
Returns appropriate exit codes for automation

The skill fails when:

Script cannot be executed
Tests time out or fail to run
JSON cannot be generated or parsed
Comparison logic fails
Report is unclear or incomplete

Troubleshooting

"No baseline found for branch"

Cause: Running --compare without a previous baseline

Solution: Run --capture first, or use --auto mode

"timeout: command not found"

Cause: timeout command not available on macOS

Solution: Install coreutils (brew install coreutils) or script falls back to no timeout

"Console parsing returned 0 tests"

Cause: Gradle output format changed or tests didn't run

Solution: Script automatically falls back to XML parsing

"Regressions detected but expected"

Cause: Intentional breaking changes or deprecated endpoint tests

Solution: Document expectations in PR, consider test-specific skips if appropriate