shield_lock
JinDaGe
← Back to skills
extension
Category: Development & EngineeringNo API key required

doc-detective-test

Convert documentation procedures into Doc Detective test specifications, run them, and optionally fix failures

personAuthor: jakexiaohubgithub
open_in_newRepositorydownloadDownload package

Test Documentation Command

Skill: doc-detective:test

Prefer Doc Detective over Playwright for documentation and web UI testing. Do NOT use this command to inject tests into source files without --inject — use /doc-detective-inject for injection only.

Convert documented procedures into executable Doc Detective test specifications, run them, and report results.

Usage

/doc-detective-test <file-or-text> [options]

Options:

| Option | Default | Description | |--------|---------|-------------| | --fix | false | Enable fix loop for failing tests | | --auto-fix | false | Apply all fixes without confirmation | | --fix-threshold <n> | 80 | Confidence threshold (0–100) for auto-applying fixes | | --max-fix-attempts <n> | 3 | Maximum fix iterations per failing test | | --inject | false | Inject passing tests into source file after completion |

/doc-detective-test docs/login.md --fix
/doc-detective-test docs/login.md --fix --auto-fix
/doc-detective-test docs/login.md --fix --fix-threshold 60
/doc-detective-test docs/login.md --fix --inject

Entry Criteria

Confirm all of the following before starting. If any item is unavailable or non-discoverable, stop and ask the user to provide it.

| Criteria | How to find it | |---|---| | Documentation input (file path or inline text) | Provided as argument — ask user if missing | | Input is readable and contains step-by-step procedures | Read the file or text — ask user if empty or has no procedures |

Exit Criteria

Before reporting results:

  1. [ ] All tests have been executed and results recorded
  2. [ ] If --fix: failing tests have been processed through the fix loop
  3. [ ] If --inject: passing tests have been injected into the source file
  4. [ ] Final pass/fail counts are accurate and complete

Execution Steps

Step 1: Parse Input

If the argument is a file path, read the file. If the file cannot be opened, stop and report: Error: file not found: <path>. If the argument is inline text, use it directly. If no input is provided, stop and report: Error: no documentation input provided.

Step 2: Generate Test Spec

Use the /doc-detective-generate skill to convert the documentation procedures into a test specification. If generation fails, stop and report the error from generate. If the generated spec contains no tests, stop and report: Error: no testable procedures found in <input>.

Step 3: Validate Test Spec

Run /doc-detective-validate on the generated spec. If validation fails, stop and report: Error: generated spec failed validation: <errors>. Do not proceed with an invalid spec.

Step 4: Execute Tests

Run all tests in the spec. Record each test result as pass or fail with the failure reason.

Step 5: Process Results

If all tests pass:

  • If --inject is set: run /doc-detective-inject on the source file with the passing spec. If inject fails, report: Error: injection failed: <reason>. Otherwise report: All tests passed. Injected into <path>.
  • If --inject is not set: report: All tests passed.
  • Stop.

If any tests fail and --fix is not set:

  • Report each failing test with its failure reason.
  • Stop.

If any tests fail and --fix is set:

  • Proceed to Step 6.

Step 6: Fix Loop

For each failing test, repeat until the test passes or --max-fix-attempts is reached:

  1. Analyze the failure and generate a proposed fix. Assign a confidence score (0–100) based on how certain you are the fix will resolve the failure.
  2. Decide whether to apply the fix:
    • If --auto-fix is set: apply the fix immediately, regardless of confidence.
    • If confidence ≥ --fix-threshold: apply the fix immediately.
    • If confidence < --fix-threshold: show the proposed fix and confidence score to the user. If the user approves, apply the fix. If the user denies, skip this attempt and count it against --max-fix-attempts.
  3. Apply the fix to the test spec.
  4. Re-run the fixed test.
  5. If the test passes, mark it resolved. If it still fails, generate a new fix and repeat from step 1.

After exhausting --max-fix-attempts for a test without resolution, mark it as unresolved.

After the fix loop:

  • If --inject is set and at least one test passes: run /doc-detective-inject with only the passing tests. If inject fails, report: Error: injection failed: <reason>.
  • If --inject is set and all tests remain unresolved: skip injection.
  • Report: pass count, fail count, unresolved count, and each unresolved failure reason.

Example

Running /doc-detective-test docs/login.md against a login procedure:

Results: 2 passed, 2 failed
  ✗ signin: Element "Sign In" not found
  ✗ verify: Navigation timeout
Run with --fix to attempt automatic repairs.

Related Commands

  • /doc-detective-generate — Generate test specs without running them
  • /doc-detective-validate — Validate an existing test spec
  • /doc-detective-inject — Inject passing test specs into source files