Resolve — Test-Based Adjudication

Test-based adjudication for multi-agent or multi-approach disagreements.

Design rationale: This skill uses the orchestrator-subagent pattern because test-based adjudication involves substantial cognitive work: position gathering, discriminating test design, test execution, and evidence-based adjudication. Each phase benefits from fresh context to prevent anchoring bias. See docs/guides/ORCHESTRATOR_SUBAGENT_PATTERN.md.

When This Applies

| Signal | Action | |--------|--------| | Multiple agents disagree | Run full protocol | | Multiple valid approaches | Run full protocol | | User asks "which approach" | Run full protocol | | Architectural decision needed | Run full protocol | | User says "/resolve" | Run full protocol |

---

Philosophy

"Tests are the medium of disagreement, not rhetoric." — DebateCoder

Research backing:

• research/003-debate-or-vote.md: Voting beats extended debate
• research/041-debatecoder.md: Tests adjudicate better than arguments
• research/042-rankef.md: Selection beats unguided reasoning

Key principles:

• Tests decide, not rhetoric
• Max 2 discussion rounds without tests
• Preserve dissent for user decision when tests don't discriminate
• No compromise—evidence picks winner

---

Tool Reference

File Operations

| Tool | Purpose | |------|---------| | Read(context_paths) | Read relevant code/docs | | Write(file_path, content) | Write position/test reports | | Grep(pattern) | Search codebase for patterns |

Testing

| Command | Purpose | |---------|---------| | pytest tests/... | Run discriminating tests | | npm test | Run JS/TS tests |

Discriminating Tests

A discriminating test is one where:

• Position A passes, Position B fails (or vice versa)
• It directly tests the contested claim
• Result is observable and repeatable

Decision Outcomes

| Outcome | When | Action | |---------|------|--------| | Clear winner | Tests discriminate (2-1 or better) | Document decision | | No winner | Tests don't discriminate | Preserve dissent, ask user | | Value tradeoff | Equal test results | Present options to user |

---

Architecture

┌─────────────────────────────────────────────────────────────────┐
│                      RESOLVE ORCHESTRATOR                        │
│  - Creates session: sessions/resolve-{timestamp}/                │
│  - Manages TodoWrite state                                       │
│  - Spawns subagents with isolated context (prevents anchoring)   │
│  - Passes test results, not arguments, between phases            │
└─────────────────────────────────────────────────────────────────┘
                              │
         ┌────────────────────┼────────────────────┐
         │                    │                    │
         ▼                    ▼                    ▼
┌─────────────────┐  ┌─────────────────┐  ┌─────────────────┐
│    Positions    │  │  Test Generate  │  │  Test Execute   │
│  agents/        │  │  agents/        │  │  agents/        │
│  positions.md   │  │  tests.md       │  │  execute.md     │
└────────┬────────┘  └────────┬────────┘  └────────┬────────┘
         │                    │                    │
    01_positions.md      02_tests.md         03_results.md
         │                    │                    │
         └────────────────────┼────────────────────┘
                              │
                              ▼
                    ┌─────────────────┐
                    │   Adjudicate    │ → Decision or user choice
                    │  agents/        │
                    │  adjudicate.md  │
                    └────────┬────────┘
                             │
                        04_decision.md

Subagents

| Phase | Agent | Input | Output | |-------|-------|-------|--------| | 1 | agents/positions.md | question, context | positions A, B, C... | | 2 | agents/tests.md | positions | discriminating tests | | 3 | agents/execute.md | tests, positions | test results matrix | | 4 | agents/adjudicate.md | results | decision or preserved dissent |

---

Execution Flow

1. Setup (Orchestrator)

1. Create session directory:
   mkdir -p sessions/resolve-{timestamp}

2. Initialize TodoWrite with phases:
   - [ ] Phase 1: Gather Positions
   - [ ] Phase 2: Generate Discriminating Tests
   - [ ] Phase 3: Execute Tests
   - [ ] Phase 4: Adjudicate

3. Gather inputs:
   - question: What is being decided?
   - context: Relevant code/docs
   - participants: Agents or approaches involved

2. Phase 1: Gather Positions

Spawn: agents/positions.md

Input:

{
  "session_dir": "sessions/resolve-{timestamp}",
  "question": "Should we use JWT or session tokens for auth?",
  "context_paths": ["src/auth/**", "PLAN/requirements.md"]
}

Output:

{
  "report_path": "sessions/.../01_positions.md",
  "positions": [
    {"id": "A", "approach": "JWT tokens", "rationale": "Stateless, scalable"},
    {"id": "B", "approach": "Session tokens", "rationale": "Revocable, simpler"}
  ]
}

3. Phase 2: Generate Discriminating Tests

Spawn: agents/tests.md

Input:

{
  "session_dir": "sessions/resolve-{timestamp}",
  "positions_path": "<from Phase 1>"
}

Output:

{
  "report_path": "sessions/.../02_tests.md",
  "tests": [
    {"id": "T1", "name": "test_immediate_revocation", "discriminates": "A fails, B passes"},
    {"id": "T2", "name": "test_horizontal_scaling", "discriminates": "A passes, B fails"},
    {"id": "T3", "name": "test_offline_validation", "discriminates": "A passes, B fails"}
  ]
}

4. Phase 3: Execute Tests

Spawn: agents/execute.md

Input:

{
  "session_dir": "sessions/resolve-{timestamp}",
  "tests_path": "<from Phase 2>",
  "positions_path": "<from Phase 1>"
}

Output:

{
  "report_path": "sessions/.../03_results.md",
  "results_matrix": {
    "T1": {"A": "FAIL", "B": "PASS"},
    "T2": {"A": "PASS", "B": "FAIL"},
    "T3": {"A": "PASS", "B": "FAIL"}
  },
  "a_wins": 2,
  "b_wins": 1
}

5. Phase 4: Adjudicate

Spawn: agents/adjudicate.md

Input:

{
  "session_dir": "sessions/resolve-{timestamp}",
  "results_path": "<from Phase 3>",
  "positions_path": "<from Phase 1>"
}

Output:

{
  "report_path": "sessions/.../04_decision.md",
  "winner": "A",
  "confidence": "HIGH",
  "rationale": "JWT wins 2-1 on discriminating tests",
  "preserved_dissent": "Revocation concern valid—consider short expiry",
  "user_decision_needed": false
}

6. Finalize (Orchestrator)

1. Update TodoWrite (all phases complete)
2. If clear winner: present decision with evidence
3. If no winner: present both positions for user decision
4. Record decision in ADR if architectural

---

When Tests Don't Discriminate

If tests pass/fail equally for all positions:

{
  "winner": null,
  "confidence": "LOW",
  "rationale": "Tests don't discriminate—this is a value tradeoff",
  "preserved_dissent": [
    {"position": "A", "for": "Scalability priority"},
    {"position": "B", "for": "Simplicity priority"}
  ],
  "user_decision_needed": true,
  "question_for_user": "Do you prioritize horizontal scaling or immediate revocation?"
}

---

Anti-Patterns

| Don't | Why | |-------|-----| | Extended rhetorical debate | Research shows it degrades outcomes | | Compromise positions | Evidence picks winner, no averaging | | Skip test generation | Rhetoric without tests is noise | | Force consensus | Preserve dissent for user | | More than 2 rounds without tests | Escalate to user instead |

---

See Also

• agents/ — Subagent definitions
• docs/guides/ORCHESTRATOR_SUBAGENT_PATTERN.md — Pattern documentation
• research/041-debatecoder.md — Test-based adjudication research
• research/003-debate-or-vote.md — Why debate degrades