Iterative Academic Writing

Write academic documents with fact verification, self-evaluate against 14 principles, and iteratively refine until quality criteria are met.

Workflow

┌─────────────────────────────────────────────────────────────┐
│                                                             │
│  [Phase 0]        [Phase 1]        [Phase 2]                │
│  Build Fact  ───→  Draft    ───→   Evaluate                 │
│  Base                                  │                    │
│                                        ↓                    │
│                                   Issues Found?             │
│                                        │                    │
│                              No ───────┴─────── Yes         │
│                              ↓                   ↓          │
│                          ✅ Complete      [Phase 3]         │
│                                            Plan             │
│                                              │              │
│                                              ↓              │
│                                        [Phase 4]            │
│                                         Rewrite             │
│                                              │              │
│                                              └───→ Phase 2  │
│                                                   (LOOP)    │
└─────────────────────────────────────────────────────────────┘

Termination Conditions:

• ✅ Success: Critical == 0 AND Hallucinations == 0
• ⚠️ Partial: Iteration == 5 with no hallucinations → output + remaining issues
• ❌ Failure: Iteration == 5 with hallucinations → cannot publish

14 Principles

Factual Accuracy (사실성) — HIGHEST PRIORITY

| ID | Principle | Violation | |----|-----------|-----------| | FA1 | Fact-based claims | Claim not in Fact Base | | FA2 | No invented numbers | Metrics/statistics without verified source | | FA3 | Verified citations | Paper reference not WebSearch-verified |

Structure (구조)

| ID | Principle | Violation | |----|-----------|-----------| | ST1 | Topic-first (두괄식) | Paragraph starts with background instead of main claim | | ST2 | Purpose-driven opening | Section doesn't state why it exists | | ST3 | Implication-focused closing | Section ends without stating significance | | ST4 | Figure/table reference | Visual used without "Figure X shows..." |

Understandability (이해도)

| ID | Principle | Violation | |----|-----------|-----------| | UN1 | Definition-first | Technical term used before "X is Y" definition | | UN2 | Compare-contrast | New approach explained without comparing to existing | | UN3 | Beginner-friendly | Requires domain knowledge to understand | | UN4 | Coherent flow | Missing logical connectors between paragraphs |

Format (형식)

| ID | Principle | Violation | |----|-----------|-----------| | FM1 | Conciseness | Unnecessary modifiers (매우, very, extremely) | | FM2 | Specific numbers | Vague quantifiers (많은, some, several) | | FM3 | Prose only | Bullet points in body text | | FM4 | Abstract-level | Exposes file paths, function names |

Issue Classification

🔴 Critical (must fix — blocks completion)

| ID | Issue | Principle | |----|-------|-----------| | C1 | Hallucination — claim not in Fact Base | FA1/FA2 | | C2 | Fact contradiction — claim differs from Fact Base | FA1 | | C3 | Topic-first violation | ST1 | | C4 | Undefined technical term | UN1 | | C5 | Not beginner-friendly | UN3 | | C6 | Logical disconnect between paragraphs | UN4 | | C7 | Bullet points in prose | FM3 |

🟡 Warning (should fix)

| ID | Issue | Principle | |----|-------|-----------| | W1 | Unverified citation | FA3 | | W2 | Missing compare-contrast | UN2 | | W3 | No implication at section end | ST3 | | W4 | Vague modifiers | FM1 | | W5 | Purpose unclear at section start | ST2 | | W6 | Vague quantifiers | FM2 | | W7 | Code elements exposed | FM4 | | W8 | Figure/table without reference | ST4 |

🔵 Suggestion (optional)

| ID | Issue | |----|-------| | S1 | Add connective words | | S2 | Add example/analogy | | S3 | Simplify sentence |

---

Phase 0: Build Fact Base (REQUIRED)

Before writing anything, collect and verify all facts. No claim may appear in the document without a Fact Base entry.

Sources to Check

| Source Type | What to Extract | Verification Method | |-------------|-----------------|---------------------| | Code/config files | Parameters, settings | Direct file inspection | | Result files | Metrics, numbers | Direct file inspection | | Papers/references | Citations, claims | WebSearch verification | | External facts | Dates, statistics | WebSearch verification |

Fact Base Format

## Fact Base

### ✅ Verified Facts
| ID | Claim | Source | Verification |
|----|-------|--------|--------------|
| F1 | Learning rate is 0.03 | config.yaml:12 | Code inspection |
| F2 | Titans paper published 2025 | arXiv:2501.00663 | WebSearch |
| F3 | BERT has 110M parameters | Original paper | WebSearch |

### ❌ Unverified (DO NOT USE IN DOCUMENT)
| Claim | Reason |
|-------|--------|
| "15% improvement" | No result file found |
| "State-of-the-art performance" | No benchmark comparison |

Fact Base Rules

1. Every number must have a source — no invented statistics
2. Every citation must be WebSearch-verified — no hallucinated papers
3. Unverified claims are forbidden — cannot appear in document
4. When in doubt, don't include — omission > hallucination

---

Phase 1: Draft

Process

1. Design structure: Outline sections with core message for each
2. List technical terms: Prepare "X is Y" definitions for all jargon
3. Write with constraint: Every factual claim must reference Fact Base

Internal Fact Check (while writing)

✍️ Writing: "LTPO uses learning rate 0.03"
   → Fact check: F1 ✓ → OK to include

✍️ Writing: "This achieved 15% improvement"  
   → Fact check: NOT IN FACT BASE ❌ → DO NOT WRITE
   → Alternative: "Experiments were conducted to measure improvement" (no number)

Output

• Draft document
• Technical term definitions used
• Fact Base reference log

After Draft

→ Proceed to Phase 2 (Evaluate)

---

Phase 2: Evaluate

Step 1: Hallucination Check (FIRST PRIORITY)

Extract every factual claim and verify against Fact Base:

### Claim Verification
| Location | Claim | Fact Base ID | Status |
|----------|-------|--------------|--------|
| Para 1 | "learning rate 0.03" | F1 | ✅ Verified |
| Para 2 | "15% improvement" | — | ❌ HALLUCINATION |
| Para 3 | "Titans (2025)" | F2 | ✅ Verified |

Step 2: Principle Check

Check each paragraph against all 14 principles.

Output Format

## Evaluation (Iteration N/5)

### Fact Check Summary
- Total claims: 15
- Verified: 12
- Hallucinations: 2 ← MUST BE 0 TO PASS

### 🔴 Critical (N)
- [C1] Para 2: "15% improvement" — NOT IN FACT BASE
- [C1] Para 4: "outperforms baseline by 2x" — NOT IN FACT BASE
- [C4] Para 3: "SAE" used without definition

### 🟡 Warning (N)
- [W2] Section 2: No compare-contrast for new method
- [W4] Para 1: "significantly" is vague

### 🔵 Suggestion (N)
- [S1] Para 2→3: Add transitional phrase

### Status: CONTINUE (2 hallucinations, 1 other critical)

Decision Logic

IF hallucinations > 0 OR critical_issues > 0:
    → Status: CONTINUE
    → Proceed to Phase 3 (Plan)
ELSE:
    → Status: COMPLETE
    → Output final document

---

Phase 3: Plan

Priority Order (STRICT)

1. Hallucinations (C1, C2) — fix first, blocks everything
2. Other Critical (C3-C7) — fix second
3. Warnings (W1-W7) — fix if iteration budget allows

Fix Planning Format

For each issue:

### Issue: [C1] Para 2 — "15% improvement" hallucination

**Cause**: Wrote number without verifying source

**Fix Options**:

Option A (Remove claim):
- Before: "이를 통해 15%의 성능 향상을 달성했다."
- After: "실험 결과 성능 향상을 확인하였다." (no specific number)

Option B (Verify and update):
- Action: Check results.json for actual metric
- If found: Add to Fact Base with source, update sentence
- If not found: Use Option A

**Selected**: Option A (no results file available)

After Planning

→ Proceed to Phase 4 (Rewrite)

---

Phase 4: Rewrite

Process

1. Apply all planned fixes in priority order
2. Re-verify every claim against Fact Base
3. Ensure no new hallucinations introduced
4. Update Fact Base if new verified facts discovered

After Rewrite

→ Return to Phase 2 (Evaluate)
   This creates the iteration loop.

---

Termination Conditions

| Condition | Action | |-----------|--------| | Critical == 0 AND Hallucinations == 0 | ✅ Success — Output final document | | Iteration == 5 AND Hallucinations == 0 | ⚠️ Partial — Output current + list remaining warnings | | Iteration == 5 AND Hallucinations > 0 | ❌ Failure — Cannot publish; list unverifiable claims |

On Success

Output the final document and offer to address remaining Suggestions.

On Failure

## ❌ Cannot Publish

The following claims could not be verified and must be removed or verified externally:

| Claim | Location | Reason |
|-------|----------|--------|
| "15% improvement" | Para 2 | No source found |
| "2x faster" | Para 5 | No benchmark data |

Please provide sources for these claims or approve their removal.

---

Hallucination Types Reference

| Type | Example | How to Detect | |------|---------|---------------| | Invented numbers | "15% improvement" | Not in Fact Base | | False attribution | "Smith et al. showed..." | WebSearch finds no such claim | | Fabricated details | "Published in March 2024" | Date not verifiable | | Exaggerated claims | "Significantly outperforms" | No comparative data | | Causal hallucination | "X caused Y" | No evidence for causation | | Citation hallucination | "arXiv:2401.99999" | Paper doesn't exist |

---

Progress Report Template

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
ITERATIVE ACADEMIC WRITING — Progress Report
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

[Phase 0] Fact Base: 12 verified facts, 2 unverified (excluded)

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
[Iteration 1/5]
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Phase 1: Draft ✓
Phase 2: Evaluate
  → Hallucinations: 2 | Critical: 1 | Warning: 3
  → Status: CONTINUE
  → Proceed to Phase 3

Phase 3: Plan ✓
  → Priority: Fix 2 hallucinations first
  → Proceed to Phase 4

Phase 4: Rewrite ✓
  → Removed unverified claims
  → Return to Phase 2

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
[Iteration 2/5]
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Phase 2: Evaluate
  → Hallucinations: 0 | Critical: 0 | Warning: 2
  → Status: COMPLETE ✅

✅ Document finalized
   Remaining: 2 warnings (optional to fix)