Redaction Tool Development Guide

An intelligent document redaction application with two-pass PII detection—regex patterns for standard formats plus AI semantic analysis for contextual data.

Live site: https://redaction-tool.casedev.app/

Architecture

src/
├── app/
│   ├── page.tsx                    # Main application UI
│   └── api/
│       ├── detect-pii/             # Two-pass PII detection
│       ├── export-pdf/             # PDF generation
│       ├── detect/                 # Database-backed detection
│       ├── export/                 # Database-backed export
│       ├── jobs/                   # Job management
│       └── upload/                 # File upload
├── components/
│   ├── redaction/
│   │   ├── PatternSelector.tsx     # Redaction type selection
│   │   ├── EntityList.tsx          # Detected entities
│   │   └── DocumentPreview.tsx     # Preview with highlights
│   ├── upload/
│   │   └── DropZone.tsx            # File upload
│   └── ui/                         # shadcn components
└── lib/
    ├── redaction/
    │   ├── detector.ts             # Two-pass detection logic
    │   └── patterns.ts             # Regex patterns & presets
    ├── case-dev/
    │   └── client.ts               # Case.dev API client
    ├── db.ts                       # Database connection
    └── utils.ts                    # Utilities

Core Workflow

Upload Doc → Extract Text → Pass 1: Regex → Pass 2: AI → Review → Export PDF
     ↓            ↓             ↓              ↓           ↓          ↓
  PDF/TXT      pdf.js       SSN, CC,       Names,      Toggle      Redacted
  images       extraction   phone, email   addresses   entities    document
                            patterns       context

Tech Stack

| Layer | Technology | |-------|-----------| | Frontend | Next.js 14, React, Tailwind CSS | | UI | shadcn/ui | | PDF Processing | pdf.js (extract), pdf-lib (generate) | | AI Detection | Case.dev LLM (GPT-4o) | | Database | PostgreSQL + Prisma (optional) |

Key Features

| Feature | Description | |---------|-------------| | Two-Pass Detection | Regex + AI for comprehensive coverage | | PII Types | SSN, bank accounts, credit cards, names, addresses, phone, email, DOB | | Presets | Pre-configured redaction profiles | | Entity Review | Toggle, edit masked values | | PDF Export | Generate redacted documents | | Audit Log | Track what was redacted |

Two-Pass Detection

See references/pii-detection.md for patterns and AI prompts.

Pass 1: Regex (Fast, High-Precision)

• SSN: XXX-XX-XXXX with validation
• Credit cards: Luhn-valid patterns
• Phone: US formats
• Email: Standard format
• Dates: Common formats

Pass 2: AI/LLM (Semantic, Aggressive)

• Non-standard formats ("SSN: one two three...")
• Contextual references ("my social is...")
• Names and addresses
• OCR errors and typos
• Obfuscated data

Redaction Presets

| Preset | Types Included | |--------|----------------| | SSNs and Financial | SSN, Account Numbers, Credit Cards | | All Personal Information | All PII types | | Contact Information Only | Phone, Email | | Financial Only | Account Numbers, Credit Cards |

Case.dev Integration

See references/casedev-redaction-api.md for API patterns.

LLM Detection

const aiEntities = await detectWithLLM(text, piiTypes);

OCR for Images

const text = await extractTextFromImage(imageUrl);

Development

Setup

npm install
cp .env.example .env
# Add CASEDEV_API_KEY
npm run dev

Environment

CASEDEV_API_KEY=sk_case_...               # Required
DATABASE_URL=postgresql://...             # Optional for job persistence

Database (Optional)

npx prisma migrate dev

API Endpoints

| Method | Endpoint | Description | |--------|----------|-------------| | POST | /api/detect-pii | Two-pass PII detection | | POST | /api/export-pdf | Generate redacted PDF | | POST | /api/upload | File upload | | GET | /api/jobs | List jobs | | GET | /api/jobs/:id | Get job status |

Common Tasks

Adding a New PII Type

1. Add regex pattern to lib/redaction/patterns.ts
2. Add to AI prompt in lib/redaction/detector.ts
3. Add UI toggle in PatternSelector.tsx
4. Add masking function

Improving AI Detection

Modify the LLM prompt to be more/less aggressive or handle specific formats.

Adding Export Format

1. Add generation function in lib/export/
2. Add endpoint in api/export-[format]/
3. Add UI option

Troubleshooting

| Issue | Solution | |-------|----------| | Regex missing PII | Check pattern, add variations | | AI too aggressive | Adjust prompt confidence | | PDF export fails | Verify pdf-lib compatibility | | OCR errors | Use higher quality images |