Consume Feed Skill

A robust ingestion engine for upstream data sources.

Phase 1: RSS (Implemented)

• "Pull the feeds now" -> ./run.sh run --mode manual
• "Add this RSS feed <url>" -> ./run.sh sources add rss --url <url>
• "Check feed ingest health" -> ./run.sh doctor

Phase 2: GitHub & NVD (Aspirational)

• "Add GitHub repo <owner>/<repo>" -> sources add github --repo <owner>/<repo>
• "Track NVD for <keyword>" -> sources add nvd --query <keyword>

Usage

Run Ingestion

# Run nightly crawl (all sources, respect intervals)
./run.sh run --mode nightly

# Run specific source immediately
./run.sh run --source <key>

Manage Sources

# List all
./run.sh sources list

# Add RSS
./run.sh sources add rss --url "https://github.blog/feed/"

Diagnosis & Initialization

# Health check
./run.sh doctor

# Force initialize search views and indexes
./run.sh doctor --init

Resilience

• Uses exponential backoff and jitter for all network requests.
• Persists checkpoints (ETags, Timestamps) to resume efficiently.
• Reuses Memory skill connection for stable, shared database access.