Musk Intelligence Report Generator

Generate daily intelligence reports covering Musk's ecosystem (Tesla, SpaceX, Starlink, xAI, Neuralink, Optimus) by collecting raw data and processing through a 3-stage LLM pipeline.

Quick Start

# Generate today's report (collect + pipeline)
cd /Users/hansel/WorkBuddy/现金收购套利/musk-intelligence-funnel
bash scripts/generate_musk_intel.sh

# Generate for specific date
bash scripts/generate_musk_intel.sh 2026-06-23

# Publish to IMA (after human review)
bash scripts/publish_musk_intel_ima.sh 2026-06-23

Required API Keys

1. RapidAPI Twitter API45 (Primary X Source) ⚠️ REQUIRED

Purpose: Fetch X list (configured in config/x_watchlist.yaml)
Get key: https://rapidapi.com/twitter-api45
Config: .env → RAPIDAPI_KEY; config/x_watchlist.yaml → lists
Test: python3 main.py test-x

# .env configuration
RAPIDAPI_KEY=your_rapidapi_key_here
TWITTER_LIST_ID=your_list_id_here  # Get from https://x.com/i/lists/YOUR_LIST_ID
X_PROVIDER=auto  # auto | rapidapi | rsshub

API Details:

Host: twitter-api45.p.rapidapi.com
Endpoint: GET /listtimeline.php?list_id={TWITTER_LIST_ID}
Rate limit: Check RapidAPI dashboard
Fallback: Automatically falls back to RSSHub if RapidAPI fails

Configure your X list:

Create an X list (e.g., "Musk Ecosystem Watch")
Add accounts to follow (e.g., @Tesla, @SpaceX, @SawyerMerritt)
Get list ID: https://x.com/i/lists/1234567890 → ID is 1234567890
Update config/x_watchlist.yaml → lists[0].id

2. LLM API (Multi-Provider Support) ⚠️ REQUIRED

Purpose: LLM processing for Filter, Analysis, Report stages
Supported Providers: OpenAI, Anthropic, Google, Moonshot (Kimi), DeepSeek, Qwen
Config: .env → LLM_API_KEY, LLM_BASE_URL, LLM_PROVIDER
Cost: Varies by provider (~¥0.6-1.40 per run for Kimi K2.7)

# .env configuration (Method 1: Auto-config by provider)
LLM_PROVIDER=moonshot  # openai | anthropic | google | moonshot | deepseek | qwen
LLM_API_KEY=your_api_key_here
LLM_BASE_URL=https://api.moonshot.cn/v1  # Auto-set by LLM_PROVIDER
LLM_MODEL=kimi-k2.7-code  # Auto-set by LLM_PROVIDER

# .env configuration (Method 2: Manual config)
LLM_API_KEY=your_api_key
LLM_BASE_URL=https://your-custom-endpoint/v1
LLM_MODEL=your-model-name

Supported Providers:

| Provider | LLM_PROVIDER | Get API Key | Default Model | |----------|----------------|-------------|---------------| | Moonshot (Kimi) | moonshot | platform.moonshot.cn | kimi-k2.7-code | | OpenAI | openai | platform.openai.com | gpt-4o | | Anthropic | anthropic | console.anthropic.com | claude-sonnet-4-20250514 | | Google | google | aistudio.google.com | gemini-2.0-flash | | DeepSeek | deepseek | platform.deepseek.com | deepseek-chat | | Qwen | qwen | dashscope.aliyun.com | qwen-max |

Model Strategy (auto-configured by LLM_PROVIDER):

Filter stage: Stable model (e.g., kimi-k2.6 for Moonshot, gpt-4o for OpenAI)
Analysis/Report stages: Reasoning model (e.g., kimi-k2.7-code for Moonshot)

Detailed Setup: See references/api_reference.md

3. RSSHub (Backup X Source) 🔧 RECOMMENDED

Purpose: Backup X list collector if RapidAPI fails
Setup: cd rsshub && docker compose up -d
Config: rsshub/.env → X Cookie
Test: curl http://localhost:1200/twitter/list/1927018058977030558?count=5

# .env configuration
RSSHUB_BASE_URL=http://localhost:1200

4. IMA API (Optional) 📤 OPTIONAL

Purpose: Upload generated reports to Tencent IMA knowledge base
Config: ~/.config/ima/client_id and api_key
Setup: See references/ima_setup.md

Workflow

Step 1: Collect Data

Collects from multiple sources with 48-hour window:

python3 main.py --date YYYY-MM-DD collect --since-last-report --hours 48

Sources:

X list (configured in config/x_watchlist.yaml) - RapidAPI primary, RSSHub fallback
X deep articles (priority authors like pmarca, 7-day window)
Media RSS (SpaceNews, Ars Technica, etc.)
SEC EDGAR (Tesla/SpaceX filings, 30-day window)

Output: data/raw/YYYY-MM-DD.json

Step 2: Run Pipeline

Process collected data through 3-stage LLM pipeline:

python3 main.py --date YYYY-MM-DD run --raw data/raw/YYYY-MM-DD.json

Stages:

Filter: Remove noise, keep product/regulatory/capacity/capital signals
Analysis: Business analysis, catalyst identification
Report: Generate daily Markdown report (NO catalyst block - removed per user preference)

Output:

output/daily/YYYY-MM-DD/01_filtered_signals.json
output/daily/YYYY-MM-DD/02_analysis.md
output/daily/YYYY-MM-DD/03_daily_report.md
output/reports/马斯克情报日报 YYYY-MM-DD.md (IMA format)

Step 3: Human Review

User reviews generated report before publishing:

# Review files
open output/daily/YYYY-MM-DD/03_daily_report.md
open output/reports/马斯克情报日报 YYYY-MM-DD.md

Review checklist:

[ ] Signals are new (not reported in previous days)
[ ] Updates marked with "新进展"
[ ] Soft signals (personal road tests, unverified rumors) handled per user feedback
[ ] Deep articles (x_deep) retained even if >48h old

Step 4: Publish to IMA (Optional)

After human approval, upload to IMA knowledge base:

bash scripts/publish_musk_intel_ima.sh YYYY-MM-DD

Or manually:

python3 scripts/upload_ima_intel.py "output/reports/马斯克情报日报 YYYY-MM-DD.md"

Scheduled Execution

Daily report generation is scheduled via macOS launchd:

Time: Every day 8:00 AM
Plist: ~/Library/LaunchAgents/com.musk.intel.update.plist
Script: scripts/generate_musk_intel.sh
Logs: /tmp/musk_intel_update.log

Management commands:

# Check status
launchctl list | grep musk

# Manual execution
launchctl start com.musk.intel.update

# View logs
tail -f /tmp/musk_intel_update.log

# Uninstall
launchctl unload ~/Library/LaunchAgents/com.musk.intel.update.plist

File Structure

musk-intelligence-funnel/
├── main.py                          # CLI entry point
├── config/                          # X lists, RSS, telemetry config
├── prompts/                        # 3-stage agent prompts
│   ├── filter_agent.md              # Filter stage prompt (K2.6)
│   ├── analysis_agent.md            # Analysis stage prompt (K2.7)
│   └── report_agent.md             # Report stage prompt (K2.7, NO catalyst)
├── src/                            # Collectors, agents, pipeline
│   ├── collectors/                  # Data collection modules
│   ├── agents/                      # Filter/Analysis/Report agents
│   └── utils/                       # Utilities (LLM client, IMA export, etc.)
├── data/
│   ├── raw/                        # Daily raw collection JSON
│   └── x_article_registry.json     # Deep article registry
├── output/
│   ├── daily/YYYY-MM-DD/           # Daily artifacts
│   └── reports/                    # IMA-formatted reports (human review)
├── rsshub/                         # Self-hosted RSSHub (Docker)
└── scripts/
    ├── generate_musk_intel.sh      # Generate report (no publish)
    └── publish_musk_intel_ima.sh   # Publish to IMA (after review)

Troubleshooting

X collection empty

# Test X connection
python3 main.py test-x

# Check RapidAPI key
echo $RAPIDAPI_KEY

# Check list_id parameter (must be pure number, not URL)
# Correct: list_id=1927018058977030558
# Wrong: id=https://x.com/i/lists/xxx

RSSHub fallback triggered

Check stats.x_provider in collection output. If rsshub, RapidAPI failed.

Fix:

Verify RAPIDAPI_KEY in .env
Check RapidAPI quota/rate limits
Or force RapidAPI: X_PROVIDER=rapidapi

Report stage hangs

Symptom: Report stage runs >10 minutes with no output

Fix: Increase timeout in src/utils/llm_client.py:

@dataclass
class LLMConfig:
    timeout: float = 1200.0  # 20 minutes (default was 600.0)

IMA upload fails

Check:

~/.config/ima/client_id and api_key exist
IMA Node skills installed at IMA_SKILLS_DIR
Knowledge base ID: ADieg6--1fzzF0MaLjTlKm_QTj2gB-_t8-FWRy9OVBw=

Duplicate reports

Check coverage_manifest.json and prior coverage loading.

Fix: Ensure src/utils/prior_coverage.py loads from:

output/daily/*/coverage_manifest.json (primary)
Obsidian knowledge base (supplementary)

Prompt Engineering Notes

Filter Agent (`prompts/filter_agent.md`)

Model: kimi-k2.6 (stable, fast)
Temperature: 0.6 (fixed when thinking disabled)
Key rules:

Preserve completeness: ALL 8 signal types must be covered
Media deep articles: if an article is forwarded ≥2 times by watched accounts, promote to watched
Freshness detection: mark fresh or stale

Report Agent (`prompts/report_agent.md`)

Model: kimi-k2.7-code (reasoning)
Temperature: default (not set)
Key rules:

NO catalyst block (removed per user preference)
Completeness: EVERY signal from Analysis must appear in report
Information density: 3-5 sentences per signal, use raw_evidence
Media articles: ≥4 sentences, turn every page
NO研判: only factual reporting, no commentary

Cost Estimation

Per run (full pipeline):

Filter: ~¥0.30 (K2.6, ~32k prompt + ~4k completion)
Analysis: ~¥0.20 (K2.7, ~4k prompt + ~6k completion)
Report: ~¥0.10 (K2.7, ~3k prompt + ~6k completion)
Total: ~¥0.60-0.80

Monthly (daily runs): ~¥18-24

Security Notes

RAPIDAPI_KEY and OPENAI_API_KEY are stored in .env (gitignored)
IMA credentials stored in ~/.config/ima/ (outside project)
No secrets in config/ or prompts/
RSSHub Cookie in rsshub/.env (gitignored)

References

API Reference - Detailed API documentation
IMA Setup Guide - IMA upload configuration
Prompt Engineering - Filter/Analysis/Report prompt details