Monitor What LLMs Say Before Users Choose Your Dev Tool

Use this skill as the main visibility workflow router for developer tools and open-source products.

Brand: DevTool Answer Monitor

Companion repo: devtool-answer-monitor

Use this when you want an agent to help you monitor how LLMs describe your product, build a reusable query pool, diagnose negative or outdated answers, and plan what to fix next.

Safety First

Treat this root skill as a read-only workflow router.
Default to quickstart replay or manual paste mode when you only need examples or scoring help.
Do not ask users to paste API keys into chat. If API collection mode is needed, tell them to configure local environment variables themselves and then hand off execution to visibility-monitor.
Review local scripts such as install.sh, quickstart.sh, and the selected runner before executing shell commands.

Start Here

Copy one of these prompts to begin:

Analyze how ChatGPT and Claude describe my API docs
Build a developer-tool answer monitoring query pool for my SDK
Find negative or outdated LLM claims about my project

30-Second Result

Typical input

product truth such as a README, docs, changelog, integrations, or positioning page
answer evidence such as copied model answers, screenshots, or cited URLs
scope such as target models, languages, regions, or a repeated query set

What this skill returns

a reusable query pool
raw evidence and a score draft plan
a monitoring summary and report outline
a repair backlog with T+7 or T+14 validation points

Companion demo and sample outputs

Zero-install demo: sample-run viewer
Public benchmarks: MinerU story and Sciverse API story
Sample outputs: leaderboard snapshot and repair trend snapshot

Trigger

Use this skill when the task is any of the following:

generate a visibility query matrix and Query Pool from product truth;
monitor how multiple LLMs mention, recommend, or misunderstand a product;
plan model-specific content placement based on datasource patterns;
check whether a draft page, FAQ, changelog, or case study is ready to influence model answers;
repair wrong, negative, outdated, or competitor-only answers;
verify whether a repair action improved metrics at T+7 or T+14;
help a user choose between quickstart replay, manual paste mode, and API collection mode.

Beginner Routing

When the user is new to the repository, route them in this order.

| Situation | Next step | |---|---| | Needs environment check first | open docs/getting-started.md and review the environment check section | | Wants environment-free first run | open docs/index.html or docs/for-beginners.md | | Wants a short explanation first | open docs/for-beginners.md | | Wants deeper onboarding | open docs/getting-started.md | | Wants the English repository overview | open README.md | | Wants the Chinese repository overview | open README.zh-CN.md |

Visibility Strategy

Always keep the workflow in this order:

| Stage | Goal | |---|---| | Query design | turn product truth into scenario matrix, three-layer keywords, and Query Pool seeds | | Monitoring | score mention, positive mention, capability accuracy, and ecosystem accuracy | | Placement | map each target model to likely datasource channels and publication surfaces | | Repair | classify bad answers into information error, negative evaluation, outdated information, or competitor insertion | | Activation | analyze whether answers help a user install, integrate, or invoke the product | | Regression | compare follow-up runs and check whether metrics improved after action |

Mode Selection

Choose the execution mode before running monitoring.

| Mode | Use when | Typical inputs | |---|---|---| | Quickstart replay | user wants the fastest first run without API setup | sample model config + sample manual responses | | Manual paste mode | user already has copied answers from chat tools | Query Pool + manual response JSON | | API collection mode | user wants repeatable real monitoring | Query Pool + model config + locally configured provider env vars |

Input Contract

Prepare as many of the following as possible before execution.

| Input | Examples | |---|---| | Product truth | README, docs, changelog, integrations, positioning | | Answer evidence | raw answers, screenshots, copied responses, cited links | | Monitoring scope | models, languages, regions, dates, repeated query set | | Publishing targets | docs, blog, GitHub, Q&A, partner channels |

Workflow Router

Choose the next sub-skill according to the user's immediate need.

| Situation | Next Skill | |---|---| | Need query design and scenario clustering | visibility-query-matrix | | Need weekly monitoring, evidence logging, report output, or shell execution after explicit user approval | visibility-monitor | | Need pre-publish content QA | visibility-content-check | | Need to repair bad answers and define regression checks | visibility-repair |

Required Reading Order

For a full program, read these repository documents in sequence:

playbooks/visibility-workflow-architecture.md
playbooks/keyword-strategy.md
playbooks/monitoring-system.md
playbooks/model-datasources.md
playbooks/content-platform-map.md
playbooks/negative-fix-sop.md

Output Contract

Always preserve the following outputs.

| Output | Description | |---|---| | Query foundation | scenario matrix, keyword layers, Query Pool | | Monitoring outputs | raw evidence, score draft, summary, report, leaderboard or overview | | Action plan | content placement priorities and repair backlog | | Regression record | T+7 and T+14 comparisons after key fixes |

Positioning

DevTool Answer Monitor is the skill layer for the devtool-answer-monitor repo.

Use the repo when you want runnable demos, scripts, and report artifacts.
Use the skill when you want an agent-guided workflow for monitoring, repair, and regression planning.

Handoff Rules

At the end of each run, preserve:

which product was optimized;
which models and languages were in scope;
which queries are reused in weekly tracking;
what the top three visibility weaknesses are;
what actions are already completed and what still needs validation.