{"slug": "answerwatch-track-changes-in-what-ai-models-recommend", "title": "Answerwatch – Track changes in what AI models recommend", "summary": "Answerwatch, a new open-source tool, lets users track changes in what AI models recommend by running the same prompt across models like OpenAI, Anthropic, and Google, storing results locally in SQLite, and generating static HTML reports showing agreement, rank gaps, and drift. Version 0.1 requires Python 3.10+, uses OpenRouter for multi-provider access, and keeps all data private with no external calls except model requests.", "body_md": "Track what AI models recommend—and see when the answer changes.\n\nYour brand, product, or category can look different depending on which model someone asks. Answerwatch makes that visible: run the same prompt across models, compare their recommendations, and keep a local audit trail as answers drift.\n\nEverything stays local in SQLite. The only external call is the model request you choose to make.\n\n```\n                   ┌─ OpenAI\nprompts ───────────├─ Anthropic ──► ranked entities ──► agreement + drift\n                   └─ Google                              │\n                                                         ▼\n                                                  static HTML report\n```\n\nRequires Python 3.10 or newer.\n\n```\npython -m pip install -e .\nanswerwatch init\nanswerwatch run --demo\nanswerwatch compare\nanswerwatch report\n```\n\nOpen the HTML path printed by the last command. The demo makes no network requests and spends no API credits. It produces a polished dashboard with agreement, consensus, rank gaps, citation domains, and raw answers.\n\n| You need to know | Answerwatch shows |\n|---|---|\n| Do models agree? | Pairwise recommendation-agreement matrix |\n| What does only one model mention? | Unique recommendations per model |\n| Is ranking materially different? | Largest rank gaps for shared entities |\n| Did the answer change this week? | Run-to-run drift with additions and removals |\n| Can I trust the analysis? | Raw-response audit trail and extraction evidence |\n\nAnswerwatch uses [OpenRouter](https://openrouter.ai/) so one API key can\nquery models from different providers.\n\n```\nexport OPENROUTER_API_KEY=\"...\"\nanswerwatch run\nanswerwatch compare\nanswerwatch report\n```\n\n`answerwatch init`\n\ncreates:\n\n```\n# answerwatch.yml\nmodels:\n  - openai/gpt-5.2\n  - anthropic/claude-sonnet-4.5\n  - google/gemini-3-flash-preview\n\nprompts: sample-prompts.yml\ndatabase: .answerwatch/answerwatch.db\n\nsettings:\n  temperature: 0.2\n  max_tokens: 1200\n  retries: 2\n# sample-prompts.yml\nprompts:\n  - id: best-running-shoes\n    text: What are the best running shoes for beginners?\n    entity: product\n```\n\nUse model identifiers currently available in OpenRouter.\n\n```\nanswerwatch init\nanswerwatch run [--demo]\nanswerwatch compare [RUN_ID] [--baseline RUN_ID|previous]\nanswerwatch report [RUN_ID] [--baseline RUN_ID|previous] [--output report.html]\n```\n\nWithout a run ID, `compare`\n\nand `report`\n\nuse the latest completed run.\n\nTo see recommendation drift, run once, change time or model settings, then:\n\n```\nanswerwatch compare --baseline previous\nanswerwatch report --baseline previous\n```\n\nVersion 0.1 asks models to put named recommendations first in a numbered list. A deterministic local parser extracts those list items. Both the raw answer and the exact extracted evidence are retained in SQLite.\n\nThis keeps the first release fast, private, and auditable. It does not claim that generated URLs are verified citations.\n\nThe default database and reports live under `.answerwatch/`\n\n. Delete that\ndirectory to remove local run history.\n\n```\nPYTHONPATH=src python -m unittest discover -s tests -v\n```\n\nPlease read [CONTRIBUTING.md](/haror1/answerwatch/blob/main/CONTRIBUTING.md) before opening a pull request.\nWe use Conventional Commit-style messages, for example:\n\n```\nfeat(report): surface consensus recommendations\nfix(extract): preserve linked entity names\ndocs: clarify live model setup\n```\n\n**Now** — local runs, recommendation comparison, static reports, and OpenRouter\nsupport.\n\n**Next** — stronger entity normalization, structured provider citations, and\nscheduled local runs.\n\n**Later** — optional hosted history, alerts, teams, and branded reports.\n\nSee [ROADMAP.md](/haror1/answerwatch/blob/main/ROADMAP.md) for the intended boundaries.\n\nMIT", "url": "https://wpnews.pro/news/answerwatch-track-changes-in-what-ai-models-recommend", "canonical_source": "https://github.com/haror1/answerwatch", "published_at": "2026-07-28 06:23:51+00:00", "updated_at": "2026-07-28 06:52:45.461423+00:00", "lang": "en", "topics": ["ai-tools", "developer-tools", "large-language-models"], "entities": ["Answerwatch", "OpenRouter", "OpenAI", "Anthropic", "Google", "SQLite", "Python"], "alternates": {"html": "https://wpnews.pro/news/answerwatch-track-changes-in-what-ai-models-recommend", "markdown": "https://wpnews.pro/news/answerwatch-track-changes-in-what-ai-models-recommend.md", "text": "https://wpnews.pro/news/answerwatch-track-changes-in-what-ai-models-recommend.txt", "jsonld": "https://wpnews.pro/news/answerwatch-track-changes-in-what-ai-models-recommend.jsonld"}}