{"slug": "audience-personas-built-from-engagement-data-scored-on-predictions", "title": "Audience personas built from engagement data, scored on predictions", "summary": "A new open-source framework called Persona Loop uses observed audience behavior to build self-improving user personas that make falsifiable predictions about content performance. The system, built for Claude Code, scores each prediction against real outcomes and automatically documents biases when a persona is wrong twice in the same direction. The project aims to replace traditional demographic-based marketing personas with behavior-grounded agents that improve through a feedback loop.", "body_md": "Build audience or user personas from observed behavior.\n\nUse them to judge your work before you ship it, and score their predictions against real outcomes so they get better over time.\n\nBuilt for Claude Code, but the persona files work as system prompts in other models and harnesses too (see `docs/customization.md`\n\n). The entire system is made up of markdown files (*.md): agent files, skill files, and data files populated with your own audience evidence.\n\nYou bring the evidence. The system start with whatever record of audience behavior you alreadhy have (analytics exports, spreadsheets, pasted numbers, screenshots) and it only improves if you keep feeding it results after each asset ships. Some platforms have systems that help automate the loop, while other require the user to manually update engagement for the feedback loop to function.\n\nMost marketing personas are based on demographics and research projects rather than observed behavior. \"Jordan, 34, VP of Engineering, cares about team velocity and developer experience.\" Jordan says he cares about this, but nobody observed Jordan doing anything to prove he does. So when you ask a persona like that whether your post, ad, lead magnet or feature will land, you get confident fiction.\n\nThe fix is grounding every persona behavioral claim or preference in observed behavior (who actually engaged, how they engaged) and requiring each persona to make a prediction that is then tested and scored against actual results. This is what feeds the feedback loop.\n\nThe loop is how personas improve. It can also spot when personas develop biases and help course correct. For example, in testing I found that one persona over-rated any post that shared a third-party benchmark. The feedback loop found the pattern in 2 misses, wrote it into that persona's file as a known bias, and the persona stopped making that mistake.\n\nThe loop is the framework. The personas judge, predict, get scored, and accumulate documented biases. And, most importantly, persona quality becomes measurable and trackable.\n\nAnything you produce for an audience whose reactions you can observe.\n\nHere are a few examples:\n\n- Social content and blog posts\n- Ad copy and landing pages\n- Email campaigns\n- Product ideas and feature framing\n- Messaging and positioning drafts\n- Product documentation and guides\n\nThe evidence source changes (social analytics, ad metrics, email stats, sales call notes, user interviews, etc.). The loop stays the same.\n\nThree skills provide the foundation for building, using, and improving persona-based agents. Feel free to customize them to your speif\n\n```\n/persona-build   →  interviews you + reads your evidence  →  writes persona agent files\n/persona-judge   →  runs your draft past every persona    →  verdict + falsifiable prediction, logged\n/persona-learn   →  ingests real results                  →  scores predictions, updates personas\n```\n\nEach persona is a subagent: a markdown file with a composite identity, behavioral claims and preference traced to evidence, a current-events section refreshed by web search, and a \"known biases\" section that grows as the scoring loop catches repeated misses. See `.cluade/agents/persona-template.md`\n\nfor an agent template.\n\nThe rule that makes it self-improving: **every judgment ends in a falsifiable prediction, and every prediction is scored against real outcomes.**\nA persona that is wrong twice in the same direction gets that bias written into its file, where it reads it before every future judgment.\n\nReal output, condensed. Before this repo went public I ran it through /persona-judge as the asset. Persona names changed to function; roster names redacted.\n\nVerdict: fix and ship.All three observed personas find the framework useful and credible. All three independently flagged the same gap before they'd act on it: no example of the system's output in the docs.\n\nDeveloper (common tiebreaker):Passes the attention test as an eval-loop pattern, not a marketing tool. Would star it as reference architecture and share it framed as \"look how they built the calibration loop\". won't clone as-is. Sharpest point: the \"8 of 10 = trustworthy\" accuracy claim had no chance baseline. A persona that always guesses the most common band looks calibrated while doing zero work.\n\nMarketer:(main targeted persona)Strongest yes. This fixes a persona doc she's personally embarrassed about. Honest self-read: stars it, saves the post, puts it in the slow-week pile. One real output example is what converts her weekend.\n\nBuilder:(secondary target persona)Strong yes. Shares with team, but not for the marketing framework. This is a great tactical example of building agent loops to improve agent performance and efficiency.\n\nConflicts:Developer predicts LOW for launch resonance, while Builder and Marketer predict MID. Developer normally wins ties, but the asset explicitly targets Marketer and Builders, so they outweigh the Developer.\n\nCredibility flags:no baseline for the 8-in-10 claim.\n\nAmplifier check:Roster member [redacted] plausibly comments, and only if the launch post invites extension rather than announcing. Roster member [redacted] expected to share.\n\nPredicted: MID, lower half, driven by marketers and founders, amplifier hook: no.Logged 2026-07-12. Marcus's falsifiable counterclaim: fewer than 10 roster engagers.\n\n*Note that you can tweak the amount of detail you get back from the main agent's summary in `/persona-judge`\n\n. I've opted for brevity.\n\nTwo of the suggested fixes are in the docs now: the accuracy baseline in `docs/methodology.md`\n\nand this example. The prediction analysis is based on launch performance.\n\n**3 skills**:`/persona-build`\n\n,`/persona-judge`\n\n,`/persona-learn`\n\n. Together they run the loop.**An annotated persona template**(`.claude/agents/persona-template.md`\n\n): the structure /persona-build fills in when it generates your persona agents.**Data file templates** in`audience/`\n\n: audience profile, performance log, engagement roster, and prediction log. You populate these; they stay local.**3 docs**: the methodology behind the system, how to collect evidence, and how to adapt it to ads, email, or product work.\n\nSee the folder map below for where each file lives.\n\n- Claude Code installed\n- Some record of how your audience actually behaves: analytics exports, engagement lists, comment threads, campaign metrics, user interview notes. More is better, but you can start thin and build up.\n\n-\n**Clone and open:**\n\n```\ngit clone https://github.com/noashavit/noas-persona-loop.git\ncd persona-loop\nclaude\n```\n\n-\n**Gather evidence. This comes first: the personas are built from it, and /persona-build stops if** Read`audience/`\n\nis empty.`docs/evidence-collection.md`\n\n, then drop what you have into`audience/`\n\n. Raw is fine: CSV exports, spreadsheets, docs with pasted numbers, screenshots of analytics. You don't need a big audience or a long history: 30 engaged people across 5-10 past assets is enough to start. -\nThe\n\n`.template.md`\n\nfiles show the shape the data ends up in; you don't have to match it yourself. Populated files stay local: the`.gitignore`\n\nexcludes them so you don't accidentally push your audience data. -\n**Build personas.** Run:\n\n```\n/persona-build\n```\n\nThe skill reads your evidence, interviews you about your audience and goals, proposes segments weighted by observed engagement (not audience share), and writes one agent file per persona into `.claude/agents/`\n\n. If you have 6+ past assets with known outcomes, it also blind-tests the draft personas against those known results (copy only, results hidden) before finalizing. There's no separate command for this, it's baked into the skill.\n\n-\n**Get persona feedback.** Paste a draft and run:\n\n```\n/persona-judge\n```\n\nYou get per-persona reactions, conflicts between personas, and one synthesized prediction logged to\n\n`audience/prediction-log.md`\n\n. -\n**Close the loop.** When real results come in automatically or via manual updates, run:\n\n```\n/persona-learn\n```\n\nThis skill scores pending predictions, updates your playbook, appends repeated misses to persona bias sections, and refreshes each persona's current-events knowledge using web search.\n\nSteps 1-3 are set up. Steps 4 and 5 repeat forever.\nThe results don't arrive on their own: either bring fresh analytics on a cadence (weekly is a good starting point) or automate collection. More abou this in `docs/evidence-collection.md`\n\n.\n\n```\npersona-loop/\n├── .claude/\n│   ├── agents/\n│   │   └── persona-template.md      # annotated template; /persona-build generates real ones\n│   └── skills/\n│       ├── persona-build/           # interview + generate personas\n│       ├── persona-judge/           # run drafts past personas, log predictions\n│       └── persona-learn/           # score predictions, update personas\n├── audience/                        # YOUR data lives here (gitignored when populated)\n│   ├── audience-profile.template.md\n│   ├── performance-log.template.md\n│   ├── engagement-roster.template.md\n│   ├── prediction-log.template.md\n│   └── evidence/                    # per-persona observed-behavior files\n├── docs/\n│   ├── methodology.md               # how the system was built, phase by phase\n│   ├── evidence-collection.md       # how to gather behavior data from any source\n│   └── customization.md             # adapting to ads, email, product ideas, other channels\n└── CLAUDE.md                        # project instructions Claude Code reads on open\n```\n\nShort version. Full reasoning in `docs/methodology.md`\n\n.\n\n**Every behavioral claim or preference traces to evidence.** If a persona file says \"responds to pricing teardowns,\" there's an observed behavior behind it. No invented preferences.**Weight personas by observed engagement, not audience composition.** In my tests, demographics said one segment was 25% of my personal audience. The engagement data said 8%. The persona was rebuilt at 8%.**Blind test against known results before trusting.** Run new personas on past assets, results hidden, and score their predictions. Personas that fail the blind test get rewritten before they judge anything live.**Predictions are mandatory and falsifiable.** Banded predictions (LOW / MID / HIGH against thresholds you define) with named reasoning. \"This will do well\" is not a prediction.**Score against the metric personas can predict.** Personas predict resonance: engagement given reach, reply rate given delivery, conversion given traffic. Distribution is a separate problem they can't see.**Wrong twice in the same direction becomes a documented bias.** Appended to the persona file, read before every future judgment.**Real people are research anchors, never characters.** Personas can cite what a real person publicly posted. They never fabricate quotes or opinions for real named people.**Persona identity changes slowly, the playbook changes often.** Results update your tactical playbook freely. Rewriting who a persona*is*requires repeated evidence.**Aspirational personas are kept and labeled.** If you want a persona for an audience you don't have yet, keep it. It gets no veto power and its predictions carry no weight until that segment shows up in your data.\n\nMIT. Use it, fork it, sell things you build with it. Attribution appreciated.\n\n[Noa Shavit](https://www.linkedin.com/in/noashavit), product marketer and AI builder based in San Francisco.", "url": "https://wpnews.pro/news/audience-personas-built-from-engagement-data-scored-on-predictions", "canonical_source": "https://github.com/noashavit/noas-persona-loop", "published_at": "2026-07-13 01:44:05+00:00", "updated_at": "2026-07-13 02:05:29.353120+00:00", "lang": "en", "topics": ["ai-tools", "developer-tools", "machine-learning"], "entities": ["Claude Code", "Persona Loop"], "alternates": {"html": "https://wpnews.pro/news/audience-personas-built-from-engagement-data-scored-on-predictions", "markdown": "https://wpnews.pro/news/audience-personas-built-from-engagement-data-scored-on-predictions.md", "text": "https://wpnews.pro/news/audience-personas-built-from-engagement-data-scored-on-predictions.txt", "jsonld": "https://wpnews.pro/news/audience-personas-built-from-engagement-data-scored-on-predictions.jsonld"}}