# Google Research Agile Deliberation: Build Image Classifiers for Fuzzy Concepts by Refining Your Own Definition > Source: > Published: 2026-11-04 00:00:00+00:00 Google Research has published Agile Deliberation, a framework that helps people define fuzzy visual categories by judging the hard cases an AI shows them. The method, a CVPR 2026 paper, turns classifier building into a loop: a vision-language model surfaces borderline images, the user rules on each, and an automatic prompt-optimization step rewrites the written definition. In a study with 18 live sessions, Google reports better labeling consistency and lower effort than manual methods. The research post gives no numeric scores, so the size of the gain is not checkable from the text alone. The post, by Otilia Stretcu and Leijie Wang, appeared on November 4, 2026 on the [Google Research blog](https://research.google/blog/agile-deliberation-how-ai-helps-humans-define-the-gray-areas/?ref=explainx). The paper is on [arXiv as 2512.10821](https://arxiv.org/abs/2512.10821?ref=explainx), titled "Agile Deliberation: Concept Deliberation for Subjective Visual Classification." ## TL;DR | Question | Answer | |---|---| | What problem does it solve? | Users start with a vague idea of a concept ("healthy food") and cannot label consistently until the idea is sharper | | How? | Two stages: concept scoping, then concept iteration with borderline images | | What powers it? | Vision-language models plus automatic prompt optimization | | Evidence | 18 live user sessions, about 1.5 hours each, two domains | | Numbers published? | Not in the blog text; findings are qualitative | | Code? | Open source in the google-research repository, with a demo notebook | | Who worked on it? | Google Research, with Google Ads, per the post | ## Why is a subjective classifier hard to build? Standard supervised learning assumes the label is settled. Show a model enough pictures marked "healthy" and "not healthy," and it learns the pattern. The trouble is the human at the start. The arXiv abstract says existing human-in-the-loop approaches "typically assume users begin with a clear, stable concept understanding," while in reality users "start with a vague idea and must iteratively refine it through concept deliberation." The authors say they found this practice through structured interviews with content moderation experts. The Google post puts the same point in plain terms: "Traditional training workflows don't account for this concept evolution." A moderator who starts with "clickbait promising rewards" will meet a borderline post in the first hour that forces a decision the written policy never made. Without a tool that surfaces such cases early, labels drift, and the classifier inherits the inconsistency. This is a close cousin of a problem builders already know from LLM work: a prompt or rubric that looks clear on paper fails on the edge cases. Our posts on [context engineering versus prompt engineering](https://explainx.ai/blog/context-engineering-vs-prompt-engineering-2026) and [prompt engineering, fine-tuning and RAG](https://explainx.ai/blog/prompt-engineering-vs-fine-tuning-vs-rag) cover the text side of that gap. ## How does Agile Deliberation work? Google describes two stages. **Stage 1: concept scoping.** The user breaks a broad concept into positive sub-categories and explicit exclusions. The post's example: "Healthy Food" might include salads and grain bowls and exclude processed foods. This gives the system a structured first draft instead of one sentence. **Stage 2: concept iteration.** Vision-language models find images that challenge the current text definition. The user judges each borderline case. That feedback feeds an automatic prompt-optimization loop that rewrites the definition used by the classifier. Google says this draws on active learning, but it selects images for semantic ambiguity rather than only model uncertainty. In other words, it is not looking for pictures the model is unsure about statistically. It is looking for pictures where the definition itself is arguable. The output is a classifier driven by an improved text definition, not a newly trained image network. That matters for cost and speed: changing a prompt is cheaper than retraining. It also means the quality depends on the underlying vision-language model. ## What did the study test? Google ran 18 live user sessions of about 1.5 hours each, with conditions in randomized order, across two domains: - **Healthy Food** , a concept with many cultural and nutritional edge cases. - **Pay-to-Play** , clickbait that promises misleading financial or digital rewards. The post says the work was done with Google Ads. It compared three baselines against Agile Deliberation: 1. Zero-shot classifiers. 2. The Modeling Collaborator approach, which automates concept decomposition. 3. Manual deliberation: spreadsheets, guessed search queries and hand-written prompts. ## What were the results? The post reports these findings, without numeric values in its text: - Participants showed significantly higher labeling consistency over time. - Their classifiers outperformed the baselines. - Gains were largest on the hardest borderline images. - Experts said the system exposed "blind spots" in their initial definitions. - Manual deliberation often failed to improve accuracy or even degraded it, according to a chart caption. - Participants reported significantly lower effort and stress, plus higher perceived success and ease of use. Every participant preferred Agile Deliberation. Notice what is missing. We could not see accuracy figures, confidence intervals or effect sizes in the blog text. The charts may carry values, but they were not readable in the text we processed. The arXiv abstract (version 2) does give headline figures: about 7.5% higher F1 than automated decomposition baselines and over 3% higher than manual deliberation. Those are the authors' numbers, so read the paper for the setup behind them. ## What people are likely to ask **Is this the same as active learning?** Not quite. Classic active learning picks the examples the model is least sure about. Agile Deliberation picks examples where the definition is arguable, which can be images the model is quite sure about but a human would dispute. The post says it draws on active learning but selects for semantic ambiguity. **Does it train a new vision model?** The post describes an automatic prompt-optimization loop that improves the text definition. It does not describe training a new image network. That keeps iteration cheap, but the result is only as good as the vision-language model reading the prompt. **Why not just write a longer prompt?** That is the manual deliberation baseline: hand-written prompts, spreadsheets and guessed search queries. Per the chart caption Google cites, that route often failed to improve accuracy or made it worse. The difference is that the tool brings the hard cases to you instead of waiting for you to think of them. **Can a small team use it?** The code and notebook are public, and the only fixed cost is reviewer time. Plan for a long first session. Google's participants spent about 1.5 hours per session. **Does it replace a policy document?** No. It helps produce one. The decisions you record while judging borderline images are exactly the clauses a good policy needs. ## What are the limits? The post has no limitations section. From what it says, a reader should keep these in mind: - **Small study.** Eighteen sessions in two domains is a start, not proof for every concept type. - **No static benchmark.** The authors argue offline datasets cannot capture how a person's concept shifts. That is a fair point, but it also makes comparison with other methods harder. - **Expert-sized effort.** Sessions ran about 1.5 hours. A team needs to budget that time per concept. - **Model dependence.** The quality of surfaced borderline cases depends on the vision-language model. Biases in that model carry into the definition. The post does not report a bias analysis. - **Subjective means contested.** If two moderators disagree, the tool refines one person's view. It does not settle policy disputes between teams. ## How to try it Google says the code is open source at `github.com/google-research/google-research/tree/master/agile_deliberation`, with an interactive notebook, `Demo.ipynb`, that walks through the Healthy Food example. A practical plan for a first test: 1. Pick one real, fuzzy category from your own work, for example "unsafe product photo" or "off-brand image." 2. Write the scoping lists first: three to six positive sub-categories and a list of explicit exclusions. 3. Run the notebook on a few hundred of your own images. 4. Record every borderline decision you make. Those become your policy notes even if you never ship the classifier. 5. Compare the final classifier with a plain zero-shot prompt on a held-out set you label separately. Step 5 is the honest check. If the refined definition does not beat the zero-shot prompt on your data, you have learned that your concept was already clear, or that your model is the limit. ## Where it fits in the wider research picture Agile Deliberation belongs to a growing group of tools that treat the human as a collaborator in the loop, not a label source. We have seen the same direction in other Google Research work: [ToolGrad](https://explainx.ai/blog/google-research-toolgrad-tool-use-dataset-generation-2026) builds tool-use data, [Generative UI for teachers](https://explainx.ai/blog/google-research-generative-ui-teachers-simulations-2026) puts the human in control of a generated interface, and [StreamVibe](https://explainx.ai/blog/streamvibe-proactive-video-agent-google-research-when-to-speak-benchmark-2026) studies when an agent should speak up. For the general question of when a person should stay in the loop, see our guide to [human-in-the-loop AI and when to let an agent run](https://explainx.ai/blog/human-in-the-loop-ai-when-to-let-agent-run-2026). The lesson that carries over to builders: the specification is part of the model. A written definition that has been tested against its own edge cases is worth more than a larger model reading a vague one. If you run content moderation, ad review or dataset curation, this is a practical pattern even if you never use Google's code. Show reviewers the ambiguous cases early, write down their rulings, and fold those rulings back into the definition. ## What to watch - Whether Google publishes numeric results or releases a hosted tool beyond the notebook. - Independent replications on other concepts, such as safety categories. - Whether the approach extends from images to video and text, where the same fuzziness appears. *Details come from the Google Research post of November 4, 2026 and the arXiv abstract. Claims about results are Google's and were not independently reproduced.*