# LongBanana: Expert Benchmark for Long-Context, Multi-Reference Image Synthesis

> Source: <https://explainx.ai/blog/longbanana-benchmark-long-context-multi-reference-image-synthesis-2026?utm_source=rss&utm_medium=feed&utm_campaign=blog>
> Published: 2026-11-09 00:00:00+00:00

**LongBanana is a new expert-verified benchmark for long-context, multi-reference image synthesis** — the pattern where an agent must read dozens of reference images, ignore distractors, and produce one output that satisfies dense instructions. The [project site](https://long-banana.com/?ref=explainx) says it contains **90 tasks**, about **32 reference images per case**, and **4,670 atomic evaluation items**, with code and data public. The paper is [arXiv 2611.06856](https://arxiv.org/abs/2611.06856?ref=explainx), listed on [Hugging Face Papers](https://huggingface.co/papers/2611.06856?ref=explainx), and marked **accepted to NeurIPS 2026** on the homepage.

That matters because image models already look good on short prompts. The harder product shape is **reference-grounded composition**: merchandising boards, animation pipelines, interior layouts, and agent loops that pick visual evidence before calling a generator. LongBanana is built to score that loop, not just aesthetic quality.

## What problem LongBanana targets

Text-to-image leaderboards reward pretty pixels. They rarely check whether the model used **the correct reference object**, kept **relationships** straight, or met **count and layout** constraints when 30 similar thumbnails compete for attention.

The LongBanana authors frame three skills:

1. **Retrieve** relevant objects from a large, heterogeneous reference set and reject hard negatives.
2. **Bind** identity, appearance, and state to the right entities.
3. **Compose** one image that satisfies global constraints — counts, spatial layout, temporal state — not only local texture match.

The benchmark is explicitly aimed at **agent-driven image generation**, where a controller plans steps, calls tools, and may revise output. That aligns with the agentic editing direction in research such as [Adobe EditAgent](https://explainx.ai/blog/adobe-editagent-image-editing-agent-specialized-tools-verification-record-78-percent-10x-compute-2026) and with broader [multimodal context budgeting](https://explainx.ai/blog/multimodal-vision-token-budget-context-window-guide-2026) questions when vision tokens stack up.

## Benchmark shape: 90 cases, three suites, nine categories

LongBanana benchmark taxonomy: three suites and nine capability categories
*Image: [LongBanana](https://long-banana.com/?ref=explainx)*

The site organizes tasks in a **3×3 taxonomy** across three suites of 30 cases each:

| Suite | Focus | Example categories | 
|---|---|---|
| Scene Composition | What is in the scene and how it connects | Semantic coverage, relational graph consistency, layout arrangement | 
| Visual Appearance | Identity and look fidelity | Global appearance conditioning, local appearance binding, harmonization | 
| Reference Reasoning | Choose and infer across long context | Selection and retrieval, temporal or state reasoning, counting and aggregation | 

Each case ships with an **entity–relation graph**. Experts turn graph nodes, edges, and whole-graph obligations into checklist items scored as **fail (0), uncertain (1), or pass (2)**. Correct omission when an entity must be absent also earns pass.

Scale highlights from the homepage and paper summary:

- **90** expert-curated cases in the reported benchmark set.
- **~32** references per case.
- **~20** constraints and**~50** evaluation items per task.
- **4,670** expert-specified atomic items in total.
- At least **30** cases include two or more hard-negative references.

Developers can browse [simulated targets and full reference boards](https://long-banana.com/?ref=explainx) in the gallery before downloading anything.

## Graph-grounded evaluation: why checklists beat one score

LongBanana task overview: references, generation, and checklist evaluation pipeline
*Image: [LongBanana](https://long-banana.com/?ref=explainx)*

LongBanana-Eval scores three layers:

- **Content fidelity (node level)** — required entities present with correct identity and appearance.
- **Relational binding (edge level)** — attributes and relationships attach to the right entities.
- **Global constraint satisfaction (graph level)** — counts, layout, and composition hold for the whole scene.

Each checklist item can carry an **evidence trace**: which references support the judgment, optional OCR or counting tools, and critic outputs before aggregation. The authors report **7,005** double-labeled judgments on a subset with **89.4%** exact agreement and **0.83** quadratic weighted kappa — unusually high for dense vision rubrics, though still human-labeled rather than independent third-party audit.

For teams used to [RULER-style effective-length tests](https://explainx.ai/blog/ruler-benchmark-long-context-evaluation-guide-2026) on text, the analogy is: LongBanana asks where **visual context** stops being usable, not where a single needle fact can be retrieved.

## LongBanana-Gen: agent harness and RPR workflow

LongBanana-Gen agent workflow with retrieve, plan, execute, and reflect
*Image: [LongBanana](https://long-banana.com/?ref=explainx)*

The paper introduces **LongBanana-Gen**, a harness for long-horizon generation. Experiments compare:

- **Direct generation** from a single model call.
- **Minimal agent harnesses** with lightweight tool loops.
- **ReAct–Plan–Exec–Reflect (RPR)** workflows that plan, execute, and repair reference-grounded errors.

The public leaderboard (November 2026 snapshot on the site) highlights that **controller choice dominates**:

- Strongest reported **checklist average** in the table:**78.4 / 100** for an OpenClaw + RPR stack with Nano Banana 2.
- Highest **task pass rate** :**47.8%** , shared by OpenClaw + RPR and OpenCode in the manuscript tables.

The site notes a subtle result: RPR raises checklist averages across harnesses, but **task pass rates can fall** for some controllers if reasoning steps break binding. Better planning is not free if the final generator loses track of references.

That is a lesson for anyone wiring image MCP tools or [ByteDance Seedream-style multi-round editors](https://explainx.ai/blog/bytedance-seedream-5-5-100-round-editing-rgba-november-2026): the orchestration layer needs its own eval, not only the base diffusion model.

## How to try LongBanana this week

1. **Read the task gallery** on[long-banana.com](https://long-banana.com/?ref=explainx) — pick one suite that matches your product (retrieval-heavy retail vs layout-heavy interiors).
2. **Clone the GitHub repo** linked from the paper page and inspect annotation format before you fine-tune.
3. **Submit or reproduce** leaderboard runs using the same 90-case split; do not mix development QC cases into reported scores.
4. **Pair with agent evals** your team already runs — coding agents are not substitutes, but[agent benchmark hygiene](https://explainx.ai/blog/ai-benchmarks-complete-guide-2026) transfers.

Hardware needs depend on your generator stack; the benchmark itself is evaluation data and harness code, not a single shipped foundation model.

## How LongBanana compares to nearby benchmarks

| Axis | Classic T2I leaderboard | Needle-style visual tests | LongBanana | 
|---|---|---|---|
| Inputs | Short text prompt | Often one hidden reference | ~32 references plus distractors | 
| Failure mode | Blur or style drift | Miss one object | Wrong reference, wrong bind, wrong count | 
| Agent loop | Optional | Rare | Central (LongBanana-Gen) | 
| Auditability | Subjective preference | Binary hit | Thousands of atomic checks | 

It does not replace [human preference evals](https://explainx.ai/blog/seth-block-tononi-nature-ai-not-conscious-silicon-commentary-2026) or product A/B tests. It gives a **regression suite** when you change retrieval, reranking, or tool policies in an image agent.

## Why teams should care in November 2026

Image APIs now advertise editing rounds, reference slots, and board-style inputs. Without a benchmark like LongBanana, teams optimize for the first frame that looks plausible, then fail in production when the wrong SKU, character, or layout variant slips through.

The Kolors / KlingAI authorship also signals that video and image labs treating **reference banks** as first-class inputs are investing in public measurement, not only demo reels. If you build on open weights from [Seedream](https://explainx.ai/blog/bytedance-seedream-5-5-100-round-editing-rgba-november-2026) or closed APIs, the harness ideas still apply.

## Scoring protocol: what the leaderboard numbers mean

The LongBanana site publishes two headline metrics in its November 2026 tables: **checklist-average score** on a 0–100 scale and **task pass rate** at a difficulty-weighted threshold (reported as P at 70% in the UI copy). Checklist averages reward incremental correctness across thousands of atomic items. Pass rate asks whether an entire case clears a stricter bar. A system can improve the average while still failing whole tasks when one global constraint breaks — exactly the failure mode product teams see when a render looks polished but violates a count or layout rule.

The human-audited pool described on the site covers **900 generated outputs** across **10 generation conditions** and **46,700 reviewed checklist labels** for those runs. That scale is expensive, which is why the authors emphasize **graph-grounded automation**: critics can call OCR, counting, layout, and identity tools before a human adjudicates borderline items. Teams borrowing the benchmark should not confuse the public leaderboard snapshot with a continuously refreshed production harness; it is a frozen manuscript table unless the repo publishes a live submission API.

Inter-rater stats matter for legal and safety buyers, not only researchers. The **89.4% exact agreement** and **0.83 quadratic weighted kappa** on double-labeled items beat many vision rubrics, but kappa still leaves room for disagreement on edge cases. If you fine-tune on LongBanana scores, hold out entire suites — not random items — so binding and retrieval skills do not leak across categories.

## Implementation notes for teams running evals

**Reference inventory design.** Each case ships with on the order of **32** images plus distractors. Your agent must declare which references it attended to if you want debugging parity with LongBanana-Eval traces. Logging retrieval indices and similarity scores helps explain checklist failures without rerunning generation.

**Controller ablations.** The RPR result is instructive: adding reflection repaired some binding errors but hurt pass rate for certain open controllers. When you A/B harnesses, report both per-item and per-task metrics. A coding-style lesson from [agent harness studies](https://explainx.ai/blog/what-does-a-harness-buy-tokens-mostly-claude-code-opencode-mini-swe-agent-study-2026) applies here — extra turns buy quality only when the tool surface preserves state.

**Compute budgeting.** Thirty-two references per step can dominate vision-token cost before diffusion runs. Pair LongBanana with [token budget guides](https://explainx.ai/blog/multimodal-vision-token-budget-context-window-guide-2026) so you do not confuse benchmark scores with economical production configs.

**NeurIPS acceptance** signals peer review of the methodology, not endorsement of any single commercial generator on the leaderboard. Reproduce from the GitHub assets linked on [long-banana.com](https://long-banana.com/?ref=explainx) before you cite rank numbers in customer decks.

## Related reading

## Sources

- LongBanana project, [homepage, leaderboard, and dataset links](https://long-banana.com/?ref=explainx) , accessed November 9, 2026.
- Cao, H. et al., [LongBanana: An Expert-Verified Benchmark for Long-Context Multi-Reference Image Synthesis](https://arxiv.org/abs/2611.06856?ref=explainx) , arXiv:2611.06856.