Long-context. Multi-reference. Expert-verified.

LongBanana

Many references. One image.
Every detail matters.

An expert-verified benchmark for long-context multi-reference image synthesis. Can a system find the right visual evidence, bind it to the right objects, and satisfy a complex instruction in one coherent image?

GitHub Leaderboard Dataset Explore the gallery

Paper coming soon · Browse all 90 benchmark cases

LAC / 001

Fashion editorial with multiple reference-grounded layouts

Arrange the sceneLayout arrangement

LAB / 001

Headphone lineup with individually specified appearances

Keep the right detailsLocal appearance binding

SEL / 001

Selected reference objects arranged on a display shelf

Find what belongsSelection & retrieval

TSR / 007

A six-panel timeline of car design evolution

Reason across timeTemporal / state reasoning

AGG / 001

Fine-dining composition with precise count requirements

Make every count countCounting & aggregation

Simulated targets from real benchmark cases. Select a case to inspect its references.

What does LongBanana measure?

Beyond looking plausible.

A convincing image can still use the wrong reference.
LongBanana checks whether the evidence and the instruction actually agree.

01 / RETRIEVE

Find the right evidence

Inspect a large, heterogeneous reference set. Select relevant objects and reject visually convincing distractors.

Reference-grounded content

02 / BIND

Preserve what belongs together

Keep identity, appearance, and state attached to the correct entity. Respect the relationships between objects.

Entities & relationships

03 / COMPOSE

Get the whole scene right

Satisfy counts, spatial arrangements, and global constraints while synthesizing one coherent final image.

Scene-wide correctness

90EXPERT-CURATED CASES

32REFERENCES PER CASE

4,670ATOMIC CHECKLIST ITEMS

9CAPABILITY CATEGORIES

The people behind the benchmark

Contributors from

Research communities across academia and industry.

KlingAI Research

South China
Normal University

University of
British Columbia

Stony Brook
University

Tsinghua
University

Hong Kong University
of Science and Technology

Carnegie Mellon
University

The University
of Hong Kong

University
of Waterloo

Meet the authors

Haoxiang Cao^1,3,*,§ · Yuxuan Zhang^2,3,10,*,§ · Penghui Du^3,* · Bo Li³ · Jiajiong Cao³ · Yingying Fan⁴ · Zimeng Wu^3,§ · Yifu Luo⁵ · Mingzhe Zheng⁶ · Junwen Miao⁷ · Huaisong Zhang^3,5,§ · Yang Yao⁸ · Yu Wang⁵ · Dongfu Jiang⁹ · Ping Nie⁹ · Wenhu Chen⁹ · Chaoqun Wang^1,† · Changqian Yu^3,‡ · Kelsey R. Allen^2,10,†

¹ South China Normal University · ² University of British Columbia · ³ KlingAI Research · ⁴ Stony Brook University · ⁵ Tsinghua University · ⁶ HKUST · ⁷ Carnegie Mellon University · ⁸ The University of Hong Kong · ⁹ University of Waterloo · ¹⁰ Vector Institute

* Equal contribution. ‡ Project lead. † Corresponding authors. § Work done during internship in KlingAI Research.

One benchmark. Three suites.

A broad test of
visual grounding.

Each suite contains three capability categories, with ten cases per category. Explore the category most relevant to your system.

SUITE 0130 cases

Scene Composition

What is in the scene, how it connects, and where it goes.

SCCSemantic Coverage & Consistency RGCRelational Graph Consistency LACLayout Arrangement Consistency

SUITE 0230 cases

Visual Appearance

Keep visual identity and appearance faithful to the references.

GACGlobal Appearance Conditioning LABLocal Appearance Binding AHCAppearance Harmonization & Coherence

SUITE 0330 cases

Reference Reasoning

Choose, infer, and count across a long visual context.

SELSelection & Retrieval TSRTemporal / State Reasoning AGGCounting & Aggregation

View the full benchmark taxonomy ↗

LongBanana taxonomy showing three suites, nine subclasses and ten cases each

From visual evidence to auditable scores

Every requirement gets a check.

Graph-grounded evaluation makes errors inspectable across content,
relationships, and scene-wide constraints.

LongBanana task and evaluation overview: reference inventory and instruction, generation and atomic checklist evaluation — LongBanana task overview · Detailed instructions and reference inventories become visually verifiable obligations.

NODE LEVEL

Content Fidelity

Are the required entities present, with the right identity and appearance?

EDGE LEVEL

Relational Binding

Are attributes and relationships bound to the correct entities?

WHOLE GRAPH

Global Constraint Satisfaction

Does the image satisfy counts, layout, and global composition?

0 Fail · 1 Uncertain · 2 Pass / Correct omission receives 2 when absence is required.

How the benchmark is builtScenario → graph → references & instruction → checklist

Experts start from realistic scenarios and construct an entity–relation graph. The same graph aligns reference inventories, synthesis instructions, and atomic evaluation criteria. At least 30 cases include two or more hard-negative references; development and quality-check cases are excluded from the reported 90-case benchmark.

Reverse-construction pipeline: scenario construction, graph construction, prompt and reference synthesis, checklist and curation — Expert curation keeps target instructions, reference evidence, and evaluation criteria aligned.

Inside LongBanana-EvalItem-specific evidence, tools, and auditable judgments

Each checklist item combines the candidate image, relevant references, allowed ordinal labels, and the graph unit being judged. Three critics assess content, relationships, and global constraints, with optional OCR, counting, layout, and identity tools. Every judgment records an evidence trace before aggregation.

LongBanana-Eval routes checklist items and reference evidence to critics and tools before aggregating scores — The independently double-labeled subset contains 7,005 labels, with 89.4% exact agreement and 0.83 quadratic weighted kappa.

The benchmark in practice

Leaderboard

Compare controllers and image generators on the same 90-case benchmark. Values follow the manuscript's main tables.

Show all categories

Avg. = checklist-average score (0–100). P@70% = difficulty-weighted task pass rate. Click a column heading to sort. Bold marks the best value in each numeric column.

STRONGEST CHECKLIST AVERAGE78.4 / 100

OpenClaw + RPR with Nano Banana 2.

HIGHEST TASK PASS RATE47.8%

Shared by OpenClaw + RPR and OpenCode.

THE CONTROLLER MATTERS

RPR improves checklist averages across the reported harnesses, but task pass rates decrease for OpenCode and Hermes-Agent. Better reasoning must preserve the correct reference bindings.

Evaluation protocol & human validation ＋

The human-audited pool covers 90 cases and 10 generation conditions: 900 outputs and 46,700 reviewed checklist labels. Subclass and overall scores use checklist averages; suite averages are the mean of their three subclass scores. The independently double-labeled subset has 89.4% exact agreement and 0.83 quadratic weighted kappa.

LongBanana-Gen

Retrieve. Plan. Reflect.

RPR combines ReAct, Plan–Exec, and Reflect to inspect evidence, organize generation, and repair reference-grounded errors.

LongBanana-Gen workflow combining evidence retrieval, planning, generation and reflection — Agentic generation workflow · The effect of RPR depends on the surrounding controller's ability to preserve reference bindings.

See the benchmark for yourself

The details are the challenge.

Explore 90 cases, their instructions, simulated targets,
and the complete 32-image reference context.

Open the case gallery

Banana-dog poses in three colorful Polaroid frames