# Can Modern VLMs Actually Reason Through Spatial Grid Puzzles?

> Source: <https://dev.to/fahmiiii/can-modern-vlms-actually-reason-through-spatial-grid-puzzles-560f>
> Published: 2026-10-11 23:14:45+00:00

## 
  
  
  What I Benchmarked

Current Frontier Visual Language Models (VLMs) excel at multi-modal reasoning and captioning, but how well do they execute step-by-step visual spatial verification?

To test this capability, I created **VCIS-Bench (Visual Constraint & Instruction Simulation)**, a synthetic 7×7 grid puzzle benchmark. Each puzzle features a strict layout: an Agent (** A**), Key (** K**), Locked Door (** D**), Goal (** G**), and Walls (**#**). The agent must follow strict cardinal mechanics—it cannot pass through walls, and it **must collect the key** on the left side of a central partition before unlocking the door to reach the goal on the right.

I evaluated the models across **3 core tasks**:

1. 
**Task 1: Overall Plan Verification:** Given an action sequence, determine whether the proposal is`VALID` or`INVALID` .
2. 
**Task 2: Spatial Grounding Verification:** Evaluate spatial awareness by verifying object coordinates relative to labeled row (**R1–R7** ) and column (**C1–C7** ) headers.
3. 
**Task 3: Step-Level Error Diagnosis:** Identify the exact 1-based step index ($1, 2, 3, \dots$) where an action plan first violates a rule (e.g., hitting a wall or passing through a door without a key).

## 
  
  
  Models Tested

I evaluated three distinct models across different parameter tiers to measure accuracy, latency, and cost-efficiency:

- 
**Claude Sonnet 5.5 (Anthropic):** Benchmark state-of-the-art vision reasoning.
- 
**Gemini 3.7 Flash (Google):** Google’s fast multi-modal flash model.
- 
**Gemma 4 26B A4B (Google):** Open-weights vision-language architecture.

These models provide a clear perspective on the performance gap between top-tier proprietary frontier models, lightweight API endpoints, and open-weights models.

## 
  
  
  Findings

Here is how the models performed across the benchmark suite:

### 
  
  
  Leaderboard Overview

| Task | Claude Sonnet 5.5 | Gemini 3.7 Flash | Gemma 4 26B A4B | 
| **Task 1: Overall Plan Verification** | **100.0%** | **100.0%** | **83.3%** | 
| **Task 2: Spatial Grounding Verification** | **100.0%** | **100.0%** | **96.7%** | 
| **Task 3: Step-Level Error Diagnosis** | **100.0%** | **92.0%** | **76.0%** | 
| **Overall Score** | **1.00** | **0.97** | **0.85** | 

*(Data evaluated on October 11, 2026)*

### 
  
  
  Key Takeaways

- 
**Step-Level Simulation is the Hardest Bottleneck:** While**Claude Sonnet 5.5** achieved a perfect**100% accuracy** across all tasks, performance dropped for smaller models on Task 3.**Gemini 3.7 Flash** reached 100% on binary verification tasks (Tasks 1 and 2), but dropped to**92.0%** when required to pinpoint the exact 1-based step index of a failure.
- 
**Spatial Coordination vs. Open-Weight Execution:****Gemma 4 26B A4B** performed strongly on Task 2 spatial coordinate checks (**96.7%** ), proving that open-weight models are proficient at vision grounding. However, its performance decreased to**76.0%** on step-level execution (Task 3) due to tracking errors across multi-step state transitions.
- 
**Cost-Efficiency Sweet Spot:** Looking at the**Score vs. Total Cost Pareto Chart** ,**Gemini 3.7 Flash** offered a favorable efficiency tradeoff—delivering a near-perfect**0.97 aggregate score** at roughly half the total cost and execution latency of Claude Sonnet 5.5.

## 
  
  
  My Benchmark

Check out the task code and interactive leaderboard runs on Kaggle Benchmarks: