# Envisioning Agentic Robotics: S0, S1, S2 and a Four-Level Autonomy Roadmap

> Source: <https://explainx.ai/blog/envisioning-agentic-robotics-survey-s0-s1-s2-roadmap-2026?utm_source=rss&utm_medium=feed&utm_campaign=blog>
> Published: 2026-11-16 00:00:00+00:00

**A survey posted to arXiv as [2611.18359](https://arxiv.org/abs/2611.18359?ref=explainx) argues robotics is entering an agentic era.** The authors—affiliated with **Grid Robotics**—say **GPT-6 Astra** and similar frontier models, wrapped in **embodied harnesses**, can **match or beat task-specific policies** on several embodied benchmarks **without** task fine-tuning. Their framework gives builders a vocabulary we lacked while vendors shipped demos: **S0 body control**, **S1 skills**, **S2 planning and verification**, plus **nine evaluation dimensions** and a **four-level roadmap** for delegating real work.

explainx.ai covered the **industry pitch** in [Berkeley–NVIDIA agentic robotics at CoRL](https://explainx.ai/blog/agentic-robotics-berkeley-nvidia-corl-2026) and [Gemini Robotics 2.5 factory demos](https://explainx.ai/blog/gemini-robotics-2-5-mercedes-kitting-asimov-jailbreak-2026). This paper is the **academic synthesis**: what to measure, what stays hard in physics, and why **harness design** can move success rates as much as **model swaps**.

## TL;DR

| Idea | Plain-language takeaway | 
|---|---|
| Paradigm | Shift from **one policy per task** to**goal pursuit with tools** | 
| Core | **Embodied System 2** = foundation model + harness (context, tools, memory, verifier) | 
| Loop | Six steps: interpret → decide → act → observe → verify → continue or recover | 
| Eval gap | Community mixes **model scores** with**system scores** ; paper pushes separation | 
| Roadmap | **L1 primitives → L4 standing roles** (breakfast demo in Figure 1) | 

## Why policy-only robotics stopped scaling

Classic stacks chained **perception, planning, and control** with hand-tuned modules. Deep RL in simulation produced impressive **locomotion** and **humanoid tracking**, but policies stayed **narrow**.

**Vision-language-action (VLA)** models such as the π series unified **see + language + act**, broadening generalization. **World-action models** added **video dynamics**. Yet the authors note performance can still **collapse** when scenes or instructions shift—exactly what [world-model research](https://explainx.ai/blog/nvidia-birddreams-driving-world-model-bev-perspective-2026) tries to address.

**Agentic robotics** proposes a different fix: keep **general reasoning** in a **foundation model**, expose **callable skills** (VLAs, planners, code) as **S1**, and let **S2** **verify** whether the world actually changed before committing to the next step.

## S0, S1, and S2: responsibilities, not SKUs

Figure 3 in the paper is the cheat sheet builders should screenshot:

### S2 — intent, orchestration, verification

**S2** decides whether to act, gather information, clarify, retry, or stop. It calls **S1 skills** or **S0 interfaces**, then checks evidence against the goal. In products like **Gemini Robotics**, a high-level VLM already does slices of this orchestration; the survey argues that without **explicit verification**, you cannot claim a full agentic stack.

The **embodied harness** is not optional fluff. It assembles **context**, exposes **tools**, maintains **plans and memory**, and runs a **verifier**. Papers cited inside the survey show **harness-only changes** moving success **9–35 points** on some tasks—even when the planner model stays fixed.

### S1 — skills and callable modules

**S1** includes **VLA policies**, **perception stacks**, **motion planners**, and even **Python cells** the model writes at runtime. The survey warns **more tools ≠ better results**: on EmbodiedSkillBench, restricting tools helped weaker models but **hurt GPT-6 Astra** until it could **selectively** use skills—mirroring lessons from [Runway Praxis video-control releases](https://explainx.ai/blog/runway-praxis-1-open-weights-video-robot-control-2026).

Robotic gripper mirroring a hand holding a cup, evoking S1 callable skills in agentic robotics

### S0 — real-time body interface

**S0** executes torques and balance loops without waiting for S2 deliberation. A **real-time event monitor** can **estop** or pause skills when humans interject or collisions occur—critical for [Ring robotics pilot safety narratives](https://explainx.ai/blog/ring-robotics-drone-cam-rover-cam-ai-pilot-november-2026).

## The agentic loop in plain language

The paper’s breakfast **red cup vs blue cup** example is intentionally mundane:

1. User asks for a cup at the table.
2. S2 plans subtasks: locate, grasp, carry, place.
3. Skills run; S0 keeps the arm stable.
4. User changes mind mid-grasp → **event monitor** interrupts S2.
5. S2 verifies cancellation before issuing conflicting commands.
6. A skill may report success while the camera shows failure → **verifier** catches the mismatch.

That loop is what separates **agentic robotics** from **open-loop VLAs**. It also explains why [OpenAI’s GPT-6 Astra lattice science post](https://explainx.ai/blog/openai-gpt-6-astra-standard-model-lattice-nathaniel-craig-science-blog-2026) matters to hardware teams: the same model family is now judged on **whether it can supervise tools**, not just answer prompts.

## Evaluation: nine dimensions and honest reporting

The authors complain the field mixes incompatible numbers—different **RoboDojo** harness settings for Astra, unlike comparisons on [GPT-6 cyber agent releases](https://explainx.ai/blog/vals-ai-cyberoffense-gpt-6-sol-daybreak-deepseek-v4-1-flash-open-weight-cyber-range-2026) where protocols are spelled out.

Their framework demands:

- **Independent outcome verification** (not trusting skill self-reports)
- **Transparent** tool and memory configs
- **Closed-loop** tests while the world still moves

For practitioners, that means publishing **harness JSON**, **skill manifests**, and **interrupt rules** beside model names—exactly what reproducible [eval harness posts](https://explainx.ai/blog/baseten-efficient-frontier-llm-inference-guide-2026) argue for in cloud inference.

Robot hand reaching toward a globe, evoking closed-loop embodied evaluation in agentic robotics

## Roadmap L1–L4: from primitives to jobs

Figure 1 maps **delegation depth**:

| Level | Responsibility | Breakfast demo | 
|---|---|---|
| **L1** | Single primitive | Flip one item | 
| **L2** | Full task | Serve one plate | 
| **L3** | Workflow | Coordinate cooking steps | 
| **L4** | Standing role | “Breakfast host” for the household | 

The authors connect **L4** to **sustained autonomy**—robots trusted with **ongoing roles**, not one-off commands. That is the commercial endgame behind [NVIDIA GR00T marketing](https://explainx.ai/blog/nvidia-birddreams-driving-world-model-bev-perspective-2026) and humanoid hype, but the paper stresses **verification** and **legal monitors** first.

## Digital agents vs physical agents: what transfers

Section 5 of the survey asks which **digital agent** assumptions survive contact with physics. Highlights readers should internalize:

- **Irreversible actions** — deleting a file is not dropping a mug; verifiers must observe**persistent state** .
- **Latency budgets** — S2 deliberation cannot block S0 balance loops; the**event monitor** exists because physics does not wait.
- **Partial observability** — wrist cameras miss occlusions; the breakfast example’s**failed grasp** is the canonical case.
- **Safety envelopes** — predefined estop rules trump model persuasion.

That is why [Gemini Robotics jailbreak sets](https://explainx.ai/blog/gemini-robotics-2-5-mercedes-kitting-asimov-jailbreak-2026) and [Asimov policy research](https://explainx.ai/blog/gemini-robotics-2-5-mercedes-kitting-asimov-jailbreak-2026) matter even when chat benchmarks look solved.

## Papers the survey anchors on (selected)

| Work | Role in the argument | 
|---|---|
| Zhang et al. 2026c **GPT-6 Astra on RoboDojo** | Early evidence frontier models beat task policies | 
| Han et al. 2026 **EmbodiedSkillBench** | Tool restriction helps weak models, hurts strong ones without selective use | 
| Wei et al. 2026a **SCOPER** | Evidence memory + verification guides recovery | 
| Harness VLA (Zhang et al. 2026f) | Memory-guided agents lift frozen VLAs | 

Builders should read **at least one** primary robotics eval (RoboDojo, EmbodiedBench, LIBERO-PRO) before quoting a single **success rate** from social media.

## Failure modes the roadmap acknowledges

The **L1–L4** ladder is not optimism—it is a **responsibility gradient**. Standing roles (L4) imply:

- **Accountability** when the robot misjudges user intent over weeks.
- **Continual learning** from physical experience without catastrophic forgetting.
- **Regulatory visibility** into what S2 logged before each irreversible act.

That matches policy conversations in [South Korea’s CSAT integrity panic](https://explainx.ai/blog/south-korea-tax-exam-ai-cheating-qualification-reforms-2026): society tolerates AI cheating headlines before it tolerates **physical harm** from autonomy.

## Open questions the survey leaves explicit

The authors do **not** claim agentic robotics is **solved**. They flag unsettled areas worth watching in 2027:

- **Who owns verification** when S1 skills are third-party VLAs with opaque latents?
- **How to benchmark standing roles (L4)** without multi-week human studies?
- **Whether execution memory** should live in S2 harness or inside S1 policies (Harness VLA work cited in the survey reports large swings).
- **Legal liability** when S2 replans after user interruptions mid-motion.

Those questions intersect [Norway laser aquaculture robots](https://explainx.ai/blog/norway-stingray-ai-laser-salmon-sea-lice-optical-delousing-2026) and warehouse autonomy posts—not just humanoid hype.

## Connection to explainx.ai coverage threads

This survey stitches together threads explainx.ai has followed separately:

- **Frontier models in physical loops** —[GPT-6 Astra science](https://explainx.ai/blog/openai-gpt-6-astra-standard-model-lattice-nathaniel-craig-science-blog-2026) plus embodied eval citations inside arXiv 2611.18359.
- **Harness beats raw parameters** — same lesson as[NVIDIA SkillEvaluator](https://explainx.ai/blog/nvidia-skillevaluator-agent-skills-three-tier-cli-guide-2026) for digital agents, now with torque limits.
- **Verification as product** — aligns with[AI homework vs exam scores](https://explainx.ai/blog/ai-homework-scores-exam-scores-learning-penalty-2026) : closed-loop checks matter more than open-loop confidence.

If you ship one takeaway to leadership: **budget for harness engineers**, not just bigger foundation weights.

Download [arXiv 2611.18359](https://arxiv.org/abs/2611.18359?ref=explainx) ([PDF](https://arxiv.org/pdf/2611.18359?ref=explainx); the authors also run a [project website](https://grid-robotics-official.github.io/agentic-robotics-roadmap?ref=explainx)) and read **Sections 2–4** before you buy another **policy-only VLA** license. The figures are dense, but they are cheaper than retooling a production line around the wrong abstraction. Skim the bibliography if you need a reading list for Q1 planning. Assign one paper per sprint to your robotics pod. Discuss whether your current vendor demo maps to L1 tasks only, not promised L4 roles.

## Glossary for standups

| Term | One-line definition | 
|---|---|
| **Agentic loop** | Six-step cycle ending in verified world state | 
| **Embodied harness** | Runtime wiring context, tools, memory, verifier | 
| **S2** | Planner that can refuse, clarify, or replan | 
| **S1** | Callable skills including VLAs and scripts | 
| **S0** | Millisecond body control | 
| **WAM** | World-action model predicting dynamics + actions | 
| **L4 role** | Ongoing responsibility, not a single command | 

## What builders should do this month

1. **Label your stack** — Map existing components to S0/S1/S2 before you rebrand a VLA as an “agent.”
2. **Invest in verifiers** — Cheap RGB checks often beat bigger models without them.
3. **Benchmark the harness** — A/B planner constants before you swap weights.
4. **Read Astra robotics evals skeptically** — Compare budgets, tool lists, interruption policies, and harness versions.
