Show Lab at the National University of Singapore released ForceHarness, an open agent harness that lets frozen frontier API models do contact-rich manipulation by closing a fast force loop beneath the model and returning physics the model can reason about — with no finetuning and no human demonstrations. On a real Franka FR3 arm that had joint torques but no wrist force sensor, the authors report GPT-6 Astra rising from 2/60 to 54/60 successful episodes across six tasks, and they summarize that as 3.3% → 90.0% zero-shot success in the paper abstract.
If you already ship vision-language-action stacks or GR00T-style sim glue, the pitch is narrow but sharp: RGB-only policies still fail when the task is decided by touch — whether a USB connector seated, a screw caught thread, or one bottle is heavier than another. ForceHarness is the missing harness layer between a frozen multimodal model and a robot arm that already has compliant control.
What problem ForceHarness targets #
Frontier models can plan pick-and-place from images. Contact-rich skills break that assumption for two reasons the authors spell out on the project page:
- Contact evolves faster than an API model chooses actions , so something must regulate force between model calls.
- Frozen models do not interpret high-rate force streams , so raw wrenches need translation into quantities models already reason about.
That is the same gap CoRL 2026 career advice keeps highlighting: language-scale models and touch-scale physics run on different clocks.
Architecture: command path vs force path #
ForceHarness diagram connecting frozen GPT-6 Astra to force primitives and physics translation for contact-rich robot manipulation Image: Show Lab, National University of Singapore
The harness splits into a blue command path (images in, tool calls out) and a teal force path (primitives run locally, return structured physics). Three pieces matter for builders:
Self-terminating force primitives
Twelve contact skills (search, wiggle, screw, slide, lift, and others) share one control loop. A target pose leads measured pose so the arm’s impedance controller turns lead into force; an admittance law holds commanded force; the primitive stops on contact, stall, resistance drop, or its internal goal, then holds still while the model thinks. That is how the system respects API latency without dropping torque safety.
Physics translation layer
Fixed estimators turn each primitive’s trace into mass, centre-of-mass offset, friction, contact height, stiffness, insertion advance — each with uncertainty windows, no hand-tuned task thresholds. The layer reports how heavy a load is, not whether that is heavy enough, so the frontier model makes the judgment call.
Force sensing without a wrist F/T sensor
With a wrist sensor, hand force is direct. Without one, the stack solves w = (Jᵀ)⁺ τ_ext from joint torques. Three runtime rules handle bias: tare at rest, settle then read, and report deltas rather than absolutes — important on arms like the FR3 setup in the paper.
Real-robot results (Franka FR3, GPT-6 Astra) #
The project page publishes a per-task success table (10 episodes each) for six contact-rich skills: USB insertion, screw driving, eggplant peeling, whiteboard wiping, bottle weight sorting, and firmness probing. No-force GPT-6 Astra totals 2/60; ForceHarness totals 54/60; GPT-6.1 Sol + ForceHarness lands 47/60. The authors note Fisher exact p < 0.01 between force and no-force conditions for GPT-6 Astra.
Those tasks are deliberately force-first: weight sort and firmness probe need sensing the camera cannot see. That aligns with Hubego’s contact-rich data pitch — internet video rarely shows fine contact.
Simulation: ForceBench and HiddenPhys #
Sim-to-real transfer illustration for ForceHarness benchmarks in Isaac Lab and HiddenPhys MuJoCo
ForceBench (Isaac Lab, RGB only)
On held-out seeds the release had not used before, GPT-6 Astra + ForceHarness averages 82.5% across eight ForceBench tasks versus 62.5% without force and 35.3% for the best trained row reported in the original ForceBench paper (200 demos per task, not reproduced by the authors). Details and seed splits are on the project page tables.
HiddenPhys (MuJoCo, hidden physical variable)
HiddenPhys is their new seven-task suite where success requires inferring a physical property the image hides. GPT-6 Astra + ForceHarness scores 135/140 total successes with 0 damaged handles in their table, matching a hand-written expert force controller (136/140, 1 damage) and beating demonstration-trained π₀.₅, ForceVLA, and Diffusion Policy rows that show higher damage counts.
They ship 1,423 expert episodes on Hugging Face dataset Jack97/ForceHarness-Data for replay and LeRobot training.
How to try it (reproduce before you trust it) #
The repo is MIT-licensed tooling around paid API calls — budget accordingly.
git clone https://github.com/showlab/ForceHarness
cd ForceHarness && pip install -e ".[sim]"
python scripts/check_sim.py
python scripts/run_episode.py --task weight_sort \
--model gpt-6-astra --paper-defaults --seed 9100
For agent-assisted setup, the authors ship skills/forceharness-setup/SKILL.md: paste their prompt into Claude Code, Codex, or Cursor so the agent installs simulators gate-by-gate and never moves hardware without confirmation. That pattern mirrors how explainx.ai covers external sandboxes for coding agents — safety gates belong in the harness, not the model card.
Paid API costs scale with episodes; start in HiddenPhys simulation before pointing at a factory arm.
Why this matters if you build agentic robotics #
| Design choice | ForceHarness implication |
|---|---|
| Frozen API model + tool loop | You can add new contact skills as tools without retraining VLA weights. |
| Datacenter inference | Accept latency ; primitives own the millisecond loop. |
| Dataset collection | Beats 200-demo force VLAs on public suites in their tables — shifts ROI towardharness engineering . |
| Safety | Damage counts on HiddenPhys are explicit; pair with ViCoNE physical-AI CVE work for deployment hardening. |
ForceHarness does not replace NVIDIA GR00T or UniRobot training stacks. It argues a complementary layer: when the model is already frontier-scale and API-only, force tools + physics summaries may outperform another fine-tune.
Relation to GPT-6 Astra robot claims #
Several 2026 robot posts cite GPT-6 Astra manipulation demos. ForceHarness is the first open release we saw that publishes full episode tables, held-out seeds, and damage metrics side by side with no-force baselines on identical harness settings. Treat marketing clips and paper tables differently until you replay seeds locally.
Limitations the authors acknowledge #
- API cost and latency cap episode length; primitives exist partly to reduce call frequency during contact.
- Simulation seeds used during ForceBench tool development are not held out; the page labels which rows are strictly held out.
- Smaller models still struggle on some tasks even with physics translation — read ablations inarXiv:2611.21099 before swapping models.
Comparison snapshot #
| System | Training data | Force path | Highlight |
|---|---|---|---|
| ForceHarness + GPT-6 Astra | None (zero-shot API) | Primitives + physics layer | 54/60 real arm successes reported |
| π₀.₅ + Prompt Fusion (ForceBench cited row) | 200 demos/task | Learned | 35.3% mean in authors’ comparison table |
| ScaleVLA 552B VLA | Robot demos + huge VLM | Not force-first | Sample efficiency on LIBERO-PRO |
| Expert force controller (HiddenPhys) | Hand-written FSM | Classical | 136/140, benchmark ceiling |
Governance and export context (why timing matters) #
This drop landed during Nvidia Megaspeed export-control headlines and Waymo regulatory resets. None of that changes how you pip install ForceHarness, but teams shipping embodied agents in regulated markets should keep hardware procurement separate from model API geography — the harness runs wherever your arm and inference endpoint live.
What to watch next #
- Independent labs replaying held-out seeds on ForceBench.
- Whether GPT-6.1 Sol + ForceHarness closes the small gap behind Astra on the real arm table.
- Integration patterns with MCP tool hosts so force tools sit beside camera tools in one agent schema.
Related reading #
Episode monitoring and debugging #
The project page describes a web monitor that runs beside real-robot episodes: live wrist force, model observations, tool calls, and structured tool results, with recordings for side-by-side review. That matters because contact failures are often one wrong primitive — not a bad high-level plan. If you already run coding-agent external sandboxes, treat force episodes the same way: log every tool boundary before you blame the base model.
Debugging checklist we extracted from the release README and paper:
- Confirm sim health with
python scripts/check_sim.pybefore spending API credits. - Replay a known-good seed from the paper defaults before random exploration.
- Compare damage counters on HiddenPhys — trained VLAs in their table show more broken handles than ForceHarness rows.
- Gate hardware behind the shipped setup skill so calibration completes before the arm moves.
FAQ for procurement teams #
Robotics leads asking vendors “Do you support force-aware API agents?” can cite ForceHarness as an open reference architecture, not a warranty. It does not certify ISO safety; it shows how tool schemas can wrap admittance control under frozen weights. Pair vendor answers with Waymo’s regulatory story: regulators care about failure modes, not benchmark slides.
Paper metadata #
| Field | Value |
|---|---|
| arXiv | 2611.21099 |
| Authors | Ke Sun, Kevin Yuchen Ma, Yike Zhang, Yanzhe Chen, Mike Zheng Shou (Show Lab, NUS) |
| Code | github.com/showlab/ForceHarness |
| Data | huggingface.co/datasets/Jack97/ForceHarness-Data |
ForceHarness is education-grade open tooling: read the paper, run HiddenPhys, and only then decide whether your product needs another VLA fine-tune or a force-aware tool harness on the model you already pay for.