# Show HN: URML – safety-eval harness for AI agents on lab and factory hardware

> Source: <https://github.com/URML-MARS/URML/tree/main/examples/physical-ai-safety-eval>
> Published: 2026-08-28 16:28:06+00:00

A small, opinionated, human-readable language for describing robot intent.

**What it measures**: whether an agent's proposed intent is admissible on declared hardware under a declared deployment envelope, with a machine-readable reason for every refusal and the evidence class of every limit a refusal relied on. **What it does not measure**: physics. URML judges declared limits and intent coherence; whether a declaration is true is the integrator's, the vendor's, or a runtime measurement's job, and the evidence tag says which.

Built for the gate Anthropic named for its [Model Hardware Standard](https://www.anthropic.com/news/model-hardware-standard-research-preview): the standard opens after "safety evaluations and best practices for AI systems that operate physical equipment" exist. The cell here is shaped like the assay in their post (a liquid handler, a plate-handling arm, a plate reader). It is not an MHS reference file and claims no compatibility; see [ docs/integrations/model-hardware-standard.md](/URML-MARS/URML/blob/main/docs/integrations/model-hardware-standard.md) for where URML sits.

- Validates each intent in
`intents.yaml`

whole, against`lab-cell.manifest.yaml`

(the device's own limits) and`deploy.envelope.yaml`

(the site's stricter limits). Strictest wins. - For every accepted program, rehearses it under a declared motion model and lets the RFC-0667 envelope monitors judge the trace (the runtime shield's view of the same envelope).
- For every refusal, prints the codes and the evidence tag of the capability the refusal leaned on (RFC-0631: declared, derived, verified).
- Lowers accepted programs, and only those, through the
`MhsAdapter`

scaffold onto read/write calls against a recording transport, proving no refused intent produced a device call.

Seven intents: two admissible (run the assay plate; park and read deck temperature) and five named failure modes an agent might propose: crushing a plate with 250 N, wandering to an undeclared room, picking an object the cell never declared, measuring on an instrument that is not there, and asking an arm to take off.

```
python run_safety_eval.py
```

Hermetic, a few seconds, no model and no device. The committed `safety-eval-report.txt`

is byte-asserted in CI.

`mhs_adapter.py`

is the shape of the adapter URML ships when the MHS specification is open: validated primitives lower onto `read(key)`

/ `write(key, value)`

over an injectable transport, with the key map as deployment configuration. **The key names and value encodings are placeholders**, not the specification. Promotion to a real `urml-mhs-runtime`

with RFC-0014 conformance fixtures waits for the open spec ([RFC-0683](/URML-MARS/URML/blob/main/docs/rfcs/0683-model-hardware-standard-outreach.md)).

Add a row to `intents.yaml`

with `expect: accept`

or `expect: refuse`

plus the codes the refusal must carry; the harness asserts its own expectations, so a corpus row that stops behaving as documented fails CI rather than a reader.