cd /news/ai-safety/show-hn-urml-safety-eval-harness-for… · home topics ai-safety article
[ARTICLE · art-114385] src=github.com ↗ pub= topic=ai-safety verified=true sentiment=· neutral

Show HN: URML – safety-eval harness for AI agents on lab and factory hardware

URML, a new open-source safety-evaluation harness for AI agents operating lab and factory hardware, validates robot intent against declared device and deployment limits, issuing machine-readable refusals with evidence tags. The tool, posted on Hacker News, is designed to complement Anthropic's Model Hardware Standard, which awaits safety-evaluation tooling, and includes seven test intents, two admissible and five failure modes, with a CI-asserted report.

read2 min views3 publishedAug 28, 2026
Show HN: URML – safety-eval harness for AI agents on lab and factory hardware
Image: Michielbdejong (auto-discovered)

A small, opinionated, human-readable language for describing robot intent.

What it measures: whether an agent's proposed intent is admissible on declared hardware under a declared deployment envelope, with a machine-readable reason for every refusal and the evidence class of every limit a refusal relied on. What it does not measure: physics. URML judges declared limits and intent coherence; whether a declaration is true is the integrator's, the vendor's, or a runtime measurement's job, and the evidence tag says which.

Built for the gate Anthropic named for its Model Hardware Standard: the standard opens after "safety evaluations and best practices for AI systems that operate physical equipment" exist. The cell here is shaped like the assay in their post (a liquid handler, a plate-handling arm, a plate reader). It is not an MHS reference file and claims no compatibility; see docs/integrations/model-hardware-standard.md for where URML sits.

  • Validates each intent in intents.yaml

whole, againstlab-cell.manifest.yaml

(the device's own limits) anddeploy.envelope.yaml

(the site's stricter limits). Strictest wins. - For every accepted program, rehearses it under a declared motion model and lets the RFC-0667 envelope monitors judge the trace (the runtime shield's view of the same envelope).

  • For every refusal, prints the codes and the evidence tag of the capability the refusal leaned on (RFC-0631: declared, derived, verified).
  • Lowers accepted programs, and only those, through the MhsAdapter

scaffold onto read/write calls against a recording transport, proving no refused intent produced a device call.

Seven intents: two admissible (run the assay plate; park and read deck temperature) and five named failure modes an agent might propose: crushing a plate with 250 N, wandering to an undeclared room, picking an object the cell never declared, measuring on an instrument that is not there, and asking an arm to take off.

python run_safety_eval.py

Hermetic, a few seconds, no model and no device. The committed safety-eval-report.txt

is byte-asserted in CI.

mhs_adapter.py

is the shape of the adapter URML ships when the MHS specification is open: validated primitives lower onto read(key)

/ write(key, value)

over an injectable transport, with the key map as deployment configuration. The key names and value encodings are placeholders, not the specification. Promotion to a real urml-mhs-runtime

with RFC-0014 conformance fixtures waits for the open spec (RFC-0683).

Add a row to intents.yaml

with expect: accept

or expect: refuse

plus the codes the refusal must carry; the harness asserts its own expectations, so a corpus row that stops behaving as documented fails CI rather than a reader.

── more in #ai-safety 4 stories · sorted by recency
── more on @urml 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
Live at https://your-agent.zahid.host
Get free account → Pricing
from €0/mo · no card required
LIVE [news/show-hn-urml-safety-…] indexed:0 read:2min 2026-08-28 ·