{"slug": "test-automation-to-agentic-validation-building-governed-ai-agents-for-physical", "title": "Test Automation to Agentic Validation: Building Governed AI Agents for Physical AI Systems", "summary": "An open-source Python project published on GitHub by DrVinay demonstrates a governed multi-agent validation workflow in which LLM-based agents inspect regressions, group failures, gather evidence, reason, run experiments and recommend engineering action inside a CI pipeline for a simulated ADAS hardware-in-the-loop (HIL) system. The project progressively adds tools, a RAG framework, an independent verifier, guardrails, Human-in-the-loop (HITL) authorization and observability through agent traces, and is built on dummy data solely to illustrate the architecture rather than serve as a production blueprint. The author argues agents should complement rather than replace standard V&V engineering processes, letting engineers focus on HITL authorization while agents handle testing at speed and scale.", "body_md": "When the world of system testers is still upskilling to move from manual to automation testing, the system and software validation landscape has taken a leapfrog to agentic validation. Currently, autonomous agents are performing reasoning and action to orchestrate a CI pipeline autonomously.\n\nIn traditional autonomous testing, engineers have created CI pipelines to invoke tests every night and analyze the published results the next morning. The drawback in this traditional method is that we need to wait until the next morning to analyze the failures, find the root cause, take engineering action, and perform the regression tests after the fix. In case of failure, the whole night of platform time is wasted without any valuable action. So, can we ask a teammate to keep an eye on the pipeline the whole night, maybe an AI teammate?\n\nCan an AI agent based on a Large Language Model (LLM) become part of the CI pipeline’s engineering workflow to help investigate, reason, and take engineering action based on the validation evidence?\n\nTo explore this, an open-source Python project is developed that progressively evolved from basic tool addition in an LLM to a governed multi-agent validation workflow.\n\nGitHub repo for the project: [https://github.com/DrVinay/ai-agents-adas-validation](https://github.com/DrVinay/ai-agents-adas-validation)\n\nA simulated ADAS hardware-in-the-loop (HIL) system demonstrates, step by step, how to equip an LLM with tools, orchestrate multiple tools, build a RAG framework, and progressively add an independent verifier, guardrails, Human-in-the-loop (HITL) authorization, and observability through agent traces. The project uses dummy data and is intended solely to illustrate the architecture; it should not be treated as a blueprint for building a production-ready system.\n\n**Smart Physical AI systems demand smart validation** The biggest challenge with a physical AI system is not building it, but the deployment, as it needs to co-live physically in an environment comprising humans, animals, etc. A missed validation opportunity can lead to a loss of life, property, etc. This paved the way for highly automated testing rather than relying on manual testing processes. Furthermore, governed AI agents are emerging as the next key enabler for more efficient validation systems.\n\n- hardware timing\n\n- Sensor noise\n\n- environment interference\n\n- system dynamics limitations\n\n- safety constraints (Strict FuSa, SOTIF guidelines)\n\n- Actuator limitations\n\n- and so on\n\nThe heavy usage of AI models in generating code for these systems has forced the need for faster and more efficient validation methods. The AI agents are helpful in investigating failures, correlating engineering data, history, software release notes, and coordinating targeted experiments. The AI agents should not replace but complement the standard V&V engineering processes, helping engineers to be more efficient.\n\n**Moving from traditional automation to agentic validation** Speed and scale are the two benefits of agentic validation. When the testing team was required to test 100s of variants of the same software and adding more engineers was not an option, the agents came to the rescue. This allowed engineers to focus on HITL authorization while agents were doing the testing. A traditional CI pipeline follows a predefined flow:\n\nPrepare test environment → Test execution → Result collection → Report publishing\n\nIn case of failures, the engineers still have to do the manual steps of initiating triaging groups, creating JIRA tickets, adding logs, etc., before doing the actual work of reasoning, action, and experimentation.\n\nAgents, on the other hand, follow:\n\nInspect regression → group failures → gather evidence → reasoning → experiment → recommend engineering action\n\nThe role of agents is important to maintain causality in triaging, i.e., executing the next engineering action based on history and evidence.\n\n**REACT-inspired architecture**\n\nThe ReAct approach was introduced by Yao et al. in “ReAct: Synergizing Reasoning and Acting in Language Models.” The ReAct workflow is:\n\nEvaluate current evidence → Select a tool → Execute action → Observe result → Update investigation → Repeat\n\nThis project's manually implemented autonomous loop (5A, 5B in Table 1) is not an exact reproduction, but is inspired by the ReAct pattern.\n\n**Core work: Building AI agents** In this work, instead of jumping directly to building a complex multi-agent framework, a gradual introduction of capabilities is demonstrated.\n\nStarting with basics, i.e., interaction with an LLM, adding tools, creating a basic RAG with semantic memory. Lower-level orchestration is demonstrated through an autonomous agent using the *OpenAI Responses API*, where the LLM selects the tool, while Python code performs the operation and returns the results. \n\nIn the bonus section, the agents were given more autonomy using the *OpenAI Agents SDK*, which provides higher-level orchestration primitives such as Agent, Runner, function tools, agent-as-tool delegation, HITL authorization, and tracing.\n\n**Investigation: A simulated ADAS HIL CI pipeline** In this project, a dummy HIL test setup is used to support our point. Consider 500 HIL tests as follows:\n\nOut of 37 failures, 30 are failing due to an ECU flashing issue, accounting for ~81% of failures. The simulated investigation identifies a change in ECU boot-ready time causing all 30 failures.\n\n**Importance of Causality** This software build also introduced a bootloader security check alongside an increase in boot-ready latency. The agent could have also pointed towards the security check and the reason for test failures, but the distinction is particularly important here for physical AI systems\n\nCorrelation between a change and a failure is not sufficient to establish causality.\n\nTherefore, the agent performed a targeted simulation experiment before reaching a conclusion.\n\n**Motivation for adding an Independent verifier to an AI agent**\n\nAdding an independent verifier is inspired by having an independent V&V team doing black-box testing or acceptance testing. The Agent provides evidence containing failure metrics, identifiers, diagnostic observations, experimental results, and a proposed action. A separate verification mechanism checks whether the claim is supported by causality and completeness, mathematical consistency, and targeted experimentation.\n\nAn autonomous agent shall be capable of recognizing insufficient evidence, obtain additional observations, and revise its conclusion.\n\n**Importance of guardrails and HITL authorization for physical AI systems** Guardrails for a physical AI system is not a good-to-have but a must-have. Eg. without a guardrail, a system can be provided 10V when the operating limit is only +-5V, leading to overheating and damaging the system. Similarly, a Human-in-the-loop (HITL) authorization is mandatory to allow the agent to take the action with modified engineering inputs. In AI, it’s called ‘\n\n- Agent reasoning: What action does the evidence suggest?\n\n- Technical verification: Is the engineering justification sufficiently supported?\n\n- Human authorization: Should this particular action be executed?\n\nThe agent itself checks the validity of its proposal, and only valid proposals are sent for human approval.\n\n**Main goal: A governed multi-agent validation workflow** The final implementation, \n\nThe design follows the responsibilities matrix: the Manager investigates, the Verifier reviews, application guardrails enforce the rules, the human authorizes to take sensitive actions, and the engineering tools execute permitted operations.\n\nImportant to note here is the pre- and post-approval of guardrails. This is required to confirm the guardrail validity in a Physical AI system: *the system state may change while the workflow is paused for human authorization*. This helps address time-to-check/time-to-execute risks. \n\nThis design is consistent with Anthropic’s “Trustworthy agents in practice,” where the broader human-control principles are discussed.\n\n**Observability: Keeping an eye on the agent’s actions** For physical AI systems, it is crucial to have control over the agent's actions, with proper evidence to justify the actions chronologically.\n\nWhich tools did the agent invoke?\n\nWhat evidences are collected?\n\nIs the conclusion approved by verifier?\n\nWhich actions are allowed or blocked by which guardrail?\n\nReason for requesting additional experiment\n\nTimestamping of all the actions.\n\nFor industry use, observability should connect agent actions to Requirement IDs, software build IDs, SUT (system-under-test) configurations, platform configurations, JIRA ticket numbers, test execution IDs, evidence references, and approval records, etc.\n\nIf an agent recommends an action, the V&V engineer must be able to reproduce the evidence and decisions that led to it.\n\n**Result of the mock drill: simulated HIL regression** After independent verification, applying guardrails, and human approval, the Agent has changed the ECU flashing timeout from 10 seconds to 15 seconds. Afterward, another human approval is obtained before performing the regression test of all 500 tests.\n\nThe main flashing failure group disappears in the post-change regression, while 7 other failures remain. This demonstrates the complete engineering workflow: investigation, evidence collection, experiment, independent verification, human authorization, controlled action, and post-change validation.\n\n**Application to other physical AI systems** This implementation focuses on ADAS/HIL testing; however, the underlying approach could be generalized to any physical AI systems.\n\nThe discussed architecture is applicable, i.e., engineering tools provide observations, an agent organizes the investigation, an independent verifier assesses conclusions, and application policy governs sensitive execution.\n\n**Conclusion** The project began with interacting with LLMs, followed by adding customized tools using Python code. and followed by adding semantic memory and implementing a REACT-inspired architecture. This demonstrates the importance of having an independent verifier. A sophisticated Physical AI system can't operate outside its boundaries; to enforce this, guardrails are introduced. To ensure the safety of the system and environment, human approval is mandatory for experimenting with complex tasks, as shown in the project.\n\nThe objective is not to replace validation engineers with AI agents. It is to provide validation engineers more power to deal with increasingly complex physical AI systems and making evidence-based decisions faster and at scale.\n\nThe project can serve as a starting point for V&V engineers to start interacting with LLMs, building and orchestrating tools, adding memory, using OpenAI APIs, getting familiar with concepts of guardrails, HITL, multi-agent systems, traces, observability, and agent-as-a-tool.\n\nComplete Python implementation: [GitHub — AI Agents for ADAS Validation](https://github.com/DrVinay/ai-agents-adas-validation)\n\n**Technical foundation and References** The implementation uses OpenAI’s Responses API and Agents SDK, along with Python, SQLite, and ChromaDB. The following sources include implementation documentation, related architectural research, and domain guidance.\n\n**1.** [OpenAI — Responses API: Function Calling](https://developers.openai.com/api/docs/guides/function-calling): Function-tool integration and application-managed tool execution loops.\n\n**2.** [OpenAI — Agents SDK](https://openai.github.io/openai-agents-python/): Agent orchestration, tools, structured outputs, and workflow execution.\n\n**3.** [OpenAI — Human-in-the-Loop](https://openai.github.io/openai-agents-python/human_in_the_loop/) : Sensitive-action approval, interruptions, RunState, and pause/resume workflows.\n\n**4.** [OpenAI — Tracing](https://openai.github.io/openai-agents-python/tracing/): Observability, tool calls, and workflow events.\n\n**5.** [Yao et al. — ReAct: Synergizing Reasoning and Acting in Language Models](https://arxiv.org/abs/2210.03629) : Inspiration for the architectural foundation for reasoning-and-action workflow.\n\n**6.** [Anthropic — Building effective agents](https://www.anthropic.com/research/building-effective-agents) — Best practices for building reliable agents.\n\n**7.** [Anthropic — Trustworthy agents in practice](https://www.anthropic.com/research/trustworthy-agents) — Research on human control, transparency, and security.\n\n**8.** [OpenAI — A practical guide to building agents](https://cdn.openai.com/business-guides-and-resources/a-practical-guide-to-building-agents.pdf) — Practical considerations for architecture, tools, guardrails, and human intervention.\n\n**9.** [ISO 34502 — Scenario-based safety evaluation framework for automated driving systems](https://www.iso.org/standard/78951.html) — Domain context; no claim of standards compliance.\n\n**10.** [ASAM OpenSCENARIO DSL](https://www.asam.net/standards/detail/openscenario-dsl/) — Structured scenario description for ADAS V&V.\n\n[Test Automation to Agentic Validation: Building Governed AI Agents for Physical AI Systems](https://pub.towardsai.net/test-automation-to-agentic-validation-building-governed-ai-agents-for-physical-ai-systems-e5c9f4fe6aa9) was originally published in [Towards AI](https://pub.towardsai.net) on Medium, where people are continuing the conversation by highlighting and responding to this story.", "url": "https://wpnews.pro/news/test-automation-to-agentic-validation-building-governed-ai-agents-for-physical", "canonical_source": "https://pub.towardsai.net/test-automation-to-agentic-validation-building-governed-ai-agents-for-physical-ai-systems-e5c9f4fe6aa9?source=rss----98111c9905da---4", "published_at": "2026-11-17 18:01:03+00:00", "updated_at": "2026-11-17 20:08:06.418587+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "ai-safety", "autonomous-vehicles", "developer-tools"], "entities": ["DrVinay", "GitHub", "ADAS", "JIRA"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/test-automation-to-agentic-validation-building-governed-ai-agents-for-physical", "markdown": "https://wpnews.pro/news/test-automation-to-agentic-validation-building-governed-ai-agents-for-physical.md", "text": "https://wpnews.pro/news/test-automation-to-agentic-validation-building-governed-ai-agents-for-physical.txt", "jsonld": "https://wpnews.pro/news/test-automation-to-agentic-validation-building-governed-ai-agents-for-physical.jsonld"}}